1
15.053
Thursday, May 2
z
Dynamic Programming
–
Review
–
More examples
Handouts:
Lecture Notes
2
Match game example
z
Suppose that there are 50 matches on a
table, and the person who picks up the
last match wins.
At each alternating turn,
my opponent or I can pick up 1, 2 or 6
matches.
Assuming that I go first, how
can I be sure of winning the game?
3
Determining the strategy using DP
z
n = number of matches left (n is the
state/stage
)
z
g(n) = 1
if you can force a win at n matches.
g(n) = 0
otherwise
g(n) = optimal value function.
At each state/stage you can make one of three
decisions:
take 1, 2 or 6 matches.
z
g(1) = g(2) = g(6) = 1
(boundary conditions)
z
g(3) = 0;
g(4) = g(5) = 1.
The recursion:
z
g(n) = 1 if g(n1) = 0 or g(n2) = 0 or g(n6) = 0;
g(n) = 0 otherwise.
z
Equivalently,
g(n) = 1 – min (g(n1), g(n2), g(n6)).
4
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
The same table
5
Principle of Optimality
z
Any optimal policy has the property that
whatever the current state and decision,
the remaining decisions must constitute
an optimal policy with regard to the state
resulting from the current decision.
z
Whatever node j is selected, the remaining
path from j to the end is the shortest path
starting at j.
6
Finding shortest paths in graphs
with no directed circuits.
1
2
5
4
3
6
7
8
5
1
3
2
1
1
2
4
7
3
6
If a network has no directed cycles, then the nodes
can be labeled so that for each arc (i,j), i < j.
Such a node labeling is called a
topological order
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document7
Finding a topological order
1
2
5
4
3
6
7
8
5
1
3
2
1
1
2
4
7
3
6
Find a node with no incoming arc.
Label it node 1.
For i = 2 to n, find a node with no incoming arc from
an unlabeled node.
Label it node i.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '05
 Prof.JamesOrlin
 Dynamic Programming, Shortest path problem, optimality, best npv

Click to edit the document details