This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 8 The Prototype Sequential Decision Process The networks studied in the unstarred sections of this chapter are particu
larly simple. They are ﬁnite and acyclic. A key property of such networks is
introduced in terms of Figure 2—2. This network is ﬁnite and acyclic, but the
labels have purposely been omitted from its nodes. These nodes can be labeled
with the integers 1 through 4 in such a way that each are (i, j) has 1' less than j.
Do so. (There is exactly one such labeling.) Labeling Figure 22 in this way
should convince you that any acyclic network consisting of N nodes can have
its nodes labeled with the integers 1 through N so that each are (i, j) has i less
than j. For the cognoscenti, We now sketch a labeling procedure that accom
plishes this. Note that at least one node must have no arcs terminating at it
(else there would be a cycle). Label this node with the integer I. Then delete
it and all arcs emanating from it. Examine the remaining network. Note that it must have at least one node at'which no arcs terminate (for the same reason). :
Label that node with the integer 2. Et cetera. FIGURE 22. SHORTEST AND LONGEST PATHS The prototype of all dynamic programming problems is to ﬁnd the shortest (or
the longest) path through a ﬁnite acyclic network. In acyclic networks, shortest
path and longest—path problems are interchangeable; either can be converted to
the other by multiplying the lengths of all the arcs by “I. A shortestpath prob
lem is now worked out in complete detail. Then a longestpath problem is
analyzed, with some of the details left to you. ShortestRoute Problem John bicycles to work regularly. Being methodical, he has decided to
calculate the fastest of several routes from his home to the ofﬁce. With some
effort, he has collected the data summarized in Figure 2—3. Node 1 represents
his home, node 9 represents his ofﬁce, and the remaining nodes represent road
junctions. Directed arcs represent roads between junctions. The number
adjacent to each arc is the travel time in minutes on the road it represents; it
takes 15 minutes to travel from junction 4 to junction 7. The arrow is a some—
what artiﬁcial device; it represents John’s prior conviction that when bicycling
to work he might wish to traverse a road in the direction of the arrow, but Would
never wish to do so in the opposing direction. Effectively, all the “streets” are Shortest and Longest Paths 9 FIG U H E 23. oneway; a version with twoway streets is described in the chapter’s starred
section. Note that the nodes are labeled such that each are (i, j) has i < j. Note
alsothat the labeling is not unique (e. g., labels 2 and 3 can be interchanged). With a cursory examination of Figure 23, you can satisfyyourself that the
fastest path from home to oﬂice is (l, 3, 4, 5, 7, 9) and that its travel time is
19 minutes. Having solved John’s problem by inspection, we now do so by
dynamic programming. The intent of this is to provide insight into dynamic
programming. Let f,r = the minimum travel time from node i to node 9 By deﬁnition, f,. is zero, and you have just observed that fI = 19. Let t‘U denote
the travel time along are (t, j), so :47 = 15. Interpret ti} as the travel time of the path from node i to node 9 that ﬁrst traverses are
(1', j) and then travels as quickly as possible from node j to node 9. As this is a
path from node i to node 9, its travel time must be at least as large as )2. In other
' words,
12 g t” +13, ii 9 But the fastest path from node i to node 9 traverses some are (1', j) ﬁrst and then
gets from node j to node 9 as quickly as possible. So some j satisﬁes the displayed
inequality as an equality, and ' (21) _ f, = min [in +JG}, ia’: 9 The set of those j over which the. righthand side of equation (21) is to be
minimized has not been represented explicitly; minimization occurs over those
j for which (i, j) is an arc. When i = 4, one minimizes the righthand side over
the values 5, 6, 7, and 8 of j. The case 1' = 9 is not covered by equation (21),
but we have already observed that ' ' (L2)  f9 = 0 Notice that equation (2—1) speciﬁes 1‘} once f, is known for every j such
that (i, j) is an arc. _The nodes in Figure 2~3 have been numbered so that every
arc (i, j) has j greater than 1'. Hence, f; can be determined from (21) once f}
is known for every j greater than 1'. Consequently, (21) allows computation of 10 The Prototype Sequential Decision Process f, in decreasing i, starting with i = 8 and ending with 1' = 1. If this is your ﬁrst
exposure to dynamic programming, you are advised to ﬁnd pencil and paper,
perform the somewhat tedious calculation just described, and compare your
results with what follows. (Alternatively, you might do the calculation directly
on Figure. 2—3.) fa=10+fg=1Q__
f1: 3+f9=
. 7+fs . 7+10
ﬁ—min{15+f9—min{15+ OHIS
f5: 4+f5 4+10
. 15+f1_ . 15+ 3_
f4—m1n 7+fa—mm 7+10—14
3+fs 3+15
f3=min{3+f4:min{3+l4=17
4+f,, 4+15
. 12+}; . 12110
2‘ 2 2 0
f2 rnin{6+er mm{6+14 2
__ . 1+fz__ . 1+20ﬁ
flam1n{2+f3—m1n{2+l7——l9 Figure 24 summarizes the information obtained from this calculation.
Travel time f, is recorded above node 1', each are (i, j) attaining the minimum
in (21) is preserved, and all other arcs are deleted. Note that Figure 2—4 pre "
scribes the fastest path from each node to node 9, not just from node 1 to node
9. It turned out to be easiest to soive John’s problem by embedding it in the
more general problem of ﬁnding the shortest path from each node to node 9. Equation (21) is the prototype of the equations of dynamic programming.
The argument used to justify (21) holds whether or not the network’s nodes FIGURE 24. Shortest and Longest Paths 11 are labeled so that each arc (1‘ i) has z' < j. It holds for any are lengths, including
negative lengths. It even holds for cyclic networks, provided that no cycle has
negative length. Although (2—1) holds more generally, the preceding computational method
works only for those acyclic networks whose nodes are labeled so that each are
(i, j) has i< j. This labeling often arises naturally, as when decision making
evolves over time. More general methods of computation are described in this
chapter’s starred section. Sequential decision prooesses usually represent decision problems involv
ing tradeoffs between immediate costs and future costs. The myopic solution
to the shortestroute problem selects for each node i the shortest directed arc
(i, j) emanating from it. The myopic solution disregards the node at which an
arc terminates, for which reason it may be far from optimal. In fact, the network
in Figure 2~3 is designed so that the myopic solution to the shortestroute
problem yields the longest path from each node to node 9. When an equation system has exactly one solution, it is said to characterize
that solution. The numbers { f1, . . . , f9} satisfy (21) and (22). Moreover, we
have computed these numbers from (21) and (2—2), which means that no other
numbers satisfy these equations. In other words, (21) and (22) characterize
the set { f1, . . . , f9} of fastest travel times to node 9. [In this, we mean that the
functional values are unique, not that the maximizing indices in (21) are unique;
ties occur when multiple paths have the same length] Shortest Path Trees The network displayed in Figure 24 has one fewer arc than node, and it
prescribes exactly one path from each node to node 9. This network is a tree
of paths to node 9. The tree’s path from node i to node 9 has length ﬂ, which
is the shortest of the lengths of all the paths from node i to node 9. Hence, the
network depicted in Figure 24 is a tree of shortest paths, or a shortestpath tree.
Rather than ﬁnding the shortest path from node 1 to node 9, we have found a
tree of shortest paths to node 9 from all other nodes. More generally, a subset of the arcs in a network is called a tree of paths
to node j if this are set contains exactly one path from each node i, with i e’: j,
to node j. Similarly, a subset of arcs is called a tree of paths from node i if this
are set contains exactly one path from node i to each node j, except j = i. No
tree can contain a cycle, because then it would have multiple paths. One can
show that every tree contains one fewer arc than the number of nodes in the
network. When the methods of dynamic programming are applied to network
optimization problems, one often gets a tree of shortest paths. LongestRoute Problem One might suspect that only extreme pessimists could be interested in the
longest path through a directed acyclic network, but this will turn out to be
false. Consider the problem of ﬁnding the longest path from node 1 of Figure DYNAMIC _
PROGRAMMING MODELS AND APPLICATIONS ERIC V. DENARDO School of Organization and Management
Yale University El 7—9 (Shortest Route Problem) The network given in Figure 7—7 give
diﬁ'erent routes for reaching city B from city A passing through a number c
other cities. The lengths of the individual routes are shown on the arrow: It is required to determine the shortest route from A to B. Formulateth
problem as a dynamic programming model. Explicitly deﬁne the stage:
states, and return function; then ﬁnd the optimal solution. ...
View
Full Document
 Fall '10
 Curry
 Dynamic Programming, Shortest path problem, travel time, John bicycles

Click to edit the document details