Dynamic Programming

# Dynamic Programming - Contents 10 Dynamic Programming(DP...

This preview shows pages 1–5. Sign up to view the full content.

Contents 10 Dynamic Programming (DP) 511 1 0 . 1S equ en t i a lD e c i s i onP r o c e s s e s .............. 5 1 1 10.2 Backwards Recursion, a Generalization of Back Substi- tu t i on . .......................... 5 2 1 1 0 . 3S t a t eSp a c e ,S t a g e s ,R e cu r s iv eEqu a t i on s........ 5 2 4 10.4 To Find Shortest Routes in a S t a g edA cy c l i cN e tw o rk . ................ 5 3 0 1 0 . 5Sh o r t e s tR ou t e s-2. ................... 5 3 4 10.6 Solving the Nonnegative Integer Kn ap s a c kP r ob l emByDP. ............... 5 3 9 10.7 Solving the 0 1Kn s a c r l emb yDP. ...... 5 4 2 1 0 . 8AD i s c r e t eR e s r c eA l l o c a t i r l em . ....... 5 4 7 1 0 . 9Ex e r c i s e s.......................... 5 5 3 1 0 . 1 0R e f e r c e s......................... 5 6 3 i

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
ii
Chapter 10 Dynamic Programming (DP) This is Chapter 10 of “Junior Level Web-Book Optimization Models for decision Making ”byKattaG .Murty . 10.1 Sequential Decision Processes So far, we have discussed methods for solving single stage or static models ; i.e., we f nd a solution at one time for the model and we are done. But in many applications we need to make a sequence of decisions one after the other. These applications deal with a process or system that is observed at the beginning of a period to be in a particular state . That point of time may be a decision point where, one out of a possible f nite set of decisions or actions is to be taken to move the system towards some goal. Two things happen, both depend on the present state of the system, and the decision taken: (i): an immediate cost is incurred (or reward earned) (ii): the action moves the system to another state in the next period. And the same process is repeated over a f nite number of periods, n say. Thus, a sequence of decisions are taken at discrete points of time. The aim is to optimize an objective function that is additive over time, to get the system to a desired f nal state. The objective may be to 511

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
512 Ch.10. Dynamic Programming minimize the sum of the costs incurred at the various decision points, or to maximize the sum of the rewards earned if the problem is posed that way. The important feature in such a sequential decision process is that the various decisions cannot be treated in isolation, since one must balance a desirable low cost at the time of a decision with the possibility of higher costs in later decisions. Here we have a multistage problem involving a f nite number of stages, n . The system may be in several possible states. As time passes, the state of the system changes depending on the sequence of decisions taken and the initial state at the beginning. Because of these changing states of the system, the approach for optimizing the performance of such a system is called dynamic programming (DP) . A selection that speci f es the action to take at each decision point is called a policy . The aim of DP is to determine an optimal policy that minimizes the total costs in all the stages (or maximizes the total reward if the problem is posed that way). DP solves such problems recursively in the number of stages n . At each decision point it selects an action that minimizes the sum of the current cost and the best future cost. We will now illustrate these basic concepts with some examples.
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 11/06/2011 for the course ISE 421 taught by Professor Km during the Spring '11 term at King Fahd University of Petroleum & Minerals.

### Page1 / 56

Dynamic Programming - Contents 10 Dynamic Programming(DP...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online