1 WW form - Introduction to Dynamic Programming Dynamic...

Info icon This preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Dynamic Programming Dynamic programming is a methodology that can be effectively applied to a subset of the general class of mathematical programming problems. It is particularly useful for problems where the variables are restricted to discrete values (though they need not necessarily be integer), there can be stochastic aspects to the problem, and the functions need not be convex. There are problem structure and computational limitations for the applicability of dynamic programming. As with many methods, dynamic programming is an efficient solution methodology within its effective applicable range which in terms of the general class of mathematical programming problems is relatively restiicted. The methodology is surprisingly robust in the sense of the broad class of problems that can be formulated as dynamic programming problems and the apparent lack of standard structural limitations such as continuity and convexity assumptions. Dynamic programming generally is used to decompose a multiple variable problem ' into a sequence of related problems each with fewer variables (usually a single variable) I than the original problem This approach is particularly appropriate for multiple period decision problems for which the methodology was originally developed. The development of dynamic programming is generally attributed to Richard Bellman (1920—1984) who published the first paper on the method in 1952 [Bellman, 1952] concerning the solution of (sequential stochastic decision problems, the first book [Bellman 1957] on the topic general methodology, and the first computationally oriented text [Bellman and Dreyfus, 1962]. A short history of the development of dynamic programming and the first problem classes _ studied is given in an article in memorial to Richard E. Bellman [Bellman and Lee, 1984]. In the chapters on dynamic programming, we present a variety of problems and solution methods. Emphasis is on the computational aspects of formulating problems and obtaining their solutions. A widevariety of different problem classes will be covered including serial, nonserial, stochastic, multiple constraints, sequencing and approximating 3 continuous variable problems. The use of dynamic programming as a theoretical analysis tool is not emphasized and can be found in various texts such as Dreyfus and Law [1977]. ' it We utilize two computer languages for this exposition, MGR/DP and MGR/ML. The MOR/DP system was developed specifically for solving dynamic programming recursion function formulations. In this language it is particularly easy to develop a dynamic programming problem and solve it directly. The language has a great deal of flexibility, but it is restricted to single state and decision variable formulations. For problems which deal with sets, such as the job sequencing and traveling salesperson problems,-the MGR/ML language is used. This more general purpose language has both set handling capabilities as well as recursive function calls which facilitate development of certain algorithms. As an introduction to the techniques involved in dynamic programming, we consider a production planning problem. This example will highlight the essential features of dynamic programming, particularly the recursive formulation and resulting backward solution technique. This problem falls into the class of network structures which have a very descriptive graphic representation that'effectively illustrates the solution method. Example Problem A company has known weekly orders for varying numbers of a given product over the next 5 weeks. These requirements are 50, 30, 60, 45, and 25. To make a production run for this product, it costs $25 to setup the machine and each unit produced costs an additional $100. The company has production capacity and storage space for as many units of the product as needed; however, it costs $1 per unit per week to store units needed in future time periods. The production capacity is such that all five future period demands could be produced in a single run._ Of course, the company would incur a considerable holding cost. Thus, the problem is to determine in which periods to make production runs and how many items to produce on each run. This analysis is to be based on a minimum cost basis. Note that the unit cost is the same regardless of the production periods and the total demand is 210 units over this period. Therefore, the total cost to produce these units is 210 times $100. This cost could be omitted from the analysis and only the controllable costs associated with machine setups and item holding analyzed. We will include the unit cost in our analysis for completeness and-it should help illustrate how to directly analyze situations where this cost varies with the time period. The production decision in period 1 is to decide how many period demands (1 to 5) are to be produced It is obvious that the demands should be produced in their order of use; to do otherwise would merely increase the holding costs over that associated with producing the needed units during the latest possible production run before they are required. The next production decision occurs when more units need be produced to meet the demand requirements. Based on the associated setup and holding costs, the optimal decision could vary from making a production’run each time period to making only one production run in the first period. We graphically represent this production planning problem solution via a network representation of the problem in Figure 1, where the circles or nodes represent a decision period and the arcs represent the decisions taken. The nodes depict both the decision and the demand points. Thus, selecting a given are from a node corresponds to a specific decision to produce a given number of period demands at this node; the arc ends at the next , period in which a production run must be made based on the decision taken at the previous node. Note that a sixth node is included in the network representation of the problem. This node is needed to complete the production of all the demands during the planning period and represents where the planning horizon ends for this problem. Figure 1: Decision diagram of the production planning problem as a network. To solve the problem, we need some notation. Let d,- be the known demand in period i, and let x,- be the decision taken at node or period 1', with the x,- value from the set {d,-, di+ dm, , d; + :15“ + + dn]. Let f,- be the optimal (minimal) cost for node 1' (period 1) to the end of the problem (node 6). We are actually interested in finding f1, but as we will see, this requires that we obtain all ffs sequentially (backwards) from f5 to f1 (we define f6 = 0). We proceed to define these functions in the order that they occur in Figure 1, starting with f1; f1 = min{25+100d1+f2, 25+100(d1+d2)+1d2+ f3. 25+100(d1+d2+d3)+1(d2+2d3)+ f4, 25+100(d1+d2+d3+d4)+1(d2+2d3+3d4)+f5_ 25+ 100(d1+d2+d3+d4+d5)+1 (d2+2d3+3d4+4d5) }. Rewriting this equation in a more readable form, we have the full set: f1 = min{clg+f2, C13+f3, €14+f4a 615+va €15+f6 }, f2 = mifliczfifs. €24+f4: €25+f5a C28+f5 ], f3 '-' mifllca4+fm 035+f5a €35+f6 }, f4 = min{c45+f5, C46+f6 }, f5 = minl055+f6 }, where cil- represents the cost in periodj of producing period demands from i to j—l (that is, xi = d; + + all-4), and we define f6 = 0.‘ A more concise representation is f, = $31k” +fj}. To actually evaluate f1 we must already know f2, , f6. These functions (including f1) are defined generally as fig=min{c,-j +fj}, fori=1,---,5. J): Note that to solve for f1, we must know f2, ,f6, and to solve for f2, we must know - f3, ,f5, and so forth. Finally, to solve for f5, we need only know f6, which we defined as 0. Thus, we can solve for f5, then from this we can solve for f4, etc., until we have worked our way back to solving for f1. This solution procedure is very common to dynamic programming and is called backward recursion. Generally, we solve dynamic programming problems by starting at the end and working backward to the start of the problem This structure occurs because of the forward recursive definition of the optimization ftmctions. Now to solve the production planning problem, we must compute Ctj and the evaluate f5, , f1, in that order. For this particular problem, the cij's are: (:12 = 25 + 100(50) = 5025 c3 = 25 + 100(50+30)+1(30) = 8055 c14 = 25 + 100(50+30+60)+1(30)+2(60) = 14175 815 = 25 + 100(50+30+60+45)+1(30)+2(60)+3(45) = 18810 c16 = 25 + 100(50+30+60+45+25)+1(30)+2(60)+3(45)+4(25) = 21410 c23 = 25 + 100(30) = 3025 C24 = 25 + 100(30+60)+1g60) = 9085 cg = 25 + 100(30+60+45)+1(60)+2(45) = 13675 _ (:25 = 25 + 100(30+60+45+25)+1(60)+2(45)+3(25) = 16250. c3,4 = 25 + 100050) = 5025 C35 = 25 + 100(60+45)+1(45) = 10570 5,6 = 25 + 100(50+45+25)+1(45)+2(25) = 13120 c45 = 25 + 100(45) = 4525 cm5 = 25 + 100(45+25)+1(25) = 7050 as, = 25 + 100(25) = 2525 Then, 1% = 0 f5 = ' 1:37:3533223050 70 0 11‘an“ .7 $016M» }1§osg f3 = m {6025 + 7050, 10570 + €525, 1312040: 13075 0 or? re 0 ‘ --5 0,, s“ 5 f2 = min {302591-i3075, 9085 a 39%01‘213675 + 2525, “62 = 16100 A 212% f1 = min {soiiiglg 8055 + 13075, 14175 + 7050, £ 18810 + 2525, 21410} = 21125 an The optimal solution cost is $21,125, and the decision taken at period 1 is to produce only :5 for that period. Notice, at this point, we do not know the values of other of the optimal decisions. It turns out, for this problem data, that the holding cost relative to the setup cost is too expensive to inventory any items. .44» 2 ~=2 ?.—-> 355%? How do we obtain the other optimal decisions (x2, , x5)? We trace the optimal solution by taking the path of the optimal decision at node 1, x1 = 50, which puts us at node 2. The optimal decision at node 2, x2 2 30, which puts us at node 3, etc. The optimal policy then is to produce only the amount needed in each time period (50, 30, 61), 45, 25). This model is generally called the Wagner—Whitin model even though their solution procedure was based on a forward-recursion dynamic programming method. The standard solution approach now is backward-recursion since it is generally easier to present and , understand. HomeWork: Resolve this problem with the setup cost of {Syd/rather than $25. '(Answer, (30, 0, 60, 70, 0) for a total cost of $21,220.) References Bellman, Richard E. 1952. "On the Theory of Dynamic Programming." Prgg. Ngl, ‘ M, Vol. 38, 716-. Bellman, Richde 1957. Dynamic Programming. Princeton University Press, Princeton, N. I. Bellman, R. E. and S. E. Dreyfus. 1962. Applied Dynamic Programming. Princeton Univ. Press, Princeton, N. J. Bellman, R. E. and ES. Lee. 1984. "History and Development of Dynamic Programming." iEEE gggntlgl Sysgems Magazine, Vol. 4, No. 4, Nov. 1984, 24- 28. Dreyfus, S. E. and A. M. Law. 1977. The Art and Theory of Dynamic Programming. Academic Press, New York. Wagner,H. M. andT. M. Whitin. 1958. "Dynamic Version of the Economic Lot Size Model", Managemengsgen 11,9; V01. 5, 89 96. ...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern