{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

notes3 - Dynamic Programming Peter Ireland EC720.01 Math...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Dynamic Programming Peter Ireland * EC720.01 - Math for Economists Boston College, Department of Economics Fall 2009 We have now studied two ways of solving dynamic optimization problems, one based on the Kuhn-Tucker theorem and the other based on the maximum principle. These two methods both lead us to the same sets of optimality conditions; they differ only in terms of how those optimality conditions are derived. Here, we will consider a third way of solving dynamic optimization problems: the method of dynamic programming. We will see, once again, that dynamic programming leads us to the same set of optimality conditions that the Kuhn-Tucker theorem does; once again, this new method differs from the others only in terms of how the optimality conditions are derived. While the maximum principle lends itself equally well to dynamic optimization problems set in both discrete time and continuous time, dynamic programming is easiest to apply in discrete time settings. On the other hand, dynamic programing, unlike the Kuhn-Tucker theorem and the maximum principle, can be used quite easily to solve problems in which optimal decisions must be made under conditions of uncertainty. Thus, in our discussion of dynamic programming, we will begin by considering dynamic programming under certainty; later, we will move on to consider stochastic dynamic pro- gramming. References: Dixit, Chapter 11. Acemoglu, Chapters 6 and 16. 1 Dynamic Programming Under Certainty 1.1 A Perfect Foresight Dynamic Optimization Problem in Dis- crete Time No uncertainty * Copyright c 2009 by Peter Ireland. Redistribution is permitted for educational and research purposes, so long as no changes are made. All copies must be provided free of charge and must include this copyright notice. 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Discrete time, infinite horizon: t = 0 , 1 , 2 , ... y t = stock, or state, variable z t = flow, or control, variable Objective function: X t =0 β t F ( y t , z t ; t ) 1 > β > 0 discount factor Constraint describing the evolution of the state variable Q ( y t , z t ; t ) y t +1 - y t or y t + Q ( y t , z t ; t ) y t +1 for all t = 0 , 1 , 2 , ... Constraint applying to variables within each period: c G ( y t , z t ; t ) for all t = 0 , 1 , 2 , ... Constraint on initial value of the state variable: y 0 given The problem: choose sequences { z t } t =0 and { y t } t =1 to maximize the objective function subject to all of the constraints. Notes: a) It is important for the application of dynamic programming that the problem is additively time separable: that is, the values of F , Q , and G at time t must depend only on the values of y t and z t at time t. b) Once again, it must be emphasized that although the constraints describing the evolution of the state variable and that apply to the variables within each period can each be written in the form of a single equation, these constraints must hold for all t = 0 , 1 , 2 , ... . Thus, each equation actually represents an infinite number of constraints.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern