This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Larry Karp Notes for Dynamics September 19, 2001 II. General Dynamic Problem in Resources, Calculus of Variations 1) Description of dynamic optimization problem 2) Statement of necessary condition (Euler equation) 3) Interpretation of Euler Equation 4) Autonomous problems and steady states 5) Derivation of Euler Equation 6) Special functional forms 7) Necessary and sufficient conditions 8) Different boundary conditions for dynamic problems 9)An example: extraction of a nonrenewable resource 1) Examples of control problems in natural resources x = stock of resources, y = "harvest" f(x) = natural growth U(x,y) flow of payoff  the integrand r = discount rate ert = discount factor T = final time xT = terminal condition
T x0 = initial condition max P1 s.t. m
t0 e r t U x (t ) , y (t ) dt state equation initial condition terminal condition x f(x) y xt 0 given x(T) xT In this problem, x is the state variable, y is the control variable, and dx/dt = f(x) y is the equation of motion. + +/U(x, y) fisheries fisheries (compet) fisheries (monopoly) fisheries (social planner) ____________________ pollution = = =  + p(t)y  c (x, y) p(y) @ y  c (x, y) w(y)  c(x,y) where wN / p(y) x = f(x) + g <contribution to pollution + v (g)  D (x) benefit damage function function x = y U(x,g) = nonrenewable resource f(x) / 0 For the C.O.V. problem write integrand as F (t, x, x) using y = f(x)  x. For example, for fishery controlled by social planner: F (t , x , x ) / w f (x) x c x , f (x) x e r t I can write P1 as T max P2 s.t. m
t0 F (t , x , x) dt x0 given xT given I have eliminated the constraint x=f(x)y by substitution. I get rid of one variable (y) and one constraint and write the problem in terms of x and its derivative dx/dt. It is common to eliminate the control variable (here y) when using COV to solve a dynamic problem. That is, instead of choosing the optimal trajectory for the control variable y (harvest), we explicitly choose dx/dt.1 2. Necessary Condition When using the Maximum Principle, we typically retain the control variable y, and we do not eliminate the equation of motion. 2:2
1 (Subscripts indicate partial derivatives.) A necessary condition for P2 is that the optimal x* solves the Euler equation: Fx (t , x , x ) Fxt Fxx d Fx(t , x , x ) dt dx d 2x Fxx dt dt 2 Fxt Fxxx Fxx x This is a second order ODE, and we have 2 B.C.'s, x(()=x( and x(T) = xT This is a Atwo point boundary value problem@ (TPBVP) 3. Interpretation of Euler Equation
t1 t1 Interpretation of Euler equation: Integrate, using or,
t1 m
t Fx ()d dF () Fx(t1) Fx(t ) m x
t m
t Fx( , x() , x()) d = Fx(t , x (t1) , x (t1) Fx(t , xt , x(t )) Benefit over t,t1 of having the extra fish in the stock Leaving one more fish in sea at t and taking it out at t1 (reallocating consumption) Mention case where Fx / 0. This inequality would hold if, for example, a larger stock does not make it any cheaper to catch fish, and the growth is independent of the stock of fish. The independence of growth on the stock of fish is not plausible. However, the independence of the growth on the stock does hold for a nonrenewable resource, such as oil, or for a resource with exogenous growth (e.g. the water in an aquifer, where additions to the stock might be independent of the amount of water currently in the aquifer). If Fx / 0 the Euler equation implies that the present value of the marginal utility of consumption is constant. Continue interpretation of Euler Equation for fish example with no stock dependent costs x = f(x)  y 2:3 T max e m
0 rt U f (x ) x dt x0 , xT given Here y is the harvest (consumption) and U(y) is utility of consuming fish. Euler equation is d e r tU N(y) e r tU N(y) f N(x) dt If I integrate EE over (t, t1) I get
t1 rt1 e r t U N(yt ) m
t e rs U N(ys )f N(xs)ds e U N(yt1) LHS = PDV of consuming extra fish at t Consider the following perturbation of the optimal trajectory. If I refrained from eating the marginal fish at time t, and then ate it at time t1 , I can eat the Aextra fish@ produced by this marginal fish over the interval (t,t1). The marginal fish (that I refrain from eating at time t) increases the growth by f N(xs). When I eat these extra fish, I obtain a PDV of marginal utility of ersUN(y s) f N(x s) at time s <t 1 (and I still have the marginal fish at time s). I eat the marginal fish at t1. At optimum, there is no benefit of reallocating consumption in this manner. In other words, the perturbation that I described yields no increase in utility. In the problem above, time enters only through the discount factor, and because of the upper limit of integration  the finite horizon  T. This fact makes it possible to eliminate t from the Euler equation, as shown below. If I write out Euler equation (which requires that I find re rtU N(y) e rtU NN (y)y e rtU N(y)f N(x) U NN (y) y f N(x) r U N(y) d rt e U N(y) and cancel ert I have: dt This expression is independent of time. 2:4 I can write LHS of the last version of the Euler Equation as d U N(y) dt U N(y) In words, the EE says that along the optimal trajectory, the proportionate rate of change of marginal utility must equal the difference between growth rate of capital stock (fish) and "rate of impatience" concave utility Y U NN > 0 UN Y y > 0 Special case U(y) = ln y, Euler equation is iff f N (x ) r > 0 y f N (x) r y 4. Autonomous Problems and Steady States Consider the previous fish problem (where harvest costs do not depend on the stock of fish), but replace the upper limit of integration T by infinity . The optimal trajectory to this infinite horizon problem converges to a steady state. At this steady state (where y is constant) it must be the case fN(x) = r. In other words, the steady state for the infinite horizon version of this problem is independent of the utility function, U. The steady state requires that the marginal growth rate of fish equals the discount rate. The control problem is said to be autonomous if and only if the horizon is infinite and time enters explicitly only through a constant discount rate. In this case, the optimal control depends only on the state, not on time. Think about why this timeindependence requires an infinite horizon. If there were a finite horizon, the amount of Atimetogo@ would affect the optimal control. If there is a finite horizon, but time does not enter the integrand except via discounting, we saw from the previous section that the Euler Equation is independent of time. However, with a finite horizon, the amount of time until we reach the terminal boundary condition is finite, and it changes as calendar time changes. Thus, the solution to the twopointboundaryvalue problem depends explicitly on time. In contrast, if the time horizon is infinite, the Atimetogo@ is always infinite  i.e. it is constant. 2:5 I will consider the (infinite horizon) autonomous problem again, and use the fish example to interpret S.S. condition. The general problem is:
" max e rt F (x , x )dt m
0 s.t. x () x0 The Euler Equation is Fx rFx Fxx x Fxx x Notice that I canceled the discount factor in order to write the Euler Equation as independent of time. By definition, variables are constant in steady state. In steady state x x 0 . So optimal steady state xs solves (5) Fx(xs , 0) rFx(xs , 0) Interpret (5) for example of fish problem in the general case where the cost of harvesting does depend on the stock of fish  in contrast to the special case that I considered at the beginning of this section. Divide both sides of (5) by r, and interpret the left side as the PDV of the gain of having one more fish in the stock; the right side equals the cost of reducing consumption by one fish at a point in time. The following discussion explains this interpretation. As before, the growth equation is x f (x) y y f(x) x Here I want to allow the function F( )  the integrand  to include a stockdependent harvest cost. F= U(y) B c(x,y) = U(f(x)  x)  c(x, f(x)  x) 8 8 utility of cost of harvesting y consuming y when stock is x I want to interpret the SS condition. Suppose that I am at the steady state and I decide to eat one less fish, and thereby increase the steady state by one fish. The costs and benefits of this marginal change should exactly balance each other at the optimal steady state  as the Euler Equation implies. If I'm at the SS and decide to consume one less fish my change in flow of welfare at the time I reduce 2:6 my consumption (consume one less fish) is Fx =  UN(y) + c2 = change in utility of consumption + change in harvest cost. Having one more fish in the stock increases the flow of future consumption (= growth in steady state) by f N (xs). The change in flow of total welfare equals change in utility from consumption UN(ys) f N(xs), minus the change in harvest cost. The harvest cost at the steady state is c(x, f(x)), so the change in cost due to the change in steady state is c1 + c2fN. In symbols: + Fx = U N f N  (c1 + c2 f N) Fx r Change in total utility discounted over infinite horizon is . Exercise. Consider the case where growth function f(x) is "inverted U". The level of stock that provides maximum sustainable yield solves f'(x) = 0. What factors determine whether the socially optimal steady state is above or below the level that maximizes sustainable yield? Comment: the necessary condition using COV is a second order ODE. For the autonomous problem (no explicit dependence on time except via constant discounting, and infinite horizon) we can use phase plane analysis to analyze the solution. In order to do this, we need to convert the second order ODE (the EE) into a system of two first order ODEs. For example, if I have the second order ODE G(x,x ) = 0, define y / x and analyze the system: ,x , G(x,y,y) (this is the equation for y in implicit form) and x = y (this is the equation for x ). 5. Derivation of Euler Eq. A function is admissible if it is continuously differentiable on [t0, T] and satisfies x(t0) = x0, x(T) = xT Suppose x*(t) is optimal. Let x(t) be another admissible soln. Define the deviation h(t) h(t) = x(t)  x*(t) 2:7 Admissibility Y x(t) = x*(t) = xT Y x(0) = x*(0) = x0 Y and h(t) differentiable h(T) = 0 h(0) = 0 (1) Then x(t) = x*(t) + ah(t) is also admissible. I rewrite the maximization problem in terms of the scalar a.
T g(a) T m
t0 F t , x (t ) , x(t ) dt m
t0 F t , x (t ) ah(t ) , x(t ) ah(t ) dt Optimality of x* implies that g(a) reaches a max at a = 0, which implies dg 0 da *a 0
T dg dF( m da da
t0 ) dt T (2) m
t0 Fx (t , x , x ) h(t ) Fx (t , x , x ) h (t ) dt 0 ____________ integrate by parts. m
T udv vu* T m vdu dh Fx dt Fx dh m m dt
t0 t0 T Fx h (t ) * t0 d F ( ) h(t )dt m dt x
t0 T 2:8 First term after last equality equals 0 by (1), implying T (2N) m
t0 Fx ( ) d F ( ) h(t )dt 0 dt x Notice that the term in brackets in the integrand is evaluated at a=0; consequently this term does not depend on the function h(t). I'll abbreviate this term by writing it as z(t). In order to show that (2') implies the Euler equation, we need the Fundamental Lemma of calculus of variations:
T If z(t) is continuous on [t0, T) and m
t0 z(t ) h(t ) dt 0 for all continuous h(t) then g(t) / 0. Sketch of proof by contradiction. Suppose z(t ) 0 , e.g. Find an interval over which z(t) 0, let h(t) 0 over only that interval. It is easy to see for this choice of h(t), the integral is not 0. This is our contradiction. Apply fundamental lemma to (2N) to obtain Euler Equation 6. Special functional forms 1) Suppose F is linearquadratic in x, x. Show Euler equation is linear ODE. Then show that this implies that dx/dt is a linear function of x. Interpret this as linear control rule. 2:9 2) Suppose time enters only through discounting: F(t,x,x) / ertF(x,x Show Euler equation ). is autonomous (no dependence on time.) In this case we have: d(Fx) dt d rt ~ ~ (e Fx ) re rtFx dt ~ ~ ~ e rtFxxx e rtFxx x set e rtFx cancel ert ~ ~ ~ ~ Y r Fx Fxx x Fx x x Fx Another special case f N(x) = constant, see K&S pg 28 Other Important special cases F = F(t, x) no dependence on x (in this case EE is 1st order ODE) F = F(x, x) no dependence on t F = F(x) no dependence on t or x F is linear in x (talk about this later) 7. Necessary and sufficient conditions We claimed that if x* is optimal, then for all continuously differentiable h(t) with h(0) = h(T) = 0, a = 0 must maximize
T g (a) T m
t0 F (t , x ah , x ah)dt Y 2 g NN(0) (3) m
t0 Fxx h 2 2Fxx hh Fxxh dt # 0 A sufficient condition is that F is jointly concave in x, x. This condition often does not hold in interesting problems. The Legendre Condition, Fxx(t, x*, x*) # 0 is necessary (It isn't sufficient). 2:10 Show necessity of Legendre condition d(h 2) 2Fxx hh / Fxx Y dt
T T 2 m
t0 Fxx h h T m
t0 T Fxx d(h 2) dt dt Fxxh 2* t0 d h 2 (Fxx )dt m dt
t0 =0 So (3) can be rewritten
T g NN(0) m
t0 Fxx d 2 Fxx h 2 Fxx(h)2 dt Qh 2 Ph dt m dt
t0 T / Qt / Pt T Lemma: Given continuous functions P(t), Q(t), a necessary condition for is P(t) # 0. (This inequality is just the Legendre condition.) Geometric argument. Suppose P > 0 over some interval. m
t0 2 Qt h 2 Pt h dt # 0 2:11 Choose h(t) so that *h (t )* arbitrarily small, but *h (t )* arbitrarily large. With this choice, the term involving P is positive and nonnegligible, and the term involving Q is negligible, so that the integral is positive. This is our contradiction. 8. Different boundary conditions T fixed, x(t) free (free end value) T free, x(T) fixed both x(t) and T free There is a scrap function S(x(t),T) and either x(T) and/or T is free x(0) may be free (jump states) t0 may be free (time you begin extracting from mine) See pg 6061 K&S for boundary conditions Argument for free terminal value of state, x(T) is free:
T g(a) m
t0 F t , x ah , x ah dt must be maximized at a = 0 Y gN(0)=0 2:12 T g N(0) m
t0 Fx h Fx h dt T second term is m
t0 Fx dh Fx h * t0 T T m
t0 h dFx dt dt We get same stuff as before, but now h(T) need not equal 0, so we have the extra condition Fx T , x (t )x (t ) 0 This condition is called a transversality condition (TC). Interpretation (fish story):If the terminal stock is a choice variable (i.e., x(T) is free) marginal utility of consuming final unit should be 0 at T, since there is no reason to keep stock. When we "lose" a B.C. it is replaced by a transversality condition. If the final time, T is free (i.e. a choice variable), transversality condition is (4) F T , x (T ) , x (T ) x Fx(T...) 0 Interpretation (fish story): If I extended the time of consumption by one unit, I would get an extra F( ) units of utility. When T is chosen optimally, this extra benefit should equal the loss incurred by changing harvest by the amount needed to hold constant the terminal value of the state  the second term of (4). (If x(T) is fixed, then by definition I have to hold its terminal value constant. If x(T) is chosen optimally, there is no benefit from changing that value  I want to hold its terminal value fixed.) 9 An example: extraction of a nonrenewable resource Extraction of a nonrenewable resource. Example: Rate of extraction is x / y with price p(y), utility of consumption U(y) U N(y) / p (y) , U NN(y) p N(y) < 0 2:13 Social planner's problem is
T max m
0 e rt U (y)dt x0 given, xT = 0, T free Euler equation Fx 0 Y e rt d rt e U N(y) dt U N (y) c , a constant This is Hotelling Rule: PDV of Marginal Utility (= price) constant, so price growing at rate of interest. (Note that whenever Fx / 0, EE is a 1st order ODE, much easier to solve than 2nd order ODE.) Legendre condition satisfied: Fx UN(x) , Fxx UNN(x) < 0. e rT U(x) xU N(x) 0 Tranversality condition: (4') Transversality condition provides a boundary condition for the ODE given to us from the Euler Equation. This boundary condition, implied by equation (4') is x = 0. This equality is, x=0, is obviously sufficient for TC. Show x=0 is necessary for TC. First consider case where T is finite (as is the case for linear example below). Suppose x(T) 0 , TC Y U(x) U N(x) 0 x This equation says that average and marginal utilities are equal at time T. However, since U is concave in x average utility is always greater than marginal utility. Thus, the hypothesis that x(T) , 0 leads to a contradiction, so in this case (T is finite) they hypothesis is false. Now consider the possibility that T is infinite (as will be the case if limy $ 0 U'(y) = 4) In this case x must approach 0 as T becomes large. (Extraction cannot remain bounded away from 0 forever because 2:14 the initial stock of the nonrenewable resource is finite. In order to see how you would actually use the transversality condition to solve a control problem, consider the previous problem, but specialize the general utility function to the quadratic case. In other words, assume linear marginal utility: U N(y) (a by) a bx The EE becomes: a bx ce rt (price grows at rate r) x(0) given, x(T) = 0 integrate Euler equation, use B.C.
T x0 0 T m
0 xdt (a ce rt ) dt m b
0 T x0 (a ce rt ) a c rT dt T (e 1) m b b br
0 This is one equation in 2 unknowns, c, and T. Use x(T) = 0 in a + bx = cerT Y a = cerT As an exercise, solve problem above for monopolist rather than social planner. 2:15 ...
View
Full
Document
This note was uploaded on 08/01/2008 for the course ARE 263 taught by Professor Karp during the Fall '06 term at Berkeley.
 Fall '06
 KARP

Click to edit the document details