sol8 - Solutions to Problem Set 8 ARE 261 December 15, 2004...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Solutions to Problem Set 8 ARE 261 December 15, 2004 Question 1 1. The dynamic programming equation (DPE) for the control problem is J t ( x t ,t )=max u t ½ 1 2 e rt ( u 2 t + x 2 t )+ J x ( x t ,t )( x t + u t ) ¾ (1) where J ( x t ,t ) is the value function, and J t ( x t ,t ) and J x ( x t ,t ) are partial derivatives of the value function. 2. J ( x T ,T )= e rT ax 2 T 2 . 3. Substitute the guess for the value function, that is J ( x t ,t )= s ( t ) x 2 t 2 ,into the DPE, di f erentiate with respect to u t , and set the expression to zero. Note that J t ( x t ,t )= ˙ s x 2 t 2 and J x ( x t ,t )= sx . This gives the following policy function u t = e rt s ( t ) x t (2) Substitute the optimal policy function into the DPE. This in turns gives the di f erential equation that solves s ( t ) ˙ s = e rt s 2 + e rt 2 s t (3) With the boundary condition that s ( T )= ae rT . It is easier to work with a transformation of s ( t ) .L e t z ( t )= e rt s ( t ) .Th en dz dt = z ( t ) 2 (2 r ) z ( t )+1 (4) With the boundary condition that z T = a . 4. λ t from the Maximum Principle is equal to J x ( x t ,t ) from the DPE. 5. We already have an autonomous di f erential equation in z ( t ) .B ys o
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
the di f erential equation for z ( t ) in terms of τ where τ = T t ,thet ime remaining. In terms of τ the di f erential equation is dz = z ( τ ) 2 +(2 r ) z ( τ ) 1 (5) This equation has two steady states, one positive and one negative. These are solved for by setting dz =0 and then solving the quadratic in z .Th e positive root is equal to z 1 = (2 r )+ (2 r ) 2 +4 2 and the negative root is equal to z 2 = (2 r ) (2 r ) 2 +4 2 . The negative root is stable while the positive root is unstable. The f gure shows the graph of dz =0 for r =1 -1 0 1 2 3 4 5 dz/d(tau) -3 -2 -1 1 2 z In the steady state, we have s = z 2 e rt , so ds dt = rz 2 e rt = rs -4 -2 0 2 4 ds/dt -4 -2 2 4
Background image of page 2
6. So long as the initial point a lies to the left of the positive root z 1 ,the di f erential equation for
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 8

sol8 - Solutions to Problem Set 8 ARE 261 December 15, 2004...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online