sol8 - Solutions to Problem Set 8 ARE 261 Question 1 1 The...

Solutions to Problem Set 8
ARE 261
December 15, 2004
Question 1
1. The dynamic programming equation (DPE) for the control problem is
− J
t
( x
t
,t )=max
u
t
½
−
1
2
e
− rt
( u
2
t
+ x
2
t
)+ J
x
( x
t
,t )( x
t
+ u
t
)
¾
(1)
where J ( x
t
,t ) is the value function, and J
t
( x
t
,t ) and J
x
( x
t
,t ) are partial
derivatives of the value function.
2. J ( x
T
,T )= − e
− rT
ax
2
T
2
.
3. Substitute the guess for the value function, that is J ( x
t
,t )= s ( t )
x
2
t
2
,into
the DPE, di f erentiate with respect to u
t
, and set the expression to zero.
Note that J
t
( x
t
,t )= ˙ s
x
2
t
2
and J
x
( x
t
,t )= sx . This gives the following
policy function
u
t
= e
rt
s ( t ) x
t
(2)
Substitute the optimal policy function into the DPE. This in turns gives
the di f erential equation that solves s ( t )
˙ s = − e
rt
s
2
+ e
− rt
− 2 s
t
(3)
With the boundary condition that s ( T )= − ae − rT . It is easier to work
with a transformation of s ( t ) .L e t z ( t )= e
rt
s ( t ) .Th en
dz
dt
= − z ( t )
2
− (2 − r ) z ( t )+1 (4)
With the boundary condition that z
T
= − a .
4. λ
t
from the Maximum Principle is equal to J
x
( x
t
,t ) from the DPE.
5. We already have an autonomous di f erential equation in z ( t ) .B ys o

the di f erential equation for z ( t ) in terms of τ where τ = T − t ,thet ime
remaining. In terms of τ the di f erential equation is
dz
dτ
= z ( τ )
2
+(2 − r ) z ( τ ) − 1 (5)
This equation has two steady states, one positive and one negative. These
are solved for by setting
dz
dτ
=0 and then solving the quadratic in z .Th e
positive root is equal to z
1
=
− (2 − r )+
√
(2 − r ) 2 +4
2
and the negative root is
equal to z
2
=
− (2 − r ) −
√
(2 − r ) 2 +4
2
. The negative root is stable while the
positive root is unstable. The f gure shows the graph of
dz
dτ
=0 for r =1
-1
0
1
2
3
4
5
dz/d(tau)
-3 -2 -1 1 2
z
In the steady state, we have
s
∗
= z
2
e
− rt
,
so
ds
dt
= − rz
2
e
− rt
= − rs
-4
-2
0
2
4
ds/dt
-4 -2 2 4

6. So long as the initial point − a lies to the left of the positive root z
1
,the
di f erential equation for