sol8 - Solutions to Problem Set 8 ARE 261 Question 1 1 The...

Solutions to Problem Set 8
ARE 261
December 15, 2004
Question 1
1. The dynamic programming equation (DPE) for the control problem is
− J
t
( x
t
, t ) = max
u
t
½
−
1
2
e
− rt
( u
2
t
+ x
2
t
) + J
x
( x
t
, t )( x
t
+ u
t
)
¾
(1)
where J ( x
t
, t ) is the value function, and J
t
( x
t
, t ) and J
x
( x
t
, t ) are partial
derivatives of the value function.
2. J ( x
T
, T ) = − e
− rT
ax
2
T
2
.
3. Substitute the guess for the value function, that is J ( x
t
, t ) = s ( t )
x
2
t
2
, into
the DPE, di ff erentiate with respect to u
t
, and set the expression to zero.
Note that J
t
( x
t
, t ) = ˙ s
x
2
t
2
and J
x
( x
t
, t ) = sx . This gives the following
policy function
u
t
= e
rt
s ( t ) x
t
(2)
Substitute the optimal policy function into the DPE. This in turns gives
the di ff erential equation that solves s ( t )
˙ s = − e
rt
s
2
+ e
− rt
− 2 s
t
(3)
With the boundary condition that s ( T ) = − ae − rT . It is easier to work
with a transformation of s ( t ) . Let z ( t ) = e
rt
s ( t ) . Then
dz
dt
= − z ( t )
2
− (2 − r ) z ( t ) + 1 (4)
With the boundary condition that z
T
= − a .
4. λ
t
from the Maximum Principle is equal to J
x
( x
t
, t ) from the DPE.
5. We already have an autonomous di ff erential equation in z ( t ) . By solv-
ing this equation “backwards” we convert the boundary condition at the
terminal period into a boundary condition at the initial time. Rede fi ne
1

the di ff erential equation for z ( t ) in terms of τ where τ = T − t , the time
remaining. In terms of τ the di ff erential equation is
dz
dτ
= z ( τ )
2
+ (2 − r ) z ( τ ) − 1 (5)
This equation has two steady states, one positive and one negative. These
are solved for by setting
dz
dτ
= 0 and then solving the quadratic in z . The
positive root is equal to z
1
=
− (2 − r )+
√
(2 − r ) 2 +4
2
and the negative root is
equal to z
2
=
− (2 − r ) −
√
(2 − r ) 2 +4
2
. The negative root is stable while the
positive root is unstable. The fi gure shows the graph of
dz
dτ
= 0 for r = 1
-1
0
1
2
3
4
5
dz/d(tau)
-3 -2 -1 1 2
z
In the steady state, we have
s
∗
= z
2
e
− rt
,
so
ds
dt
= − rz
2
e
− rt
= − rs
-4
-2
0
2
4
ds/dt
-4 -2 2 4
s
2

6. So long as the initial point − a lies to the left of the positive root z
1
, the
di ff erential equation for s will converge to the steady state as T → ∞ .