This preview shows pages 1–3. Sign up to view the full content.
HANDOUT ON DYNAMIC
PROGRAMMING
1E
x
a
m
p
l
e
We’ll start with a simple growth example where a planner chooses how much
people should consume and how much to invest in a productive technology.
Preferences are given by
u
(
c
t
)=log(
c
t
) and what is not eaten out of production
today, say
k
t
+1
,
becomes productive tomorrow
y
t
+1
=
f
(
k
t
+1
)=
k
α
t
+1
where
α
≤
1
.
If people discount the future at rate
β<
1 and there is a general
time horizon represented by
T
(which could be
∞
)
,thesequencevers
iono
fthe
programming problem can be written as:
v
T
(
k
0
m
a
x
{
c
t
,k
t
+1
}
T
t
=0
T
X
t
=0
β
t
u
(
c
t
)
s.t.c
t
+
k
t
+1
=
f
(
k
t
)
,
∀
t
c
t
≥
0
,k
t
+1
≥
0
0
given.
One way to solve this problem is to write down the system of
f
rst order
conditions for each
t
∈
{
0
, ..., T
}
.
An alternative approach is to use dynamic
programming. First consider what happens if a person enters the last period
with
k
T
units of capital and acts optimally. In that case they solve:
v
T
(
k
T
m
a
x
c
T
≥
0
,k
T
+1
≥
0
log(
c
T
)
s.t.c
T
+
k
T
+1
≤
k
α
T
The solution is simple,
k
T
+1
=0and
c
T
=
k
α
T
.
In that case,
v
T
(
k
T
δ
(0) log(
k
α
T
)+
γ
(0)
where
δ
(0) = 1 and
γ
(0) = 0 and the argument in the function denotes the
number of periods remaining.
Now consider the problem if a person enters the next to last period
t
=
T
−
1
with
k
T
−
1
units of capital and acts optimally. The problem can be stated as:
v
T
−
1
(
k
T
−
1
m
a
x
c
T
−
1
≥
0
,k
T
≥
0
log(
c
T
−
1
βv
T
(
k
T
)
s.t.c
T
−
1
+
k
T
≤
k
α
T
−
1
Since utility is strictly increasing in consumption, the constraint will hold with
equality and we can substitute out consumption to yield the problem
v
T
−
1
(
k
T
−
1
)=max
k
T
≥
0
log(
k
α
T
−
1
−
k
T
T
(
k
T
)(
1
)
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentSubstituting
v
T
(
k
T
)=
α
log(
k
T
) into the objective, the foc are given by
1
k
α
T
−
1
−
k
T
=
αβ
k
T
⇐⇒
k
T
=
αβk
α
T
−
1
(1 +
αβ
)
(2)
Substituting the decision rule (2) back into the objective (1) yields
v
T
−
1
(
k
T
−
1
)=l
o
g
μ
k
α
T
−
1
−
αβk
α
T
−
1
(1 +
αβ
)
¶
+
αβ
log
μ
αβk
α
T
−
1
1+
αβ
¶
v
T
−
1
(
k
T
−
1
)=(1+
αβ
)log
¡
k
α
T
−
1
¢
−
(1 +
αβ
)log(1+
αβ
)+
αβ
log(
αβ
)
v
T
−
1
(
k
T
−
1
δ
(1)
α
log (
k
T
−
1
γ
(1)
(3)
where the coe
ﬃ
cients are obviously de
f
ned as
δ
(1) = (1 +
αβ
)and
γ
(1) =
αβ
log(
αβ
)
−
(1 +
αβ
αβ
)
.
By induction, let there be
τ
periods remaining (i.e.
t
=
T
−
τ
):
v
T
−
τ
(
k
T
−
τ
)= max
k
T
−
τ
+1
≥
0
log(
k
α
T
−
τ
−
k
T
−
τ
+1
βv
T
−
τ
+1
(
k
T
−
τ
+1
)
which upon substituting the
τ
period analogue of (3) given by
v
T
−
τ
+1
(
k
T
−
τ
+1
δ
(
τ
−
1)
α
log (
k
T
−
τ
+1
γ
(
τ
−
1) into the above yields
v
T
−
τ
(
k
T
−
τ
k
T
−
τ
+1
≥
0
log(
k
α
T
−
τ
−
k
T
−
τ
+1
)+
β
[
δ
(
τ
−
1)
α
log (
k
T
−
τ
+1
γ
(
τ
−
1)]
(4)
The foc is
1
k
α
T
−
τ
−
k
T
−
τ
+1
=
αβδ
(
τ
−
1)
k
T
−
τ
+1
k
T
−
τ
+1
=
αβδ
(
τ
−
1)
k
α
T
−
τ
(1 +
αβδ
(
τ
−
1))
(5)
Substituting the decision rule (5) back into the objective (4) yields
v
T
−
τ
(
k
T
−
τ
o
g
μ
k
α
T
−
τ
−
αβδ
(
τ
−
1)
k
α
T
−
τ
(1 +
αβδ
(
τ
−
1))
¶
+
αβδ
T
−
τ
+1
log
μ
αβδ
(
τ
−
1)
k
α
T
This is the end of the preview. Sign up
to
access the rest of the document.
 Spring '07
 CORBAE

Click to edit the document details