This preview shows pages 1–2. Sign up to view the full content.
ARE 261
October 22, 2001
1) The control problem is
max
¡
1
2
Z
T
o
e
¡
rt
(
u
2
+
x
2
)
dt
¡
e
¡
rT
ax
2
2
s:t:
_
x
=
x
+
u
,
x
0
given.
where
a
is a parameter.
a)Write down the dynamic programing equation.
b)Write the boundary condition for this equation; that is, what is the value
function at t = T?
c) “Guess” that the value function is quadratic:
J
(
x;t
) =
s
(
t
)
x
2
=
2
.
Find the di¤erential equation that
s
(
t
)
solves, and the boundary condition
for this equation. (This is called a Ricatti di¤erential equation.)
At this point it is useful to recognize that you can simplify the Ricatti
di¤erential equation by using a transformation of
s
.
De…ne
z
(
t
) =
e
rt
s
(
t
)
.
Using this de…nition and your previous equation for
_
s
, …nd the equation for
_
z
and the boundary condition for
z
.
It is convenient to work with
z
rather than
s
because the di¤erential equa
tion for
z
does not depend on calendar time,
t
– that is, the equation for
z
is
autonomous.
d) Remember that you previously solved (essentially) this problem using the
Maximum Principle.
There you “guessed” that the costate variable was linear
in the state.
What is the relation between these two approaches?
e) De…ne
¿
=
T
¡
t
, the “time to go” (until the end of the horizon)
We
want to consider the limiting form of the original problem as
T
! 1
.
From
the de…nition of
¿
we have
d¿
=
¡
dt
.
By working with
¿
(“time to go”) rather
than with
t
(the calendar time) we have “reversed the clock”.
Note that as
T
! 1
,
¿
! 1
for any …nite
t
(…nite calendar time).
Use your previous
equation for
dz
dt
to …nd the equation for
dz
d¿
.
Graph this equation as a function
of
z
.
Notice that there are two steady states.
Which of these is stable in the
reversed system (by “reversed system” I mean that the independent variable is
¿
rather than
t
)?
The algebraic (as opposed to di¤erential) equation for the
steady stae value of
z
is known as the algebraic Ricatti equation.
f) For what set of values of the parameter
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
 Fall '06
 KARP

Click to edit the document details