*_TGSOI S:\AAe760\Website\handouts\Lec4_99.doc
9/13/99
''+ 4GZ[XGR 8KYU[XIK +IUTUSOIY 2KIZ[XK
Review of Pontryagins Maximum Principle: Last time I gave a fairly narrow presentation of the Maximum Principle. A more general statement replaces the optimality condition Hu = 0 with the more general statement
max H $
u
t 0, T
.
(1)
Why is this important? For more complex problems problems in which, for instance, control or state variables are bounded, relevant necessary conditions can be formulated by appending these additional inequality constraints to the Hamiltonian, and solving (1). That this is true can be seen by repeating the proof presented last time, with the additional constraints appended to the Lagrangian. We will go through this exercise sometime in the next couple of weeks. At the same time we will see that additional constraints also imply additional transversality conditions. A note on the current value adjoint variable Recall the discussion last time about our simple profit maximization problem with the Hamiltonian
H = e rt pf u$ wu ( r u) .
(2)
The optimality condition for this problem is
e rt pf u$ w = .
(3)
To make this condition more tractable in analytical manipulations of the necessary conditions, it is often useful to replace with the current value adjoint variable (as opposed to the present value adjoint variable. We define e rt = ; note that is the present value of . Substitution of this expression into (3) and then cancellation yields:
1
S:\AAe760\Website\handouts\Lec4_99.doc
9/13/99
e rt pf u$ w = e rt pf u$ w =
.
(4)
So at any time t, the stock is extracted until the marginal net benefit of its current consumption equals the marginal user cost of current consumption, as measured at time t.
Interpretation of the adjoint variable It is axiomatic that * t $ is the marginal cost of meeting the constraint posed by the equation of motion. What does this mean in a dynamic context? In a static context, we have the resource constraint f(u)=x, with Lx = , in which case * is the marginal change in the maximum value of the objective function that results from a marginal increase in the resource it is the shadow price of the resource x.
In our dynamic world, we have instead the equation of motion f x , u = x ; in discrete time this may be stated,
1 6
f 1 xt , u 6 = xt + xt f 1 xt , u 6 + xt = xt +
(5)
and so here * t $ is the marginal change in L that results from a marginal change in the resource stock at time t + . It follows that as 0 , * t $ is properly interpreted to be the shadow price of the resource stock at time t. We can show this with a bit more rigordefine
L* =
I
T
0
>H3 x
*
, u* , * , t 8 + x * Cdt
+ S 3 x * 1T 6, T 8 + *V 3 x * 1T 6, T 8 * 1T 6 x * 1T 6 + * 106 x0
(6)
This, of course, is the Lagrangian representation of our maximization problem, after integration by parts, and recognizing that x(0) is given. Now note,
L * = * 0$ , x 0
(7)
2
S:\AAe760\Website\handouts\Lec4_99.doc
9/13/99
so we see that at the instant, initial * t $ is indeed the shadow price of a marginal increase in the resource stock. Now suppose we follow the optimal path of the state variable to time t : diagram here
At t we have a new initial state, x t = x * t , and the solution of this new problem would remain u t $, t t T , since we start at time t with the stock along the path
*
16
16
x * t $ . Of course, this means we stay on x * t $ in the interval t t T . Then we have the
new Lagrangian at time t : L* =
T
I> 2
t
C + S 2 x 1T 6, T 7 + V 2 x 1T 6, T 7 1T 6 x 1T 6 + 1t 6 x 1t 6
H x * , u* , * , t + x * dt
* * * * * * *
7
,
(8)
and so we see that at time t ,
L * = * 1t 6 . x * 1t 6
(9)
Interpretation of the Transversality Condition With this interpretation of the adjoint variable in mind, recall the transversality condition associated with the path of the adjoint variable:
* 1T 6 = S x 3 x * 1T 6, T 8 + Vx 3 x * 1T 6, T 8 .
(10)
This indicates that .
So new stock at the last instant is valuable only if the salvage function is not the zero function, or if it allows us to reduce the cost of meeting the terminal surface constraint. Two special cases
3
S:\AAe760\Website\handouts\Lec4_99.doc
9/13/99
The importance of the transversality condition Recall the claim that the transversality condition is often necessary to anchor the terminal path. We want to show heuristically that the transversality condition is typically necessary to solve a control problem (otherwise, our necessary conditions are trivial; any path can be found which satisfies them). Consider the following typical problem:
u t $U 0
max
I
T
B2 x1t 6, u1t 6, t 7dt
s. t . x1t 6 = f 2 x1t 6, u1t 67
(11)
x106 = x 0 T fixed
We form the Hamiltonian
The Pontryagin necessary conditions for this problem are
So now we attempt a solution of thi...