{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lec4 - MIT OpenCourseWare http/ocw.mit.edu 16.323...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
MIT OpenCourseWare http://ocw.mit.edu 16.323 Principles of Optimal Control Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
16.323 Lecture 4 HJB Equation DP in continuous time HJB Equation Continuous LQR Factoids: for symmetric R u T R u = 2 u T R u ∂R u = R u
Background image of page 2
Spr 2008 16.323 4–1 DP in Continuous Time Have analyzed a couple of approximate solutions to the classic control problem of minimizing: t f min J = h ( x ( t f ) , t f ) + g ( x ( t ) , u ( t ) , t ) dt t 0 subject to x ˙ = a ( x , u , t ) x ( t 0 ) = given m ( x ( t f ) , t f ) = 0 set of terminal conditions u ( t ) ∈ U set of possible constraints Previous approaches discretized in time, state, and control actions Useful for implementation on a computer, but now want to consider the exact solution in continuous time Result will be a nonlinear partial differential equation called the Hamilton-Jacobi-Bellman equation ( HJB ) a key result. First step: consider cost over the interval [ t, t f ] , where t t f of any control sequence u ( τ ) , t τ t f t f J ( x ( t ) , t, u ( τ )) = h ( x ( t f ) , t f ) + g ( x ( τ ) , u ( τ ) , τ ) t Clearly the goal is to pick u ( τ ) , t τ t f to minimize this cost. J ( x ( t ) , t ) = min J ( x ( t ) , t, u ( τ )) u ( τ ) ∈U t τ t f June 18, 2008
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Spr 2008 16.323 4–2 Approach: Split time interval [ t, t f ] into [ t, t + Δ t ] and [ t + Δ t, t f ] , and are specifically interested in the case where Δ t 0 Identify the optimal cost-to-go J ( x ( t + Δ t ) , t + Δ t ) Determine the “stage cost” in time [ t, t + Δ t ] Combine above to find best strategy from time t . Manipulate result into HJB equation. Split: t f J ( x ( t ) , t ) = min h ( x ( t f ) , t f ) + g ( x ( τ ) , u ( τ ) , τ )) t u ( τ ) ∈U t τ t f t t t f = min h ( x ( t f ) , t f ) + g ( x , u , τ ) + g ( x , u , τ ) t t t u ( τ ) ∈U t τ t f Implicit here that at time t t , the system will be at state x ( t t ) . But from the principle of optimality , we can write that the optimal cost-to-go from this state is: J ( x ( t + Δ t ) , t + Δ t ) Thus can rewrite the cost calculation as: �� t t J ( x ( t ) , t ) = min g ( x , u , τ ) + J ( x ( t + Δ t ) , t + Δ t ) t u ( τ ) ∈U t τ t t June 18, 2008
Background image of page 4
Spr 2008 16.323 4–3 Assuming J ( x ( t + Δ t ) , t + Δ t ) has bounded second derivatives in both arguments, can expand this cost as a Taylor series about x ( t ) , t ∂J J ( x ( t + Δ t ) , t + Δ t ) J ( x ( t ) , t ) + ( x ( t ) , t ) Δ t ∂t ∂J + ( x ( t ) , t ) ( x ( t + Δ t ) x ( t )) x Which for small Δ t can be compactly written as: J ( x ( t + Δ t ) , t + Δ t ) J ( x ( t ) , t ) + J ( x ( t ) , t t t + J x ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t t Substitute this into the cost calculation with a small Δ t to get J ( x ( t ) , t ) = u ( t ) ∈U { g ( x ( t ) , u ( t ) , t t + J ( x ( t ) , t ) min + J ( x ( t ) , t t + J ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t t } t x Extract the terms that are independent of u ( t ) and cancel 0 = J ( x ( t ) , t )+ min ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t ) } u ( t ) ∈U { g ( x ( t ) , u ( t ) , t ) + J t x This is a partial differential equation in J ( x ( t ) , t ) that is solved backwards in time with an initial condition that J ( x ( t f ) , t f ) = h ( x ( t f )) for x ( t f )
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}