{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# lec4 - MIT OpenCourseWare http/ocw.mit.edu 16.323...

This preview shows pages 1–6. Sign up to view the full content.

MIT OpenCourseWare http://ocw.mit.edu 16.323 Principles of Optimal Control Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
16.323 Lecture 4 HJB Equation DP in continuous time HJB Equation Continuous LQR Factoids: for symmetric R u T R u = 2 u T R u ∂R u = R u
Spr 2008 16.323 4–1 DP in Continuous Time Have analyzed a couple of approximate solutions to the classic control problem of minimizing: t f min J = h ( x ( t f ) , t f ) + g ( x ( t ) , u ( t ) , t ) dt t 0 subject to x ˙ = a ( x , u , t ) x ( t 0 ) = given m ( x ( t f ) , t f ) = 0 set of terminal conditions u ( t ) ∈ U set of possible constraints Previous approaches discretized in time, state, and control actions Useful for implementation on a computer, but now want to consider the exact solution in continuous time Result will be a nonlinear partial differential equation called the Hamilton-Jacobi-Bellman equation ( HJB ) a key result. First step: consider cost over the interval [ t, t f ] , where t t f of any control sequence u ( τ ) , t τ t f t f J ( x ( t ) , t, u ( τ )) = h ( x ( t f ) , t f ) + g ( x ( τ ) , u ( τ ) , τ ) t Clearly the goal is to pick u ( τ ) , t τ t f to minimize this cost. J ( x ( t ) , t ) = min J ( x ( t ) , t, u ( τ )) u ( τ ) ∈U t τ t f June 18, 2008

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Spr 2008 16.323 4–2 Approach: Split time interval [ t, t f ] into [ t, t + Δ t ] and [ t + Δ t, t f ] , and are specifically interested in the case where Δ t 0 Identify the optimal cost-to-go J ( x ( t + Δ t ) , t + Δ t ) Determine the “stage cost” in time [ t, t + Δ t ] Combine above to find best strategy from time t . Manipulate result into HJB equation. Split: t f J ( x ( t ) , t ) = min h ( x ( t f ) , t f ) + g ( x ( τ ) , u ( τ ) , τ )) t u ( τ ) ∈U t τ t f t t t f = min h ( x ( t f ) , t f ) + g ( x , u , τ ) + g ( x , u , τ ) t t t u ( τ ) ∈U t τ t f Implicit here that at time t t , the system will be at state x ( t t ) . But from the principle of optimality , we can write that the optimal cost-to-go from this state is: J ( x ( t + Δ t ) , t + Δ t ) Thus can rewrite the cost calculation as: �� t t J ( x ( t ) , t ) = min g ( x , u , τ ) + J ( x ( t + Δ t ) , t + Δ t ) t u ( τ ) ∈U t τ t t June 18, 2008
Spr 2008 16.323 4–3 Assuming J ( x ( t + Δ t ) , t + Δ t ) has bounded second derivatives in both arguments, can expand this cost as a Taylor series about x ( t ) , t ∂J J ( x ( t + Δ t ) , t + Δ t ) J ( x ( t ) , t ) + ( x ( t ) , t ) Δ t ∂t ∂J + ( x ( t ) , t ) ( x ( t + Δ t ) x ( t )) x Which for small Δ t can be compactly written as: J ( x ( t + Δ t ) , t + Δ t ) J ( x ( t ) , t ) + J ( x ( t ) , t t t + J x ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t t Substitute this into the cost calculation with a small Δ t to get J ( x ( t ) , t ) = u ( t ) ∈U { g ( x ( t ) , u ( t ) , t t + J ( x ( t ) , t ) min + J ( x ( t ) , t t + J ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t t } t x Extract the terms that are independent of u ( t ) and cancel 0 = J ( x ( t ) , t )+ min ( x ( t ) , t ) a ( x ( t ) , u ( t ) , t ) } u ( t ) ∈U { g ( x ( t ) , u ( t ) , t ) + J t x This is a partial differential equation in J ( x ( t ) , t ) that is solved backwards in time with an initial condition that J ( x ( t f ) , t f ) = h ( x ( t f )) for x ( t f )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}