L10-StochOptControl

# J t x min w x u t dt j t dt xt dt u where

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ith a corresponding distribution of costs from which we can calculate an expected cost. J (t , x ) = min w (x , u , t )dt + J (t + dt , xt +dt ) u Where the expectation occurs because, while we know our location and control precisely, we do not know precisely where this will take us. I’ll discuss u (x , t ) here. Even though we’ve precomputed it but it’s useful when buﬀeted by (speciﬁcally unanticipated but generally modeled) noise. Nick Jones Elements of Stochastic Optimal Control: ICDNS Stochastic Hamilton Jacobi Bellman equation J (t , x ) = min w (x , u , t )dt + J (t + dt , xt +dt ) u We can Taylor expand J (t + dt , xt +dt ). We have to go to second order in dx since &lt; dx 2 &gt;= O(dt ) (basics of SDEs). J (t +dt , xt +dt ) = J (x , t )+dt ∂t J (t , x )+ dx ∂x J (t , x )+ 1 2 dx 2 ∂x J (t , x ) 2 Noting that &lt; dx &gt;= f (x , u , t )dt and &lt; dx 2 &gt;= ν (t , x , u )dt we obtain: 1 2 −∂t J (t , x ) = min(w (x , u , t ) + f (x , u , t )∂x J (t , x ) + ν (t , x , u )∂x J (t , x )) u 2 This is the Stochastic HJB equation. We have picked up a diﬀusion-like term in our dynamics. Nick Jones Elements of Stochastic Optimal Control: ICDNS Linear Quadratic Gaussian Control Linear dynamics, quadratic costs, Gaussian (white) noise. Linear Dynamics with Gaussian noise: dx = [Ax + Bu ]dt + Fd ξ Quadratic Cost: 1 1 w (x , u , t ) = u T Ru + x T Qx 2 2 Final Cost: 1 W (x ) = x T QT x 2 Where we can have A, B , F , R , Q time varying. Nick Jones Elements of Stochastic Optimal Control: ICDNS Linear Quadratic Gaussian Control II One can show (using the stochastic HJB) that the optimal cost-to-go is of the form: 1 J (x , t ) = x T V (t )x + a(t ) 2 (with V symmetric) and the corresponding optimal control: u (x , t ) = −R −1 B T...
View Full Document

Ask a homework question - tutors are online