This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ith a corresponding distribution of costs
from which we can calculate an expected cost.
J (t , x ) = min w (x , u , t )dt + J (t + dt , xt +dt )
u Where the expectation occurs because, while we know our location
and control precisely, we do not know precisely where this will take
us. I’ll discuss u (x , t ) here. Even though we’ve precomputed it
but it’s useful when buﬀeted by (speciﬁcally unanticipated but
generally modeled) noise.
Nick Jones Elements of Stochastic Optimal Control: ICDNS Stochastic Hamilton Jacobi Bellman equation
J (t , x ) = min w (x , u , t )dt + J (t + dt , xt +dt )
u We can Taylor expand J (t + dt , xt +dt ). We have to go to second
order in dx since < dx 2 >= O(dt ) (basics of SDEs).
J (t +dt , xt +dt ) = J (x , t )+dt ∂t J (t , x )+ dx ∂x J (t , x )+ 1
2
dx 2 ∂x J (t , x )
2 Noting that < dx >= f (x , u , t )dt and < dx 2 >= ν (t , x , u )dt we
obtain:
1
2
−∂t J (t , x ) = min(w (x , u , t ) + f (x , u , t )∂x J (t , x ) + ν (t , x , u )∂x J (t , x ))
u
2 This is the Stochastic HJB equation. We have picked up a
diﬀusionlike term in our dynamics.
Nick Jones Elements of Stochastic Optimal Control: ICDNS Linear Quadratic Gaussian Control
Linear dynamics, quadratic costs, Gaussian (white) noise.
Linear Dynamics with Gaussian noise:
dx = [Ax + Bu ]dt + Fd ξ
Quadratic Cost:
1
1
w (x , u , t ) = u T Ru + x T Qx
2
2
Final Cost:
1
W (x ) = x T QT x
2
Where we can have A, B , F , R , Q time varying. Nick Jones Elements of Stochastic Optimal Control: ICDNS Linear Quadratic Gaussian Control II
One can show (using the stochastic HJB) that the optimal
costtogo is of the form:
1
J (x , t ) = x T V (t )x + a(t )
2
(with V symmetric) and the corresponding optimal control:
u (x , t ) = −R −1 B T...
View Full
Document
 Spring '14
 NickJones
 Gate

Click to edit the document details