{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

L10-StochOptControl

# L10-StochOptControl - Elements of Stochastic Optimal...

This preview shows pages 1–7. Sign up to view the full content.

Elements of Stochastic Optimal Control: ICDNS MSci/MSc Nick Jones [email protected]

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Aspects of Stochastic Optimal Control In this lecture we will particularly briskly investigate selected topics in Stochastic Optimal Control. Our objective will be obtain enough background so that we can consider a rejection sampling approach to solving a class of Stochastic Optimal Control problems in Ref. [3]. We’ll try and keep notation broadly consistent with Refs. [2, 3]. We will then investigate this last in our practical lecture. Note: we will have one more lecture on topics in Control in which we will investigate the bridge between topics in inference and control. This lecture will be the ﬁrst of 5 - the remaining three will be on classic theory to study the driving of systems and the ﬁnal lecture will draw together some of the strands of the course. Nick Jones Elements of Stochastic Optimal Control: ICDNS
Tasks Beyond asking you to check the more obvious statements in the handouts I have made the following requests. The objective here is to ensure you’ve actually followed up on the material and understand it. Beautiful expositions are not required - just demonstrations of understanding.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Noisy dynamics We now consider Gaussian perturbations to our dynamical system. dx = f ( x , u , t ) dt + d ξ These can be distinct for diﬀerent state variables and element of d ξ can be correlated between them: < d ξ i d ξ j > = ν ij ( t , x , u ) dt (though uncorrelated in time). Note that the control and noise need not be independent. Nick Jones Elements of Stochastic Optimal Control: ICDNS
Expected cost and optimal cost-to-go dx = f ( x , u , t ) dt + d ξ C ( x 0 , u (0 T )) = h W ( x T ) + Z T 0 dt w ( x , u , t ) i Any particular control u (0 T ) (with i.c. x 0 ) thus speciﬁes an ensemble of trajectories with a corresponding distribution of costs from which we can calculate an expected cost. J ( t , x ) = min u w ( x , u , t ) dt + h J ( t + dt , x t + dt ) i Where the expectation occurs because, while we know our location and control precisely, we do not know precisely where this will take us. I’ll discuss u 0 ( x , t ) here. Even though we’ve precomputed it but it’s useful when buﬀeted by (speciﬁcally unanticipated but generally modeled) noise. Nick Jones Elements of Stochastic Optimal Control: ICDNS

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Stochastic Hamilton Jacobi Bellman equation J ( t , x ) = min u w ( x , u , t ) dt + h J ( t + dt , x t + dt ) i We can Taylor expand J ( t + dt , x t + dt ). We have to go to second order in dx since < dx 2 > = O ( dt ) (basics of SDEs). h J ( t + dt , x t + dt ) i = J ( x , t )+ dt t J ( t , x )+ h dx i x J ( t , x )+ 1 2 h dx 2 i 2 x J ( t , x ) Noting that < dx > = f ( x , u , t ) dt and < dx 2 > = ν ( t , x , u ) dt we obtain: - t J ( t , x ) = min u ( w ( x , u , t ) + f ( x , u , t ) x J ( t , x ) + 1 2 ν ( t , x , u ) 2 x J ( t , x )) This is the Stochastic HJB equation. We have picked up a diﬀusion-like term in our dynamics.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 18

L10-StochOptControl - Elements of Stochastic Optimal...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online