L10-StochOptControl

L10-StochOptControl - Elements of Stochastic Optimal...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Elements of Stochastic Optimal Control: ICDNS MSci/MSc Nick Jones [email protected]
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Aspects of Stochastic Optimal Control In this lecture we will particularly briskly investigate selected topics in Stochastic Optimal Control. Our objective will be obtain enough background so that we can consider a rejection sampling approach to solving a class of Stochastic Optimal Control problems in Ref. [3]. We’ll try and keep notation broadly consistent with Refs. [2, 3]. We will then investigate this last in our practical lecture. Note: we will have one more lecture on topics in Control in which we will investigate the bridge between topics in inference and control. This lecture will be the first of 5 - the remaining three will be on classic theory to study the driving of systems and the final lecture will draw together some of the strands of the course. Nick Jones Elements of Stochastic Optimal Control: ICDNS
Background image of page 2
Tasks Beyond asking you to check the more obvious statements in the handouts I have made the following requests. The objective here is to ensure you’ve actually followed up on the material and understand it. Beautiful expositions are not required - just demonstrations of understanding.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Noisy dynamics We now consider Gaussian perturbations to our dynamical system. dx = f ( x , u , t ) dt + d ξ These can be distinct for different state variables and element of d ξ can be correlated between them: < d ξ i d ξ j > = ν ij ( t , x , u ) dt (though uncorrelated in time). Note that the control and noise need not be independent. Nick Jones Elements of Stochastic Optimal Control: ICDNS
Background image of page 4
Expected cost and optimal cost-to-go dx = f ( x , u , t ) dt + d ξ C ( x 0 , u (0 T )) = h W ( x T ) + Z T 0 dt w ( x , u , t ) i Any particular control u (0 T ) (with i.c. x 0 ) thus specifies an ensemble of trajectories with a corresponding distribution of costs from which we can calculate an expected cost. J ( t , x ) = min u w ( x , u , t ) dt + h J ( t + dt , x t + dt ) i Where the expectation occurs because, while we know our location and control precisely, we do not know precisely where this will take us. I’ll discuss u 0 ( x , t ) here. Even though we’ve precomputed it but it’s useful when buffeted by (specifically unanticipated but generally modeled) noise. Nick Jones Elements of Stochastic Optimal Control: ICDNS
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Stochastic Hamilton Jacobi Bellman equation J ( t , x ) = min u w ( x , u , t ) dt + h J ( t + dt , x t + dt ) i We can Taylor expand J ( t + dt , x t + dt ). We have to go to second order in dx since < dx 2 > = O ( dt ) (basics of SDEs). h J ( t + dt , x t + dt ) i = J ( x , t )+ dt t J ( t , x )+ h dx i x J ( t , x )+ 1 2 h dx 2 i 2 x J ( t , x ) Noting that < dx > = f ( x , u , t ) dt and < dx 2 > = ν ( t , x , u ) dt we obtain: - t J ( t , x ) = min u ( w ( x , u , t ) + f ( x , u , t ) x J ( t , x ) + 1 2 ν ( t , x , u ) 2 x J ( t , x )) This is the Stochastic HJB equation. We have picked up a diffusion-like term in our dynamics.
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 18

L10-StochOptControl - Elements of Stochastic Optimal...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online