Homework4sol.pdf - IEOR-E4707 Spring 2018 Homework 4...

• Homework Help
• jackdaniels2018
• 5
• 100% (4) 4 out of 4 people found this document helpful

This preview shows page 1 - 3 out of 5 pages.

IEOR-E4707, Spring 2018: Homework 4 Solutions March 21, 2018 Problem 1. Consider the following stochastic control problem (the state, control and the Brownian motion are all one-dimensional): max J ( u ) = E [ Z 1 0 - x 2 t + 1 2 u 2 t ] dt subject to dx t = u t dW t , x 0 = 1 u t [ - 1 , 1] , t [0 , 1] a) Express E [ x 2 t ] in terms of the control u , and hence find the optimal control of the problem. b) Write down the HJB equation for this stochastic control problem, and solve it. c) Hence find the optimal control and compare with the result in a). Solution:
1
b) The HJB equation for this problem is, v t + sup u [ - 1 , 1] { 1 2 u 2 v xx - x 2 + 1 2 u 2 } = 0 v ( T, x ) = 0 There are two cases, v xx + 1 > 0 or v xx + 1 0. If v xx + 1 > 0, then u * t = 1 , - 1 the HJB equation becomes, v t + 1 2 ( v xx + 1) - x 2 = 0 with the boundary condition we solve for v ( t, x ) v ( t, x ) = ( t - 1) x 2 - t 2 2 + t 2 its derivative v xx + 1 = 2 t - 1, because we assume v xx + 1 > 0 thus this solution is valid only when t > 1 2 . Now if v xx + 1 0, then u * t = 0, the HJB equation becomes, v t - x 2 = 0 with the boundary condition we solve for v ( t, x ) v ( t, x ) = ( t - 1) x 2 similar as first case, this solution is valid only when t 1 2 . Thus by discussion above, the optimal control is, u * t = ( 0 t 1 2 1 , - 1 t > 1 2 (3) c) The results are same in part a) and part b).