# hw4_sol.pdf - IEOR-E4707 Spring 2019 Homework 4 Solutions...

• 4

This preview shows page 1 - 3 out of 4 pages.

IEOR-E4707, Spring 2019: Homework 4 Solutions March 26, 2019 Problem 1. Consider the following stochastic control problem (the state, control and the Brownian motion are all one-dimensional): max J ( u ) = E [ Z 1 0 - x 2 t + 1 2 u 2 t ] dt subject to dx t = u t dW t , x 0 = 1 u t [ - 1 , 1] , t [0 , 1] a) Express E [ x 2 t ] in terms of the control u , and hence find the optimal control of the problem. b) Write down the HJB equation for this stochastic control problem, and solve it. c) Hence find the optimal control and compare with the result in a). Solution:
1
b) The HJB equation for this problem is, v t + sup u [ - 1 , 1] { 1 2 u 2 v xx - x 2 + 1 2 u 2 } = 0 v ( T, x ) = 0 There are two cases, v xx + 1 > 0 or v xx + 1 0. If v xx + 1 > 0, then u * t = 1 , - 1 the HJB equation becomes, v t + 1 2 ( v xx + 1) - x 2 = 0 with the boundary condition we solve for v ( t, x ) v ( t, x ) = ( t - 1) x 2 - t 2 2 + t 2 its derivative v xx + 1 = 2 t - 1, because we assume v xx + 1 > 0 thus this solution is valid only when t > 1 2 . Now if v xx + 1 0, then u * t = 0, the HJB equation becomes, v t - x 2 = 0 with the boundary condition we solve for v ( t, x ) v ( t, x ) = ( t - 1) x 2