This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE292 Spring 2006 Analysis & Control of Markov Chains May 21, 2006 Prof. Ben Van Roy Homework Assignment 5 : Solutions 5.2 a) We have a linear quadratic problem with imperfect state information, where A k ,B b ,Q k , and R k are all 1 . Consequently, the optimal control law is given by: μ * k ( I k ) = L k E [ x k  I k ] where L k = ( R k + B k K k +1 B b ) 1 B k K k +1 A k = K k +1 1 + K k +1 with K N = 1 . Moreover the Ricatti equation may be simplified here to yield K k = 1 + 2 K k +1 1 + K k +1 Now, we claim that in fact E [ x k  I k ] = x k . To see this simply note that if z ∈ { 2 1 / 4 , 2 + 1 / 4 } then it must be that x = 2 and if z ∈ { 2 1 / 4 , 2 + 1 / 4 } , then it must be that x = 2 . Proceeding by induction, if we know x k 1 perfectly given I k 1 then with knowledge of u k 1 and given z k , we know w k + v k perfectly since w k + v k = z k x k 1 u k 1 But knowing w k + v k here is sufficient to know w k exactly since we have w k = 1 iff w k + v k ∈ { 3 / 4 , 5 / 4 } and w k = 1 iff w k + v k ∈ { 5 / 4 , 3 / 4 } . Consequently, we know x k = x k 1 + u k 1 + w k perfectly given I k ....
View
Full
Document
This note was uploaded on 02/15/2011 for the course EECS 6.231 taught by Professor Bertsekas during the Spring '10 term at MIT.
 Spring '10
 Bertsekas

Click to edit the document details