{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Hw3_ORIE361_solns_2008

Hw3_ORIE361_solns_2008 - k = 1,X k-1 = 0,X k-2 = 0,X 1 = 0...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
ORIE 361 – Homework 3 (Introduction to Discrete Time Markov Chains) Instructor: Mark E. Lewis due February 13, 2008 (drop box) Answers. 1. No, the chain is not Markov. Note P ( X n = 2 | X n - 1 = 1 , X n - 2 = 2) = 0, since, { X n - 2 = 2 } implies { Y n - 2 = 1 } and { X n - 1 = 1 , Y n - 2 = 1 } implies { Y n - 1 = 0 } which in turn implies X n cannot be 2. On the other hand, P ( X n = 2 | X n - 1 = 1 , X n - 2 = 0) = 1 2 , since { X n - 2 = 0 } implies { Y n - 2 = 0 } and { X n - 1 = 1 , Y n - 2 = 0 } implies { Y n - 1 = 1 } . The Markov property would require P ( X n = 2 | X n - 1 = 1 , X n - 2 = 2) = P ( X n = 2 | X n - 1 = 1 , X n - 2 = 0) = P ( X n = 2 | X n - 1 = 1) , which is obviously not the case. 2. Let the state on any day be the number of coin that is flipped on that day. ( 1 if coin 1 is flipped on that day). So the transition matrix is P = 1 2 . 7 . 3 . 6 . 4 . So, P 3 = 1 2 . 667 . 333 . 666 . 334 . Hence, The required probability = 1 2 [ P 3 11 + P 3 21 ] = 0 . 6665 . 3. (a) The classes are { 0 } , { 5 } , { 1 } , { 2 } , { 3,4 } (b) 0 and 5 are the recurrent states and 1,2,3,4 are the transient states. 4. P( T 1 =1) = P( X 1 = 1 | X 0 = 1) = P 11 = 1 - β . k
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .. ,X 1 = 0 | X = 1) = P 10 P k-2 00 P 01 = βα k-2 (1-α ) 1 E ( T 1 ) = ∞ X k =1 kP ( T 1 = k ) = 1-β + β ∞ X k =2 kα k-2 (1-α ) = 1-β + β (2 + α 1-α ) = 1 + β 1-α 5. (a) The classes are { } , { d } , { 1,2,. . . , d-1 } (b) 0 and d are the recurrent states and 1,2, .. . , d-1 are the transient states. 6. (a) The only class is the whole state space. This chain is irreducible. (b) All the states are recurrent. (c) Let f be the function that relates the each state to its cost, i.e. f (0) = 20 ,f (1) = 30 ,f (2) = 50 ,f (3) = 300 E ( f ( X 1 ) | X = 0) = 3 X k =0 f ( k ) P ( X 1 = k | X = 0) = 0 . 6 f (0) + 0 . 2 f (1) + 0 . 1 f (2) + 0 . 1 f (3) = 53 2...
View Full Document

{[ snackBarMessage ]}