Hw3_ORIE361_solns_2008 - k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .....

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
ORIE 361 – Homework 3 (Introduction to Discrete Time Markov Chains) Instructor: Mark E. Lewis due February 13, 2008 (drop box) Answers. 1. No, the chain is not Markov. Note P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 2) = 0, since, { X n - 2 = 2 } implies { Y n - 2 = 1 } and { X n - 1 = 1 ,Y n - 2 = 1 } implies { Y n - 1 = 0 } which in turn implies X n cannot be 2. On the other hand, P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 0) = 1 2 , since { X n - 2 = 0 } implies { Y n - 2 = 0 } and { X n - 1 = 1 ,Y n - 2 = 0 } implies { Y n - 1 = 1 } . The Markov property would require P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 2) = P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 0) = P ( X n = 2 | X n - 1 = 1) , which is obviously not the case. 2. Let the state on any day be the number of coin that is flipped on that day. ( 1 if coin 1 is flipped on that day). So the transition matrix is P = 1 2 ± . 7 . 3 . 6 . 4 ² . So, P 3 = 1 2 ± . 667 . 333 . 666 . 334 ² . Hence, The required probability = 1 2 [ P 3 11 + P 3 21 ] = 0 . 6665 . 3. (a) The classes are { 0 } , { 5 } , { 1 } , { 2 } , { 3,4 } (b) 0 and 5 are the recurrent states and 1,2,3,4 are the transient states. 4. P( T 1 =1) = P( X 1 = 1 | X 0 = 1) = P 11 = 1 - β . k 2, P ( T 1 = k ) = P ( X
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .. ,X 1 = 0 | X = 1) = P 10 P k-2 00 P 01 = k-2 (1- ) 1 E ( T 1 ) = X k =1 kP ( T 1 = k ) = 1- + X k =2 k k-2 (1- ) = 1- + (2 + 1- ) = 1 + 1- 5. (a) The classes are { } , { d } , { 1,2,. . . , d-1 } (b) 0 and d are the recurrent states and 1,2, .. . , d-1 are the transient states. 6. (a) The only class is the whole state space. This chain is irreducible. (b) All the states are recurrent. (c) Let f be the function that relates the each state to its cost, i.e. f (0) = 20 ,f (1) = 30 ,f (2) = 50 ,f (3) = 300 E ( f ( X 1 ) | X = 0) = 3 X k =0 f ( k ) P ( X 1 = k | X = 0) = 0 . 6 f (0) + 0 . 2 f (1) + 0 . 1 f (2) + 0 . 1 f (3) = 53 2...
View Full Document

This note was uploaded on 03/01/2010 for the course ORIE 361 taught by Professor Lewis,m. during the Spring '07 term at Cornell University (Engineering School).

Page1 / 2

Hw3_ORIE361_solns_2008 - k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .....

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online