Hw3_ORIE361_solns_2008

# Hw3_ORIE361_solns_2008 - k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .....

This preview shows pages 1–2. Sign up to view the full content.

ORIE 361 – Homework 3 (Introduction to Discrete Time Markov Chains) Instructor: Mark E. Lewis due February 13, 2008 (drop box) Answers. 1. No, the chain is not Markov. Note P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 2) = 0, since, { X n - 2 = 2 } implies { Y n - 2 = 1 } and { X n - 1 = 1 ,Y n - 2 = 1 } implies { Y n - 1 = 0 } which in turn implies X n cannot be 2. On the other hand, P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 0) = 1 2 , since { X n - 2 = 0 } implies { Y n - 2 = 0 } and { X n - 1 = 1 ,Y n - 2 = 0 } implies { Y n - 1 = 1 } . The Markov property would require P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 2) = P ( X n = 2 | X n - 1 = 1 ,X n - 2 = 0) = P ( X n = 2 | X n - 1 = 1) , which is obviously not the case. 2. Let the state on any day be the number of coin that is ﬂipped on that day. ( 1 if coin 1 is ﬂipped on that day). So the transition matrix is P = 1 2 ± . 7 . 3 . 6 . 4 ² . So, P 3 = 1 2 ± . 667 . 333 . 666 . 334 ² . Hence, The required probability = 1 2 [ P 3 11 + P 3 21 ] = 0 . 6665 . 3. (a) The classes are { 0 } , { 5 } , { 1 } , { 2 } , { 3,4 } (b) 0 and 5 are the recurrent states and 1,2,3,4 are the transient states. 4. P( T 1 =1) = P( X 1 = 1 | X 0 = 1) = P 11 = 1 - β . k 2, P ( T 1 = k ) = P ( X

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .. ,X 1 = 0 | X = 1) = P 10 P k-2 00 P 01 = k-2 (1- ) 1 E ( T 1 ) = X k =1 kP ( T 1 = k ) = 1- + X k =2 k k-2 (1- ) = 1- + (2 + 1- ) = 1 + 1- 5. (a) The classes are { } , { d } , { 1,2,. . . , d-1 } (b) 0 and d are the recurrent states and 1,2, .. . , d-1 are the transient states. 6. (a) The only class is the whole state space. This chain is irreducible. (b) All the states are recurrent. (c) Let f be the function that relates the each state to its cost, i.e. f (0) = 20 ,f (1) = 30 ,f (2) = 50 ,f (3) = 300 E ( f ( X 1 ) | X = 0) = 3 X k =0 f ( k ) P ( X 1 = k | X = 0) = 0 . 6 f (0) + 0 . 2 f (1) + 0 . 1 f (2) + 0 . 1 f (3) = 53 2...
View Full Document

## This note was uploaded on 03/01/2010 for the course ORIE 361 taught by Professor Lewis,m. during the Spring '07 term at Cornell University (Engineering School).

### Page1 / 2

Hw3_ORIE361_solns_2008 - k = 1 ,X k-1 = 0 ,X k-2 = 0 ,. .....

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online