{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Hw2_ORIE361_2008_solns

# Hw2_ORIE361_2008_solns - ORIE 361 Homework 2(Introduction...

This preview shows pages 1–4. Sign up to view the full content.

ORIE 361 – Homework 2 (Introduction to Discrete Time Markov Chains) Instructor: Mark E. Lewis due February 5, 2007 (drop box) Answers. 1. The state space is Ω = { 0 , 1 , . . . , d } and P i,j = ( 2 i j )( 2 d - 2 i d - j ) ( 2 d d ) j = 0 , 1 , . . . , d, i = 1 , 2 , . . . , d - 1 0 and d are the two absorbing states. P 0 , 0 = 1 , P d,d = 1 2. Let, the state be the number of umbrellas he has at his present location. The state space = { 0,1, . . . ,r } . The transition probabilities are P 0 ,r = 1 , P 0 ,i = 0 i = 0 , 1 , 2 , . . . , r - 1 . P i,r - i = 1 - p, P i,r - i +1 = p i = 1 , 2 , . . . , r. P i,j = 0 ifj / ∈ { r - i, r - i + 1 } i = 1 , 2 , . . . , r. 3. i) In order to have X k = 0, based on our definition of the process, we need to have 2 , 3 , .... , k = 0. So, P ( X k = 0) = P ( 2 = 0 , 3 = 0 , .... , k = 0) = 1 2 k - 1 ii) P ( X 5 = 1 | X 4 = 1 , X 3 = 0) = P ( X 4 + X 3 + 5 = 1 | X 4 = 1 , X 3 = 0) = P ( 5 = 0 | X 4 = 1 , X 3 = 0) = P ( 5 = 1) = 1 2 iii) P ( X 5 = 1 | X 4 = 1) = P ( X 5 = 1 | X 4 = 1 , X 3 = 1) P ( X 3 = 1 | X 4 = 1) + P ( X 5 = 1 | X 4 = 1 , X 3 = 0) P ( X 3 = 0 | X 4 = 1) 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
= 0 + 1 2 . 1 2 = 1 4 b) If the process was Markov we should have P ( X 5 = 1 | X 4 = 1 , X 3 = 0) = P ( X 5 = 1 | X 4 = 1) . Since, from the calculations above, this does not hold, the process is not Markov. What makes the process not Markovian is that the value of the process at time n , does not depend only on the value of the process at time n - 1 but also at time n - 2. 4. Check the graph below. 2
Figure 1: Probability diagrams

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}