{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

HW8_Sol

# HW8_Sol - ECEN 646 Homework 8 Solutions 1 Cinlar...

This preview shows pages 1–3. Sign up to view the full content.

ECEN 646 Homework 8 Solutions 1 C ¸inlar Exercise (4.2), Chapter 5 (a) Since X is a Markov chain, the transitions are conditionally inde- pendent of the past given the present. Therefore, the required probability can be written as P { X 5 = b,X 6 = r,X 7 = b,X 8 = b | X 4 = w } = P { X 5 = b | X 4 = w } P { X 6 = r | X 5 = b } P { X 7 = b | X 6 = r } P { X 8 = b | X 7 = b } = P (2 , 3) P (3 , 1) P (1 , 3) P (3 , 3) = (0 . 6)(0 . 8)(1)(0 . 2) = 0 . 096 (b) We can define a new variable Y = f ( X ) which will once again be Markov since the function f ( . ) is invertible. This new chain will have the state space E = { 2 , 4 , 7 , 3 } . The transition probability remains unchanged by this relabelling of states. E [ f ( X 5 ) f ( X 6 ) | X 4 = y ] = E [ Y 5 Y 6 | Y 4 = 3] = summationdisplay i E summationdisplay j E i jP ( Y 5 = i,Y 6 = j | Y 4 = 3) = summationdisplay i E summationdisplay j E i jP ( Y 5 = i | Y 4 = 3) P ( Y 6 = j | Y 5 = i ) = 14 . 41 2 C ¸inlar Exercise (4.4), Chapter 5 (a) It is clear that if we start from either state 1 or state 3, we remain forever within these 2 states. Therefore, the set { 1 , 3 } is a closed, irreducible set. Also, starting at states 2 and 4, we enter the states { 1 , 3 } sooner or later. Therefore the states 2 and 4 are transient. States 1 and 3 are recurrent positive. Since, P (1 , 1) > 0, state 1 is aperiodic. Since states 1 and 3 belong to the same irreducible set, state 3 is aperiodic as well. 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
States 1 and 3 - Recurrent positive, aperiodic. States 2 and 4 - Transient. (b) The set { 1 , 3 , 4 } is closed and irreducible. Starting at states 2 or 5 sooner or later lands us in this set. Also P (3 , 3) > 0. These yield States 1,3,4 - Recurrent positive, aperiodic States 2,5 - Transient. (c) The states can be split into 2 closed irreducible sets as { 1 , 3 , 4 } and { 2 , 5 } . We have P (1 , 1) > 0 and P (2 , 2) > 0. This means that both sets are aperiodic and recurrent positive. Therefore all the states in this chain are recurrent positive and aperiodic. (d) The sets { 1 , 3 , 6 } and { 2 , 5 } are closed and irreducible. Starting at states 4 or 7 leads us into one of these 2 sets. Therefore, 4 and 7 are transient states. Also, P (1 , 1) > 0 and P (5 , 5) > 0. Combining these observations, we have States 1,3,6 - Recurrent positive, aperiodic States 2,5 - Recurrent positive, aperiodic States 4,7 - Transient (e) The set { 1 , 3 , 5 }
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}