HW8_Sol - ECEN 646 Homework 8 Solutions 1 C ¸inlar...

This preview shows pages 1–3. Sign up to view the full content.

ECEN 646 Homework 8 Solutions 1 C ¸inlar Exercise (4.2), Chapter 5 (a) Since X is a Markov chain, the transitions are conditionally inde- pendent of the past given the present. Therefore, the required probability can be written as P { X 5 = b,X 6 = r,X 7 = b,X 8 = b | X 4 = w } = P { X 5 = b | X 4 = w } P { X 6 = r | X 5 = b } P { X 7 = b | X 6 = r } P { X 8 = b | X 7 = b } = P (2 , 3) P (3 , 1) P (1 , 3) P (3 , 3) = (0 . 6)(0 . 8)(1)(0 . 2) = 0 . 096 (b) We can define a new variable Y = f ( X ) which will once again be Markov since the function f ( . ) is invertible. This new chain will have the state space E = { 2 , 4 , 7 , 3 } . The transition probability remains unchanged by this relabelling of states. E [ f ( X 5 ) f ( X 6 ) | X 4 = y ] = E [ Y 5 Y 6 | Y 4 = 3] = summationdisplay i E summationdisplay j E i jP ( Y 5 = i,Y 6 = j | Y 4 = 3) = summationdisplay i E summationdisplay j E i jP ( Y 5 = i | Y 4 = 3) P ( Y 6 = j | Y 5 = i ) = 14 . 41 2 C ¸inlar Exercise (4.4), Chapter 5 (a) It is clear that if we start from either state 1 or state 3, we remain forever within these 2 states. Therefore, the set { 1 , 3 } is a closed, irreducible set. Also, starting at states 2 and 4, we enter the states { 1 , 3 } sooner or later. Therefore the states 2 and 4 are transient. States 1 and 3 are recurrent positive. Since, P (1 , 1) > 0, state 1 is aperiodic. Since states 1 and 3 belong to the same irreducible set, state 3 is aperiodic as well. 1 This study resource was shared via CourseHero.com

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
States 1 and 3 - Recurrent positive, aperiodic. States 2 and 4 - Transient. (b) The set { 1 , 3 , 4 } is closed and irreducible. Starting at states 2 or 5 sooner or later lands us in this set. Also P (3 , 3) > 0. These yield States 1,3,4 - Recurrent positive, aperiodic States 2,5 - Transient. (c) The states can be split into 2 closed irreducible sets as { 1 , 3 , 4 } and { 2 , 5 } . We have P (1 , 1) > 0 and P (2 , 2) > 0. This means that both sets are aperiodic and recurrent positive. Therefore all the states in this chain are recurrent positive and aperiodic. (d) The sets { 1 , 3 , 6 } and { 2 , 5 } are closed and irreducible. Starting at states 4 or 7 leads us into one of these 2 sets. Therefore, 4 and 7 are transient states. Also, P (1 , 1) > 0 and P (5 , 5) > 0. Combining these observations, we have States 1,3,6 - Recurrent positive, aperiodic States 2,5 - Recurrent positive, aperiodic States 4,7 - Transient (e) The set { 1 , 3 , 5 } is closed and irreducible as is the set { 4 , 6 } . The tran- sitions in { 1 , 3 , 5 } are such that we can come back to the starting state only in an even number of steps.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern