HW8_Sol - ECEN 646 Homework 8 Solutions 1 C ¸inlar...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
ECEN 646 Homework 8 Solutions 1 C ¸inlar Exercise (4.2), Chapter 5 (a) Since X is a Markov chain, the transitions are conditionally inde- pendent of the past given the present. Therefore, the required probability can be written as P { X 5 = b,X 6 = r,X 7 = b,X 8 = b | X 4 = w } = P { X 5 = b | X 4 = w } P { X 6 = r | X 5 = b } P { X 7 = b | X 6 = r } P { X 8 = b | X 7 = b } = P (2 , 3) P (3 , 1) P (1 , 3) P (3 , 3) = (0 . 6)(0 . 8)(1)(0 . 2) = 0 . 096 (b) We can define a new variable Y = f ( X ) which will once again be Markov since the function f ( . ) is invertible. This new chain will have the state space E = { 2 , 4 , 7 , 3 } . The transition probability remains unchanged by this relabelling of states. E [ f ( X 5 ) f ( X 6 ) | X 4 = y ] = E [ Y 5 Y 6 | Y 4 = 3] = summationdisplay i E summationdisplay j E i jP ( Y 5 = i,Y 6 = j | Y 4 = 3) = summationdisplay i E summationdisplay j E i jP ( Y 5 = i | Y 4 = 3) P ( Y 6 = j | Y 5 = i ) = 14 . 41 2 C ¸inlar Exercise (4.4), Chapter 5 (a) It is clear that if we start from either state 1 or state 3, we remain forever within these 2 states. Therefore, the set { 1 , 3 } is a closed, irreducible set. Also, starting at states 2 and 4, we enter the states { 1 , 3 } sooner or later. Therefore the states 2 and 4 are transient. States 1 and 3 are recurrent positive. Since, P (1 , 1) > 0, state 1 is aperiodic. Since states 1 and 3 belong to the same irreducible set, state 3 is aperiodic as well. 1 This study resource was shared via CourseHero.com
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
States 1 and 3 - Recurrent positive, aperiodic. States 2 and 4 - Transient. (b) The set { 1 , 3 , 4 } is closed and irreducible. Starting at states 2 or 5 sooner or later lands us in this set. Also P (3 , 3) > 0. These yield States 1,3,4 - Recurrent positive, aperiodic States 2,5 - Transient. (c) The states can be split into 2 closed irreducible sets as { 1 , 3 , 4 } and { 2 , 5 } . We have P (1 , 1) > 0 and P (2 , 2) > 0. This means that both sets are aperiodic and recurrent positive. Therefore all the states in this chain are recurrent positive and aperiodic. (d) The sets { 1 , 3 , 6 } and { 2 , 5 } are closed and irreducible. Starting at states 4 or 7 leads us into one of these 2 sets. Therefore, 4 and 7 are transient states. Also, P (1 , 1) > 0 and P (5 , 5) > 0. Combining these observations, we have States 1,3,6 - Recurrent positive, aperiodic States 2,5 - Recurrent positive, aperiodic States 4,7 - Transient (e) The set { 1 , 3 , 5 } is closed and irreducible as is the set { 4 , 6 } . The tran- sitions in { 1 , 3 , 5 } are such that we can come back to the starting state only in an even number of steps.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern