soln8 - ECE320 Solution Notes 8 Cornell University 1 In a...

• Homework Help
• 501844684_ch
• 7

This preview shows pages 1–3. Sign up to view the full content.

ECE320 Solution Notes 8 Spring 2006 Cornell University T.L.Fine 1. In a Markov chain, let X = { ξ 1 , ξ 2 , ξ 3 } and π (1) = [ . 2 , . 3 , . 5] , P = . 5 . 5 0 . 3 . 3 . 4 0 . 6 . 4 . (a) Evaluate P ( X 2 = ξ 2 ). π (2) = π (1) P , P ( X 2 = ξ 2 ) = π (2) (2) = . 49 . (b) Draw a state transition diagram. .5 .3 .5 .3 .4 .4 .6 1 2 3 ξ ξ ξ Figure 1: (c) Classify the three states as to whether they are absorbing, persistent, or transient. All three states are persistent in that with probability one you will always eventually return to each of them. None of them are absorbing in that with positive probability (actually probability one) you will eventually exit from each of these three states. (d) What are the periodicities d i for ξ i ? As each state has a self-loop, a positive probability of returning to itself in one step, the periodicities are all equal to 1. (e) Identify the communicating classes. There is a single communicating class containing all three states; each state is accessible in no more than two steps for each other state. 1

This preview has intentionally blurred sections. Sign up to view the full version.

(f) Identify the closed communicating classes. As there is a single communicat- ing class in a finite Markov chain, it must be closed. Alternatively, as all states reside in a single communicating class, it is obviously closed. (g) If there is a stationary or limiting initial distribution π , then determine π . If there is no such limiting initial distribution, then provide reasons. This solution exists because the Markov chain is irreducible (a single closed communicating class) and aperiodic (period 1 for all states). The unique stationary or limiting distribution satisfies π = π P ⇐⇒ π (1) = . 5 π (1) + . 3 π (2) + 0 , π (2) = . 5 π (1) + . 3 π (2) + . 6 π (3) , π (3) = 0 + . 4 π (2) + . 4 π (3) . The unique solution, normalized to sum to 1 as required of a pmf , is π (1) = 9 24 , π (2) = 15 24 , π (3) = 10 24 . (h) If there is a stationary limiting distribution, what is the expected time between returns to state ξ 2 ? As the Markov chain is irreducible and aperiodic. ET 2 , 2 = 1 π (2) = 24 15 . 2. Repeat Problem 1 for P = . 5 . 5 0 0 . 6 . 4 0 . 2 . 8 . (a) P ( X 2 = ξ 2 ) = π (2) (2) = . 2( . 5) + . 3( . 6) + . 5( . 2) = . 38 . (b) (c) State ξ 1 is transient; with probability one you will eventually leave this state, never to return. The other two states are both persistent but neither is absorbing as there is a positive probability of leaving each of ξ 2 , ξ 3 in one step.
This is the end of the preview. Sign up to access the rest of the document.
• Spring '06
• FINE
• Probability theory, Markov chain, Cornell University

{[ snackBarMessage ]}

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern