{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

SolutionsII - ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II Spring 2007 1. Since this Markov chain has finitely many states, it’s convenient to solve parts of the problem using the transition matrix P defined by [ P ] ij = P ( i, j ), where P ( i, j ) is the probability of making a one-step transition from i to j . For this example, P = 2 6 6 4 r p 0 q q r p 0 0 q r p p 0 q r 3 7 7 5 . (a) We could do this by computing P 2 , the square of the transition matrix. Then P (2) ( i, j ) = [ P 2 ] ij for all i and j . However, it’s easier to look at the picture. To make a two-step transition from 1 to 2, you can either go 1 to 1 to 2 or 1 to 2 to 2. These are mutually exclusive possibilities, so their probabilities add to give P (2) (1 , 2) = rp + pr = 2 pr . The same reasoning applies to two-step transitions from 2 to 3, 3 to 4, and 4 to 1, so P (2) (1 , 2) = P (2) (2 , 3) = P (2) (3 , 4) = P (2) (4 , 1) = 2 pr Similarly, using the q ’s instead of the p , we get P (2) (1 , 4) = P (2) (2 , 1) = P (2) (3 , 2) = P (2) (4 , 3) = 2 qr For the i -to- i transitions, the only way to make any of them in two steps is to stay put or move to an “adjacent” state and back. For each i , this yields P (2) ( i, i ) = r 2 + 2 pq . To make a 1-to-3 transition, you can go either 1 to 2 to 3 or 1 to 4 to 3. The 3-to-1 transitions follow those paths in reverse. Because of the symmetries in the probabilities, the results come out the same, namely P (2) (1 , 3) = P (2) (3 , 1) = p 2 + q 2 . You’ll find that the same holds for the remaining two possibilities: P (2) (2 , 4) = P (2) (4 , 2) = p 2 + q 2 . Note that P 4 j =1 P (2) ( i, j ) = 1 for all i . Just to check this for i = 1, the sum is r 2 + 2 pq + 2 pr + p 2 + q 2 + 2 pq = ( p + q + r ) 2 = 1 , (b) Here are the possible 4-step trips from 1 to 3 that don’t hit 3 until step 4, along with their probabilities: 1-1-1-2-3: r 2 p 2 1-1-2-2-3: rprp = r 2 p 2 1-2-2-2-3: pr 2 p = r 2 p 2 1-2-1-2-3: pqpp = p 3 q 1-1-1-4-3: r 2 q 2 1-1-4-4-3: r 2 q 2 1-4-4-4-3: r 2 q 2 1-4-1-4-3: q 3 p Add these together and you get f (4) (1 , 3). 1
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
(c) The chain is “generic” in the sense that it has only one recurrence class, and that class is positively recurrent, so the chain has a unique stationary distribution π * , which satisfies 4 X i =1 π * i P ( i, j ) = π * j for all j . Thinking of π * as a row vector, that’s the same as π * P = π * , where P is the transition matrix above. The equations read ( r - 1) π * 1 + * 2 + * 4 = 0 * 1 + ( r - 1) π * 2 + * 3 = 0 * 2 + ( r - 1) π * 3 + * 4 = 0 * 1 + * 3 + ( r - 1) π * 4 = 0 The unique distribution solving these equations is π * i = 1 / 4 for all i . This might not seem too surprising considering how symmetric the chain is. (d) From the convergence theorems in class, 1 m j = π * j for all j , so the limiting fraction of the time the chain spends in each state is just 1/4 for every state. Again, not so surprising. 2. (a) States 3 and 4 are clearly transient, because you can leave the “3-4 cluster” (and therefore never return) with positive probability. States 1 and 2 are recurrent — starting from state 1, the only way you could leave 1 and never return is to go to 2 and stay there forever, which has probability (1 - δ ) = 0, where I’m assuming and δ are both positive. Similarly, 5, 6, and 7 are recurrent. The only
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern