SolutionsII - ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II Spring 2007 1. Since this Markov chain has finitely many states, its convenient to solve parts of the problem using the transition matrix P defined by [ P ] ij = P ( i, j ), where P ( i, j ) is the probability of making a one-step transition from i to j . For this example, P = 2 6 6 4 r p q q r p q r p p q r 3 7 7 5 . (a) We could do this by computing P 2 , the square of the transition matrix. Then P (2) ( i, j ) = [ P 2 ] ij for all i and j . However, its easier to look at the picture. To make a two-step transition from 1 to 2, you can either go 1 to 1 to 2 or 1 to 2 to 2. These are mutually exclusive possibilities, so their probabilities add to give P (2) (1 , 2) = rp + pr = 2 pr . The same reasoning applies to two-step transitions from 2 to 3, 3 to 4, and 4 to 1, so P (2) (1 , 2) = P (2) (2 , 3) = P (2) (3 , 4) = P (2) (4 , 1) = 2 pr Similarly, using the q s instead of the p , we get P (2) (1 , 4) = P (2) (2 , 1) = P (2) (3 , 2) = P (2) (4 , 3) = 2 qr For the i-to- i transitions, the only way to make any of them in two steps is to stay put or move to an adjacent state and back. For each i , this yields P (2) ( i, i ) = r 2 + 2 pq . To make a 1-to-3 transition, you can go either 1 to 2 to 3 or 1 to 4 to 3. The 3-to-1 transitions follow those paths in reverse. Because of the symmetries in the probabilities, the results come out the same, namely P (2) (1 , 3) = P (2) (3 , 1) = p 2 + q 2 . Youll find that the same holds for the remaining two possibilities: P (2) (2 , 4) = P (2) (4 , 2) = p 2 + q 2 . Note that P 4 j =1 P (2) ( i, j ) = 1 for all i . Just to check this for i = 1, the sum is r 2 + 2 pq + 2 pr + p 2 + q 2 + 2 pq = ( p + q + r ) 2 = 1 , (b) Here are the possible 4-step trips from 1 to 3 that dont hit 3 until step 4, along with their probabilities: 1-1-1-2-3: r 2 p 2 1-1-2-2-3: rprp = r 2 p 2 1-2-2-2-3: pr 2 p = r 2 p 2 1-2-1-2-3: pqpp = p 3 q 1-1-1-4-3: r 2 q 2 1-1-4-4-3: r 2 q 2 1-4-4-4-3: r 2 q 2 1-4-1-4-3: q 3 p Add these together and you get f (4) (1 , 3). 1 (c) The chain is generic in the sense that it has only one recurrence class, and that class is positively recurrent, so the chain has a unique stationary distribution * , which satisfies 4 X i =1 * i P ( i, j ) = * j for all j . Thinking of * as a row vector, thats the same as * P = * , where P is the transition matrix above. The equations read ( r- 1) * 1 + q * 2 + p * 4 = p * 1 + ( r- 1) * 2 + q * 3 = p * 2 + ( r- 1) * 3 + q * 4 = q * 1 + p * 3 + ( r- 1) * 4 = The unique distribution solving these equations is * i = 1 / 4 for all i . This might not seem too surprising considering how symmetric the chain is....
View Full Document

This note was uploaded on 02/01/2010 for the course ECE 496 taught by Professor Delchamps during the Spring '07 term at Cornell University (Engineering School).

Page1 / 6

SolutionsII - ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online