This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ECE 496 SOLUTIONS TO HOMEWORK ASSIGNMENT II Spring 2007 1. Since this Markov chain has finitely many states, its convenient to solve parts of the problem using the transition matrix P defined by [ P ] ij = P ( i, j ), where P ( i, j ) is the probability of making a onestep transition from i to j . For this example, P = 2 6 6 4 r p q q r p q r p p q r 3 7 7 5 . (a) We could do this by computing P 2 , the square of the transition matrix. Then P (2) ( i, j ) = [ P 2 ] ij for all i and j . However, its easier to look at the picture. To make a twostep transition from 1 to 2, you can either go 1 to 1 to 2 or 1 to 2 to 2. These are mutually exclusive possibilities, so their probabilities add to give P (2) (1 , 2) = rp + pr = 2 pr . The same reasoning applies to twostep transitions from 2 to 3, 3 to 4, and 4 to 1, so P (2) (1 , 2) = P (2) (2 , 3) = P (2) (3 , 4) = P (2) (4 , 1) = 2 pr Similarly, using the q s instead of the p , we get P (2) (1 , 4) = P (2) (2 , 1) = P (2) (3 , 2) = P (2) (4 , 3) = 2 qr For the ito i transitions, the only way to make any of them in two steps is to stay put or move to an adjacent state and back. For each i , this yields P (2) ( i, i ) = r 2 + 2 pq . To make a 1to3 transition, you can go either 1 to 2 to 3 or 1 to 4 to 3. The 3to1 transitions follow those paths in reverse. Because of the symmetries in the probabilities, the results come out the same, namely P (2) (1 , 3) = P (2) (3 , 1) = p 2 + q 2 . Youll find that the same holds for the remaining two possibilities: P (2) (2 , 4) = P (2) (4 , 2) = p 2 + q 2 . Note that P 4 j =1 P (2) ( i, j ) = 1 for all i . Just to check this for i = 1, the sum is r 2 + 2 pq + 2 pr + p 2 + q 2 + 2 pq = ( p + q + r ) 2 = 1 , (b) Here are the possible 4step trips from 1 to 3 that dont hit 3 until step 4, along with their probabilities: 11123: r 2 p 2 11223: rprp = r 2 p 2 12223: pr 2 p = r 2 p 2 12123: pqpp = p 3 q 11143: r 2 q 2 11443: r 2 q 2 14443: r 2 q 2 14143: q 3 p Add these together and you get f (4) (1 , 3). 1 (c) The chain is generic in the sense that it has only one recurrence class, and that class is positively recurrent, so the chain has a unique stationary distribution * , which satisfies 4 X i =1 * i P ( i, j ) = * j for all j . Thinking of * as a row vector, thats the same as * P = * , where P is the transition matrix above. The equations read ( r 1) * 1 + q * 2 + p * 4 = p * 1 + ( r 1) * 2 + q * 3 = p * 2 + ( r 1) * 3 + q * 4 = q * 1 + p * 3 + ( r 1) * 4 = The unique distribution solving these equations is * i = 1 / 4 for all i . This might not seem too surprising considering how symmetric the chain is....
View
Full
Document
This note was uploaded on 02/01/2010 for the course ECE 496 taught by Professor Delchamps during the Spring '07 term at Cornell University (Engineering School).
 Spring '07
 DELCHAMPS
 Algorithms

Click to edit the document details