test-sol - STA 447/2006S, Winter 2008: In-Class Test...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
STA 447/2006S, Winter 2008: In-Class Test (DRAFT) SOLUTIONS 1. [8 points] Let ( p ij ) be the transition probabilities for random walk on the graph whose vertices are V = { 1 , 2 , 3 , 4 } , with a single edge between each of the four pairs (1,2), (2,3), (3,1), and (3,4), and no other edges. Compute (with full explanation) lim n →∞ p ( n ) 13 . Solution: Since 1 2 3 4 , the graph is connected, so the random walk is irreducible. Also, state 3 is aperiodic since e.g. it is possible to get from 3 to 3 in two steps ( 3 4 3 ) or in three steps ( 3 1 2 3 ), and gcd (2 , 3) = 1 . So, by irreducibility, all states are aperiodic. Also, from class, the chain has stationary distribution given by π u = d ( u ) / v d ( v ) = d ( u ) / 2 | E | . In this case, d (1) = d (2) = 2 , d (3) = 3 , and d (4) = 1 , so v d ( v ) = 2 + 2 + 3 + 1 = 8 (or equivalently, 2 | E | = 2 · 4 = 8 ). Hence, by the Markov chain convergence theorem, lim n →∞ p ( n ) 13 = π 3 = d (3) / 8 = 3 / 8 . 2. Consider the Markov chain with state space S = { 1 , 2 , 3 } , and transition probabilities given by p 11 = 1 / 6, p 12 = 1 / 3, p 13 = 1 / 2, p 22 = p 33 = 1, and p ij = 0 otherwise. (a) [4 points] Compute (with explanation) f 12 (i.e., the probability, starting from 1, that the chain will eventually visit 2). Solution: When the chain leaves the state 1, it goes to either 2 or 3 and then stays there. So, f 12 = P ( X 1 = 2 | X 0 = 1 , X 1 6 = 1) = p 12 / (1 - p 11 ) = 1 / 3 / (1 - 1 / 6) = (1 / 3) / (5 / 6) = 2 / 5 . Alternatively, since f 32 = 0 (because p 33 = 1 ), we have that f 12 = p 12 + j 6 =2 p 1 j f j 2 = p 12 + p 11 f 12 + p 13 f 32 = (1 / 3) + (1 / 6) f 12 + (1 / 2)(0) , so (5 / 6) f 12 = 1 / 3 , so f 12 = 2 / 5 . (b)
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 03/27/2010.

Page1 / 4

test-sol - STA 447/2006S, Winter 2008: In-Class Test...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online