{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# test-sol - STA 447/2006S Winter 2008 In-Class Test(DRAFT...

This preview shows pages 1–2. Sign up to view the full content.

STA 447/2006S, Winter 2008: In-Class Test (DRAFT) SOLUTIONS 1. [8 points] Let ( p ij ) be the transition probabilities for random walk on the graph whose vertices are V = { 1 , 2 , 3 , 4 } , with a single edge between each of the four pairs (1,2), (2,3), (3,1), and (3,4), and no other edges. Compute (with full explanation) lim n →∞ p ( n ) 13 . Solution: Since 1 2 3 4 , the graph is connected, so the random walk is irreducible. Also, state 3 is aperiodic since e.g. it is possible to get from 3 to 3 in two steps ( 3 4 3 ) or in three steps ( 3 1 2 3 ), and gcd (2 , 3) = 1 . So, by irreducibility, all states are aperiodic. Also, from class, the chain has stationary distribution given by π u = d ( u ) / v d ( v ) = d ( u ) / 2 | E | . In this case, d (1) = d (2) = 2 , d (3) = 3 , and d (4) = 1 , so v d ( v ) = 2 + 2 + 3 + 1 = 8 (or equivalently, 2 | E | = 2 · 4 = 8 ). Hence, by the Markov chain convergence theorem, lim n →∞ p ( n ) 13 = π 3 = d (3) / 8 = 3 / 8 . 2. Consider the Markov chain with state space S = { 1 , 2 , 3 } , and transition probabilities given by p 11 = 1 / 6, p 12 = 1 / 3, p 13 = 1 / 2, p 22 = p 33 = 1, and p ij = 0 otherwise. (a) [4 points] Compute (with explanation) f 12 (i.e., the probability, starting from 1, that the chain will eventually visit 2). Solution: When the chain leaves the state 1, it goes to either 2 or 3 and then stays there. So, f 12 = P ( X 1 = 2 | X 0 = 1 , X 1 = 1) = p 12 / (1 - p 11 ) = 1 / 3 / (1 - 1 / 6) = (1 / 3) / (5 / 6) = 2 / 5 . Alternatively, since f 32 = 0 (because p 33 = 1 ), we have that f 12 = p 12 + j =2 p 1 j f j 2 = p 12 + p 11 f 12 + p 13 f 32 = (1 / 3) + (1 / 6) f 12 + (1 / 2)(0) , so (5 / 6) f 12 = 1 / 3 , so f 12 = 2 / 5 . (b) [3 points] Prove that p ( n ) 12 1 / 3, for any positive integer n .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 4

test-sol - STA 447/2006S Winter 2008 In-Class Test(DRAFT...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online