This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MS&E 221 Problem Session 1 Solutions CA: Hongsong Yuan Jan 28, 2011 Problem 1( Expected Hitting Time ) A spider and an insect are sitting on the opposite vertices of a cube. In each time period, the spider moves to one of its adjacent vertices with even probability, while the insect stays at the same vertex all the time. What is the expected time for the spider to catch the insect? The location of the spider forms a Markov chain with 8 states. The transition probabilities are: P ij = ‰ 1 / 3 , if i,j are adjacent , otherwise Let h ( i ) be the hitting time of state 8, given the chain starts with state i . By first transition analysis, we have h ( i ) = 1 + X j P ij h ( j ) , i = 1 , 2 , ··· , 7 h (8) = 0 By symmetry, h (2) = h (4) = h (6) , h (3) = h (5) = h (7) . So the system of linear equations reduces to h (1) = 1 + 1 3 · 3 h (2) h (2) = 1 + 1 3 · 2 h (3) + 1 3 h (1) h (3) = 1 + 1 3 · 2 h (2) + 1 3 · Solving this system, we get h (1) = 10 . 1 Problem 2( Verifying Recurrence/Transience ) Consider a Markov chain with the following tran sition matrix: P = . 2 0 0 . 8 0 0 1 0 . 2 . 8 0 0 0 . 5 0 . 5 0 0 . 6 . 4 0 0 0 0 1 1 0 0 1. Identify the communicating classes. 2. Identify the positive recurrent, null recurrent and transient states. 3. Identify the periodicity for each closed class. 1. The communicating classes are { 1 , 3 , 5 } , { 4 } , { 2 , 6 , 7 } . 2. The classes { 1 , 3 , 5 } and { 2 , 6 , 7 } are closed and finite, so they are recurrent. { 4 } is not closed, so it is transient....
View
Full
Document
This note was uploaded on 02/03/2011 for the course MS&E 221 taught by Professor Ramesh during the Winter '11 term at Stanford.
 Winter '11
 Ramesh

Click to edit the document details