problem_session_1_10-11_sol

problem_session_1_10-11_sol - MS&E 221 Problem...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: MS&E 221 Problem Session 1 Solutions CA: Hongsong Yuan Jan 28, 2011 Problem 1( Expected Hitting Time ) A spider and an insect are sitting on the opposite vertices of a cube. In each time period, the spider moves to one of its adjacent vertices with even probability, while the insect stays at the same vertex all the time. What is the expected time for the spider to catch the insect? The location of the spider forms a Markov chain with 8 states. The transition probabilities are: P ij = 1 / 3 , if i,j are adjacent , otherwise Let h ( i ) be the hitting time of state 8, given the chain starts with state i . By first transition analysis, we have h ( i ) = 1 + X j P ij h ( j ) , i = 1 , 2 , , 7 h (8) = 0 By symmetry, h (2) = h (4) = h (6) , h (3) = h (5) = h (7) . So the system of linear equations reduces to h (1) = 1 + 1 3 3 h (2) h (2) = 1 + 1 3 2 h (3) + 1 3 h (1) h (3) = 1 + 1 3 2 h (2) + 1 3 Solving this system, we get h (1) = 10 . 1 Problem 2( Verifying Recurrence/Transience ) Consider a Markov chain with the following tran- sition matrix: P = . 2 0 0 . 8 0 0 1 0 . 2 . 8 0 0 0 . 5 0 . 5 0 0 . 6 . 4 0 0 0 0 1 1 0 0 1. Identify the communicating classes. 2. Identify the positive recurrent, null recurrent and transient states. 3. Identify the periodicity for each closed class. 1. The communicating classes are { 1 , 3 , 5 } , { 4 } , { 2 , 6 , 7 } . 2. The classes { 1 , 3 , 5 } and { 2 , 6 , 7 } are closed and finite, so they are recurrent. { 4 } is not closed, so it is transient....
View Full Document

Page1 / 5

problem_session_1_10-11_sol - MS&E 221 Problem...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online