# hw 4 - game as follows A state i is chosen at random...

This preview shows page 1. Sign up to view the full content.

205B homework, week 5; due Thursday February 26 These are miscellaneous questions on Markov chains, not necessarily closely connected to this week’s class material. 1. Let X n be the Markov chain on states 0 , 1 ,...,K with transition matrix p ( i,i + 1) = 2 / 3 and p ( i,i - 1) = 1 / 3; 1 i K - 1 p (0 , 0) = p ( K,K ) = 1 and initial state i 0 6 = 0 ,K . Let X * n be the process X n conditioned on the event { X m = K ultimately } . (a) Prove carefully that X * n is Markov. (b) Find its transition matrix. (c) Find the distribution of min n 0 X * n . 2. Let S be a ﬁnite set. Let p ( i,j ) be an irreducible Markov transition matrix on P , with stationary distribution π . Consider a cat-and-mouse
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: game, as follows. A state i is chosen at random according to π ; the cat and mouse are both placed at i , but before the cat can do anything the mouse jumps to another state chosen according to p ( i, · ). Thereafter, the mouse doesn’t move. The cat now searches for the mouse by moving at random according to the “time-reversed” Markov chain, i.e. the chain with transition matrix q ( i,j ) = π ( j ) p ( j,i ) /π ( i ) . Find a simple formula for the expected number of steps taken by the cat until it ﬁnds the mouse. 4...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online