Notes - Copyright c 2009 by Karl Sigman 1 Time-reversible...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Copyright c 2009 by Karl Sigman 1 Time-reversible Markov chains In these notes we study positive recurrent Markov chains { X n : n } for which, when in steady-state (stationarity), yield the same Markov chain (in distribution) if time is reversed. The fundamental condition required is that for each pair of states i,j the long-run rate at which the chain makes a transition from state i to state j equals the long-run rate at which the chain makes a transition from state j to state i ; i P i,j = j P j,i . 1.1 Two-sided stationary extensions of Markov chains For a positive recurrent Markov chain { X n : n N } with transition matrix P and stationary distribution , let { X * n : n N } denote a stationary version of the chain, that is, one in which X . It turns out that we can extend this process to have time n take on negative values as well, that is, extend it to { X * n : n Z } . This is a way of imagining/assuming that the chain started off initially in the infinite past, and we call this a two-sided extension of our process. To get such an extension 1 we start by shifting the origin to be time k 1 and extending the process k time units into the past: Define X * n ( k ) = X * n + k ,- k n < . Note how { X * n ( k ) : n N } has the same distribution as { X * n : n N } by stationarity, and in fact this extension on- k n < is still stationary too. Now as k , the process { X * n ( k ) :- k n < } converges (in distribution) to a truly two-sided extension and it remains stationary; we get the desired two-sided stationary extension { X * n : n Z } . And for each time n Z it holds that P ( X * n = j ) = j , j S . 1.2 Time-reversibility: Time-reversibility equations Let { X * n : n Z } be a two-sided extension of a positive recurrent Markov chain with transition matrix P and stationary distribution . The Markov property is stated as the future is independent of the past given the present state, and thus can be re-stated as the past is...
View Full Document

Page1 / 4

Notes - Copyright c 2009 by Karl Sigman 1 Time-reversible...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online