This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Copyright c 2009 by Karl Sigman 1 Timereversible Markov chains In these notes we study positive recurrent Markov chains { X n : n ≥ } for which, when in steadystate (stationarity), yield the same Markov chain (in distribution) if time is reversed. The fundamental condition required is that for each pair of states i,j the longrun rate at which the chain makes a transition from state i to state j equals the longrun rate at which the chain makes a transition from state j to state i ; π i P i,j = π j P j,i . 1.1 Twosided stationary extensions of Markov chains For a positive recurrent Markov chain { X n : n ∈ N } with transition matrix P and stationary distribution π , let { X * n : n ∈ N } denote a stationary version of the chain, that is, one in which X ∼ π . It turns out that we can extend this process to have time n take on negative values as well, that is, extend it to { X * n : n ∈ Z } . This is a way of imagining/assuming that the chain started off initially in the infinite past, and we call this a twosided extension of our process. To get such an extension 1 we start by shifting the origin to be time k ≥ 1 and extending the process k time units into the past: Define X * n ( k ) = X * n + k , k ≤ n < ∞ . Note how { X * n ( k ) : n ∈ N } has the same distribution as { X * n : n ∈ N } by stationarity, and in fact this extension on k ≤ n < ∞ is still stationary too. Now as k → ∞ , the process { X * n ( k ) : k ≤ n < ∞} converges (in distribution) to a truly twosided extension and it remains stationary; we get the desired twosided stationary extension { X * n : n ∈ Z } . And for each time n ∈ Z it holds that P ( X * n = j ) = π j , j ∈ S . 1.2 Timereversibility: Timereversibility equations Let { X * n : n ∈ Z } be a twosided extension of a positive recurrent Markov chain with transition matrix P and stationary distribution π . The Markov property is stated as “the future is independent of the past given the present state”, and thus can be restated as “the past is...
View
Full Document
 Fall '11
 N/A
 Economics, Markov chain, Karl Sigman

Click to edit the document details