160a_timerev

160a_timerev - Winter 2010 Pstat160a Hand-out MC#6 Time...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Winter 2010 Pstat160a Hand-out MC#6 Time reversible Markov Chains 1. Definition Consider an irreducible, aperiodic (=ergodic) Markov chain X n with transition probability matrix P and stationary distribution π . Suppose the chain is in equilibrium distribution (steady state) that is P ( X n = i ) = π i . For instance, the chain starts with stationary distribution that is P ( X = i ) = π i , or the chain is run for a long time. Instead of observing just which states X n is, observe transitions. Then the probability that you see a transition from i to j is π i P ij (the chain needs to be in i and moves to j ). A Markov chain is said to be reversible (or time reversible ) if (and only if! ) it is as likely to see a transition i to j than j to i that is X n reversible: π i P ij = π j P ji for all i,j ∈ S Detailed balance equations (1) Important remarks: 1) This in no way means that P ij = P ji that is that the chain is equality likely to move from i to j than j to i !!!! 2) The result implies that if we can find π that satisfies the detailed balance equations (1), the X n is reversible and π is the stationary distribution. When it works ( X n reversible), this is a much simpler way to find π ! Indeed, we can check that (1) implies that pi i is π is the stationary distribution, just take ∑ j on both sides of (1) and you get the general balance equation for an ergodic chain (equation (1) in handout MC #5). The converse if of course not generally true (only for reversiblefor an ergodic chain (equation (1) in handout MC #5)....
View Full Document

This note was uploaded on 01/10/2011 for the course STAT 160A taught by Professor Bonnet during the Winter '10 term at UCSB.

Page1 / 3

160a_timerev - Winter 2010 Pstat160a Hand-out MC#6 Time...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online