Markov

# We want to show that stochastic processes j chang

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1.76 Proposition. Suppose a Markov chain has an accessible atom distribution . Then P fT 1g = 1. and a stationary Proof: Let B = fx : Px fT = 1g 0g; these are the states from which it is possible to go forever without hitting . We want to show that B  = 0. Since is recurrent, if the chain starts from state , then with probability 1 it will return to in nitely many times. Therefore, the set B cannot be accessible from , for if it were, there would be positive probability, starting from , of eventually entering the set B and then never returning to . Thus, by the previous proposition, B  = 0. 1.77 Definition. Let  and  be two probability measures on a set S. We say that  is absolutely continuous with respect to  if A = 0 for all A S such that  A = 0, that is, each set having probability 0 under  also has probability 0 under . Suppose a chain fXt g with transition kernel P and an aperiodic, accessible atom has a stationary distribution . Let t denote the distribution of Xt and start the chain in any distribution 0 that is absolutely continuous with respect to . Then kt , k ! 0 as t ! 1. 1.78 Theorem. Proof: We use the coupling technique from before; much of the reasoning remains the same, so I'll just give a sketch. Again, we run two independent copies of the chain, fXt g and fXt g, starting in the initial distributions 0 and , respectively. We want to show that Stochastic Processes J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-40 with probability 1 the two chains eventually couple; in fact we claim that they eventually visit the state at the same time. By using the aperiodicity assumption together with the number-theoretic lemma from before, we see that the bivariate chain fXt ; Xt  : t = 0; 1; : : :g has an accessible atom  ; . The bivariate chain has a stationary distribution: the obvious product distribution   . So by Proposition 1.76, if the bivariate chain were started out in its stationary distribution   , it would eventually hit its atom  ;  with probability 1. That is, letting A denote the set of pairs of states x; y such that Px;y fT ;  1g = 1, we have   A = 1. From this, the absolute continuity of 0 with respect to  implies that 0  A = 1 observe that   Ac  = 0 implies 0  A = 0 . Thus, P0  fT ;  1g = 1, as claimed. 1.79 Exercise. Do we really need the hypothesis about the absolute continuity of 0 ? Here is an example although somewhat technical and arti cial that shows how things can go wrong without it. Let the state space S be the unit interval 0; 1 . Let B = f2,n : n = 1; 2; : : :g. De ne the distribution  to have probability mass 1 2 on the point 1 and density 1 2 on the rest of the interval, 0; 1. For each state x 2 B , take the next-state distribution P x;  to = ,n 2 B , de ne P 2,n ;  to have mass 2n+1 ,2 on the point 2,n+1 and the be . For x = 2 2n+1 ,1 remaining mass 1=2n+1 , 1 on the point 1. Show that the state 1 is an accessible atom, and that  is a stationary distribution for the chain. But what happens if we start...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online