Markov

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: all x 2 S. t=0 1.73 Example. In Example 1.69, the state 0 is an accessible atom. Our goal in this section is a Basic Limit Theorem for chains that have atoms. Although it is natural to think that most chains of interest do not have atoms, so that the theory developed in this section would not often apply, we will see in the next section how a surprisingly large class of chains may be viewed as chains with an atom. 1.74 Proposition. Suppose a chain with an accessible atom bution . Then f g 0 and is recurrent. has a stationary distri- is accessible, it follows that for each state x there is S t such that a t x; f g g 0. That is, de ning Gt = fx : P 0g, we have Gt = S. So there is an n such that Gn  0, which gives Proof: Since P t x; f Z f g = dxP n x; f g  Z Gn dxP n x; f g 0: The integral of a positive function over a set of positive measure is positive. The proof that is recurrent is like what we did before for countable state spaces. Since P fXt = g = f g 0 for all t, de ning N = P1 I fXt = g, we get E  N  = 1. But t=0 E N   E x N  for all states x; recall that starting from R we get to count at least one visit to for sure! So, averaging over , we get E N   dxE x N  = E  N , so that E N  = 1. This implies the recurrence of , by the geometric trials argument from before. 1.75 Proposition. Suppose the chain fXt g has an accessible atom distribution . Let B be a set that is not accessible from , that is, Then B  = 0. P and a stationary 1g = 0. fTB Proof: De ne B ;n = fx 2 B : Px fT  ng  g: S By the assumption that is an accessible atom, m;n B1=m;n = B . Thus, we will be done if we show that B ;n  = 0 for each n and each 0. So consider a xed n and 0. Stochastic Processes J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS Page 1-39 Starting from any x 2 B ;n, with probability at least , the chain goes to within n steps, and then never returns to B ;n. The last statement about not returning to B ;n follows by de nition of B and the fact that B ;n B . So each time we enter P ;n , there is probability B at least that within n steps we leave B ;n forever. De ning N = 1 I fXt 2 B ;ng to be t=0 the total number of visits to the set B ;n, a bit of thought shows that Py N  n= for each E y 2 S. Here is one way to see this. Look at the total number N0 = 1 I fXrn 2 B ;ng r=0 of visits to B ;n at times 0; n; 2n; : : :. Then Py fN0 1g  1 , , Py fN0 2g  1 , 2 , and so on. So 1 1 X X E y N0 = PfN0 r g  1 , r = 1= : r=0 r=0 P Similarly, for each 0 k n, the number Nk = 1 I fXk+rn 2 B ;ng of visits at times r=0 k; k + n; k +2n; : : : satis es E y Nk  1= . Thus, N = N0 + N1 +    + Nn,1 has expected value at most n= , starting from any state y. So E  N  n= . This implies that B ;n  = 0: if B ;n  were positive, then clearly E  N would be in nite, which we have just shown is not the case. The previous result implies that a stationary chain with an accessible atom will not enter a set of states that is not accessible from . ...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online