Stochastic

# If not pick a site w 2 s t r1 and repeat the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: nd we have x ! z by deﬁnition, so Lemma 1.4 implies y ! z . Cx is closed and irreducible so all states in Cx are recurrent. Let R1 = Cx . If S T R1 = ;, we are done. If not, pick a site w 2 S T R1 and repeat the procedure. ******* The rest of this section is devoted to the proof of Theorem 1.7. To do this, it is enough to prove the following two results. Lemma 1.9. If x is recurrent and x ! y , then y is recurrent. Lemma 1.10. In a ﬁnite closed set there has to be at least one recurrent state. To prove these results we need to introduce a little more theory. Recall the time of the k th visit to y deﬁned by k k Ty = min{n > Ty 1 : Xn = y } and ⇢xy = Px (Ty < 1) the probability we ever visit y at some time n 1 when we start from x. Using the strong Markov property as in the proof of (1.4) gives k Px (Ty < 1) = ⇢xy ⇢k 1 . yy (1.5) Let N (y ) be the number of visits to y at times n compute EN (y ). 1. Using (1.5) we can 16 CHAPTER 1. MARKOV CHAINS Lemma 1.11. Ex N (y )...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online