Stochastic

1 since i r 1 i r r2 for x y 62 a i number

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 0, so to investigate the convergence of p (x, y ) it is enough, by the decomposition theorem, to suppose the chain is irreducible and all states are recurrent. The period of a state is the greatest common divisor of Ix = {n 1 : pn (x, x) > 0}. If the period is 1, x is said to be aperiodic. A simple su cient condition to be aperiodic is that p(x, x) > 0. To compute the period it is useful to note that if ⇢xy > 0 and ⇢yx > 0 then x and y have the same period. In particular all of the states in an irreducible set have the same period. The three main results about the asymptotic behavior of Markov chains are: n Theorem 1.19. Suppose p is irreducible, aperiodic, and has a stationary distribution ⇡ . Then as n ! 1, pn (x, y ) ! ⇡ (y ). Theorem 1.21. Suppose p is irreducible and recurrent. If Nn (y ) be the number of visits to y up to time n, then Nn (y ) 1 ! n Ey T y Theorem 1.23. Suppose p is irreducible, has stationary distribution ⇡ , and P x |f (x)|⇡ (x) < 1 then n X 1X f (Xm ) ! f (x)⇡ (x) n m=1 x 62 CHAPTER 1. MA...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online