This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 1 ⇢xy
⇢yy so y is recurrent if and only if Ey N (y ) = 1. Theorems 1.5 and 1.7 allow us to decompose the state space and simplify
the study of Markov chains.
Theorem 1.8. If the state space S is ﬁnite, then S can be written as a disjoint
union T [ R1 [ · · · [ Rk , where T is a set of transient states and the Ri , 1 i k ,
are closed irreducible sets of recurrent states.
Stationary distributions
A stationary measure is a nonnegative solution of µp = µ A stationary
distribution is a nonnegative solution of ⇡ p = ⇡ normalized so that the entries
sum to 1. The ﬁrst question is: do these things exist?
Theorem 1.20. Suppose p is irreducible and recurrent. Let x 2 S and let
Tx = inf {n 1 : Xn = x}.
µx (y ) = 1
X Px (Xn = y, Tx > n) n=0 deﬁnes a stationary measure with 0 < µx (y ) < 1 for all y . 61 1.11. CHAPTER SUMMARY If the state space S is ﬁnite and irreducible there is a unique stationary distribution. More generally if Ex Tx < 1, i.e., x is positive recurrent then µ...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 The Land

Click to edit the document details