The period of a state is the greatest common divisor

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1 ⇢xy ⇢yy so y is recurrent if and only if Ey N (y ) = 1. Theorems 1.5 and 1.7 allow us to decompose the state space and simplify the study of Markov chains. Theorem 1.8. If the state space S is finite, then S can be written as a disjoint union T [ R1 [ · · · [ Rk , where T is a set of transient states and the Ri , 1 i k , are closed irreducible sets of recurrent states. Stationary distributions A stationary measure is a nonnegative solution of µp = µ A stationary distribution is a nonnegative solution of ⇡ p = ⇡ normalized so that the entries sum to 1. The first question is: do these things exist? Theorem 1.20. Suppose p is irreducible and recurrent. Let x 2 S and let Tx = inf {n 1 : Xn = x}. µx (y ) = 1 X Px (Xn = y, Tx > n) n=0 defines a stationary measure with 0 < µx (y ) < 1 for all y . 61 1.11. CHAPTER SUMMARY If the state space S is finite and irreducible there is a unique stationary distribution. More generally if Ex Tx < 1, i.e., x is positive recurrent then µ...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online