# 1 pn x y y to state the next resultp need a denition

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: sider now the basketball chain (Example 1.10): HH HM MH MM HH 3/4 0 2/ 3 0 HM 1/4 0 1/3 0 MH 0 2/3 0 1 /2 MM 0 1 /3 0 1 /2 26 CHAPTER 1. MARKOV CHAINS Lemma 1.17 implies that HH and MM are aperiodic. Since this chain is irreducible it follows from Lemma 1.18 that HM and MH are aperiodic. ******* We now come to the main results of the chapter. We ﬁrst list the assumptions. All of these results hold when S is ﬁnite or inﬁnite. • I : p is irreducible • A : aperiodic, all states have period 1 • R : all states are recurrent • S : there is a stationary distribution ⇡ Theorem 1.19. Convergence theorem. Suppose I , A, S . Then as n ! 1, pn (x, y ) ! ⇡ (y ). To state the next resultP need a deﬁnition. We day that µ(x) we 0 is a stationary measure if x µ(x)p(x, y ) = µ(y ). If S is ﬁnite we can normalize µ to be a stationary distribution. Theorem 1.20. Suppose I and R. Then there is a stationary measure with µ(x) > 0 for all x. The next result describes the “limiting fraction of time we spend...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online