This preview shows page 1. Sign up to view the full content.
Unformatted text preview: sider now the basketball chain (Example 1.10):
HH
HM
MH
MM HH
3/4
0
2/ 3
0 HM
1/4
0
1/3
0 MH
0
2/3
0
1 /2 MM
0
1 /3
0
1 /2 26 CHAPTER 1. MARKOV CHAINS Lemma 1.17 implies that HH and MM are aperiodic. Since this chain is
irreducible it follows from Lemma 1.18 that HM and MH are aperiodic.
*******
We now come to the main results of the chapter. We ﬁrst list the assumptions. All of these results hold when S is ﬁnite or inﬁnite.
• I : p is irreducible
• A : aperiodic, all states have period 1
• R : all states are recurrent
• S : there is a stationary distribution ⇡
Theorem 1.19. Convergence theorem. Suppose I , A, S . Then as n ! 1,
pn (x, y ) ! ⇡ (y ).
To state the next resultP need a deﬁnition. We day that µ(x)
we
0 is a
stationary measure if x µ(x)p(x, y ) = µ(y ). If S is ﬁnite we can normalize
µ to be a stationary distribution.
Theorem 1.20. Suppose I and R. Then there is a stationary measure with
µ(x) > 0 for all x.
The next result describes the “limiting fraction of time we spend...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 The Land

Click to edit the document details