This preview shows page 1. Sign up to view the full content.
Unformatted text preview: follows,
lemmas are a means to prove the more important conclusions called theorems.
The two are numbered in the same sequence to make results easier to ﬁnd.
Lemma 1.4. If x ! y and y ! z , then x ! z .
Proof. Since x ! y there is an m so that pm (x, y ) > 0. Similarly there is
an n so that pn (y, z ) > 0. Since pm+n (x, z ) pm (x, y )pn (y, z ) it follows that
x ! z.
Theorem 1.5. If ⇢xy > 0, but ⇢yx < 1, then x is transient. 14 CHAPTER 1. MARKOV CHAINS Proof. Let K = min{k : pk (x, y ) > 0} be the smallest number of steps we can
take to get from x to y . Since pK (x, y ) > 0 there must be a sequence y1 , . . . yK 1
so that
p(x, y1 )p(y1 , y2 ) · · · p(yK 1 , y ) > 0
Since K is minimal all the yi 6= y (or there would be a shorter path), and we
have
Px (Tx = 1) p(x, y1 )p(y1 , y2 ) · · · p(yK 1 , y )(1 ⇢yx ) > 0
so x is transient. We will see later that Theorem 1.5 allows us to to identify all the transient
states when the state space is ﬁnite. An immediate consequence...
View
Full
Document
 Spring '10
 DURRETT
 The Land

Click to edit the document details