# The two are numbered in the same sequence to make

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: follows, lemmas are a means to prove the more important conclusions called theorems. The two are numbered in the same sequence to make results easier to ﬁnd. Lemma 1.4. If x ! y and y ! z , then x ! z . Proof. Since x ! y there is an m so that pm (x, y ) &gt; 0. Similarly there is an n so that pn (y, z ) &gt; 0. Since pm+n (x, z ) pm (x, y )pn (y, z ) it follows that x ! z. Theorem 1.5. If ⇢xy &gt; 0, but ⇢yx &lt; 1, then x is transient. 14 CHAPTER 1. MARKOV CHAINS Proof. Let K = min{k : pk (x, y ) &gt; 0} be the smallest number of steps we can take to get from x to y . Since pK (x, y ) &gt; 0 there must be a sequence y1 , . . . yK 1 so that p(x, y1 )p(y1 , y2 ) · · · p(yK 1 , y ) &gt; 0 Since K is minimal all the yi 6= y (or there would be a shorter path), and we have Px (Tx = 1) p(x, y1 )p(y1 , y2 ) · · · p(yK 1 , y )(1 ⇢yx ) &gt; 0 so x is transient. We will see later that Theorem 1.5 allows us to to identify all the transient states when the state space is ﬁnite. An immediate consequence...
View Full Document

Ask a homework question - tutors are online