# Form a matrix a by taking the rst k 1 columns of p i

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: let Ty = min{n 1: Xn = y } and let ⇢xy = Px (Ty &lt; 1) When x 6= y this is the probability Xn ever visits y starting at x. When x = y this is the probability Xn returns to y when it starts at y . We restrict to times n 1 in the deﬁnition of Ty so that we can say: y is recurrent if ⇢yy = 1 and transient if ⇢yy &lt; 1. Transient states in a ﬁnite state space can all be identiﬁed using Theorem 1.5. If ⇢xy &gt; 0, but ⇢yx &lt; 1, then x is transient. Once the transient states are removed we can use Theorem 1.7. If C is a ﬁnite closed and irreducible set, then all states in C are recurrent. Here A is closed if x 2 A and y 62 A implies p(x, y ) = 0, and B is irreducible if x, y 2 B implies ⇢xy &gt; 0. The keys to the proof of Theorem 1.7 are: (i) If x is recurrent and ⇢xy &gt; 0 then y is recurrent, and (ii) In a ﬁnite closed set there has to be at least one recurrent state. To prove these results, it was useful to know that if N (y ) is the number of visits to y at times n 1 then 1 X pn (x, y ) = Ex N (y ) = n=1...
View Full Document

Ask a homework question - tutors are online