# xn gives x p xt 1 z xt y t n p xn1 z

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: according to the values of X0 , . . . , Xn gives X P (XT +1 = z, XT = y, T = n) = P (Xn+1 = z, Xn = xn , . . . , X0 = x0 ) = X x2Vn x2Vn P (Xn+1 = z |Xn = xn , . . . , X0 = x0 )P (Xn = xn , . . . , X0 = x0 ) where in the second step we have used the multiplication rule P (A \ B ) = P (B |A)P (A) 12 CHAPTER 1. MARKOV CHAINS For any (x0 , . . . , xn ) 2 Vn we have T = n and XT = y so xn = y . Using the Markov property, (1.1), and recalling the deﬁnition of Vn shows the above X P (XT +1 = z, T = n, XT = y ) = p(y, z ) P (Xn = xn , . . . , X0 = x0 ) x2Vn = p(y, z )P (T = n, XT = y ) Dividing both sides by P (T = n, XT = y ) gives the desired result. 1 Let Ty = Ty and for k 2 let k k Ty = min{n > Ty 1 : Xn = y } (1.3) be the time of the k th return to y . The strong Markov property implies that the conditional probability we will return one more time given that we have returned k 1 times is ⇢yy . This and induction implies that k Py (Ty < 1) = ⇢k yy (1.4) At this point, there are two possibilities: (i) ⇢yy < 1: The...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online