# During the next decade 6 of the homeowners became

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: RKOV CHAINS Chains with absorbing states In this case there are two interesting questions. Where does the chain get absorbed? How long does it take? Let Vy = min{n 0 : Xn = y } be the time of the ﬁrst visit to y , i.e., now being there at time 0 counts. Theorem 1.27. Consider a Markov chain with ﬁnite state space S . Let a and b be two points in S , and let C = S {a, b}. Suppose h(a) = 1, h(b) = 0, and that for x 2 C we have X h(x) = p(x, y )h(y ) y If ⇢xa + ⇢xb &gt; 0 for all x 2 C , then h(x) = Px (Va &lt; Vb ). Let r(x, y ) be the part of the matrix p(x, y ) with x, y 2 C . Since h(a) = 1 and h(b) = 0, the equation for h can be written for x 2 C as h(x) = r(x, a) + X r(x, y )h(y ) y so if we let v be the column vector with entries r(x, a) then the last equation says (I r)h = v and h = (I r) 1 v. Theorem 1.28. Consider a Markov chain with ﬁnite state space S . Let A ⇢ S and VA = inf {n 0 : Xn 2 A}. Suppose g (a) = 0 for all a 2 A, and that for x 2 C = S A we have X g (x) =...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online