During the next decade 6 of the homeowners became

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: RKOV CHAINS Chains with absorbing states In this case there are two interesting questions. Where does the chain get absorbed? How long does it take? Let Vy = min{n 0 : Xn = y } be the time of the first visit to y , i.e., now being there at time 0 counts. Theorem 1.27. Consider a Markov chain with finite state space S . Let a and b be two points in S , and let C = S {a, b}. Suppose h(a) = 1, h(b) = 0, and that for x 2 C we have X h(x) = p(x, y )h(y ) y If ⇢xa + ⇢xb > 0 for all x 2 C , then h(x) = Px (Va < Vb ). Let r(x, y ) be the part of the matrix p(x, y ) with x, y 2 C . Since h(a) = 1 and h(b) = 0, the equation for h can be written for x 2 C as h(x) = r(x, a) + X r(x, y )h(y ) y so if we let v be the column vector with entries r(x, a) then the last equation says (I r)h = v and h = (I r) 1 v. Theorem 1.28. Consider a Markov chain with finite state space S . Let A ⇢ S and VA = inf {n 0 : Xn 2 A}. Suppose g (a) = 0 for all a 2 A, and that for x 2 C = S A we have X g (x) =...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online