This preview shows page 1. Sign up to view the full content.
Unformatted text preview: RKOV CHAINS Chains with absorbing states
In this case there are two interesting questions. Where does the chain get
absorbed? How long does it take? Let Vy = min{n 0 : Xn = y } be the time
of the ﬁrst visit to y , i.e., now being there at time 0 counts.
Theorem 1.27. Consider a Markov chain with ﬁnite state space S . Let a and
b be two points in S , and let C = S {a, b}. Suppose h(a) = 1, h(b) = 0, and
that for x 2 C we have
X
h(x) =
p(x, y )h(y )
y If ⇢xa + ⇢xb > 0 for all x 2 C , then h(x) = Px (Va < Vb ). Let r(x, y ) be the part of the matrix p(x, y ) with x, y 2 C . Since h(a) = 1 and
h(b) = 0, the equation for h can be written for x 2 C as
h(x) = r(x, a) + X r(x, y )h(y ) y so if we let v be the column vector with entries r(x, a) then the last equation
says (I r)h = v and
h = (I r) 1 v.
Theorem 1.28. Consider a Markov chain with ﬁnite state space S . Let A ⇢ S
and VA = inf {n 0 : Xn 2 A}. Suppose g (a) = 0 for all a 2 A, and that for
x 2 C = S A we have
X
g (x) =...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 The Land

Click to edit the document details