Unformatted text preview: in with transition probability
q (i, j )
r(i, j ) =
i goes through the same sequence of states as Xt but stays in each one for on
unit of time. Let VA = min{t : Xt 2 A} be the time of the ﬁrst visit to A.
Then h(i) = Pi (X (VA ) = a) satisﬁes h(a) = 1, h(b) = 0 for b 2 A {a} and
X
h(i) =
r(i, j )h(j ) for i 62 A.
j The expected hitting time g (i) = Ei VA satsiﬁes g (a) = 0 for a 2 A and
g (i) = 1
i + X r(i, j )g (j ) j for i 62 A. One can also work directly with the transition rates. In the ﬁrst case the
equation with the same boundary conditions (h(a) = 1, h(b) = 0 for b 2 A {a}
and for i 62 A) is
X
Q(i, j )h(j ) = 0 for i 62 A
i In the second case, if if we let R be the part of the Qmatrix where i, j 62 A
then
g = R 11
where 1 is a column vector of all 1’s.
P
Stationary distributions. A stationary distribution has
i ⇡ (i) = 1 and
satisﬁes ⇡ pt = ⇡ for all t > 0, which is equivalent to ⇡ Q = 0. To solve these
equations mechanically, we replace the last column of Q by all 1’s to deﬁne a
matrix A and then ⇡ will be the last row of...
View
Full
Document
 Spring '10
 DURRETT
 The Land

Click to edit the document details