Winter 2010
Pstat160a
Handout MC#3
Applications of making a state absorbing
1. General Remarks
If a Markov Chain
Y
n
has an absorbing state
a
and transition matrix
P
y
whose
n
th
power is
P
n
y
, then
P
n
y
(
i,a
) represent
not only the usual
n
steps probability, but also the probability that
Y
n
reached
a
in
n
or less steps
, since the chain
“get stuck” in
a
. Similarly,
P
n
y
(
i,j
)
, i,j
6
=
a
represent the probability that
Y
n
never visited
a
in the ﬁrst
N
steps. Now
suppose that the chain
X
n
has the same transition probability than
Y
n
, except that
a
is not absorbing. The chains
X
n
and
Y
n
behave the same way until they reach the state
a
, that is their transition probabilities are identical up to that
point.
2. Visit to a state in less than
n
steps
Starting with a Markov chain with matrix
P
, suppose we want to ﬁnd the probability that the chain visit a particular
state in
n
steps or less. It should be clear by taking complement that this will give us also the probability that the chain
never visit that state in the ﬁrst
n
steps.
The basic idea is to make this state absorbing (call it
a
) and leave the rest of the chain intact. The resulting new chain
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '10
 bonnet
 Probability, Probability theory, yn, Markov chain

Click to edit the document details