This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ass function :5; 0; :5. Similarly, if Xt = 3, we use the mass
~
function 0; 1; 0. Finally, if Xt = , we know that Xt was either 2 or 3 and C = Heads, so
we use the mass function :375; 0; :625.
Again we have decomposed each transition of the given chain, according to P , into 2
stages, as depicted in Figure 1.81. These stages make transitions according to the matrices
U and V , given by
01 2 31
01 2 3 0 1
1 :1 :5 :4
1100
B5
5C
U = 2 @ 0 :2 0 :8 A; V = 2 B :0 0 :0 C:
A
3@
1
3 0 0 :2 :8
:375 0 :625
We started with a set of states R = f2; 3g. For each i 2 R, we then wrote P i;
as a mixture of some xed probability mass function = :375; 0; :625 with some other
probability mass function Qi; in our example Q2; = :5; 0; :5 and Q3; = 0; 1; 0 .
P i; = 0:8 + 0:2Qi; .
We have broken down each transition of the chain into two stages. Starting from the
state Xt in the rst stage we note whether or not Xt is in the set R, and if so we toss the
biased coin. If the coin toss comes up Heads, we move to state , and otherwise we stay
~
where we are; the result is the state we have called Xt . Then we draw the next state Xt+1
from the appropriate distribution. The point is that we have introduced a new state that
we can reach by hitting any state in the set R and then getting a Heads from the coin toss.
This is the key in general state spaces: if we can take the set R to be large enough, the set
R will have positive probability of being hit, even though each individual state in R may
have probability 0 of being hit. And if R is hit with positive probability, then so is , since
hitting only requires hitting R and a Heads from the coin toss.
Note also that we could have chosen R in di erent ways. For example, consider taking
R to be the whole state space f1; 2; 3g. In that case we have P i; :1; 0; :4 = :5:2; 0; :8 for all i 2 R:
So we can take = :2; 0; :8 and for each i 2 R = f1; 2; 3g write P i; as a mixture
P i; = 0:5 + 0:5Qi; ;
where Q1; = 0; 1; 0, Q2; = :6; 0; :4, and Q3; = :4; :4; :2. The way of running
the chain that corresponds to this decomposition of the transition probabilities is as follows.
~
Starting from any state Xt , toss a coin with PfHeadsg = 0:5. If Heads, de ne Xt = ,
~
with Xt = Xt otherwise. Then choose Xt+1 according to the probability mass function if
~ t = and according to Qi; if Xt = i 2 S.
~
X
1.83 Exercise.
Stochastic Processes J. Chang, March 30, 1999 1. MARKOV CHAINS Page 144 a Suppose we have a nitestate Markov chain and we are considering taking our set R
to consist of 2 states R = fi; j g. Express and PfHeadsg" in terms of the ith and
j th rows of the probability transition matrix of the chain. In particular, show that p =
1 , kP i; , P j; k.
b Consider the frog chain. What happens when we try to take R = f1; 2g? 1.10.3 Harris Chains A Markov chain fXt g with transition kernel P is a Harris chain if there is a set R S,
a probability measure on S, and a positive number such that
1 Px fXt 2 R fo...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 Multiplication, Markov Chains

Click to edit the document details