# And if r is hit with positive probability then so is

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ass function :5; 0; :5. Similarly, if Xt = 3, we use the mass ~ function 0; 1; 0. Finally, if Xt = , we know that Xt was either 2 or 3 and C = Heads, so we use the mass function :375; 0; :625. Again we have decomposed each transition of the given chain, according to P , into 2 stages, as depicted in Figure 1.81. These stages make transitions according to the matrices U and V , given by 01 2 31 01 2 3 0 1 1 :1 :5 :4 1100 B5 5C U = 2 @ 0 :2 0 :8 A; V = 2 B :0 0 :0 C: A 3@ 1 3 0 0 :2 :8 :375 0 :625 We started with a set of states R = f2; 3g. For each i 2 R, we then wrote P i;  as a mixture of some xed probability mass function = :375; 0; :625 with some other probability mass function Qi;  in our example Q2;  = :5; 0; :5 and Q3;  = 0; 1; 0 . P i;  = 0:8 + 0:2Qi; . We have broken down each transition of the chain into two stages. Starting from the state Xt in the rst stage we note whether or not Xt is in the set R, and if so we toss the biased coin. If the coin toss comes up Heads, we move to state , and otherwise we stay ~ where we are; the result is the state we have called Xt . Then we draw the next state Xt+1 from the appropriate distribution. The point is that we have introduced a new state that we can reach by hitting any state in the set R and then getting a Heads from the coin toss. This is the key in general state spaces: if we can take the set R to be large enough, the set R will have positive probability of being hit, even though each individual state in R may have probability 0 of being hit. And if R is hit with positive probability, then so is , since hitting only requires hitting R and a Heads from the coin toss. Note also that we could have chosen R in di erent ways. For example, consider taking R to be the whole state space f1; 2; 3g. In that case we have P i;   :1; 0; :4 = :5:2; 0; :8 for all i 2 R: So we can take = :2; 0; :8 and for each i 2 R = f1; 2; 3g write P i;  as a mixture P i;  = 0:5 + 0:5Qi; ; where Q1;  = 0; 1; 0, Q2;  = :6; 0; :4, and Q3;  = :4; :4; :2. The way of running the chain that corresponds to this decomposition of the transition probabilities is as follows. ~ Starting from any state Xt , toss a coin with PfHeadsg = 0:5. If Heads, de ne Xt = , ~ with Xt = Xt otherwise. Then choose Xt+1 according to the probability mass function if ~ t = and according to Qi;  if Xt = i 2 S. ~ X 1.83 Exercise. Stochastic Processes J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-44 a Suppose we have a nite-state Markov chain and we are considering taking our set R to consist of 2 states R = fi; j g. Express and PfHeadsg&quot; in terms of the ith and j th rows of the probability transition matrix of the chain. In particular, show that p = 1 , kP i;  , P j; k. b Consider the frog chain. What happens when we try to take R = f1; 2g? 1.10.3 Harris Chains A Markov chain fXt g with transition kernel P is a Harris chain if there is a set R S, a probability measure on S, and a positive number such that 1 Px fXt 2 R fo...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online