F g 0 2 is positive recurrent 3 for all a 2 a 4 for aa

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: fXt+1 2 Ag 1. MARKOV CHAINS   I fXt 2 Ag : That is, P x; Adx is the expected number of visits made by the segment X1 ; : : : ; XT  to the set A. Is this the same as A, which is the expected number of visits made by the segment X0 ; : : : ; XT ,1  to the set A? The answer is yes! In fact, since X0 = XT = , the two segments X1 ; : : : ; XT  = X1 ; : : : ; XT ,1 ;  and X0 ; : : : ; XT ,1  =  ; X1 ; : : : ; XT ,1  consist of precisely the same states, just visited in a di erent order. Of course the mere di erence in ordering leaves the number of visits to the set A unchanged between the two segments. 1.89 Proposition. Suppose a Markov chain has an accessible atom distribution . Then f g = 1=E T . and a stationary Proof: By the same proof as the SLLN P before, using the cycle idea, we know that if the chain is started in the state , then 1=n n=1 I fXt = g ! 1=E T  with probability 1. t Combining this with Proposition 1.76, here is what we know. If the chain is started out in the distribution , then with probability 1 it hits at some nite time, after which, with probability 1, the long run fraction of visits to converges to 1=E T . We have used this type of reasoning before: the nite amount of time it takes the chain to hit does not have any e ect on the limiting long-run fraction of time the chain spends in the state . Thus, for a chain started in the distribution ,  P lim 1=n n!1 n X t=1  I fXt = g = 1=E T  = 1: By the Bounded Convergence Theorem,  E as n ! 1. But for each n,  E 1=n n X t=1 1=n n X t=1  I fXt = g ! 1=E T   I fXt = g = 1=n n X t=1 P fXt = g = f g: This, f g = 1=E T . *** ALTERNATIVELY, do it this way..... 1.90 Theorem. distribution . Then Stochastic Processes Suppose the chain fXt g has an accessible atom and a stationary J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS 1. f...
View Full Document

Ask a homework question - tutors are online