Unformatted text preview: fXt+1 2 Ag 1. MARKOV CHAINS I fXt 2 Ag : That is, P x; Adx is the expected number of visits made by the segment X1 ; : : : ; XT
to the set A. Is this the same as A, which is the expected number of visits made
by the segment X0 ; : : : ; XT ,1 to the set A? The answer is yes! In fact, since X0 =
XT = , the two segments X1 ; : : : ; XT = X1 ; : : : ; XT ,1 ; and X0 ; : : : ; XT ,1 =
; X1 ; : : : ; XT ,1 consist of precisely the same states, just visited in a di erent order. Of
course the mere di erence in ordering leaves the number of visits to the set A unchanged
between the two segments.
1.89 Proposition. Suppose a Markov chain has an accessible atom distribution . Then f g = 1=E T . and a stationary Proof: By the same proof as the SLLN P
before, using the cycle idea, we know that if the chain is started in the state , then 1=n n=1 I fXt = g ! 1=E T with probability 1.
Combining this with Proposition 1.76, here is what we know. If the chain is started out
in the distribution , then with probability 1 it hits at some nite time, after which, with
probability 1, the long run fraction of visits to converges to 1=E T . We have used this
type of reasoning before: the nite amount of time it takes the chain to hit does not have
any e ect on the limiting long-run fraction of time the chain spends in the state . Thus,
for a chain started in the distribution , P lim 1=n n!1 n
t=1 I fXt = g = 1=E T = 1: By the Bounded Convergence Theorem, E as n ! 1. But for each n, E 1=n n
t=1 1=n n
t=1 I fXt = g ! 1=E T I fXt = g = 1=n n
t=1 P fXt = g = f g: This, f g = 1=E T .
*** ALTERNATIVELY, do it this way.....
1.90 Theorem. distribution . Then
Stochastic Processes Suppose the chain fXt g has an accessible atom and a stationary J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS
View Full Document