{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Markov

# In fact a bit of thought shows that ni the total

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: t is, 1 X Ni = I fXn = ig: n=0 1.36 Theorem. The state i is recurrent if and only if E i Ni  = 1. Proof: We already know that if i is recurrent, then Pi fNi = 1g = 1; that is, starting from i, the chain visits i in nitely many times with probability one. But of course the last display implies that E i Ni  = 1. To prove the converse, suppose that i is transient, so that q := Pi fTi = 1g 0. Considering the sample path of the Markov chain as a succession of cycles" as in the proof of Theorem 1.35, we see that each cycle has probability q of never ending, so that there are no more cycles, and no more visits to i. In fact, a bit of thought shows that Ni , the total number of visits to i including the visit at time 0 , has a geometric distribution with success probability" q, and hence expected value 1=q, which is nite, since q 0. 1.37 Corollary. If j is transient, then limn!1 P n i; j  = 0 for all states i. Proof: Supposing j is transient, we know that E j Nj  state i 6= j , we have Stochastic Processes E i Nj  = Pi 1. Starting from an arbitrary fTj 1gE i Nj j Tj 1: J. Chang, March 30, 1999 1.6. IRREDUCIBILITY, PERIODICITY, AND RECURRENCE Page 1-19 However, E i Nj j Tj 1 = E j Nj ; this is clear intuitively since, starting from i, if the Markov chain hits j at the nite time Tj , then it probabilistically restarts" at time Tj . Exercise:P a formal argument. Thus, E i Nj   E j Nj  1, so that in fact we have give E i Nj  = 1 P n i; j  1, which implies the conclusion of the Corollary. n=1 1.38 Example A drunk man will find his way home, but a drunk bird may get lost forever," or, recurrence and transience of random walks . The quotation is from Yale's own professor Kakutani, as told by R. Durrett in his probability book. We'll consider a certain model of a random walk in d dimensions, and show that the walk is recurrent if d = 1 or d = 2, and the walk is transient if d  3. In one dimension, our random walk is the simple, symmetric" random walk on the integers, which takes steps of +1 and ,1 with probability 1 2 each. That is, letting X1 ; X2 ; : : : be iid taking the values 1 with probability 1 2, we de ne the position of the random walk at time n to be Sn = X1 +    + Xn . What is a random walk in d dimensions? Here is what we will take it to be: the position of such a random walk at time n is Sn = Sn 1; : : : ; Snd 2 Zd; where the coordinates Sn 1; : : : ; Sn d are independent simple, symmetric random walks in Z. That is, to form a random walk in Zd, simply concatenate d independent one-dimensional random walks into a d-dimensional vector process. Thus, our random walk Sn may be written as Sn = X1 +    + Xn , where X1 ; X2 ; : : : are iid taking on the 2d values 1; : : : ; 1 with probability 2,d each. This might not be the rst model that would come to your mind. Another natural model would be to have the random walk take a step by choosing one of the d coordinate directions at random probability 1=d each and then taking a step of +...
View Full Document

{[ snackBarMessage ]}