This preview shows page 1. Sign up to view the full content.
Unformatted text preview: t
is,
1
X
Ni = I fXn = ig:
n=0 1.36 Theorem. The state i is recurrent if and only if E i Ni = 1.
Proof: We already know that if i is recurrent, then
Pi fNi = 1g = 1; that is, starting from i, the chain visits i in nitely many times with probability one. But
of course the last display implies that E i Ni = 1. To prove the converse, suppose that
i is transient, so that q := Pi fTi = 1g 0. Considering the sample path of the Markov
chain as a succession of cycles" as in the proof of Theorem 1.35, we see that each cycle
has probability q of never ending, so that there are no more cycles, and no more visits to i.
In fact, a bit of thought shows that Ni , the total number of visits to i including the visit
at time 0 , has a geometric distribution with success probability" q, and hence expected
value 1=q, which is nite, since q 0.
1.37 Corollary. If j is transient, then limn!1 P n i; j = 0 for all states i.
Proof: Supposing j is transient, we know that E j Nj state i 6= j , we have
Stochastic Processes E i Nj = Pi 1. Starting from an arbitrary fTj 1gE i Nj j Tj 1:
J. Chang, March 30, 1999 1.6. IRREDUCIBILITY, PERIODICITY, AND RECURRENCE Page 119 However, E i Nj j Tj 1 = E j Nj ; this is clear intuitively since, starting from i, if the
Markov chain hits j at the nite time Tj , then it probabilistically restarts" at time Tj .
Exercise:P a formal argument. Thus, E i Nj E j Nj 1, so that in fact we have
give
E i Nj = 1 P n i; j 1, which implies the conclusion of the Corollary.
n=1
1.38 Example A drunk man will find his way home, but a drunk bird may
get lost forever," or, recurrence and transience of random walks . The quotation is from Yale's own professor Kakutani, as told by R. Durrett in his probability
book. We'll consider a certain model of a random walk in d dimensions, and show that the
walk is recurrent if d = 1 or d = 2, and the walk is transient if d 3.
In one dimension, our random walk is the simple, symmetric" random walk on the integers, which takes steps of +1 and ,1 with probability 1 2 each. That is, letting X1 ; X2 ; : : :
be iid taking the values 1 with probability 1 2, we de ne the position of the random walk
at time n to be Sn = X1 + + Xn . What is a random walk in d dimensions? Here is what
we will take it to be: the position of such a random walk at time n is
Sn = Sn 1; : : : ; Snd 2 Zd;
where the coordinates Sn 1; : : : ; Sn d are independent simple, symmetric random walks in
Z. That is, to form a random walk in Zd, simply concatenate d independent onedimensional
random walks into a ddimensional vector process.
Thus, our random walk Sn may be written as Sn = X1 + + Xn , where X1 ; X2 ; : : :
are iid taking on the 2d values 1; : : : ; 1 with probability 2,d each. This might not be
the rst model that would come to your mind. Another natural model would be to have
the random walk take a step by choosing one of the d coordinate directions at random
probability 1=d each and then taking a step of +...
View
Full
Document
 Spring '10
 DURRETT
 Multiplication, Markov Chains

Click to edit the document details