Unformatted text preview: lassiﬁcation of States We begin with some important notation. We are often interested in the behavior
of the chain for a ﬁxed initial state, so we will introduce the shorthand
Px (A) = P (AX0 = x)
Later we will have to consider expected values for this probability and we will
denote them by Ex .
Let Ty = min{n 1 : Xn = y } be the time of the ﬁrst return to y (i.e.,
being there at time 0 doesn’t count), and let
⇢yy = Py (Ty < 1)
be the probability Xn returns to y when it starts at y . Note that if we didn’t
exclude n = 0 this probability would always be 1.
Intuitively, the Markov property implies that the probability Xn will return
to y at least twice is ⇢2 , since after the ﬁrst return, the chain is at y , and the
yy
probability of a second return following the ﬁrst is again ⇢yy .
To show that the reasoning in the last paragraph is valid, we have to introduce a deﬁnition and state a theorem. We say that T is a stopping time if
the occurrence (or nonoccurrence) of the event “we stop at time n,” {T = n},
can be determined by looking at the values o...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details