This preview shows page 1. Sign up to view the full content.
Unformatted text preview: of hitting 3 on the next step so
P3 (T3 > n) (0.9)n ! 0 as n ! 1
i.e., we will return to 3 with probability 1. The last argument applies even more
strongly to states 1 and 2, since the probability of jumping to them on the next
step is always at least 0.2. Thus all three states are recurrent.
The last argument generalizes to the give the following useful fact.
Lemma 1.3. Suppose Px (Ty k ) ↵ > 0 for all x in the state space S . Then Px (Ty > nk ) (1 ↵)n Generalizing from our experience with the last two examples, we will introduce some general results that will help us identify transient and recurrent
Deﬁnition 1.1. We say that x communicates with y and write x ! y if
there is a positive probability of reaching y starting from x, that is, the probability
⇢xy = Px (Ty < 1) > 0
Note that the last probability includes not only the possibility of jumping from
x to y in one step but also going from x to y after visiting several other states in
between. The following property is simple but useful. Here and in what...
View Full Document
- Spring '10
- The Land