Stochastic

# 1 ie we will return to 3 with probability 1 the last

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: of hitting 3 on the next step so P3 (T3 > n) (0.9)n ! 0 as n ! 1 i.e., we will return to 3 with probability 1. The last argument applies even more strongly to states 1 and 2, since the probability of jumping to them on the next step is always at least 0.2. Thus all three states are recurrent. The last argument generalizes to the give the following useful fact. Lemma 1.3. Suppose Px (Ty k ) ↵ > 0 for all x in the state space S . Then Px (Ty > nk ) (1 ↵)n Generalizing from our experience with the last two examples, we will introduce some general results that will help us identify transient and recurrent states. Deﬁnition 1.1. We say that x communicates with y and write x ! y if there is a positive probability of reaching y starting from x, that is, the probability ⇢xy = Px (Ty < 1) > 0 Note that the last probability includes not only the possibility of jumping from x to y in one step but also going from x to y after visiting several other states in between. The following property is simple but useful. Here and in what...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online