1 ie we will return to 3 with probability 1 the last

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: of hitting 3 on the next step so P3 (T3 > n) (0.9)n ! 0 as n ! 1 i.e., we will return to 3 with probability 1. The last argument applies even more strongly to states 1 and 2, since the probability of jumping to them on the next step is always at least 0.2. Thus all three states are recurrent. The last argument generalizes to the give the following useful fact. Lemma 1.3. Suppose Px (Ty k ) ↵ > 0 for all x in the state space S . Then Px (Ty > nk ) (1 ↵)n Generalizing from our experience with the last two examples, we will introduce some general results that will help us identify transient and recurrent states. Definition 1.1. We say that x communicates with y and write x ! y if there is a positive probability of reaching y starting from x, that is, the probability ⇢xy = Px (Ty < 1) > 0 Note that the last probability includes not only the possibility of jumping from x to y in one step but also going from x to y after visiting several other states in between. The following property is simple but useful. Here and in what...
View Full Document

Ask a homework question - tutors are online