Stochastic

# 15 13 classification of states a set a is closed if it

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: n’t communicate with it, so 3 is transient. To conclude that all the remaining states are recurrent we will introduce two deﬁnitions and a fact. 15 1.3. CLASSIFICATION OF STATES A set A is closed if it is impossible to get out, i.e., if i 2 A and j 62 A then p(i, j ) = 0. In Example 1.14, {1, 5} and {4, 6, 7} are closed sets. Their union, {1, 4, 5, 6, 7} is also closed. One can add 3 to get another closed set {1, 3, 4, 5, 6, 7}. Finally, the whole state space {1, 2, 3, 4, 5, 6, 7} is always a closed set. Among the closed sets in the last example, some are obviously too big. To rule them out, we need a deﬁnition. A set B is called irreducible if whenever i, j 2 B , i communicates with j . The irreducible closed sets in the Example 1.14 are {1, 5} and {4, 6, 7}. The next result explains our interest in irreducible closed sets. Theorem 1.7. If C is a ﬁnite closed and irreducible set, then all states in C are recurrent. Before entering into an explanation of this result, we note that Theorem 1.7 tells us that 1,...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online