Stochastic

15 13 classification of states a set a is closed if it

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: n’t communicate with it, so 3 is transient. To conclude that all the remaining states are recurrent we will introduce two definitions and a fact. 15 1.3. CLASSIFICATION OF STATES A set A is closed if it is impossible to get out, i.e., if i 2 A and j 62 A then p(i, j ) = 0. In Example 1.14, {1, 5} and {4, 6, 7} are closed sets. Their union, {1, 4, 5, 6, 7} is also closed. One can add 3 to get another closed set {1, 3, 4, 5, 6, 7}. Finally, the whole state space {1, 2, 3, 4, 5, 6, 7} is always a closed set. Among the closed sets in the last example, some are obviously too big. To rule them out, we need a definition. A set B is called irreducible if whenever i, j 2 B , i communicates with j . The irreducible closed sets in the Example 1.14 are {1, 5} and {4, 6, 7}. The next result explains our interest in irreducible closed sets. Theorem 1.7. If C is a finite closed and irreducible set, then all states in C are recurrent. Before entering into an explanation of this result, we note that Theorem 1.7 tells us that 1,...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online