5 and 17 is su cient to classify the states in any

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 5, 4, 6, and 7 are recurrent, completing our study of the Example 1.14 with the results we had claimed earlier. In fact, the combination of Theorem 1.5 and 1.7 is su cient to classify the states in any finite state Markov chain. An algorithm will be explained in the proof of the following result. Theorem 1.8. If the state space S is finite, then S can be written as a disjoint union T [ R1 [ · · · [ Rk , where T is a set of transient states and the Ri , 1 i k , are closed irreducible sets of recurrent states. Proof. Let T be the set of x for which there is a y so that x ! y but y 6! x. The states in T are transient by Theorem 1.5. Our next step is to show that all the remaining states, S T , are recurrent. Pick an x 2 S T and let Cx = {y : x ! y }. Since x 62 T it has the property if x ! y , then y ! x. To check that Cx is closed note that if y 2 Cx and y ! z , then Lemma 1.4 implies x ! z so z 2 Cx . To check irreducibility, note that if y, z 2 Cx , then by our first observation y ! x a...
View Full Document

Ask a homework question - tutors are online