This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: IEOR 4106: Professor Whitt Lecture Notes, Thursday, February 5, 2009 More on Markov Chains 1. Classification of States See Section 4.3. Concepts: 1. State j is accessible from state i if it is possible to get to j from i in some finite number of steps. (notation: i j ) 2. States i and j communicate if both j is accessible from i and i is accessible from j . (notation: i ! j ) 3. A subset A of states in the Markov chain is a communication class if every pair of states in the subset communicate. 4. A communication class A of states in the Markov chain is closed if no state outside the class is accessible from a state in the class. 5. A communication class A of states in the Markov chain is open if it is not closed; i.e., if it is possible for the Markov chain to leave that communicating class. 6. A Markov chain is irreducible if the entire chain is a single communicating class. 7. A Markov chain is reducible if there are two or more communication classes in the chain; i.e., if it is not irreducible. 8. A Markov chain transition matrix P is in canonical form if the states are relabelled (reordered) so that the states within closed communication classes appear together first, and...
View
Full
Document
 Spring '08
 Whitt

Click to edit the document details