This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: IEOR 4106: Professor Whitt Lecture Notes, Thursday, February 5, 2009 More on Markov Chains 1. Classification of States See Section 4.3. Concepts: 1. State j is accessible from state i if it is possible to get to j from i in some finite number of steps. (notation: i ˆ j ) 2. States i and j communicate if both j is accessible from i and i is accessible from j . (notation: i ! j ) 3. A subset A of states in the Markov chain is a communication class if every pair of states in the subset communicate. 4. A communication class A of states in the Markov chain is closed if no state outside the class is accessible from a state in the class. 5. A communication class A of states in the Markov chain is open if it is not closed; i.e., if it is possible for the Markov chain to leave that communicating class. 6. A Markov chain is irreducible if the entire chain is a single communicating class. 7. A Markov chain is reducible if there are two or more communication classes in the chain; i.e., if it is not irreducible. 8. A Markov chain transition matrix P is in canonical form if the states are re-labelled (re-ordered) so that the states within closed communication classes appear together first, and...
View Full Document
- Spring '08
- Markov Chains, Markov chain