lect0205 - IEOR 4106 Professor Whitt Lecture Notes Thursday...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IEOR 4106: Professor Whitt Lecture Notes, Thursday, February 5, 2009 More on Markov Chains 1. Classification of States See Section 4.3. Concepts: 1. State j is accessible from state i if it is possible to get to j from i in some finite number of steps. (notation: i ˆ j ) 2. States i and j communicate if both j is accessible from i and i is accessible from j . (notation: i ! j ) 3. A subset A of states in the Markov chain is a communication class if every pair of states in the subset communicate. 4. A communication class A of states in the Markov chain is closed if no state outside the class is accessible from a state in the class. 5. A communication class A of states in the Markov chain is open if it is not closed; i.e., if it is possible for the Markov chain to leave that communicating class. 6. A Markov chain is irreducible if the entire chain is a single communicating class. 7. A Markov chain is reducible if there are two or more communication classes in the chain; i.e., if it is not irreducible. 8. A Markov chain transition matrix P is in canonical form if the states are re-labelled (re-ordered) so that the states within closed communication classes appear together first, and...
View Full Document

Page1 / 3

lect0205 - IEOR 4106 Professor Whitt Lecture Notes Thursday...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online