This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Stat 333 Classification of States and Classes in Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j ∈ S Let i ∈ S and j ∈ S be two states. We say j is accessible from i if there exists n ≥ 0 such that P ( n ) ij > 0. We denote this by i → j . We say that i and j communicate if j is accessible from i and i is accessible from j . We denote this by i ↔ j . Theorem 1: Communication is an equivalence relation. That is: • i ↔ i (reflexive) • if i ↔ j then j ↔ i (symmetric) • if i ↔ j and j ↔ k then i ↔ k (transitive) Thus communication divides the state space S into disjoint classes. Each class consists of states which communicate only with each other. A Markov chain consisting of only one class is called irreducible. Thus in an irreducible chain all states communicate with each other. A class C is called closed if it is not possible to leave the class. i.e., P ij = 0 for every i ∈ C and j / ∈ C . A class C is called open if leaving it is possible. i.e., there exists some i ∈ C and...
View Full Document
- Winter '08
- Markov Chains, Markov chain