This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Stat 333 Classification of States and Classes in Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j S Let i S and j S be two states. We say j is accessible from i if there exists n 0 such that P ( n ) ij > 0. We denote this by i j . We say that i and j communicate if j is accessible from i and i is accessible from j . We denote this by i j . Theorem 1: Communication is an equivalence relation. That is: i i (reflexive) if i j then j i (symmetric) if i j and j k then i k (transitive) Thus communication divides the state space S into disjoint classes. Each class consists of states which communicate only with each other. A Markov chain consisting of only one class is called irreducible. Thus in an irreducible chain all states communicate with each other. A class C is called closed if it is not possible to leave the class. i.e., P ij = 0 for every i C and j / C . A class C is called open if leaving it is possible. i.e., there exists some i C and...
View Full Document
- Winter '08
- Markov Chains