This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Stat 333 Classification of States and Classes in Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j ∈ S Let i ∈ S and j ∈ S be two states. We say j is accessible from i if there exists n ≥ 0 such that P ( n ) ij > 0. We denote this by i → j . We say that i and j communicate if j is accessible from i and i is accessible from j . We denote this by i ↔ j . Theorem 1: Communication is an equivalence relation. That is: • i ↔ i (reflexive) • if i ↔ j then j ↔ i (symmetric) • if i ↔ j and j ↔ k then i ↔ k (transitive) Thus communication divides the state space S into disjoint classes. Each class consists of states which communicate only with each other. A Markov chain consisting of only one class is called irreducible. Thus in an irreducible chain all states communicate with each other. A class C is called closed if it is not possible to leave the class. i.e., P ij = 0 for every i ∈ C and j / ∈ C . A class C is called open if leaving it is possible. i.e., there exists someif leaving it is possible....
View
Full Document
 Spring '08
 Chisholm
 Markov Chains, Markov chain

Click to edit the document details