1 Stationary distributions and the limit theorem Definition 1.1. The vector is called a stationary distribution of the chain if has entries ( j : j S ) such that: (a) j for all j , and j j = 1 , (b) = P , which is to say that j = i i p ij for all j (the balance equations ). Note: P n = for all n . If X has distribution then X n has distribution for all n . 1 Proposition 1.2. An irreducible chain has a stationary distribution if and only if all the states are non-null persistent; in this case, is the unique stationary distribution and is given by i = 1 i for each i S , where i is the mean recurrence time of i . We will carry out the proof of this in several steps. 2 Fix a state k and let i ( k ) be the mean number of visits of the chain to the state i between two successive visits to state k ; that is i ( k ) = E ( N i | X = k ) where N i = X n =1 1 { X n = i } { T k n } and T k is the time of the first return to state k . We write ( k ) for the vector ( i ( k ) : i S ) . Clearly T k = i S N i , and hence k = X i S i ( k ) 3 Lemma 1.3. For any state k of an irreducible persistent chain, the vector ( k ) satisfies i ( k ) < for all i , and furthermore ( k ) = ( k ) P . 4 Proof. We show first that i ( k ) < when i 6 = k . Observe that k ( k ) = 1 . We write l ki ( n ) = P ( X n = i,T k n | X = k ) . Clearly f kk ( m + n ) l ki ( m ) f ik ( n ) . By irreducibility of the chain, there exists n such that f ik ( n ) > . So for n 2 i ( k ) = X m =1 l ki ( m ) 1 f ik ( n ) X m =1 f kk ( m + n ) 1 f ik ( n ) < as required. Next observe that l ki (1) = p ki , and l ki ( n ) = X j : j 6 = k P ( X n = i,X n- 1 = j,T k n | X = k ) = X j : j 6 = k l kj ( n- 1) p ji . 5 Summing over n 2 , we obtain i ( k ) = p ki + X j : j 6 = k X n 2 l kj ( n- 1) p ji = k ( k ) p ki + X j : j 6 = k j ( k ) p ji , since k ( k ) = 1 . 6 For any irreducible chain, the vector ( k ) satisfies ( k ) = ( k ) P , and furthermore that the components of ( k ) are non-negative with sum k . Hence, if k < , the vector with entries i = i ( k ) / k satisfies = P and furthermore has non-negative entries which sum to 1 ; that is to say, is a stationary distribution. We have proved that every non-null persistent irreducible chain has a stationary distribution. 7 Proposition 1.4. If the chain is irreducible and persistent, there exists a positive root x of the equation x = xP , which is unique up to a multiplicative constant. The chain is non-null if i x i < and null if i x i = ....
View Full Document