This preview shows pages 1–9. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 1 Stationary distributions and the limit theorem Definition 1.1. The vector is called a stationary distribution of the chain if has entries ( j : j S ) such that: (a) j for all j , and j j = 1 , (b) = P , which is to say that j = i i p ij for all j (the balance equations ). Note: P n = for all n . If X has distribution then X n has distribution for all n . 1 Proposition 1.2. An irreducible chain has a stationary distribution if and only if all the states are nonnull persistent; in this case, is the unique stationary distribution and is given by i = 1 i for each i S , where i is the mean recurrence time of i . We will carry out the proof of this in several steps. 2 Fix a state k and let i ( k ) be the mean number of visits of the chain to the state i between two successive visits to state k ; that is i ( k ) = E ( N i  X = k ) where N i = X n =1 1 { X n = i } { T k n } and T k is the time of the first return to state k . We write ( k ) for the vector ( i ( k ) : i S ) . Clearly T k = i S N i , and hence k = X i S i ( k ) 3 Lemma 1.3. For any state k of an irreducible persistent chain, the vector ( k ) satisfies i ( k ) < for all i , and furthermore ( k ) = ( k ) P . 4 Proof. We show first that i ( k ) < when i 6 = k . Observe that k ( k ) = 1 . We write l ki ( n ) = P ( X n = i,T k n  X = k ) . Clearly f kk ( m + n ) l ki ( m ) f ik ( n ) . By irreducibility of the chain, there exists n such that f ik ( n ) > . So for n 2 i ( k ) = X m =1 l ki ( m ) 1 f ik ( n ) X m =1 f kk ( m + n ) 1 f ik ( n ) < as required. Next observe that l ki (1) = p ki , and l ki ( n ) = X j : j 6 = k P ( X n = i,X n 1 = j,T k n  X = k ) = X j : j 6 = k l kj ( n 1) p ji . 5 Summing over n 2 , we obtain i ( k ) = p ki + X j : j 6 = k X n 2 l kj ( n 1) p ji = k ( k ) p ki + X j : j 6 = k j ( k ) p ji , since k ( k ) = 1 . 6 For any irreducible chain, the vector ( k ) satisfies ( k ) = ( k ) P , and furthermore that the components of ( k ) are nonnegative with sum k . Hence, if k < , the vector with entries i = i ( k ) / k satisfies = P and furthermore has nonnegative entries which sum to 1 ; that is to say, is a stationary distribution. We have proved that every nonnull persistent irreducible chain has a stationary distribution. 7 Proposition 1.4. If the chain is irreducible and persistent, there exists a positive root x of the equation x = xP , which is unique up to a multiplicative constant. The chain is nonnull if i x i < and null if i x i = ....
View Full Document
 Spring '08
 Evans

Click to edit the document details