**preview**has

**blurred**sections. Sign up to view the full version! View Full Document

**Unformatted text preview: **1 Stationary distributions and the limit theorem Definition 1.1. The vector π is called a stationary distribution of the chain if π has entries ( π j : j S ) such that: (a) π j for all j , and j π j = 1 , (b) π = π P , which is to say that π j = i π i p ij for all j (the balance equations ). Note: π P n = π for all n . If X has distribution π then X n has distribution π for all n . 1 Proposition 1.2. An irreducible chain has a stationary distribution π if and only if all the states are non-null persistent; in this case, π is the unique stationary distribution and is given by π i = 1 μ i for each i S , where μ i is the mean recurrence time of i . We will carry out the proof of this in several steps. 2 Fix a state k and let ρ i ( k ) be the mean number of visits of the chain to the state i between two successive visits to state k ; that is ρ i ( k ) = E ( N i | X = k ) where N i = X n =1 1 { X n = i } { T k n } and T k is the time of the first return to state k . We write ρ ( k ) for the vector ( ρ i ( k ) : i S ) . Clearly T k = i S N i , and hence μ k = X i S ρ i ( k ) 3 Lemma 1.3. For any state k of an irreducible persistent chain, the vector ρ ( k ) satisfies ρ i ( k ) < for all i , and furthermore ρ ( k ) = ρ ( k ) P . 4 Proof. We show first that ρ i ( k ) < when i 6 = k . Observe that ρ k ( k ) = 1 . We write l ki ( n ) = P ( X n = i,T k n | X = k ) . Clearly f kk ( m + n ) l ki ( m ) f ik ( n ) . By irreducibility of the chain, there exists n such that f ik ( n ) > . So for n 2 ρ i ( k ) = X m =1 l ki ( m ) 1 f ik ( n ) X m =1 f kk ( m + n ) 1 f ik ( n ) < as required. Next observe that l ki (1) = p ki , and l ki ( n ) = X j : j 6 = k P ( X n = i,X n- 1 = j,T k n | X = k ) = X j : j 6 = k l kj ( n- 1) p ji . 5 Summing over n 2 , we obtain ρ i ( k ) = p ki + X j : j 6 = k X n 2 l kj ( n- 1) p ji = ρ k ( k ) p ki + X j : j 6 = k ρ j ( k ) p ji , since ρ k ( k ) = 1 . 6 For any irreducible chain, the vector ρ ( k ) satisfies ρ ( k ) = ρ ( k ) P , and furthermore that the components of ρ ( k ) are non-negative with sum μ k . Hence, if μ k < , the vector π with entries π i = ρ i ( k ) /μ k satisfies π = π P and furthermore has non-negative entries which sum to 1 ; that is to say, π is a stationary distribution. We have proved that every non-null persistent irreducible chain has a stationary distribution. 7 Proposition 1.4. If the chain is irreducible and persistent, there exists a positive root x of the equation x = xP , which is unique up to a multiplicative constant. The chain is non-null if i x i < and null if i x i = ....

View Full Document