Stat150_Spring08_Markov_convergence - 1 Stationary...

Stat150_Spring08_Markov_convergence
Download Document
Showing pages : 1 - 9 of 55
This preview has blurred sections. Sign up to view the full version! View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1 Stationary distributions and the limit theorem Definition 1.1. The vector π is called a stationary distribution of the chain if π has entries ( π j : j S ) such that: (a) π j for all j , and j π j = 1 , (b) π = π P , which is to say that π j = i π i p ij for all j (the balance equations ). Note: π P n = π for all n . If X has distribution π then X n has distribution π for all n . 1 Proposition 1.2. An irreducible chain has a stationary distribution π if and only if all the states are non-null persistent; in this case, π is the unique stationary distribution and is given by π i = 1 μ i for each i S , where μ i is the mean recurrence time of i . We will carry out the proof of this in several steps. 2 Fix a state k and let ρ i ( k ) be the mean number of visits of the chain to the state i between two successive visits to state k ; that is ρ i ( k ) = E ( N i | X = k ) where N i = X n =1 1 { X n = i } { T k n } and T k is the time of the first return to state k . We write ρ ( k ) for the vector ( ρ i ( k ) : i S ) . Clearly T k = i S N i , and hence μ k = X i S ρ i ( k ) 3 Lemma 1.3. For any state k of an irreducible persistent chain, the vector ρ ( k ) satisfies ρ i ( k ) < for all i , and furthermore ρ ( k ) = ρ ( k ) P . 4 Proof. We show first that ρ i ( k ) < when i 6 = k . Observe that ρ k ( k ) = 1 . We write l ki ( n ) = P ( X n = i,T k n | X = k ) . Clearly f kk ( m + n ) l ki ( m ) f ik ( n ) . By irreducibility of the chain, there exists n such that f ik ( n ) > . So for n 2 ρ i ( k ) = X m =1 l ki ( m ) 1 f ik ( n ) X m =1 f kk ( m + n ) 1 f ik ( n ) < as required. Next observe that l ki (1) = p ki , and l ki ( n ) = X j : j 6 = k P ( X n = i,X n- 1 = j,T k n | X = k ) = X j : j 6 = k l kj ( n- 1) p ji . 5 Summing over n 2 , we obtain ρ i ( k ) = p ki + X j : j 6 = k X n 2 l kj ( n- 1) p ji = ρ k ( k ) p ki + X j : j 6 = k ρ j ( k ) p ji , since ρ k ( k ) = 1 . 6 For any irreducible chain, the vector ρ ( k ) satisfies ρ ( k ) = ρ ( k ) P , and furthermore that the components of ρ ( k ) are non-negative with sum μ k . Hence, if μ k < , the vector π with entries π i = ρ i ( k ) /μ k satisfies π = π P and furthermore has non-negative entries which sum to 1 ; that is to say, π is a stationary distribution. We have proved that every non-null persistent irreducible chain has a stationary distribution. 7 Proposition 1.4. If the chain is irreducible and persistent, there exists a positive root x of the equation x = xP , which is unique up to a multiplicative constant. The chain is non-null if i x i < and null if i x i = ....
View Full Document