{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Stat150_Spring08_Markov_classification

Stat150_Spring08_Markov_classification - Statistics 150...

This preview shows pages 1–7. Sign up to view the full content.

Statistics 150: Spring 2007 February 3, 2007 0-0

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1 Classification of states Definition State i is called persistent (or recurrent ) if P ( X n = i for some n 1 | X 0 = i ) = 1 , which is to say that the probability of eventual return to i , having started from i , is 1. If this probability is strictly less than 1 , the state is called transient . 1
Let f ij ( n ) = P ( X 1 = j, X 2 = j, . . . , X n - 1 = j, X n = j | X 0 = i ) be the probability that the first visit to state j , starting from i , takes place at the n th step. Define f ij = n =1 f ij ( n ) to be the probability that the chain ever visits j , starting from i . Of course, j is persistent if and only if f ij = 1 . 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
We seek a criterion for persistence in term of the n -step transition probabilities. We define the generating functions P ij = n =0 s n p ij ( n ) , F ij ( s ) = n =0 s n f ij ( n ) with the convention that p ij (0) = δ ij , and f ij (0) = 0 for all i and j . Clearly f ij = F ij (1) . 3
Proposition 1.1. 1. P ii ( s ) = 1 + F ii ( s ) P ii ( s ) ; 2. P ij ( s ) = F ij ( s ) P jj ( s ) if i = j . Proof. We have p ij ( m ) = m r =1 f ij ( r ) p jj ( m - r ) , m = 1 , 2 , . . . . Multiply throughout by s m , where | s | < 1 , and sum over m ( 1) to find that P ij ( s ) - δ ij = F ij ( s ) P ij ( s ) as required. 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Corollary 1.2. 1. State j is persistent if n p jj ( n ) = , and if this holds then n p ij ( n ) = for all i such that f ij > 0 .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}