{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Stat150_Spring08_Markov_classification

Stat150_Spring08_Markov_classification - Statistics 150...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Statistics 150: Spring 2007 February 3, 2007 0-0
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
1 Classification of states Definition State i is called persistent (or recurrent ) if P ( X n = i for some n 1 | X 0 = i ) = 1 , which is to say that the probability of eventual return to i , having started from i , is 1. If this probability is strictly less than 1 , the state is called transient . 1
Background image of page 2
Let f ij ( n ) = P ( X 1 = j, X 2 = j, . . . , X n - 1 = j, X n = j | X 0 = i ) be the probability that the first visit to state j , starting from i , takes place at the n th step. Define f ij = n =1 f ij ( n ) to be the probability that the chain ever visits j , starting from i . Of course, j is persistent if and only if f ij = 1 . 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
We seek a criterion for persistence in term of the n -step transition probabilities. We define the generating functions P ij = n =0 s n p ij ( n ) , F ij ( s ) = n =0 s n f ij ( n ) with the convention that p ij (0) = δ ij , and f ij (0) = 0 for all i and j . Clearly f ij = F ij (1) . 3
Background image of page 4
Proposition 1.1. 1. P ii ( s ) = 1 + F ii ( s ) P ii ( s ) ; 2. P ij ( s ) = F ij ( s ) P jj ( s ) if i = j . Proof. We have p ij ( m ) = m r =1 f ij ( r ) p jj ( m - r ) , m = 1 , 2 , . . . . Multiply throughout by s m , where | s | < 1 , and sum over m ( 1) to find that P ij ( s ) - δ ij = F ij ( s ) P ij ( s ) as required. 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Corollary 1.2. 1. State j is persistent if n p jj ( n ) = , and if this holds then n p ij ( n ) = for all i such that f ij > 0 .
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}