# Lec20 - Stat 150 Stochastic Processes Spring 2009 Lecture...

This preview shows pages 1–3. Sign up to view the full content.

Stat 150 Stochastic Processes Spring 2009 Lecture 20: Markov Chains: Examples Lecturer: Jim Pitman A nice formula : E i (number of hits on j before T i ) = π j π i Domain of truth: P is irreducible. π is an invariant measure: πP = π , π j 0 for all j , and π j > 0 Either π j < or (weaker) P i ( T i < ) = 1 (recurrent). Positive recurrent case: if E i ( T i ) < , then π can be a probability measure. Null recurrent case: if P i ( T i < ) = 1 but E i ( T i ) = , then π cannot be a probability measure. The formula above is a reﬁnement of the formula E i ( T i ) = 1 i for a positive recurrent chain with invariant π with j π j = 1. To see this, observe that N ij : = number of hits on j before T i N ij = X n =0 1 ( X n = j,n < T i ) T i = X n =0 1 ( n < T i ) = X n =0 X j S 1 ( X n = j ) | {z } =1 1 ( n < T i ) = X j S X n =0 1 ( X n = i ) = X j S N ij 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Lecture 20: Markov Chains: Examples 2 Then E i T i = E i X j N ij = X j E i N ij = X j π j π i = 1 π i with X j π j = 1 Strong Markov property : Chain refreshes at each visit to i = numbers of visits to j in successive i-blocks are iid.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 5

Lec20 - Stat 150 Stochastic Processes Spring 2009 Lecture...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online