Lec20 - Stat 150 Stochastic Processes Spring 2009 Lecture...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Stat 150 Stochastic Processes Spring 2009 Lecture 20: Markov Chains: Examples Lecturer: Jim Pitman A nice formula : E i (number of hits on j before T i ) = π j π i Domain of truth: P is irreducible. π is an invariant measure: πP = π , π j 0 for all j , and π j > 0 Either π j < or (weaker) P i ( T i < ) = 1 (recurrent). Positive recurrent case: if E i ( T i ) < , then π can be a probability measure. Null recurrent case: if P i ( T i < ) = 1 but E i ( T i ) = , then π cannot be a probability measure. The formula above is a refinement of the formula E i ( T i ) = 1 i for a positive recurrent chain with invariant π with j π j = 1. To see this, observe that N ij : = number of hits on j before T i N ij = X n =0 1 ( X n = j,n < T i ) T i = X n =0 1 ( n < T i ) = X n =0 X j S 1 ( X n = j ) | {z } =1 1 ( n < T i ) = X j S X n =0 1 ( X n = i ) = X j S N ij 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 20: Markov Chains: Examples 2 Then E i T i = E i X j N ij = X j E i N ij = X j π j π i = 1 π i with X j π j = 1 Strong Markov property : Chain refreshes at each visit to i = numbers of visits to j in successive i-blocks are iid.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 5

Lec20 - Stat 150 Stochastic Processes Spring 2009 Lecture...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online