Stat150_Spring08_Markov_cts_stationary

Stat150_Spring08_Markov_cts_stationary - Statistics 150...

This preview shows pages 1–7. Sign up to view the full content.

Statistics 150: Spring 2007 March 25, 2007 0-1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities p j = lim t →∞ P ij ( t ) are given by p j = π j j i π i i where the π j are the unique nonnegative solution of π j = X i π i p ij , X i π i = 1 . 1
We see that the p j are unique nonnegative solution of ν j p j = X i ν i p i p ij , X j p j = 1 , or, equivalently, using q ij = ν i p ij , ν j p j = X i p i q ij , X j p j = 1 . 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Another way of obtaining the equations for the p i , is by way of the forward equations P 0 ij ( t ) = X k 6 = j q kj P ik ( t ) - ν j P ij ( t ) . If we assume that the limiting probabilities p j = lim t →∞ P ij ( t ) exists, then P 0 ij ( t ) would necessarily converge to 0 as t → ∞ . Hence, assuming that we can interchange limit and summation in the above, we obtain upon letting t → ∞ , 0 = X k 6 = j p k q kj - ν j p j . 3
Let us now determine the limiting probabilities for a birth and death process. The relevant equations are λ 0 p 0 = μ 1 p 1 , λ n p n = μ n +1 p n +1 + ( λ n - 1 p n - 1 - μ n p n ) , n 1 . or, equivalently, λ 0 p 0 = μ 1 p 1 , λ 1 p 1 = μ 2 p 2 + ( λ 0 p 0 - μ 1 p 1 ) = μ 2 p 2 , λ 2 p 2 = μ 3 p 3 + ( λ 1 p 1 - μ 2 p 2 ) = μ 3 p 3 , λ n p n = μ n +1 p n +1 + ( λ n - 1 p n - 1 - μ n p n ) = μ n +1 p n +1 . 4

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
p 0 yields p 1 = λ 0 μ 1 p 0 , p 2 = λ 1 μ 2 p 1 = λ 1 λ 0 μ 2 μ 1 p 0 , p 3 = λ 2 μ 3 p 2 = λ 2 λ 1 λ 0 μ 3 μ 2 μ 1 p 0 , p n = λ n - 1 μ n p n - 1 = λ n - 1 λ n - 2 ··· λ 1 λ 0 μ n μ n - 1 ··· μ 2 μ 1 p 0 . 5
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 05/07/2008 for the course STAT 150 taught by Professor Evans during the Spring '08 term at Berkeley.

Page1 / 22

Stat150_Spring08_Markov_cts_stationary - Statistics 150...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online