Stat150_Spring08_Markov_cts_stationary

Stat150_Spring08_Markov_cts_stationary - Statistics 150...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Statistics 150: Spring 2007 March 25, 2007 0-1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities p j = lim t →∞ P ij ( t ) are given by p j = π j j i π i i where the π j are the unique nonnegative solution of π j = X i π i p ij , X i π i = 1 . 1
Background image of page 2
We see that the p j are unique nonnegative solution of ν j p j = X i ν i p i p ij , X j p j = 1 , or, equivalently, using q ij = ν i p ij , ν j p j = X i p i q ij , X j p j = 1 . 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Another way of obtaining the equations for the p i , is by way of the forward equations P 0 ij ( t ) = X k 6 = j q kj P ik ( t ) - ν j P ij ( t ) . If we assume that the limiting probabilities p j = lim t →∞ P ij ( t ) exists, then P 0 ij ( t ) would necessarily converge to 0 as t → ∞ . Hence, assuming that we can interchange limit and summation in the above, we obtain upon letting t → ∞ , 0 = X k 6 = j p k q kj - ν j p j . 3
Background image of page 4
Let us now determine the limiting probabilities for a birth and death process. The relevant equations are λ 0 p 0 = μ 1 p 1 , λ n p n = μ n +1 p n +1 + ( λ n - 1 p n - 1 - μ n p n ) , n 1 . or, equivalently, λ 0 p 0 = μ 1 p 1 , λ 1 p 1 = μ 2 p 2 + ( λ 0 p 0 - μ 1 p 1 ) = μ 2 p 2 , λ 2 p 2 = μ 3 p 3 + ( λ 1 p 1 - μ 2 p 2 ) = μ 3 p 3 , λ n p n = μ n +1 p n +1 + ( λ n - 1 p n - 1 - μ n p n ) = μ n +1 p n +1 . 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
p 0 yields p 1 = λ 0 μ 1 p 0 , p 2 = λ 1 μ 2 p 1 = λ 1 λ 0 μ 2 μ 1 p 0 , p 3 = λ 2 μ 3 p 2 = λ 2 λ 1 λ 0 μ 3 μ 2 μ 1 p 0 , p n = λ n - 1 μ n p n - 1 = λ n - 1 λ n - 2 ··· λ 1 λ 0 μ n μ n - 1 ··· μ 2 μ 1 p 0 . 5
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 22

Stat150_Spring08_Markov_cts_stationary - Statistics 150...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online