Then is called a stationary distribution for

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: tationary distribution." 1.5 Stationary distributions Suppose a distribution  on S is such that, if our Markov chain starts out with initial distribution 0 = , then we also have 1 = . That is, if the distribution at time 0 is , then the distribution at time 1 is still . Then  is called a stationary distribution for Stochastic Processes J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-10 the Markov chain. From 1.12 we see that the de nition of stationary distribution amounts to saying that  satis es the equation  = P; 1.18 that is, j  = X i2S iP i; j  for all j 2 S: In the case of an in nite state space, 1.18 is an in nite system of equations. Also from equations 1.12 we can see that if the Markov chain has initial distribution 0 = , then we have not only 1 = , but also n =  for all n. That is, a Markov chain started out in a stationary distribution  stays in the distribution  forever; that's why the distribution  is called stationary." 1.19 Example. If the N  N probability transition matrix P is symmetric, then the uniform distribution i = 1=N for all i is stationary. More generally, the uniform distribution is stationary if the matrix P is doubly stochastic , that is, the column-sums of P are 1 we already know the row-sums of P are all 1. It should not be surprising that  appears as the limit in Theorem 1.17. It is easy to see that if n approaches a limiting distribution as n ! 1, then that limiting distribution must be stationary. To see this, suppose that limn!1 n = , and let n ! 1 in the ~ equation n+1 = n P to obtain  = P , which says that  is stationary. ~~ ~ 1.20 Exercise For the mathematically inclined . What happens in the case of a countably in nite state space? Does the sort of argument in the previous paragraph still work? Computing stationary distributions is an algebra problem. Since most people are accustomed to solving linear systems of the form Ax = b, let us take the transpose of the equation P , I  = 0, getting the equation P T , I T = 0. For example, for the matrix P from 1.2, we get the equation 0 ,1 1=3 @ 1 ,1 1 0 1 1 A @ 2 A = 0; 0 ,1 1=3 @ 0 ,2=3 10 1=3 1=3 0 2=3 ,2=3 or 3 1 1= 3 1 2=3 A @ 2 A = 0; 0 2=3 ,2=3 3 which has solutions of the form  = const2=3; 1; 1. For the unique solution that satis es P the constraint i = 1, take the constant to be 3 8, so that  = 1=4; 3=8; 3=8. Here is another way, aside from solving the linear equations, to approach the problem of nding a stationary distribution; this idea can work particularly well with computers. If Stochastic Processes J. Chang, March 30, 1999 1.5. STATIONARY DISTRIBUTIONS Page 1-11 we believe the Basic Limit Theorem, we should see the stationary distribution in the limit as we run the chain for a long time. Let's try it: Here are some calculations of powers of the transition matrix P from 1.2: 0 0:246914 0:407407 0:345679 1 P 5 = @ 0:251029 0:36214 0:386831 A ; 0:251029 0:366255 0:382716 0 0:250013 0:37...
View Full Document

Ask a homework question - tutors are online