Unformatted text preview: tationary distribution." 1.5 Stationary distributions
Suppose a distribution on S is such that, if our Markov chain starts out with initial
distribution 0 = , then we also have 1 = . That is, if the distribution at time 0 is ,
then the distribution at time 1 is still . Then is called a stationary distribution for
Stochastic Processes J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-10 the Markov chain. From 1.12 we see that the de nition of stationary distribution amounts
to saying that satis es the equation = P; 1.18
that is, j = X
i2S iP i; j for all j 2 S: In the case of an in nite state space, 1.18 is an in nite system of equations. Also from
equations 1.12 we can see that if the Markov chain has initial distribution 0 = , then
we have not only 1 = , but also n = for all n. That is, a Markov chain started out in
a stationary distribution stays in the distribution forever; that's why the distribution
is called stationary."
1.19 Example. If the N N probability transition matrix P is symmetric, then the uniform distribution i = 1=N for all i is stationary. More generally, the uniform
distribution is stationary if the matrix P is doubly stochastic , that is, the column-sums of
P are 1 we already know the row-sums of P are all 1.
It should not be surprising that appears as the limit in Theorem 1.17. It is easy to
see that if n approaches a limiting distribution as n ! 1, then that limiting distribution
must be stationary. To see this, suppose that limn!1 n = , and let n ! 1 in the
equation n+1 = n P to obtain = P , which says that is stationary.
1.20 Exercise For the mathematically inclined . What happens in the case of a
countably in nite state space? Does the sort of argument in the previous paragraph still work? Computing stationary distributions is an algebra problem. Since most people are accustomed to solving linear systems of the form Ax = b, let us take the transpose of the
equation P , I = 0, getting the equation P T , I T = 0. For example, for the matrix
P from 1.2, we get the equation 0 ,1 1=3
@ 1 ,1 1 0 1 1
A @ 2 A = 0; 0 ,1 1=3
@ 0 ,2=3 10 1=3
0 2=3 ,2=3 or 3 1 1= 3
2=3 A @ 2 A = 0;
0 2=3 ,2=3
which has solutions of the form = const2=3; 1; 1. For the unique solution that satis es
the constraint i = 1, take the constant to be 3 8, so that = 1=4; 3=8; 3=8.
Here is another way, aside from solving the linear equations, to approach the problem
of nding a stationary distribution; this idea can work particularly well with computers. If
Stochastic Processes J. Chang, March 30, 1999 1.5. STATIONARY DISTRIBUTIONS Page 1-11 we believe the Basic Limit Theorem, we should see the stationary distribution in the limit
as we run the chain for a long time. Let's try it: Here are some calculations of powers of
the transition matrix P from 1.2:
0 0:246914 0:407407 0:345679 1
P 5 = @ 0:251029 0:36214 0:386831 A ;
0:251029 0:366255 0:382716
0 0:250013 0:37...
View Full Document