This preview shows page 1. Sign up to view the full content.
Unformatted text preview: repeatedly bumps against the
re ecting barrier at 0. An example with = ,0:3 is shown in the gure.
Original random walk S and reflected random walk W
2 w
s
w
s 1 w w s w
s 1 0 w
s
w
s w w s w w
w w
s w w w w w s w
s s w
w s
s s 2 s
s s
s 3 s 4 s
s
0 Stochastic Processes 5 10 15 20 J. Chang, March 30, 1999 1. MARKOV CHAINS Page 136 Notice a qualitative di erence between this process and the previous examples: here we have
an atom , in the sense that there is a state 0, here that is hit with positive probability.
A Markov chain fX0 ; X1 ; : : :g is determined by a state space S, an initial distribution
0, and a probability transition rule. The state space is a set, and the initial distribution
is a probability measure on that set. For each x 2 S, the probability transition rule, or
transition kernel," speci es a probability measure on S. That is, the transition kernel P
of the chain gives conditional probabilities like
P x; A = PfXt+1 2 A j Xt = xg:
Letting t denote the distribution of Xt , we have t+1 = t P , that is, Z t+1A = t dxP x; A
As you might suspect by now, much of the theory we have developed for countable state
spaces extends to more general state spaces, with sums replaced by integrals.
A stationary distribution is a probability distribution on S that satis es the equation Z for all A dxP x; A = A S. 1.70 Example Autoregressive process, continued . Continuing with Example 1.68, suppose ,1 1. Sensibly suspecting the family of Normal distributions as
the plausible candidates for a stationary distribution here, let us try out the distribution
= N ; 2 and see what the values of and have to be. Assuming Xt,1 and Xt
are distributed according to and noting that Zt is independent of Xt,1 ; by equating the
means and variances of the left and right side of Xt = Xt,1 + Zt we obtain the equations
= and 2 = 2 2 + 2 , which imply = 0 and 2 = 2=1 , 2. Denoting the
distribution at time t by t = N t ; t2 , we ask: does t approach as t ! 1? Let's
compute t and t explicitly. Applying the relations t = t,1 and t2 = 2 t2,1 + 2 to
t = 1; 2; : : : gives
2
2
1 = 0 ; 1 = 2 0 + 2 ;
2
2
2 = 2 0; 2 = 4 0 + 2 2 + 2 ;
2
2
3 = 3 0; 3 = 6 0 + 4 2 + 2 2 + 2 ;
..
.
2
t = t 0 ; t2 = 2t 0 + 2t,2 + 2t,4 + + 2 + 1 2 ;
..
.
P
Thus, t ! 0 and t2 ! 2 1 2k = 2 =1 , 2 , and we have established convergence to
k=0
the stationary distribution N 0; 2 =1 , 2. So here is a continuousstatespace Markov
Stochastic Processes J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS Page 137 chain for which we have found a stationary distribution and established convergence to
stationarity.
The last example was nice and easy, but we have shamelessly exploited the special features of this problem. In particular, the Normality assumptions allowed us to do explicit
computations of the distributions t and . However, what happens, for example, if the
random variables fZt g are not Normally distributed? Presumably under some...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 Multiplication, Markov Chains

Click to edit the document details