A stationary distribution is a probability

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: repeatedly bumps against the re ecting barrier at 0. An example with  = ,0:3 is shown in the gure. Original random walk S and reflected random walk W 2 w s w s 1 w w s w s -1 0 w s w s w w s w w w w s w w w w w s w s s w w s s s -2 s s s s -3 s -4 s s 0 Stochastic Processes 5 10 15 20 J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-36 Notice a qualitative di erence between this process and the previous examples: here we have an atom , in the sense that there is a state 0, here that is hit with positive probability. A Markov chain fX0 ; X1 ; : : :g is determined by a state space S, an initial distribution 0, and a probability transition rule. The state space is a set, and the initial distribution is a probability measure on that set. For each x 2 S, the probability transition rule, or transition kernel," speci es a probability measure on S. That is, the transition kernel P of the chain gives conditional probabilities like P x; A = PfXt+1 2 A j Xt = xg: Letting t denote the distribution of Xt , we have t+1 = t P , that is, Z t+1A = t dxP x; A As you might suspect by now, much of the theory we have developed for countable state spaces extends to more general state spaces, with sums replaced by integrals. A stationary distribution  is a probability distribution on S that satis es the equation Z for all A dxP x; A = A S. 1.70 Example Autoregressive process, continued . Continuing with Example 1.68, suppose ,1 1. Sensibly suspecting the family of Normal distributions as the plausible candidates for a stationary distribution here, let us try out the distribution  = N ; 2  and see what the values of  and have to be. Assuming Xt,1 and Xt are distributed according to  and noting that Zt is independent of Xt,1 ; by equating the means and variances of the left and right side of Xt = Xt,1 + Zt we obtain the equations  =  and 2 = 2 2 + 2 , which imply  = 0 and 2 = 2=1 , 2. Denoting the distribution at time t by t = N t ; t2 , we ask: does t approach  as t ! 1? Let's compute t and t explicitly. Applying the relations t = t,1 and t2 = 2 t2,1 + 2 to t = 1; 2; : : : gives 2 2 1 = 0 ; 1 = 2 0 + 2 ; 2 2 2 = 2 0; 2 = 4 0 + 2 2 + 2 ; 2 2 3 = 3 0; 3 = 6 0 + 4 2 + 2 2 + 2 ; .. . 2 t = t 0 ; t2 = 2t 0 +  2t,2 + 2t,4 +    + 2 + 1 2 ; .. . P Thus, t ! 0 and t2 ! 2 1 2k = 2 =1 , 2 , and we have established convergence to k=0 the stationary distribution N 0; 2 =1 , 2. So here is a continuous-state-space Markov Stochastic Processes J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS Page 1-37 chain for which we have found a stationary distribution and established convergence to stationarity. The last example was nice and easy, but we have shamelessly exploited the special features of this problem. In particular, the Normality assumptions allowed us to do explicit computations of the distributions t and . However, what happens, for example, if the random variables fZt g are not Normally distributed? Presumably under some...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online