{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Markov

A stationary distribution is a probability

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: repeatedly bumps against the re ecting barrier at 0. An example with  = ,0:3 is shown in the gure. Original random walk S and reflected random walk W 2 w s w s 1 w w s w s -1 0 w s w s w w s w w w w s w w w w w s w s s w w s s s -2 s s s s -3 s -4 s s 0 Stochastic Processes 5 10 15 20 J. Chang, March 30, 1999 1. MARKOV CHAINS Page 1-36 Notice a qualitative di erence between this process and the previous examples: here we have an atom , in the sense that there is a state 0, here that is hit with positive probability. A Markov chain fX0 ; X1 ; : : :g is determined by a state space S, an initial distribution 0, and a probability transition rule. The state space is a set, and the initial distribution is a probability measure on that set. For each x 2 S, the probability transition rule, or transition kernel," speci es a probability measure on S. That is, the transition kernel P of the chain gives conditional probabilities like P x; A = PfXt+1 2 A j Xt = xg: Letting t denote the distribution of Xt , we have t+1 = t P , that is, Z t+1A = t dxP x; A As you might suspect by now, much of the theory we have developed for countable state spaces extends to more general state spaces, with sums replaced by integrals. A stationary distribution  is a probability distribution on S that satis es the equation Z for all A dxP x; A = A S. 1.70 Example Autoregressive process, continued . Continuing with Example 1.68, suppose ,1 1. Sensibly suspecting the family of Normal distributions as the plausible candidates for a stationary distribution here, let us try out the distribution  = N ; 2  and see what the values of  and have to be. Assuming Xt,1 and Xt are distributed according to  and noting that Zt is independent of Xt,1 ; by equating the means and variances of the left and right side of Xt = Xt,1 + Zt we obtain the equations  =  and 2 = 2 2 + 2 , which imply  = 0 and 2 = 2=1 , 2. Denoting the distribution at time t by t = N t ; t2 , we ask: does t approach  as t ! 1? Let's compute t and t explicitly. Applying the relations t = t,1 and t2 = 2 t2,1 + 2 to t = 1; 2; : : : gives 2 2 1 = 0 ; 1 = 2 0 + 2 ; 2 2 2 = 2 0; 2 = 4 0 + 2 2 + 2 ; 2 2 3 = 3 0; 3 = 6 0 + 4 2 + 2 2 + 2 ; .. . 2 t = t 0 ; t2 = 2t 0 +  2t,2 + 2t,4 +    + 2 + 1 2 ; .. . P Thus, t ! 0 and t2 ! 2 1 2k = 2 =1 , 2 , and we have established convergence to k=0 the stationary distribution N 0; 2 =1 , 2. So here is a continuous-state-space Markov Stochastic Processes J. Chang, March 30, 1999 1.10. GENERAL STATE SPACE MARKOV CHAINS Page 1-37 chain for which we have found a stationary distribution and established convergence to stationarity. The last example was nice and easy, but we have shamelessly exploited the special features of this problem. In particular, the Normality assumptions allowed us to do explicit computations of the distributions t and . However, what happens, for example, if the random variables fZt g are not Normally distributed? Presumably under some...
View Full Document

{[ snackBarMessage ]}

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern