445Lecture7.pdf - Lecture 7 Mariana Olvera-Cravioto UNC...

This preview shows 1 out of 8 pages.

Lecture 7 Mariana Olvera-Cravioto UNC Chapel Hill [email protected] January 31st, 2019 STOR 445, Introduction to Stochastic Modeling Lecture 7 1/15
Image of page 1

Subscribe to view the full document.

Starting a Markov chain at a random state I Suppose that at time zero we start the Markov chain { X n : n 0 } at a random state X 0 . I Let q i = P ( X 0 = i ) be the PMF of the initial state. I What is the probability that the chain is in state j at time n ? STOR 445, Introduction to Stochastic Modeling Lecture 7 2/15
Image of page 2
Starting a Markov chain at a random state I Suppose that at time zero we start the Markov chain { X n : n 0 } at a random state X 0 . I Let q i = P ( X 0 = i ) be the PMF of the initial state. I What is the probability that the chain is in state j at time n ? I By conditioning on the value of X 0 we obtain P ( X n = j ) = X i P ( X n = j, X 0 = i ) = X i P ( X n = j | X 0 = i ) P ( X 0 = i ) = X i p ( n ) i,j q i = X i ( P n ) i,j q i , where P is the one-step transition matrix and ( P n ) i,j is the ( i, j ) th component of the matrix P n . STOR 445, Introduction to Stochastic Modeling Lecture 7 2/15
Image of page 3

Subscribe to view the full document.

The weather chain revisited I Recall the oversimplified weather chain from Lecture 5. I The chain had three states: { 1 - Sunny, 2 - Cloudy, 3 - Rainy } , and one-step transition matrix: P = 0 . 7 0 . 3 0 0 . 3 0 . 5 0 . 2 0 . 6 0 . 3 0 . 1 I Suppose we choose the initial state X 0 according to the PMF: q 1 = 0 . 5 , q 2 = 0 . 5 , q 3 = 0 . I Compute P ( X 1 = j ) for j = 1 , 2 , 3 . STOR 445, Introduction to Stochastic Modeling Lecture 7 3/15
Image of page 4
The weather chain revisited... continued I Let q = (0 . 5 , 0 . 5 , 0) and note that we can get all the probabilities we want by computing qP = (0 . 5 , 0 . 5 , 0) 0 . 7 0 . 3 0 0 . 3 0 . 5 0 . 2 0 . 6 0 . 3 0 . 1 = (0 . 5 , 0 . 4 , 0 . 1) , where P ( X 1 = 1) = 0 . 5 , P ( X 1 = 2) = 0 . 4 , and P ( X 1 = 3) = 0 . 1 . STOR 445, Introduction to Stochastic Modeling Lecture 7 4/15
Image of page 5

Subscribe to view the full document.

The weather chain revisited... continued I Let q = (0 . 5 , 0 . 5 , 0) and note that we can get all the probabilities we want by computing qP = (0 . 5 , 0 . 5 , 0) 0 . 7 0 . 3 0 0 . 3 0 . 5 0 . 2 0 . 6 0 . 3 0 . 1 = (0 . 5 , 0 . 4 , 0 . 1) , where P ( X 1 = 1) = 0 . 5 , P ( X 1 = 2) = 0 . 4 , and P ( X 1 = 3) = 0 . 1 . I Now let q = (0 . 5417 , 0 . 3750 , 0 . 0833) , and note that for this choice of initial distribution we get qP = (0 . 5417 , 0 . 3750 , 0 . 0833) 0 . 7 0 . 3 0 0 . 3 0 . 5 0 . 2 0 . 6 0 . 3 0 . 1 = q I In other words, q is such that the distribution of X 1 is the same as that of X 0 , and therefore, the distribution of X n is the same for all n ! STOR 445, Introduction to Stochastic Modeling Lecture 7 4/15
Image of page 6
Stationary distributions I Let { X n : n 0 } be a Markov chain having one-step transition matrix P . I We have seen that some Markov chains exhibit a long-run behavior that “forgets” its initial state, and hence have the property that X n has the same distribution for all n sufficiently large.
Image of page 7

Subscribe to view the full document.

Image of page 8
You've reached the end of this preview.
  • Spring '07
  • AndrewB.Nobel
  • lim, Markov chain

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern