445Lecture9.pdf - Lecture 9 Mariana Olvera-Cravioto UNC...

This preview shows 1 out of 9 pages.

Lecture 9 Mariana Olvera-Cravioto UNC Chapel Hill [email protected] February 7th, 2019 STOR 445, Introduction to Stochastic Modeling Lecture 9 1/16
Image of page 1

Subscribe to view the full document.

Time reversible Markov chains I Consider an irreducible, aperiodic Markov chain { X n : n 0 } having stationary distribution π . I Let’s look at the process in reverse, i.e., { X n , X n - 1 , X n - 2 , . . . } . STOR 445, Introduction to Stochastic Modeling Lecture 9 2/16
Image of page 2
Time reversible Markov chains I Consider an irreducible, aperiodic Markov chain { X n : n 0 } having stationary distribution π . I Let’s look at the process in reverse, i.e., { X n , X n - 1 , X n - 2 , . . . } . I This is also a Markov chain! STOR 445, Introduction to Stochastic Modeling Lecture 9 2/16
Image of page 3

Subscribe to view the full document.

Transition probabilities I To compute its transition probabilities { q ( i, j ) : i, j 0 } note that q i,j = P ( X m = j | X m +1 = i ) = P ( X m = j, X m +1 = i ) P ( X m +1 = i ) = P ( X m +1 = i | X m = j ) P ( X m = j ) P ( X m +1 = i ) = p j,i P ( X m = j ) P ( X m +1 = i ) I Where by stationarity, P ( X m = k ) = π k , and therefore, q i,j = p j,i π j π i , i, j 0 . STOR 445, Introduction to Stochastic Modeling Lecture 9 3/16
Image of page 4
Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16
Image of page 5

Subscribe to view the full document.

Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . I Interpretation: The chains { X 0 , X 1 , X 2 , . . . , X n , . . . } and { X n , X n - 1 , X n - 2 , . . . , X 0 , . . . } have the same distribution, i.e., they’re undistinguishable! STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16
Image of page 6
Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . I Interpretation: The chains { X 0 , X 1 , X 2 , . . . , X n , . . . } and { X n , X n - 1 , X n - 2 , . . . , X 0 , . . . } have the same distribution, i.e., they’re undistinguishable! I It follows that they must have the same stationary probabilities. STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16
Image of page 7

Subscribe to view the full document.

Using the time reversible equations I If we can find a nonnegative solution x = ( x 0 , x 1 , x 2 , . . . ) to the time reversibility equations x i p i,j = x j p j,i for all i, j 0 , X i =0 x i = 1 , then the Markov chain { X n : n 0 } is time reversible and has stationary probabilities π i = x i for all i 0 .
Image of page 8
Image of page 9
You've reached the end of this preview.
  • Spring '07
  • AndrewB.Nobel
  • Markov chain, Πi

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern