445Lecture9.pdf

# 445Lecture9.pdf - Lecture 9 Mariana Olvera-Cravioto UNC...

• Notes
• 27

This preview shows 1 out of 9 pages.

Lecture 9 Mariana Olvera-Cravioto UNC Chapel Hill [email protected] February 7th, 2019 STOR 445, Introduction to Stochastic Modeling Lecture 9 1/16

Subscribe to view the full document.

Time reversible Markov chains I Consider an irreducible, aperiodic Markov chain { X n : n 0 } having stationary distribution π . I Let’s look at the process in reverse, i.e., { X n , X n - 1 , X n - 2 , . . . } . STOR 445, Introduction to Stochastic Modeling Lecture 9 2/16
Time reversible Markov chains I Consider an irreducible, aperiodic Markov chain { X n : n 0 } having stationary distribution π . I Let’s look at the process in reverse, i.e., { X n , X n - 1 , X n - 2 , . . . } . I This is also a Markov chain! STOR 445, Introduction to Stochastic Modeling Lecture 9 2/16

Subscribe to view the full document.

Transition probabilities I To compute its transition probabilities { q ( i, j ) : i, j 0 } note that q i,j = P ( X m = j | X m +1 = i ) = P ( X m = j, X m +1 = i ) P ( X m +1 = i ) = P ( X m +1 = i | X m = j ) P ( X m = j ) P ( X m +1 = i ) = p j,i P ( X m = j ) P ( X m +1 = i ) I Where by stationarity, P ( X m = k ) = π k , and therefore, q i,j = p j,i π j π i , i, j 0 . STOR 445, Introduction to Stochastic Modeling Lecture 9 3/16
Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16

Subscribe to view the full document.

Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . I Interpretation: The chains { X 0 , X 1 , X 2 , . . . , X n , . . . } and { X n , X n - 1 , X n - 2 , . . . , X 0 , . . . } have the same distribution, i.e., they’re undistinguishable! STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16
Time reversibility I If the Markov chain { X n : n 1 } satisfies q i,j = p i,j for all i, j 0 , then we say that it is time reversible . I Equivalently, the time reversibility condition can be written as π i p i,j = π j p j,i for all i, j 0 , known as the detailed balance condition . I Interpretation: The chains { X 0 , X 1 , X 2 , . . . , X n , . . . } and { X n , X n - 1 , X n - 2 , . . . , X 0 , . . . } have the same distribution, i.e., they’re undistinguishable! I It follows that they must have the same stationary probabilities. STOR 445, Introduction to Stochastic Modeling Lecture 9 4/16

Subscribe to view the full document.

Using the time reversible equations I If we can find a nonnegative solution x = ( x 0 , x 1 , x 2 , . . . ) to the time reversibility equations x i p i,j = x j p j,i for all i, j 0 , X i =0 x i = 1 , then the Markov chain { X n : n 0 } is time reversible and has stationary probabilities π i = x i for all i 0 .
You've reached the end of this preview.
• Spring '07
• AndrewB.Nobel
• Markov chain, Πi

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern