173Lecture9 - Lecture 9 Mariana Olvera-Cravioto UC Berkeley...

This preview shows page 1 - 7 out of 21 pages.

The preview shows page 5 - 7 out of 21 pages.
Lecture 9Mariana Olvera-CraviotoUC Berkeley[email protected]February 14th, 2017IND ENG 173, Introduction to Stochastic ProcessesLecture 91/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IInterpretation:The chains{X0, X1, X2, . . . , Xn, . . .}and{Xn, Xn-1, Xn-2, . . . , X0, . . .}have the same distribution, i.e., they’reundistinguishable!IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IInterpretation:The chains{X0, X1, X2, . . . , Xn, . . .}and{Xn, Xn-1, Xn-2, . . . , X0, . . .}have the same distribution, i.e., they’reundistinguishable!IIt follows that the reversed process also has stationary distribution.IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Using the time reversible equationsIIf we can find a nonnegative solutionx= (x0, x1, x2, . . .)to the timereversibility equationsxip(i, j) =xjp(j, i)for alli, j0,1Xi=0xi= 1,then the Markov chain{Xn:n0}is time reversible and has stationaryprobabilitiesi=xifor alli0.IThese equations are simpler than the systemP=,Pii= 1, andsometimes we can “guess” thex.IND ENG 173, Introduction to Stochastic ProcessesLecture 93/11
Example: A reflected random walkIConsider a random walk on the set{0,1,2, . . . , N}where on each stepwe toss a (biased) coin to determine whether we go “up” one unit or“down” one unit.IWhen the random walk reaches zero orNit gets “reflected”, i.e., from 0it always goes to 1 and fromNit always goes toN-1.I

Upload your study docs or become a

Course Hero member to access this document

Upload your study docs or become a

Course Hero member to access this document

End of preview. Want to read all 21 pages?

Upload your study docs or become a

Course Hero member to access this document

Term
Spring
Professor
Leachman
Tags
Markov chain, IND ENG

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture