173Lecture9 - Lecture 9 Mariana Olvera-Cravioto UC Berkeley...

• Notes
• 21

This preview shows page 1 - 7 out of 21 pages.

The preview shows page 5 - 7 out of 21 pages.
Lecture 9Mariana Olvera-CraviotoUC Berkeley[email protected]February 14th, 2017IND ENG 173, Introduction to Stochastic ProcessesLecture 91/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IInterpretation:The chains{X0, X1, X2, . . . , Xn, . . .}and{Xn, Xn-1, Xn-2, . . . , X0, . . .}have the same distribution, i.e., they’reundistinguishable!IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Time reversibilityIWe say that the Markov chain{Xn:n1}istime reversibleif itsstationary distributionsatisfiesip(i, j) =jp(j, i)for alli, j0.IThe equations above are known as thedetailed balance condition.IInterpretation:The chains{X0, X1, X2, . . . , Xn, . . .}and{Xn, Xn-1, Xn-2, . . . , X0, . . .}have the same distribution, i.e., they’reundistinguishable!IIt follows that the reversed process also has stationary distribution.IND ENG 173, Introduction to Stochastic ProcessesLecture 92/11
Using the time reversible equationsIIf we can find a nonnegative solutionx= (x0, x1, x2, . . .)to the timereversibility equationsxip(i, j) =xjp(j, i)for alli, j0,1Xi=0xi= 1,then the Markov chain{Xn:n0}is time reversible and has stationaryprobabilitiesi=xifor alli0.IThese equations are simpler than the systemP=,Pii= 1, andsometimes we can “guess” thex.IND ENG 173, Introduction to Stochastic ProcessesLecture 93/11
Example: A reflected random walkIConsider a random walk on the set{0,1,2, . . . , N}where on each stepwe toss a (biased) coin to determine whether we go “up” one unit or“down” one unit.IWhen the random walk reaches zero orNit gets “reflected”, i.e., from 0it always goes to 1 and fromNit always goes toN-1.I

Course Hero member to access this document

Course Hero member to access this document

End of preview. Want to read all 21 pages?

Course Hero member to access this document

Term
Spring
Professor
Leachman
Tags
Markov chain, IND ENG