173Lecture9 - Lecture 9 Mariana Olvera-Cravioto UC Berkeley...

Lecture 9
Mariana Olvera-Cravioto
UC Berkeley
[email protected]
February 14th, 2017
IND ENG 173, Introduction to Stochastic Processes Lecture 9 1/11

Time reversibility
I
We say that the Markov chain { X
n
: n ≥ 1 } is time reversible if its
stationary distribution ⇡ satisfies
⇡
i
p ( i, j ) = ⇡
j
p ( j, i ) for all i, j ≥ 0 .
I
The equations above are known as the detailed balance condition .
IND ENG 173, Introduction to Stochastic Processes Lecture 9 2/11

Time reversibility
I
We say that the Markov chain { X
n
: n ≥ 1 } is time reversible if its
stationary distribution ⇡ satisfies
⇡
i
p ( i, j ) = ⇡
j
p ( j, i ) for all i, j ≥ 0 .
I
The equations above are known as the detailed balance condition .
I
Interpretation: The chains { X
0
, X
1
, X
2
, . . . , X
n
, . . . } and
{ X
n
, X
n - 1
, X
n - 2
, . . . , X
0
, . . . } have the same distribution, i.e., they’re
undistinguishable!
IND ENG 173, Introduction to Stochastic Processes Lecture 9 2/11

Time reversibility
I
We say that the Markov chain { X
n
: n ≥ 1 } is time reversible if its
stationary distribution ⇡ satisfies
⇡
i
p ( i, j ) = ⇡
j
p ( j, i ) for all i, j ≥ 0 .
I
The equations above are known as the detailed balance condition .
I
Interpretation: The chains { X
0
, X
1
, X
2
, . . . , X
n
, . . . } and
{ X
n
, X
n - 1
, X
n - 2
, . . . , X
0
, . . . } have the same distribution, i.e., they’re
undistinguishable!
I
It follows that the reversed process also has stationary distribution ⇡ .
IND ENG 173, Introduction to Stochastic Processes Lecture 9 2/11

Using the time reversible equations
I
If we can find a nonnegative solution x = ( x
0
, x
1
, x
2
, . . . ) to the time
reversibility equations
x
i
p ( i, j ) = x
j
p ( j, i ) for all i, j ≥ 0 ,
1 X
i =0
x
i
= 1 ,
then the Markov chain { X
n
: n ≥ 0 } is time reversible and has stationary
probabilities
⇡
i
= x
i
for all i ≥ 0 .
I
These equations are simpler than the system ⇡ P = ⇡ ,
P
i
⇡
i
= 1 , and
sometimes we can “guess” the x .
IND ENG 173, Introduction to Stochastic Processes Lecture 9 3/11

Example: A reflected random walk
I
Consider a random walk on the set { 0 , 1 , 2 , . . . , N } where on each step
we toss a (biased) coin to determine whether we go “up” one unit or
“down” one unit.
I
When the random walk reaches zero or N it gets “reflected”, i.e., from 0
it always goes to 1 and from N it always goes to N - 1 .
I