Lecture 12: 03/05/2007
Recall:
Metropolis Algorithm
S: big discrete set of states
A probability distribution on S:
( )∝ ∈( )
p x
e
x kT
Algorithm constructs a Markov Chain using “proposal followed by accept/reject.”
Start with symmetric Q
ij
; Q
ij
=prob{given I, propose j}
Accept
j as next state with probability
( , )=
< ( ) (

)
> ( )
α i j
1 if ϵj
ϵ i e
ϵj ϵi kT if ϵj
ϵ i
Overall transition probabilities for Markov chain are: P(I,j)=Q(I,j)α(I,j)
Recall detailed balance
ideas given a probabilitiy distribution π on S, their πis stationary distribution for
MC with transition probs P(I,j)
if
, =
,
,
πiPi j
Pj iπjfor all i j
(Detailed balance conditions)
Claim
: with P(I,j)= Q(I,j)α(I,j), defined as above, stationary distribution is precisely π
*
, where
*= 
∈

πi
e ϵikTj
Se ϵjkT
Ie, “π
*
=P”
Proof
: Check detailed balance conditions:
Case 1:
> ( )
ϵi
ϵ j
.
Then P(I,j)=Q(I,j) and P(j,i)=Q(j,i)
(

)
e
ϵi ϵj kT
Since Q(j,i)=Q(I,j), this says
P(I,j)=Q(I,j)=Q(j,i)=P(j,i)
(

)
e ϵi ϵj kT
Note:
(

)
= ( ) ( )=
*
*
e ϵi ϵj kT
p j p i
πj πi
Hence:
*
, =
,
*
πi Pi j
Pj iπj
Bottom Line
: π
*
, the Boltzmann distribution, is the (unique*) stationary distribution for Markov chain
(easy to argue similarly when
>
.
ϵj
ϵi
*unique because, by construction, the Markov chain is generic
every state positively recurrent, only one
recurrence class, hence only one stationary distribution.