Lecture 12: 03/05/2007
Recall:
Metropolis Algorithm
S: big discrete set of states
A probability distribution on S:
( )∝ ∈( )
p x
e
x kT
Algorithm constructs a Markov Chain using “proposal followed by accept/reject.”
Start with symmetric Q
ij
; Q
ij
=prob{given I, propose j}
Accept
j as next state with probability
( , )=
< ( ) (

)
> ( )
α i j
1 if ϵj
ϵ i e
ϵj ϵi kT if ϵj
ϵ i
Overall transition probabilities for Markov chain are: P(I,j)=Q(I,j)α(I,j)
Recall detailed balance
ideas given a probabilitiy distribution π on S, their πis stationary distribution for
MC with transition probs P(I,j)
if
, =
,
,
πiPi j
Pj iπjfor all i j
(Detailed balance conditions)
Claim
: with P(I,j)= Q(I,j)α(I,j), defined as above, stationary distribution is precisely π
*
, where
*= 
∈

πi
e ϵikTj
Se ϵjkT
Ie, “π
*
=P”
Proof
: Check detailed balance conditions:
Case 1:
> ( )
ϵi
ϵ j
.
Then P(I,j)=Q(I,j) and P(j,i)=Q(j,i)
(

)
e
ϵi ϵj kT
Since Q(j,i)=Q(I,j), this says
P(I,j)=Q(I,j)=Q(j,i)=P(j,i)
(

)
e ϵi ϵj kT
Note:
(

)
= ( ) ( )=
*
*
e ϵi ϵj kT
p j p i
πj πi
Hence:
*
, =
,
*
πi Pi j
Pj iπj
Bottom Line
: π
*
, the Boltzmann distribution, is the (unique*) stationary distribution for Markov chain
(easy to argue similarly when
>
.
ϵj
ϵi
*unique because, by construction, the Markov chain is generic
every state positively recurrent, only one
recurrence class, hence only one stationary distribution.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 DELCHAMPS
 Algorithms, Tn, Markov chain, Markov chain Monte Carlo, Hastings, Boltzmann

Click to edit the document details