Lecture 12: 03/05/2007
Recall:
Metropolis Algorithm
S: big discrete set of states
A probability distribution on S:
( )∝ ∈( )
p x
e
x kT
Algorithm constructs a Markov Chain using “proposal followed by accept/reject.”
Start with symmetric Q
ij
; Q
ij
=prob{given I, propose j}
Accept
j as next state with probability
( , )=
< ( ) -(
-
)
> ( )
α i j
1 if ϵj
ϵ i e
ϵj ϵi kT if ϵj
ϵ i
Overall transition probabilities for Markov chain are: P(I,j)=Q(I,j)α(I,j)
Recall detailed balance
ideas given a probabilitiy distribution π on S, their πis stationary distribution for
MC with transition probs P(I,j)
if
, =
,
,
πiPi j
Pj iπjfor all i j
(Detailed balance conditions)
Claim
: with P(I,j)= Q(I,j)α(I,j), defined as above, stationary distribution is precisely π
*
, where
*= -
∈
-
πi
e ϵikTj
Se ϵjkT
Ie, “π
*
=P”
Proof
: Check detailed balance conditions:
Case 1:
> ( )
ϵi
ϵ j
.
Then P(I,j)=Q(I,j) and P(j,i)=Q(j,i)
-(
-
)
e
ϵi ϵj kT
Since Q(j,i)=Q(I,j), this says
P(I,j)=Q(I,j)=Q(j,i)=P(j,i)
(
-
)
e ϵi ϵj kT
Note:
(
-
)
= ( ) ( )=
*
*
e ϵi ϵj kT
p j p i
πj πi
Hence:
*
, =
,
*
πi Pi j
Pj iπj
Bottom Line
: π
*
, the Boltzmann distribution, is the (unique*) stationary distribution for Markov chain
(easy to argue similarly when
>
.
ϵj
ϵi
*unique because, by construction, the Markov chain is generic
-every state positively recurrent, only one
recurrence class, hence only one stationary distribution.
This preview
has intentionally blurred sections.
Sign up to view the full version.
This is the end of the preview.
Sign up
to access the rest of the document.
- Spring '07
- DELCHAMPS
- Algorithms, Tn, Markov chain, Markov chain Monte Carlo, Hastings, Boltzmann
