This preview shows pages 1–3. Sign up to view the full content.
Copyright c
±
2010 by Karl Sigman
1
Markov Chain Monte Carlo Methods (MCMC)
There are many applications in which it is desirable to simulate from a probability distribution
π
(say) in which all speciﬁcs of the distribution (cdf, density, etc.) are not known a priori, and
hence standard simulation methods (such as the inverse transform method, acceptance rejection,
etc.) cannot be readily applied. For example, one might know how to simulate a complicated
stochastic process
{
X
n
}
for which a priori it is known to have a limiting (stationary) distribution
π
, but for which explicitly/analytically ﬁnding
π
is not possible. Coming up with an algorithm
that yields a copy of a rv distributed
exactly
as
π
in this context is called
perfect
or
exact
simulation. We of course could
estimate
π
by using the fact that we can simulate the stochastic
process
{
X
n
}
, and use the approximation (for large
N
)
π
i
≈
1
N
N
X
n
=1
I
{
X
n
=
i
}
.
(1)
An important related case is in the framework when a desired distribution
π
is not naturally
part of any stochastic process and is only known up to a multiplicative constant (normalizing
factor
C >
0). For example, consider the discrete case, and suppose we want the following
distribution
π
i
=
a
i
/C, i
∈ S
,
(2)
where each
a
i
>
0 can be computed but the normalizing constant
C
=
X
i
∈S
a
i
,
is uncomputable hence unknown.
π
might, for example, be a uniform distribution over a very
complicated (but ﬁnite) set involving a graph with various edges, weights and arcs. (Combina
torial objects are notoriously like this.) In such a uniform case,
a
i
= 1 for all
i
, but computing
C
can be hopeless.
The
Markov chain Monte Carlo method (MCMC)
allows us to approximate
π
as deﬁned in
(2) by constructing (and simulating) an irreducible positive recurrent Markov chain
{
X
n
}
with
state space
S
and transition matrix
P
= (
P
ij
)
such that its stationary distribution is the desired
π
,
that is, such that
π
=
πP
. Thus we can simply simulate the Markov chain out
N
steps,
N
large, and use the approximation
π
i
≈
1
N
N
X
n
=1
I
{
X
n
=
i
}
,
as pointed out in (1). (This is not
perfect
simulation, but a perfect simulation might be possible
later applied to the constructed Markov chain.) Perhaps our goal is to compute the mean of
π
in which case we could use the approximation
X
i
∈S
iπ
i
≈
1
N
N
X
n
=1
X
n
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentMore generally, we might want to compute an expected value of
π
against a function
g
=
g
(
i
),
π
(
g
)
def
=
X
i
∈S
g
(
i
)
π
i
,
in which case we could use the approximation
π
(
g
)
≈
1
N
N
X
n
=1
g
(
X
n
)
.
A classic MCMC construction is known as the
This is the end of the preview. Sign up
to
access the rest of the document.
 Spring '07
 sigman

Click to edit the document details