4703-10-Notes-MCMC - Copyright c 2010 by Karl Sigman 1...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Copyright c ± 2010 by Karl Sigman 1 Markov Chain Monte Carlo Methods (MCMC) There are many applications in which it is desirable to simulate from a probability distribution π (say) in which all specifics of the distribution (cdf, density, etc.) are not known a priori, and hence standard simulation methods (such as the inverse transform method, acceptance rejection, etc.) cannot be readily applied. For example, one might know how to simulate a complicated stochastic process { X n } for which a priori it is known to have a limiting (stationary) distribution π , but for which explicitly/analytically finding π is not possible. Coming up with an algorithm that yields a copy of a rv distributed exactly as π in this context is called perfect or exact simulation. We of course could estimate π by using the fact that we can simulate the stochastic process { X n } , and use the approximation (for large N ) π i 1 N N X n =1 I { X n = i } . (1) An important related case is in the framework when a desired distribution π is not naturally part of any stochastic process and is only known up to a multiplicative constant (normalizing factor C > 0). For example, consider the discrete case, and suppose we want the following distribution π i = a i /C, i ∈ S , (2) where each a i > 0 can be computed but the normalizing constant C = X i ∈S a i , is uncomputable hence unknown. π might, for example, be a uniform distribution over a very complicated (but finite) set involving a graph with various edges, weights and arcs. (Combina- torial objects are notoriously like this.) In such a uniform case, a i = 1 for all i , but computing C can be hopeless. The Markov chain Monte Carlo method (MCMC) allows us to approximate π as defined in (2) by constructing (and simulating) an irreducible positive recurrent Markov chain { X n } with state space S and transition matrix P = ( P ij ) such that its stationary distribution is the desired π , that is, such that π = πP . Thus we can simply simulate the Markov chain out N steps, N large, and use the approximation π i 1 N N X n =1 I { X n = i } , as pointed out in (1). (This is not perfect simulation, but a perfect simulation might be possible later applied to the constructed Markov chain.) Perhaps our goal is to compute the mean of π in which case we could use the approximation X i ∈S i 1 N N X n =1 X n . 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
More generally, we might want to compute an expected value of π against a function g = g ( i ), π ( g ) def = X i ∈S g ( i ) π i , in which case we could use the approximation π ( g ) 1 N N X n =1 g ( X n ) . A classic MCMC construction is known as the
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/14/2012 for the course IEOR 4703 taught by Professor Sigman during the Spring '07 term at Columbia.

Page1 / 6

4703-10-Notes-MCMC - Copyright c 2010 by Karl Sigman 1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online