intro_Gibbs

# intro_Gibbs - Introduction to Gibbs Sampling October 8 2010...

This preview shows pages 1–8. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Gibbs Sampling October 8, 2010 Readings: Hoff 6 October 7, 2010 Monte Carlo Sampling I We have seen that Monte Carlo sampling is a useful tool for sampling from prior and posterior distributions I By limiting attention to conjugate prior distributions, all models have had tractable posterior distributions so sampling was not really necessary (although convenient) I What if we want to use a non-conjugate prior distribution? I What if we cannot sample from the joint posterior distribution? Semi-Conjugate Examples Normal sampling model Y i | μ, φ iid ∼ N( μ, 1 φ ) But now assume that μ is independent of φ a priori: μ ∼ N( m , 1 / p ) φ ∼ G ( ν / 2 , ν s 2 / 2) Posterior Distribution: p ( μ, φ | Y ) ∝ Y j p ( y i | μ, φ ) p ( μ ) p ( φ ) Factorization of Joint p ( μ, φ | Y ) ∝ φ n / 2 exp- φ 2 s 2 ( n- 1) φ ν / 2- 1 exp(- φν s 2 ) · exp- φ 2 n (¯ y- μ ) 2 exp( p ( μ- m ) 2 ) = p ( μ | φ, Y ) p ( φ | Y ) = p ( μ | Y ) p ( φ | μ, Y ) First Factorization For μ | φ, Y complete the square to show that μ | φ, Y ∼ N n φ ¯ y + p m n φ + p , ( n φ + p )- 1 Can we recognize the marginal distribution for φ ? No! Second Factorization p ( μ, φ | Y ) ∝ φ n / 2 exp- φ 2 s 2 ( n- 1) φ ν / 2- 1 exp(- φν s 2 ) · exp- φ 2 n (¯ y- μ ) 2 exp( p ( μ- m ) 2 ) Can recognize φ | μ, Y ∼ G n + ν 2 , ( n- 1) s 2 + n (¯ y- μ ) 2 + ν s 2 2 Sampling? I Suppose we were given a value of μ that comes from the marginal posterior distribution (say μ (1) = ¯ y ) I We could then draw a value of φ from the conditional Gamma distribution given μ = μ (1) which would give us a draw from the joint distribution ( μ (1) , φ (1) ) I φ (1) could be viewed as a draw from the marginal distribution (based on the first factorization), so if we now use the conditional distribution of μ | φ, Y to draw a new μ (2) we have another sample from the joint distribution....
View Full Document

{[ snackBarMessage ]}

### Page1 / 15

intro_Gibbs - Introduction to Gibbs Sampling October 8 2010...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online