This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Introduction to Gibbs Sampling October 8, 2010 Readings: Hoff 6 October 7, 2010 Monte Carlo Sampling I We have seen that Monte Carlo sampling is a useful tool for sampling from prior and posterior distributions I By limiting attention to conjugate prior distributions, all models have had tractable posterior distributions so sampling was not really necessary (although convenient) I What if we want to use a nonconjugate prior distribution? I What if we cannot sample from the joint posterior distribution? SemiConjugate Examples Normal sampling model Y i  μ, φ iid ∼ N( μ, 1 φ ) But now assume that μ is independent of φ a priori: μ ∼ N( m , 1 / p ) φ ∼ G ( ν / 2 , ν s 2 / 2) Posterior Distribution: p ( μ, φ  Y ) ∝ Y j p ( y i  μ, φ ) p ( μ ) p ( φ ) Factorization of Joint p ( μ, φ  Y ) ∝ φ n / 2 exp φ 2 s 2 ( n 1) φ ν / 2 1 exp( φν s 2 ) · exp φ 2 n (¯ y μ ) 2 exp( p ( μ m ) 2 ) = p ( μ  φ, Y ) p ( φ  Y ) = p ( μ  Y ) p ( φ  μ, Y ) First Factorization For μ  φ, Y complete the square to show that μ  φ, Y ∼ N n φ ¯ y + p m n φ + p , ( n φ + p ) 1 Can we recognize the marginal distribution for φ ? No! Second Factorization p ( μ, φ  Y ) ∝ φ n / 2 exp φ 2 s 2 ( n 1) φ ν / 2 1 exp( φν s 2 ) · exp φ 2 n (¯ y μ ) 2 exp( p ( μ m ) 2 ) Can recognize φ  μ, Y ∼ G n + ν 2 , ( n 1) s 2 + n (¯ y μ ) 2 + ν s 2 2 Sampling? I Suppose we were given a value of μ that comes from the marginal posterior distribution (say μ (1) = ¯ y ) I We could then draw a value of φ from the conditional Gamma distribution given μ = μ (1) which would give us a draw from the joint distribution ( μ (1) , φ (1) ) I φ (1) could be viewed as a draw from the marginal distribution (based on the first factorization), so if we now use the conditional distribution of μ  φ, Y to draw a new μ (2) we have another sample from the joint distribution....
View
Full Document
 Fall '09
 Conditional Probability, 175, Markov chain Monte Carlo, Posterior probability

Click to edit the document details