MIT15_097S12_lec15

642 on reaching the stationary distribution

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: e are able to draw samples from the conditional distribution p(θj |θ1 , . . . , θj −1 , θj +1 , . . . , θd , y ). In fact, it is often the case in hierarchical models that their structure al­ lows us to determine analytically these conditional posterior distributions. This can be done for LDA, and the derivation is quite lengthy but can be found on the Wikipedia article for LDA. The Gibbs’ sampler updates the posterior variables θ1 , θ2 , . . . , θd one at a time. At each step, all of them are held constant in their current state except for one (j ), which is then updated by drawing from its (known) conditional posterior distribu­ tion, p(θj |θ1 , . . . , θj −1 , θj +1 , . . . , θd , y ). We then hold it in its state and move on to the next variable to update it similarly. When we do this iterative up­ dating for long enough, we eventually simulate draws from the full posterior. The full algorithm is: 0 0 Step 1. Initialize θ0 = [θ1 , . . . , θd ]. Set t = 1. t t t t t Step 2. For j ∈ {1, . . . , d}, sample θj from p...
View Full Document

This note was uploaded on 03/24/2014 for the course MIT 15.097 taught by Professor Cynthiarudin during the Spring '12 term at MIT.

Ask a homework question - tutors are online