Unformatted text preview: e
are able to draw samples from the conditional distribution
p(θj θ1 , . . . , θj −1 , θj +1 , . . . , θd , y ).
In fact, it is often the case in hierarchical models that their structure al
lows us to determine analytically these conditional posterior distributions.
This can be done for LDA, and the derivation is quite lengthy but can
be found on the Wikipedia article for LDA. The Gibbs’ sampler updates
the posterior variables θ1 , θ2 , . . . , θd one at a time. At each step, all of
them are held constant in their current state except for one (j ), which is
then updated by drawing from its (known) conditional posterior distribu
tion, p(θj θ1 , . . . , θj −1 , θj +1 , . . . , θd , y ). We then hold it in its state and move
on to the next variable to update it similarly. When we do this iterative up
dating for long enough, we eventually simulate draws from the full posterior.
The full algorithm is:
0
0
Step 1. Initialize θ0 = [θ1 , . . . , θd ]. Set t = 1.
t
t
t
t
t
Step 2. For j ∈ {1, . . . , d}, sample θj from p...
View
Full
Document
This note was uploaded on 03/24/2014 for the course MIT 15.097 taught by Professor Cynthiarudin during the Spring '12 term at MIT.
 Spring '12
 CynthiaRudin

Click to edit the document details