This preview shows page 1. Sign up to view the full content.
Unformatted text preview: richlet(β ), where β ∈ JW is the prior hyperparameter.
3. For document i = 1, . . . , m:
For word j = 1, . . . , ni :
Choose the topic for this word zi,j ∼ Multinomial(θi ).
Choose the word wi,j ∼ Multinomial(φzi,j ).
5.2 Graphical representation Hierarchical models are illustrated with a node for every variable and arcs
between nodes to indicate the dependence between variables. Here’s the one
for LDA: Graphs representing hierarchical models must be acyclic. For any node x,
we deﬁne Parents(x) as the set of all nodes with arcs to x. The hierarchical
model consists of, for every node x, the distribution p(x|Parents(x)). Deﬁne
Descendants(x) as all nodes that can be reached from x and Non-descendants(x)
as all other nodes. Because the graph is acyclic and the distribution for each
node depends only on its parents, given Parents(x), x is conditionally inde
pendent from Non-descendants(x). This is a powerful fact about hierarchical
models that is important for doing inference. In the graph for LDA, this
means that, for example, zi,j is independent of α, given θi...
View Full Document
This note was uploaded on 03/24/2014 for the course MIT 15.097 taught by Professor Cynthiarudin during the Spring '12 term at MIT.
- Spring '12