MIT15_097S12_lec15

M ni mm pwz i1 j 1 m ni mm i1 j 1 m

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: richlet(β ), where β ∈ JW is the prior hyperparameter. 3. For document i = 1, . . . , m: For word j = 1, . . . , ni : Choose the topic for this word zi,j ∼ Multinomial(θi ). Choose the word wi,j ∼ Multinomial(φzi,j ). 5.2 Graphical representation Hierarchical models are illustrated with a node for every variable and arcs between nodes to indicate the dependence between variables. Here’s the one for LDA: Graphs representing hierarchical models must be acyclic. For any node x, we deﬁne Parents(x) as the set of all nodes with arcs to x. The hierarchical model consists of, for every node x, the distribution p(x|Parents(x)). Deﬁne Descendants(x) as all nodes that can be reached from x and Non-descendants(x) as all other nodes. Because the graph is acyclic and the distribution for each node depends only on its parents, given Parents(x), x is conditionally inde­ pendent from Non-descendants(x). This is a powerful fact about hierarchical models that is important for doing inference. In the graph for LDA, this means that, for example, zi,j is independent of α, given θi...
View Full Document

This note was uploaded on 03/24/2014 for the course MIT 15.097 taught by Professor Cynthiarudin during the Spring '12 term at MIT.

Ask a homework question - tutors are online