*This preview shows
pages
1–9. Sign up
to
view the full content.*

This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*This
** preview**
has intentionally

**sections.**

*blurred***to view the full version.**

*Sign up*
**Unformatted text preview: **Machine Learning Srihari 1 1 Factor Graphs and Inference Sargur Srihari [email protected] Machine Learning Srihari 2 2 Topics 1. Factor Graphs 1. Factors in probability distributions 2. Deriving them from graphical models 2. Exact Inference Algorithms for Tree graphs 1. The sum-product algorithm 1. For finding marginal probabilities 2. The max-sum algorithm 1. For setting of variables that maximizes a probability 2. For value of that probability 3. Exact inference in general graphs Machine Learning Srihari 3 Factors in Joint Distributions • Joint distribution written as – Where x s is a subset of the variables • Special cases: – Directed graphs: f s ( x s ) are conditional distributions – Undirected graphs : • factors are potential functions over maximal cliques ) x ( ) x ( s s s f p ∏ = p ( x ) = p ( x k | pa k ) k = 1 K ∏ p ( x ) = 1 Z ψ C C ∏ ( x C ) p(a,b,c)=p(c|a,b)p(b|a)p(a) P ( A , B , C , D ) α exp − ε 1 ( A , B ) − ε 2 ( B , C ) − ε 3 ( C , D ) − ε 4 ( D , A ) [ ] Partition function Z is sum of RHS over all values of A,B,C,D Log-linear Model Machine Learning Srihari 4 4 Factor Graph Motivation • Joint distributions – Can be expressed as product of factors over subsets • Factor graphs make this explicit – By introducing additional nodes for factors • Sum-prod inference algorithm (to be defined) applicable to: – undirected, directed trees, polytrees – Has simple and general form with factor graphs – Used for determining marginal probabilities Machine Learning Srihari 5 5 Factor Graph Example • Factorization p( x )=f a (x 1 ,x 2 )f b (x 1 ,x 2 )f c (x 2 ,x 3 )f d (x 3 ) • Corresponding factor graph Two factors f a and f b of the same variables Machine Learning Srihari 6 6 Factor graphs properties • They are bipartite since 1. Two types of nodes 2. All links go between nodes of opposite type • Representable as two rows of nodes – Variables on top – Factor nodes at bottom • Other intuitive representations used – When derived from directed/ undirected graphs Machine Learning Srihari Deriving factor graphs from Graphical Models • Undirected Graph • Directed Graph 7 Machine Learning Srihari 8 8 Conversion of Undirected to Factor Graph • Steps in converting distribution expressed as undirected graph: 1. Create variable nodes corresponding to nodes in original 2. Create factor nodes for maximal cliques x s 3. Factors f s ( x s ) set equal...

View
Full
Document