lec4 - CSci 5512: Artificial Intelligence II Instructor:...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CSci 5512: Artificial Intelligence II Instructor: Arindam Banerjee January 30, 2012 Instructor: Arindam Banerjee 9(2171: 21:2: 2173., 21:4: 21:5) : j";(:i:1)j'B(:1:2)va(:i:1,1:2,:i:3)fU(:i:3,:i:4)ffl(:t:3::1:5) @ @ @\@ @ fA f3 f0 f1) fie; o Bipartite graph over variables and local functions 0 Edge E “is an argument of" relation 0 Encodes an efficient algorithm Instructor: Arindam Banerjee — Factor Graphs 0 Many problems deal with global function of many variables 0 Global function “factors” into product of local functions 0 Efficient algorithms take advantage of such factorization o Factorization can be visualized as a factor graph Instructor: Arindam Banerjee Bayes Nets to Factor Graphs @ Q Q @@ Q fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) fE(X3,X5) = P(X5IX3) fD(X3,X4) = P(X4IX3) Instructor: Arindam Banerjee Marginalize Product of Functions MPF using Distributive Law 0 We focus on two examples: g1(x1) and g3(X3) 0 Many problems involve “marginalize product of functions” ° From dlstr'butlve law (MPF) o Inference in Bayesian networks :- Compute p(X1|X4,X5) = £4(X1)Z (fB(X2)fC(X1,X2,X3) fD(X3:X4)) fE(X3,X5)>) g1 (X1) 0 Need to compute p(x1,x4,x_r,) and p(X4,X5) ~x1 o Marginalization of joint distribution is a MPF problem 0 Several other problems use MPF 0 Also a Prediction/Filtering in dynamic Bayes nets . Viterbi decoding in hidden Markov models g3(x3) :- Error correcting codes = fA(X1)fB(X2)f-C(XlaX21X3)> fD(X3,X4)) fE(X3,X5)) '"SWCW Arindam Banedee Marginalize Product of Functions (Contd.) Message Passing Example: Computing g1(x1) o The “not-sum” notation f.i(J:1)>< Zorn} (foo-o >< f(-‘(I'1:f2:£3) X (2%“) Maw-4)) >< (ZNMJ‘E(£3¢5))) Z h(X19X2,X3) = Z h(X13X23X3) ~x2 X1,X3 /\ Jj1 ' 0 Recall J14 ,T , ,3 g(X1,X2,X3,X4,X5) = fA(X1)fB(X2)fC(X1,X2aX3)fD(X3aX4)fE(X3aX5) ‘19 1'3 JB J'D Jo 0 Computing marginal function using not—sum notations gi(Xi) = Zg(X15XZaX3aX4aX5) NXI' Instructor: Arindam Banerjee Instructor: Arindam Banerjee Message Passing Example: Computing g2(X2) Sum-Product Algorithm 93(x3) : (Z ri(mfu(x2)rnm:m:m) >< ( Z 15(x3:x4))><(2 rho-3: m) Nth} mm with) o The overall strategy is simple message passing 0 To compute g;(x,-), form a rooted tree at X,- 0 Apply the following two rules: 0 Product Rule: At a variable node, take the product of descendants o Sum-product Rule: At a factor node, take the product of f with descendants; then perform not—sum over the parent of f 0 Known as the sum-product algorithm Instructor: Arindam Banerjee Instructor: Arindam Banerjee — Local Transformation for Message Passing Computing All Marginals TO Parent :1: To Parent 0 Interested in computing all marginal functions gi(x,-) l :> (>8 for From Children From Children 0 One option is to repeat the sum-product for every single node 0 Complexity of 0(n2) 0 Repeat computations can be avoided o Sum-product algorithm for general trees Instructor: Arindam Banerjee Instructor: Arindam Banerjee Sum Product Updates Example: Step 2 ‘"'fil(I)\{f} 0 Variable to local function: IU’X1—)fc (X1) : MfA—)X1 (X1) [ix—“()0 = H lib—IX I‘X2—>fc(x2) = life—MAX?) h€n(x)\f 0 Local function to variable: p’fD—)X3(X3) = Z IrD(X3aX4).MX4—>fo (X4) ~x3 “MM = 2 (fix) H Hymn) wadxs) = Z fo<xs,xs)wafE(Xs) NX y€n(f)\{x} ~X3 Instructor: Arindam Banerjee Instructor: Arindam Banerjee — — Example: Step 1 Example: Step 3 MfA—)X1 (X1) = fA(X1) = f ursaxxxz) = fB(x2) Mcaxalxs) C(Xl,X2,xsmxflfAnmxHsz) p’X4—H‘D (X4) — 1 MX3_H¢-C = MFDHXE’ (X3)#fE_>X3 MX5—>fE (X5) — 1 Instructor: Arindam Banerjee Instructor: Arindam Banerjee Example: Step 4 Example: Termination Marginal function is the product of all incoming messages ufc—m (X1) = Z fc(X1aX2: X3)iu’X2—>fc (X2).U'X3—>fc (X3) ~x1 g1 (X1) = MFA—m1 (X1)Mfc—>x1 (X1) ,qu_)X2 (X2) = Z fC(X1,X2,X3)lLX1_)fC (X1)}.LX3_H€C (X3) g2 (X2) = FIFE—W2 (X2)/J'fC—>X2 (X2) Nxz 33 (X3) = MfC—>X3 (X3)/~“fD—>X3 (X3):u’fE—*X3 (X3) MX3—>fD (X3) = :u'fc—>X3 (X3):U'fE—>X3 (X3) g2 (X2) = “firm (X4) :uxa—H‘E (X3) = lufc—ma (X3),U'fD—>xa (X3) g5(X5) = praxs (X5) Instructor: Arindam Banerjee Instructor: Arindam Banerjee — Example: Step 5 Belief Propagation in Bayes Nets @ Q Q Q Q @ Mx1—>fA(X1) = :ufc—>X1(X1) @ a Q Q IU’XZ —>f3 (X2) = lu’fC —>X2 (X2) Mfg—mm) = :fD(X3aX4)/1'X3—>fp(x4) ~X4 fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) [WE—Ms (X5) 2 IrD(X3’ X5)MX3—>f5 (X5) ~X5 fD(X3,X4) = P(X4IX3) fE(X3,X5) = P(X5IX3) Instructor: Arindam Banerjee Instructor: Arindam Banerjee ...
View Full Document

This note was uploaded on 02/07/2012 for the course CSCI 5512 taught by Professor Staff during the Spring '08 term at Minnesota.

Page1 / 5

lec4 - CSci 5512: Artificial Intelligence II Instructor:...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online