# lec4 - CSci 5512 Artificial Intelligence II Instructor...

This preview shows pages 1–5. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CSci 5512: Artificial Intelligence II Instructor: Arindam Banerjee January 30, 2012 Instructor: Arindam Banerjee 9(2171: 21:2: 2173., 21:4: 21:5) : j";(:i:1)j'B(:1:2)va(:i:1,1:2,:i:3)fU(:i:3,:i:4)fﬂ(:t:3::1:5) @ @ @\@ @ fA f3 f0 f1) fie; o Bipartite graph over variables and local functions 0 Edge E “is an argument of" relation 0 Encodes an efficient algorithm Instructor: Arindam Banerjee — Factor Graphs 0 Many problems deal with global function of many variables 0 Global function “factors” into product of local functions 0 Efficient algorithms take advantage of such factorization o Factorization can be visualized as a factor graph Instructor: Arindam Banerjee Bayes Nets to Factor Graphs @ Q Q @@ Q fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) fE(X3,X5) = P(X5IX3) fD(X3,X4) = P(X4IX3) Instructor: Arindam Banerjee Marginalize Product of Functions MPF using Distributive Law 0 We focus on two examples: g1(x1) and g3(X3) 0 Many problems involve “marginalize product of functions” ° From dlstr'butlve law (MPF) o Inference in Bayesian networks :- Compute p(X1|X4,X5) = £4(X1)Z (fB(X2)fC(X1,X2,X3) fD(X3:X4)) fE(X3,X5)>) g1 (X1) 0 Need to compute p(x1,x4,x_r,) and p(X4,X5) ~x1 o Marginalization of joint distribution is a MPF problem 0 Several other problems use MPF 0 Also a Prediction/Filtering in dynamic Bayes nets . Viterbi decoding in hidden Markov models g3(x3) :- Error correcting codes = fA(X1)fB(X2)f-C(XlaX21X3)> fD(X3,X4)) fE(X3,X5)) '"SWCW Arindam Banedee Marginalize Product of Functions (Contd.) Message Passing Example: Computing g1(x1) o The “not-sum” notation f.i(J:1)>< Zorn} (foo-o >< f(-‘(I'1:f2:£3) X (2%“) Maw-4)) >< (ZNMJ‘E(£3¢5))) Z h(X19X2,X3) = Z h(X13X23X3) ~x2 X1,X3 /\ Jj1 ' 0 Recall J14 ,T , ,3 g(X1,X2,X3,X4,X5) = fA(X1)fB(X2)fC(X1,X2aX3)fD(X3aX4)fE(X3aX5) ‘19 1'3 JB J'D Jo 0 Computing marginal function using not—sum notations gi(Xi) = Zg(X15XZaX3aX4aX5) NXI' Instructor: Arindam Banerjee Instructor: Arindam Banerjee Message Passing Example: Computing g2(X2) Sum-Product Algorithm 93(x3) : (Z ri(mfu(x2)rnm:m:m) >< ( Z 15(x3:x4))><(2 rho-3: m) Nth} mm with) o The overall strategy is simple message passing 0 To compute g;(x,-), form a rooted tree at X,- 0 Apply the following two rules: 0 Product Rule: At a variable node, take the product of descendants o Sum-product Rule: At a factor node, take the product of f with descendants; then perform not—sum over the parent of f 0 Known as the sum-product algorithm Instructor: Arindam Banerjee Instructor: Arindam Banerjee — Local Transformation for Message Passing Computing All Marginals TO Parent :1: To Parent 0 Interested in computing all marginal functions gi(x,-) l :> (>8 for From Children From Children 0 One option is to repeat the sum-product for every single node 0 Complexity of 0(n2) 0 Repeat computations can be avoided o Sum-product algorithm for general trees Instructor: Arindam Banerjee Instructor: Arindam Banerjee Sum Product Updates Example: Step 2 ‘"'ﬁl(I)\{f} 0 Variable to local function: IU’X1—)fc (X1) : MfA—)X1 (X1) [ix—“()0 = H lib—IX I‘X2—>fc(x2) = life—MAX?) h€n(x)\f 0 Local function to variable: p’fD—)X3(X3) = Z IrD(X3aX4).MX4—>fo (X4) ~x3 “MM = 2 (ﬁx) H Hymn) wadxs) = Z fo<xs,xs)wafE(Xs) NX y€n(f)\{x} ~X3 Instructor: Arindam Banerjee Instructor: Arindam Banerjee — — Example: Step 1 Example: Step 3 MfA—)X1 (X1) = fA(X1) = f ursaxxxz) = fB(x2) Mcaxalxs) C(Xl,X2,xsmxﬂfAnmxHsz) p’X4—H‘D (X4) — 1 MX3_H¢-C = MFDHXE’ (X3)#fE_>X3 MX5—>fE (X5) — 1 Instructor: Arindam Banerjee Instructor: Arindam Banerjee Example: Step 4 Example: Termination Marginal function is the product of all incoming messages ufc—m (X1) = Z fc(X1aX2: X3)iu’X2—>fc (X2).U'X3—>fc (X3) ~x1 g1 (X1) = MFA—m1 (X1)Mfc—>x1 (X1) ,qu_)X2 (X2) = Z fC(X1,X2,X3)lLX1_)fC (X1)}.LX3_H€C (X3) g2 (X2) = FIFE—W2 (X2)/J'fC—>X2 (X2) Nxz 33 (X3) = MfC—>X3 (X3)/~“fD—>X3 (X3):u’fE—*X3 (X3) MX3—>fD (X3) = :u'fc—>X3 (X3):U'fE—>X3 (X3) g2 (X2) = “ﬁrm (X4) :uxa—H‘E (X3) = lufc—ma (X3),U'fD—>xa (X3) g5(X5) = praxs (X5) Instructor: Arindam Banerjee Instructor: Arindam Banerjee — Example: Step 5 Belief Propagation in Bayes Nets @ Q Q Q Q @ Mx1—>fA(X1) = :ufc—>X1(X1) @ a Q Q IU’XZ —>f3 (X2) = lu’fC —>X2 (X2) Mfg—mm) = :fD(X3aX4)/1'X3—>fp(x4) ~X4 fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) [WE—Ms (X5) 2 IrD(X3’ X5)MX3—>f5 (X5) ~X5 fD(X3,X4) = P(X4IX3) fE(X3,X5) = P(X5IX3) Instructor: Arindam Banerjee Instructor: Arindam Banerjee ...
View Full Document

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern