This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: CSci 5512: Artificial Intelligence II Instructor: Arindam Banerjee January 30, 2012 Instructor: Arindam Banerjee 9(2171: 21:2: 2173., 21:4: 21:5) : j";(:i:1)j'B(:1:2)va(:i:1,1:2,:i:3)fU(:i:3,:i:4)fﬂ(:t:3::1:5) @ @ @\@ @
fA f3 f0 f1) fie; o Bipartite graph over variables and local functions
0 Edge E “is an argument of" relation
0 Encodes an efficient algorithm Instructor: Arindam Banerjee —
Factor Graphs 0 Many problems deal with global function of many variables 0 Global function “factors” into product of local functions 0 Efficient algorithms take advantage of such factorization o Factorization can be visualized as a factor graph Instructor: Arindam Banerjee Bayes Nets to Factor Graphs @ Q Q @@ Q fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) fE(X3,X5) = P(X5IX3) fD(X3,X4) = P(X4IX3) Instructor: Arindam Banerjee Marginalize Product of Functions MPF using Distributive Law 0 We focus on two examples: g1(x1) and g3(X3) 0 Many problems involve “marginalize product of functions” ° From dlstr'butlve law (MPF)
o Inference in Bayesian networks : Compute p(X1X4,X5) = £4(X1)Z (fB(X2)fC(X1,X2,X3) fD(X3:X4)) fE(X3,X5)>) g1 (X1) 0 Need to compute p(x1,x4,x_r,) and p(X4,X5) ~x1
o Marginalization of joint distribution is a MPF problem
0 Several other problems use MPF 0 Also
a Prediction/Filtering in dynamic Bayes nets
. Viterbi decoding in hidden Markov models g3(x3) : Error correcting codes = fA(X1)fB(X2)fC(XlaX21X3)> fD(X3,X4)) fE(X3,X5)) '"SWCW Arindam Banedee
Marginalize Product of Functions (Contd.) Message Passing Example: Computing g1(x1)
o The “notsum” notation f.i(J:1)>< Zorn} (fooo >< f(‘(I'1:f2:£3) X (2%“) Maw4)) >< (ZNMJ‘E(£3¢5)))
Z h(X19X2,X3) = Z h(X13X23X3)
~x2 X1,X3 /\
Jj1 '
0 Recall J14
,T , ,3
g(X1,X2,X3,X4,X5) = fA(X1)fB(X2)fC(X1,X2aX3)fD(X3aX4)fE(X3aX5) ‘19 1'3
JB J'D Jo
0 Computing marginal function using not—sum notations gi(Xi) = Zg(X15XZaX3aX4aX5) NXI' Instructor: Arindam Banerjee Instructor: Arindam Banerjee Message Passing Example: Computing g2(X2) SumProduct Algorithm 93(x3) : (Z ri(mfu(x2)rnm:m:m) >< ( Z 15(x3:x4))><(2 rho3: m) Nth} mm with) o The overall strategy is simple message passing 0 To compute g;(x,), form a rooted tree at X, 0 Apply the following two rules: 0 Product Rule:
At a variable node, take the product of descendants o Sumproduct Rule: At a factor node, take the product of f with descendants; then perform not—sum over the parent
of f 0 Known as the sumproduct algorithm Instructor: Arindam Banerjee Instructor: Arindam Banerjee —
Local Transformation for Message Passing Computing All Marginals TO Parent :1: To Parent
0 Interested in computing all marginal functions gi(x,) l :> (>8 for From Children From Children 0 One option is to repeat the sumproduct for every single node 0 Complexity of 0(n2)
0 Repeat computations can be avoided o Sumproduct algorithm for general trees Instructor: Arindam Banerjee Instructor: Arindam Banerjee Sum Product Updates Example: Step 2 ‘"'ﬁl(I)\{f} 0 Variable to local function: IU’X1—)fc (X1) : MfA—)X1 (X1)
[ix—“()0 = H lib—IX I‘X2—>fc(x2) = life—MAX?)
h€n(x)\f
0 Local function to variable: p’fD—)X3(X3) = Z IrD(X3aX4).MX4—>fo (X4)
~x3
“MM = 2 (ﬁx) H Hymn) wadxs) = Z fo<xs,xs)wafE(Xs)
NX y€n(f)\{x} ~X3 Instructor: Arindam Banerjee Instructor: Arindam Banerjee — —
Example: Step 1 Example: Step 3 MfA—)X1 (X1) = fA(X1)
= f
ursaxxxz) = fB(x2) Mcaxalxs) C(Xl,X2,xsmxﬂfAnmxHsz)
p’X4—H‘D (X4) — 1 MX3_H¢C = MFDHXE’ (X3)#fE_>X3 MX5—>fE (X5) — 1 Instructor: Arindam Banerjee Instructor: Arindam Banerjee Example: Step 4 Example: Termination Marginal function is the product of all incoming messages ufc—m (X1) = Z fc(X1aX2: X3)iu’X2—>fc (X2).U'X3—>fc (X3)
~x1 g1 (X1) = MFA—m1 (X1)Mfc—>x1 (X1)
,qu_)X2 (X2) = Z fC(X1,X2,X3)lLX1_)fC (X1)}.LX3_H€C (X3) g2 (X2) = FIFE—W2 (X2)/J'fC—>X2 (X2)
Nxz 33 (X3) = MfC—>X3 (X3)/~“fD—>X3 (X3):u’fE—*X3 (X3)
MX3—>fD (X3) = :u'fc—>X3 (X3):U'fE—>X3 (X3) g2 (X2) = “ﬁrm (X4)
:uxa—H‘E (X3) = lufc—ma (X3),U'fD—>xa (X3) g5(X5) = praxs (X5) Instructor: Arindam Banerjee Instructor: Arindam Banerjee —
Example: Step 5 Belief Propagation in Bayes Nets @ Q Q Q
Q @
Mx1—>fA(X1) = :ufc—>X1(X1) @ a Q Q IU’XZ —>f3 (X2) = lu’fC —>X2 (X2) Mfg—mm) = :fD(X3aX4)/1'X3—>fp(x4)
~X4 fA(X1) = P(X1) fB(X2) = P(X2) fc(X1,X2,X3) = P(X3IX1,X2) [WE—Ms (X5) 2 IrD(X3’ X5)MX3—>f5 (X5)
~X5 fD(X3,X4) = P(X4IX3) fE(X3,X5) = P(X5IX3) Instructor: Arindam Banerjee Instructor: Arindam Banerjee ...
View
Full Document
 Spring '08
 Staff

Click to edit the document details