# Lecture17 - CS440/ECE448 Intro to Artiﬁcial Intelligence...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CS440/ECE448: Intro to Artiﬁcial Intelligence! Lecture 17:  More on exact inference in Bayes Nets! Prof. Julia Hockenmaier! [email protected]! ! http://cs.illinois.edu/fa11/cs440! ! ! D=lo D=hi I=lo I=hi 0.6 0.7 0.3 0.4 Intelligence! Difﬁculty! G=A G=B G=C I=lo,D=lo 0.3 0.4 SAT! Grade! SAT=lo SAT=hi I=lo 0.95 I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.3 0.8 0.08 0.02 I=hil,D=hi 0.5 0.05 I=hi 0.2 0.3 0.2 Letter! L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 CS440/ECE448: Intro AI! 2! D=lo D=hi 0.6 0.4 Intelligence! Difﬁculty! G=A G=B G=C I=lo,D=lo 0.3 0.4 0.3 Grade! SAT! I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.08 0.02 I=hil,D=hi 0.5 0.3 0.2 Letter! L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 What is the probability of getting a strong letter if you are an intelligent student?! CS440/ECE448: Intro AI! 3! Review: inference in Bayes Nets" What is the probability of getting a strong letter  if you are an intelligent student?    ! ! P( X = x | e) P( X = x | e) = P (e ) –  We want to compute the probability of a set of query variables X (= the letter) given an event e (= being intelligent)! –  An event = an assignment of values to a set of evidence variables E (= intelligence)! CS440/ECE448: Intro AI! 4! Review: conditional probability" ! P ( X | Y ) = P ( X, Y ) ! P(Y ) ! P( Y ) can be computed by marginalization:! ! P ( X = x, Y = y ) ! P (Y = y ) = x ! If we can compute P(X,Y ), we can also compute P(X | Y )! ! CS440/ECE448: Intro AI! ! 5! Computing inferences   in Bayes Nets" What is the probability of getting a strong letter if you are an intelligent student?! ! What about the other, hidden, variables h ? ! We have to marginalize them out.! ! P ( X = x, E = e) = ! P ( x, h, e) ! H =h CS440/ECE448: Intro AI! 6! The Burglary example" The alarm can go off if there is a burglary or if there is an earthquake.! If the alarm goes off, John might call.! If the alarm goes off, Mary might call.! Burglary! Earthquake! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 7! The Burglary example" B=t Burglary! B=f .001 0.999 E=t Earthquake! E=f .002 0.998 B A J=t J=f t .9 .1 f .05 .95 t .95 .05 t MaryCalls! A=f f .94 .06 f JohnCalls! A=t t Alarm! E t .29 .71 f f .999 .001 A M=t M=f t .7 .3 f .01 .99 What is the probability of a burglary if John and Mary call?! CS440/ECE448: Intro AI! 8! Inference by enumeration" What is the probability of a burglary if John and Mary call?! ! P (b, j, m ) P (b | j, m ) = ! P ( j, m ) ! What is the joint probability P(b,j,m)?! P (b, j, m ) = !! P (b, j, m, e, a ) e CS440/ECE448: Intro AI! a 9! Inference by enumeration" What is the joint probability P(b,j,m,e,a)?! ! P (b, j, m, a, e) ! = P (b)P (e)P (a | b, e)P ( j | a )P (m | a ) ! ! So, ! P (b, j, m ) = !! P (e)P (b)P (a | b, e)P ( j | a )P (m | a ) e a 5 terms! 4 summands = 20 computations! CS440/ECE448: Intro AI! 10! Inference by enumeration" !! P(e)P(b)P(a | b, e)P( j | a)P(m | a) e a = P (b)!! P (e)P (a | b, e)P ( j | a )P (m | a ) e a = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) e a 3 terms! 2 summands: 6 computations! 2 terms: 7 = 1 + 6 computations! 2 summands: 14 computations! 2 terms: 15 = 1+ 14 computations! CS440/ECE448: Intro AI! 11! Or, visually….:" P(b) .001 P(e) .002 P(¬a|b,e) .05 P(a|b,e) .95 P(¬e) .998 P(a|b,¬e) .94 P(¬a|b,¬e) .06 P(j|a) .90 P( j|¬a) .05 P( j|a) .90 P( j|¬a) .05 P(m|a) .70 P(m|¬a) .01 P(m|a) .70 P(m|¬a) .01 CS440/ECE448: Intro AI! 12! Enumeration is inefﬁcient…" … it repeatedly evaluates the same subexpressions. ! Time complexity for n Boolean variables: O(2 n)! ! Insights:" … itʼs better to evaluate each subexpression only once and save the result! … often, the number of subexpressions will be relatively small! ! CS440/ECE448: Intro AI! 13! ! Variable elimination" We want to compute! ! P (b, j, m, a, e) ! = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) ! e a ! –  We have ordered the terms so that when we sum over a variable, we sum only over terms that depend on this variable.! –  Now we evaluate this equation in right-to-left order and store intermediate results! CS440/ECE448: Intro AI! 14! Factors" Variable elimination stores intermediate results in factors.! ! A factor is a table of probability distributions  that is indexed by the variables of interest. ! CS440/ECE448: Intro AI! 15! Factor graphs" We can also draw each Bayes Net as a factor graph:! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 16! Factor graphs" Each CPT is one factor (= black box)! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 17! Factor graphs" The factor graph is a hypergraph! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 18! As factors" P (b, j, m, a, e) = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) e a = f 1( B )! f 2( E )! f 3( A, B, E ) f 4( A ) f 5( A ) e CS440/ECE448: Intro AI! a 19! Evaluate the innermost sum" f6(B,E) = " f3(A,B,E) ! f4(A) ! f5(A) a = f3(a,B,E) ! f4(a) ! f5(a) + f3(¬a,B,E) ! f4(¬a) ! f5(¬a) P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e CS440/ECE448: Intro AI! 20! So…" P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e CS440/ECE448: Intro AI! 21! Evaluate the next sum" f7(B)=" f2(E) ! f6(B,E) e P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e = f1(B) ! f7(B) CS440/ECE448: Intro AI! 22! Complexity of variable elimination" Polytree networks:" only one (undirected) path between each pair of nodes.! ! Complexity is linear in the size of the network (number of CPT entries)! ! CS440/ECE448: Intro AI! 23! Complexity of variable elimination" Multiply connected networks:" multiple (undirected) paths between some pair of nodes.! ! Complexity is exponential in the size of the network (number of CPT entries)! –  (Similar to ﬁnding number of models in propositional logic)! CS440/ECE448: Intro AI! 24! ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online