Lecture17 - CS440/ECE448 Intro to Artificial Intelligence...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CS440/ECE448: Intro to Artificial Intelligence! Lecture 17:
 More on exact inference in Bayes Nets! Prof. Julia Hockenmaier! [email protected]! ! http://cs.illinois.edu/fa11/cs440! ! ! D=lo D=hi I=lo I=hi 0.6 0.7 0.3 0.4 Intelligence! Difficulty! G=A G=B G=C I=lo,D=lo 0.3 0.4 SAT! Grade! SAT=lo SAT=hi I=lo 0.95 I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.3 0.8 0.08 0.02 I=hil,D=hi 0.5 0.05 I=hi 0.2 0.3 0.2 Letter! L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 CS440/ECE448: Intro AI! 2! D=lo D=hi 0.6 0.4 Intelligence! Difficulty! G=A G=B G=C I=lo,D=lo 0.3 0.4 0.3 Grade! SAT! I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.08 0.02 I=hil,D=hi 0.5 0.3 0.2 Letter! L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 What is the probability of getting a strong letter if you are an intelligent student?! CS440/ECE448: Intro AI! 3! Review: inference in Bayes Nets" What is the probability of getting a strong letter
 if you are an intelligent student?
 
 ! ! P( X = x | e) P( X = x | e) = P (e ) –  We want to compute the probability of a set of query variables X (= the letter) given an event e (= being intelligent)! –  An event = an assignment of values to a set of evidence variables E (= intelligence)! CS440/ECE448: Intro AI! 4! Review: conditional probability" ! P ( X | Y ) = P ( X, Y ) ! P(Y ) ! P( Y ) can be computed by marginalization:! ! P ( X = x, Y = y ) ! P (Y = y ) = x ! If we can compute P(X,Y ), we can also compute P(X | Y )! ! CS440/ECE448: Intro AI! ! 5! Computing inferences 
 in Bayes Nets" What is the probability of getting a strong letter if you are an intelligent student?! ! What about the other, hidden, variables h ? ! We have to marginalize them out.! ! P ( X = x, E = e) = ! P ( x, h, e) ! H =h CS440/ECE448: Intro AI! 6! The Burglary example" The alarm can go off if there is a burglary or if there is an earthquake.! If the alarm goes off, John might call.! If the alarm goes off, Mary might call.! Burglary! Earthquake! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 7! The Burglary example" B=t Burglary! B=f .001 0.999 E=t Earthquake! E=f .002 0.998 B A J=t J=f t .9 .1 f .05 .95 t .95 .05 t MaryCalls! A=f f .94 .06 f JohnCalls! A=t t Alarm! E t .29 .71 f f .999 .001 A M=t M=f t .7 .3 f .01 .99 What is the probability of a burglary if John and Mary call?! CS440/ECE448: Intro AI! 8! Inference by enumeration" What is the probability of a burglary if John and Mary call?! ! P (b, j, m ) P (b | j, m ) = ! P ( j, m ) ! What is the joint probability P(b,j,m)?! P (b, j, m ) = !! P (b, j, m, e, a ) e CS440/ECE448: Intro AI! a 9! Inference by enumeration" What is the joint probability P(b,j,m,e,a)?! ! P (b, j, m, a, e) ! = P (b)P (e)P (a | b, e)P ( j | a )P (m | a ) ! ! So, ! P (b, j, m ) = !! P (e)P (b)P (a | b, e)P ( j | a )P (m | a ) e a 5 terms! 4 summands = 20 computations! CS440/ECE448: Intro AI! 10! Inference by enumeration" !! P(e)P(b)P(a | b, e)P( j | a)P(m | a) e a = P (b)!! P (e)P (a | b, e)P ( j | a )P (m | a ) e a = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) e a 3 terms! 2 summands: 6 computations! 2 terms: 7 = 1 + 6 computations! 2 summands: 14 computations! 2 terms: 15 = 1+ 14 computations! CS440/ECE448: Intro AI! 11! Or, visually….:" P(b) .001 P(e) .002 P(¬a|b,e) .05 P(a|b,e) .95 P(¬e) .998 P(a|b,¬e) .94 P(¬a|b,¬e) .06 P(j|a) .90 P( j|¬a) .05 P( j|a) .90 P( j|¬a) .05 P(m|a) .70 P(m|¬a) .01 P(m|a) .70 P(m|¬a) .01 CS440/ECE448: Intro AI! 12! Enumeration is inefficient…" … it repeatedly evaluates the same subexpressions. ! Time complexity for n Boolean variables: O(2 n)! ! Insights:" … itʼs better to evaluate each subexpression only once and save the result! … often, the number of subexpressions will be relatively small! ! CS440/ECE448: Intro AI! 13! ! Variable elimination" We want to compute! ! P (b, j, m, a, e) ! = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) ! e a ! –  We have ordered the terms so that when we sum over a variable, we sum only over terms that depend on this variable.! –  Now we evaluate this equation in right-to-left order and store intermediate results! CS440/ECE448: Intro AI! 14! Factors" Variable elimination stores intermediate results in factors.! ! A factor is a table of probability distributions
 that is indexed by the variables of interest. ! CS440/ECE448: Intro AI! 15! Factor graphs" We can also draw each Bayes Net as a factor graph:! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 16! Factor graphs" Each CPT is one factor (= black box)! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 17! Factor graphs" The factor graph is a hypergraph! Earthquake! Burglary! Alarm! JohnCalls! CS440/ECE448: Intro AI! MaryCalls! 18! As factors" P (b, j, m, a, e) = P (b)! P (e)! P (a | b, e)P ( j | a )P (m | a ) e a = f 1( B )! f 2( E )! f 3( A, B, E ) f 4( A ) f 5( A ) e CS440/ECE448: Intro AI! a 19! Evaluate the innermost sum" f6(B,E) = " f3(A,B,E) ! f4(A) ! f5(A) a = f3(a,B,E) ! f4(a) ! f5(a) + f3(¬a,B,E) ! f4(¬a) ! f5(¬a) P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e CS440/ECE448: Intro AI! 20! So…" P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e CS440/ECE448: Intro AI! 21! Evaluate the next sum" f7(B)=" f2(E) ! f6(B,E) e P (b, j, m, a, e) = f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A) e a = f1(B) ! " f2(E) ! f6(B,E) e = f1(B) ! f7(B) CS440/ECE448: Intro AI! 22! Complexity of variable elimination" Polytree networks:" only one (undirected) path between each pair of nodes.! ! Complexity is linear in the size of the network (number of CPT entries)! ! CS440/ECE448: Intro AI! 23! Complexity of variable elimination" Multiply connected networks:" multiple (undirected) paths between some pair of nodes.! ! Complexity is exponential in the size of the network (number of CPT entries)! –  (Similar to finding number of models in propositional logic)! CS440/ECE448: Intro AI! 24! ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online