Unformatted text preview: CS440/ECE448: Intro to Artiﬁcial Intelligence! Lecture 17:
More on exact
inference in Bayes
Nets!
Prof. Julia Hockenmaier!
[email protected]!
!
http://cs.illinois.edu/fa11/cs440!
!
! D=lo D=hi I=lo I=hi 0.6 0.7 0.3 0.4 Intelligence! Difﬁculty! G=A G=B G=C I=lo,D=lo 0.3 0.4 SAT! Grade! SAT=lo SAT=hi I=lo 0.95 I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.3 0.8 0.08 0.02 I=hil,D=hi 0.5 0.05 I=hi 0.2 0.3 0.2 Letter!
L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 CS440/ECE448: Intro AI! 2! D=lo D=hi 0.6 0.4 Intelligence! Difﬁculty!
G=A G=B G=C I=lo,D=lo 0.3 0.4 0.3 Grade! SAT! I=lo,D=hi 0.05 0.25 0.7 I=hi, D=lo 0.9 0.08 0.02 I=hil,D=hi 0.5 0.3 0.2 Letter! L=w L=s G=A 0.1 0.9 G=B 0.4 0.6 G=C 0.99 0.01 What is the probability of getting a strong
letter if you are an intelligent student?!
CS440/ECE448: Intro AI! 3! Review: inference in Bayes Nets"
What is the probability of getting a strong letter
if you are an intelligent student?
!
! P( X = x  e)
P( X = x  e) =
P (e ) – We want to compute the probability of a set of
query variables X (= the letter) given an event e
(= being intelligent)!
– An event = an assignment of values to a set of
evidence variables E (= intelligence)!
CS440/ECE448: Intro AI! 4! Review: conditional probability"
! P ( X  Y ) = P ( X, Y )
!
P(Y )
!
P( Y ) can be computed by marginalization:!
!
P ( X = x, Y = y )
! P (Y = y ) =
x
!
If we can compute P(X,Y ), we can also
compute P(X  Y )!
!
CS440/ECE448: Intro AI! ! 5! Computing inferences
in Bayes Nets"
What is the probability of getting a strong
letter if you are an intelligent student?!
!
What about the other, hidden, variables h ? !
We have to marginalize them out.!
!
P ( X = x, E = e) = ! P ( x, h, e)
!
H =h CS440/ECE448: Intro AI! 6! The Burglary example"
The alarm can go off if there is a burglary or if
there is an earthquake.!
If the alarm goes off, John might call.!
If the alarm goes off, Mary might call.!
Burglary! Earthquake!
Alarm! JohnCalls!
CS440/ECE448: Intro AI! MaryCalls!
7! The Burglary example"
B=t Burglary! B=f .001 0.999 E=t Earthquake! E=f .002 0.998 B A J=t J=f t .9 .1 f .05 .95 t .95 .05 t MaryCalls! A=f f .94 .06 f JohnCalls! A=t t Alarm! E t .29 .71 f f .999 .001 A M=t M=f t .7 .3 f .01 .99 What is the probability of a burglary if John and Mary call?!
CS440/ECE448: Intro AI! 8! Inference by enumeration"
What is the probability of a burglary if John
and Mary call?!
!
P (b, j, m )
P (b  j, m ) =
!
P ( j, m )
!
What is the joint probability P(b,j,m)?! P (b, j, m ) = !! P (b, j, m, e, a )
e
CS440/ECE448: Intro AI! a 9! Inference by enumeration"
What is the joint probability P(b,j,m,e,a)?!
!
P (b, j, m, a, e)
!
= P (b)P (e)P (a  b, e)P ( j  a )P (m  a )
!
!
So, ! P (b, j, m ) = !! P (e)P (b)P (a  b, e)P ( j  a )P (m  a )
e a 5 terms!
4 summands = 20 computations! CS440/ECE448: Intro AI! 10! Inference by enumeration" !! P(e)P(b)P(a  b, e)P( j  a)P(m  a)
e a = P (b)!! P (e)P (a  b, e)P ( j  a )P (m  a )
e a = P (b)! P (e)! P (a  b, e)P ( j  a )P (m  a )
e a 3 terms!
2 summands: 6 computations! 2 terms: 7 = 1 + 6 computations!
2 summands: 14 computations!
2 terms: 15 = 1+ 14 computations!
CS440/ECE448: Intro AI! 11! Or, visually….:"
P(b)
.001 P(e)
.002 P(¬ab,e)
.05 P(ab,e)
.95 P(¬e)
.998
P(ab,¬e)
.94 P(¬ab,¬e)
.06 P(ja)
.90 P( j¬a)
.05 P( ja)
.90 P( j¬a)
.05 P(ma)
.70 P(m¬a)
.01 P(ma)
.70 P(m¬a)
.01 CS440/ECE448: Intro AI! 12! Enumeration is inefﬁcient…"
… it repeatedly evaluates the same
subexpressions. !
Time complexity for n Boolean variables: O(2 n)! !
Insights:"
… itʼs better to evaluate each subexpression
only once and save the result!
… often, the number of subexpressions will
be relatively small!
!
CS440/ECE448: Intro AI!
13!
! Variable elimination"
We want to compute!
!
P (b, j, m, a, e)
!
= P (b)! P (e)! P (a  b, e)P ( j  a )P (m  a )
!
e
a
!
– We have ordered the terms so that when we
sum over a variable, we sum only over terms
that depend on this variable.!
– Now we evaluate this equation in righttoleft
order and store intermediate results!
CS440/ECE448: Intro AI! 14! Factors"
Variable elimination stores intermediate
results in factors.!
!
A factor is a table of probability distributions
that is indexed by the variables of interest. ! CS440/ECE448: Intro AI! 15! Factor graphs"
We can also draw each Bayes Net as a factor graph:! Earthquake! Burglary! Alarm! JohnCalls!
CS440/ECE448: Intro AI! MaryCalls!
16! Factor graphs"
Each CPT is one factor (= black box)! Earthquake! Burglary! Alarm! JohnCalls!
CS440/ECE448: Intro AI! MaryCalls!
17! Factor graphs"
The factor graph is a hypergraph! Earthquake! Burglary! Alarm! JohnCalls!
CS440/ECE448: Intro AI! MaryCalls!
18! As factors"
P (b, j, m, a, e)
= P (b)! P (e)! P (a  b, e)P ( j  a )P (m  a )
e a = f 1( B )! f 2( E )! f 3( A, B, E ) f 4( A ) f 5( A )
e CS440/ECE448: Intro AI! a 19! Evaluate the innermost sum"
f6(B,E) = " f3(A,B,E) ! f4(A) ! f5(A)
a = f3(a,B,E) ! f4(a) ! f5(a)
+ f3(¬a,B,E) ! f4(¬a) ! f5(¬a)
P (b, j, m, a, e)
= f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A)
e a = f1(B) ! " f2(E) ! f6(B,E)
e
CS440/ECE448: Intro AI! 20! So…"
P (b, j, m, a, e)
= f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A)
e a = f1(B) ! " f2(E) ! f6(B,E)
e CS440/ECE448: Intro AI! 21! Evaluate the next sum"
f7(B)=" f2(E) ! f6(B,E)
e P (b, j, m, a, e)
= f1(B) ! " f2(E) ! " f3(A,B,E) ! f4(A) ! f5(A)
e a = f1(B) ! " f2(E) ! f6(B,E)
e = f1(B) ! f7(B)
CS440/ECE448: Intro AI! 22! Complexity of variable
elimination"
Polytree networks:"
only one (undirected) path between each
pair of nodes.!
!
Complexity is linear in the size of the
network (number of CPT entries)!
! CS440/ECE448: Intro AI! 23! Complexity of variable
elimination"
Multiply connected networks:"
multiple (undirected) paths between some
pair of nodes.!
!
Complexity is exponential in the size of the
network (number of CPT entries)!
– (Similar to ﬁnding number of models in
propositional logic)! CS440/ECE448: Intro AI! 24! ...
View
Full Document
 Spring '08
 Levinson,S
 Conditional Probability, Burglary, Bayesian network, Intro AI

Click to edit the document details