lecture18-annotated - Machine Learning 10-701/15-781, Fall...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Eric Xing © Eric Xing @ CMU, 2006-2008 1 Machine Learning Machine Learning 10 10 -701/15 701/15 -781, Fall 2008 781, Fall 2008 Bayesian Network II Bayesian Network II Inference Inference Eric Xing Eric Xing Lecture 18, November 12, 2008 Reading: Chap. 8, C.B book X1 X2 X3 X4 X5 X6 X7 X8 Visit to Asia Tuberculosis Tuberculosis or Cancer XRay Result Dyspnea Bronchitis Lung Cancer Smoking Eric Xing © Eric Xing @ CMU, 2006-2008 2 ) , , , , | ( ) , , , | ( ) , , | ( ) , | ( ) | ( ) ( = ) , , , , , ( 5 4 3 2 1 6 4 3 2 1 5 3 2 1 4 2 1 3 1 2 1 6 5 4 3 2 1 X X X X X X P X X X X X P X X X X P X X X P X X P X P X X X X X X P X 1 X 2 X 3 X 4 X 5 X 6 p ( X 6 | X 2 , X 5 ) p ( X 1 ) p ( X 5 | X 4 ) p ( X 4 | X 1 ) p ( X 2 | X 1 ) p ( X 3 | X 2 ) P ( X 1 , X 2 , X 3 , X 4 , X 5 , X 6 ) = P ( X 1 ) P ( X 2 | X 1 ) P ( X 3 | X 2 ) P ( X 4 | X 1 ) P ( X 5 | X 4 ) P ( X 6 | X 2 , X 5 ) Recap of BN Representation z Joint probability dist. on multiple variables: z If X i 's are independent : ( P ( X i | ·)= P ( X i )) z If X i 's are conditionally independent (as described by a GM ), the joint can be factored to simpler products, e.g., = = i i X P X P X P X P X P X P X P X X X X X X P ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) , , , , , ( 6 5 4 3 2 1 6 5 4 3 2 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Eric Xing © Eric Xing @ CMU, 2006-2008 3 Summary z Represent dependency structure with a directed acyclic graph z Node <-> random variable z Edges encode dependencies z Absence of edge -> conditional independence z Plate representation z A BN is a database of prob. Independence statement on variables z The factorization theorem of the joint probability z Local specification Æ globally consistent distribution z Local representation for exponentially complex state-space z Support efficient inference and learning – next lecture Eric Xing © Eric Xing @ CMU, 2006-2008 4 Inference and Learning z We now have compact representations of probability distributions: BN z A BN M describes a unique probability distribution P z Typical tasks: z Task 1: How do we answer queries about P ? z We use inference as a name for the process of computing answers to such queries z Task 2: How do we estimate a plausible model M from data D ? i. We use learning as a name for the process of obtaining point estimate of . ii. But for Bayesian , they seek p ( | ), which is actually an inference problem. iii. When not all variables are observable, even computing point estimate of M need to do inference to impute the missing data .
Background image of page 2
3 Eric Xing © Eric Xing @ CMU, 2006-2008 5 z Computing statistical queries regarding the network, e.g.: z Is node X independent on node Y given nodes Z,W ? z What is the probability of X=true if (Y=false and Z=true)? z What is the joint distribution of (X,Y) if Z=false? z What is the likelihood of some full assignment? z What is the most likely assignment of values to all or a subset the nodes of the network? z General purpose algorithms exist to fully automate such computation z Computational cost depends on the topology of the network z Exact inference: z The junction tree algorithm z Approximate inference; z Loopy belief propagation, variational inference, Monte Carlo sampling Probabilistic Inference Eric Xing © Eric Xing @ CMU, 2006-2008 6 ∑∑ ) ( ) , , ( ) ( 1 1 xx k k , ,x , x P P P v x v H v x X X x H K K = = Inferential Query 1: Likelihood z Most of the queries one may ask involve
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/26/2010 for the course MACHINE LE 10701 taught by Professor Ericp.xing during the Fall '08 term at Carnegie Mellon.

Page1 / 19

lecture18-annotated - Machine Learning 10-701/15-781, Fall...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online