{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lecture18-annotated

# lecture18-annotated - Machine Learning 10-701/15-781 Fall...

This preview shows pages 1–4. Sign up to view the full content.

1 Eric Xing © Eric Xing @ CMU, 2006-2008 1 Machine Learning Machine Learning 10 10 -701/15 701/15 -781, Fall 2008 781, Fall 2008 Bayesian Network II Bayesian Network II Inference Inference Eric Xing Eric Xing Lecture 18, November 12, 2008 Reading: Chap. 8, C.B book X1 X2 X3 X4 X5 X6 X7 X8 Visit to Asia Tuberculosis Tuberculosis or Cancer XRay Result Dyspnea Bronchitis Lung Cancer Smoking Eric Xing © Eric Xing @ CMU, 2006-2008 2 ) , , , , | ( ) , , , | ( ) , , | ( ) , | ( ) | ( ) ( = ) , , , , , ( 5 4 3 2 1 6 4 3 2 1 5 3 2 1 4 2 1 3 1 2 1 6 5 4 3 2 1 X X X X X X P X X X X X P X X X X P X X X P X X P X P X X X X X X P X 1 X 2 X 3 X 4 X 5 X 6 p ( X 6 | X 2 , X 5 ) p ( X 1 ) p ( X 5 | X 4 ) p ( X 4 | X 1 ) p ( X 2 | X 1 ) p ( X 3 | X 2 ) P ( X 1 , X 2 , X 3 , X 4 , X 5 , X 6 ) = P ( X 1 ) P ( X 2 | X 1 ) P ( X 3 | X 2 ) P ( X 4 | X 1 ) P ( X 5 | X 4 ) P ( X 6 | X 2 , X 5 ) Recap of BN Representation z Joint probability dist. on multiple variables: z If X i 's are independent : ( P ( X i | ·)= P ( X i )) z If X i 's are conditionally independent (as described by a GM ), the joint can be factored to simpler products, e.g., = = i i X P X P X P X P X P X P X P X X X X X X P ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) , , , , , ( 6 5 4 3 2 1 6 5 4 3 2 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Eric Xing © Eric Xing @ CMU, 2006-2008 3 Summary z Represent dependency structure with a directed acyclic graph z Node <-> random variable z Edges encode dependencies z Absence of edge -> conditional independence z Plate representation z A BN is a database of prob. Independence statement on variables z The factorization theorem of the joint probability z Local specification Æ globally consistent distribution z Local representation for exponentially complex state-space z Support efficient inference and learning – next lecture Eric Xing © Eric Xing @ CMU, 2006-2008 4 Inference and Learning z We now have compact representations of probability distributions: BN z A BN M describes a unique probability distribution P z Typical tasks: z Task 1: How do we answer queries about P ? z We use inference as a name for the process of computing answers to such queries z Task 2: How do we estimate a plausible model M from data D ? i. We use learning as a name for the process of obtaining point estimate of . ii. But for Bayesian , they seek p ( | ), which is actually an inference problem. iii. When not all variables are observable, even computing point estimate of M need to do inference to impute the missing data .
3 Eric Xing © Eric Xing @ CMU, 2006-2008 5 z Computing statistical queries regarding the network, e.g.: z Is node X independent on node Y given nodes Z,W ? z What is the probability of X=true if (Y=false and Z=true)? z What is the joint distribution of (X,Y) if Z=false? z What is the likelihood of some full assignment? z What is the most likely assignment of values to all or a subset the nodes of the network? z General purpose algorithms exist to fully automate such computation z Computational cost depends on the topology of the network z Exact inference: z The junction tree algorithm z Approximate inference; z Loopy belief propagation, variational inference, Monte Carlo sampling Probabilistic Inference Eric Xing © Eric Xing @ CMU, 2006-2008 6 ∑∑ ) ( ) , , ( ) ( 1 1 xx k k , ,x , x P P P v x v H v x X X x H K K = = Inferential Query 1: Likelihood z Most of the queries one may ask involve

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 19

lecture18-annotated - Machine Learning 10-701/15-781 Fall...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online