Probabilistic Graphical Models
David Sontag
New York University
Lecture 1, January 31, 2013
David Sontag (NYU)
Graphical Models
Lecture 1, January 31, 2013
1 / 44
One of the most exciting advances in machine learning (AI, signal
processing, coding, contro
Probabilistic Graphical Models
David Sontag
New York University
Lecture 2, February 7, 2013
David Sontag (NYU)
Graphical Models
Lecture 2, February 7, 2013
1 / 31
Bayesian networks
Reminder of last lecture
A Bayesian network is specied by a directed acycl
Mixture Models & EM algorithm
Probabilis5c Graphical Models, Lecture 13
David Sontag
New York University
Slides adapted from Carlos Guestrin, Dan Klein, Luke Ze@lemoyer,
Dan Weld, Vibhav Gogate, and Andrew Moore
Mixtur
Probabilistic Graphical Models
David Sontag
New York University
Lecture 12, April 23, 2013
David Sontag (NYU)
Graphical Models
Lecture 12, April 23, 2013
1 / 24
What notion of best should learning be optimizing?
This depends on what we want to do
1
Densit
Probabilistic Graphical Models
David Sontag
New York University
Lecture 8, March 28, 2012
David Sontag (NYU)
Graphical Models
Lecture 8, March 28, 2012
1 / 14
From last lecture: Variational methods
Suppose that we have an arbitrary graphical model:
p(x; )
Markov chain Monte Carlo
Lecture 9
David Sontag
New York University
Slides adapted from Eric Xing and Qirong Ho (CMU)
Limitations of Monte Carlo
Direct (unconditional) sampling
Hard to get rare events in high-dimensional spaces
Infeasible for MRFs, unless
Probabilistic Graphical Models
David Sontag
New York University
Lecture 13, May 2, 2013
David Sontag (NYU)
Graphical Models
Lecture 13, May 2, 2013
1 / 14
Today: learning with partially observed data
Identiability
Overview of EM (expectation maximization)
Probabilistic Graphical Models
David Sontag
New York University
Lecture 11, April 18, 2013
Acknowledgements: Partially based on slides by Eric Xing at CMU and Andrew
McCallum at UMass Amherst
David Sontag (NYU)
Graphical Models
Lecture 11, April 18, 2013
Probabilistic Graphical Models
David Sontag
New York University
Lecture 10, April 11, 2013
David Sontag (NYU)
Graphical Models
Lecture 10, April 11, 2013
1 / 22
Summary so far
Representation of directed and undirected networks
Inference in these networks:
Probabilistic Graphical Models
David Sontag
New York University
Lecture 7, March 14, 2012
David Sontag (NYU)
Graphical Models
Lecture 7, March 14, 2012
1 / 22
Approximate marginal inference
Given the joint p(x1 , . . . , xn ) represented as a graphical mo
Probabilistic Graphical Models
David Sontag
New York University
Lecture 3, February 14, 2013
David Sontag (NYU)
Graphical Models
Lecture 3, February 14, 2013
1 / 33
Undirected graphical models
Reminder of lecture 2
An alternative representation for joint
Probabilistic Graphical Models
David Sontag
New York University
Lecture 4, February 21, 2013
David Sontag (NYU)
Graphical Models
Lecture 4, February 21, 2013
1 / 29
Conditional random elds (CRFs)
A CRF is a Markov network on variables X Y, which species t
Probabilistic Graphical Models
David Sontag
New York University
Lecture 6, March 7, 2013
David Sontag (NYU)
Graphical Models
Lecture 6, March 7, 2013
1 / 25
Todays lecture
1
Dual decomposition
2
MAP inference as an integer linear program
3
Linear programm
Probabilistic Graphical Models
David Sontag
New York University
Lecture 5, Feb. 28, 2013
David Sontag (NYU)
Graphical Models
Lecture 5, Feb. 28, 2013
1 / 22
Todays lecture
1
2
Using VE for conditional queries
Running-time of variable elimination
Eliminati