CSCI573: Probabilistic Reasoning, Spring 2014
Review of Probability and Introduction to Bayesian Network
Lecturer: Fei Sha
1
Scribes: Zhiyun Lu, Ida Johnsson
Review of Probability
1.1
Basic Concepts in Probability
sample space , possible outcome set.
ev
CSCI573: Probabilistic Reasoning, Spring 2014
Bayesian/Markov Network Parameter Learning
Lecturer: Fei Sha
1
Scribes: Chen Zhang and Kuan Liu
Review
Belief update
Message: belief on the cliques.
Loopy belief propagation.
Learning
In general a hard pr
CSCI573: Probabilistic Reasoning, Spring 2014
Mean Field Approximation
Lecturer: Fei Sha
1
Scribes: Akshay Gadde, Jia Zhuo
Review
EM algorithm for learning from incomplete data
Key insight:
log P (x|) H[Q(z); ] =
Q(z) log P (x, z|)
z
Q(z) log Q(z),
Q
z
A
CSCI573: Probabilistic Reasoning, Spring 2014
Learning with Incomplete Data
Lecturer: Fei Sha
1
Scribes: Ai He, Dehua Cheng
Review
1. MAP Inference
x = arg max p(x)
x
or
y = arg max p(y|x)
y
2. Algorithm
(a) Sum-Product Max-Product (Max-Sum)
(b) Backtrack
CSCI573: Probabilistic Reasoning, Spring 2014
Generalized form of Variational Method (02 April 2014)
Lecturer: Fei Sha
1
Scribes: Aamir Anis and Ruchir Travadi
Review
Mean-eld approximation
Given a distribution P (x), nd an approximation Q(x) satisfying
m
CSCI573
Probabilistic
Reasoning
Fei Sha
[email protected]
About this
course
CSCI573 Probabilistic Reasoning
Administration
Syllabus
What to expect?
Overview
What are
PGMs?
Applications of
PGMs
Fei Sha
[email protected]
Review basic
concepts in
probability
Janua
Fei Sha
CSCI573 Spring 2014 Projects
[email protected]
1 What is a project for this course?
Please feel free to discuss potential project ideas with me. Roughly speaking, there are three types
of projects:
Application domain specic Projects which apply grap
CSCI573: Probabilistic Reasoning, Spring 2014
Topic
Lecturer: Fei Sha
1
Scribes: Scriber Names
First thing
We have discussed the basics about this course. A crucial component is the project. Please read the note
on the project.
2
Second thing
What is a gr
Fei Sha
CSCI573 Spring 2014 Syllabus
[email protected]
Introduction The chief objective is to teach modern methods of probabilistic reasoning that are
commonly used in many parts of computer science, including but not limited to articial intelligence.
Difcul
CSCI573: Probabilistic Reasoning, Spring 2014
Map Inference: Most Probable Explanation and Marginal MAP
Lecturer: Fei Sha
1
Scribes: Hilmi Egilmez and Shuyang Gao
Review
Learning for Markov Networks
No global decomposition
Convex optimization
gradient
CSCI573: Probabilistic Reasoning, Spring 2014
Belief Update Algorithm, Learning B.N. with Fully Observed Data
Lecturer: Fei Sha
1
Scribes: Karol Hausman, Timmy Mbaya
Review
Clique (Junction) Tree
Undirected graph as data structures with the following pr
CSCI573: Probabilistic Reasoning, Spring 2014
Lecture 4 January 27, 2014
Lecturer: Fei Sha
1
Scribes: Shankhoneer Chakrovarty and Yun Ling
Content
Review
Factorization view
Independency view
Local: Xi Non-Descendants of Xi |P a(Xi )
F actorization In
CSCI573: Probabilistic Reasoning, Spring 2014
Plate Notation, CPD, Multivariate Gaussians and The Exponential Family
Lecturer: Fei Sha
1
Scribes: Bharath Sankaran, Gabriel Mel
Contents
Review
Conditional Random Fields (CRF)
Template Models (e.g. DBN) m
CSCI573: Probabilistic Reasoning, Spring 2014
Conjugacy in Exponential Family, Variable Elimination
Lecturer: Fei Sha
1
Scribes: Soravit Beer Changpinyo, Wei-Lun Harry Chao
Review: Exponential Family
The general form of the exponential family is as follow
CSCI573: Probabilistic Reasoning, Spring 2014
Lecture 9: Exponential Family
Lecturer: Fei Sha
1
Scribes: Joseph Veloce and Amir Tahmasebi
Exponential Family
1.1
General Form
p(x|) =
T
1
h(x)e (x)
Z()
= h(x)e
T
(X)A()
where A() = log Z(), and (X) represent
CSCI573: Probabilistic Reasoning, Spring 2014
Variable Elimination
Lecturer: Fei Sha
1
Scribes: Xing Shi and Paul Miyazaki
Review
Conjugate prior and sucient statistics for exponential family.
Variable elimination (belongs to exact inference techniques)
CSCI573: Probabilistic Reasoning, Spring 2014
MRF, Template/Plate Notation (2/5/2014)
Lecturer: Fei Sha
1
Scribes: Alireza Bagheri Garakani, Chet Corcos
Conditional Random Field (CRF)
One of the most popularly used graphical models (See gure 1)
Why use