10-601
Machine Learning
Graphical models and Bayesian
networks
Reading: Bishop 8.1 and 8.2.2
Independence
Independence allows for easier models, learning and
inference
For example, with 3 binary variables we only need 3
parameters rather than 7.
The savin
10601
Machine Learning
Semi supervised learning
Can Unlabeled Data improve
supervised learning?
Important question! In many cases, unlabeled data is
plentiful, labeled data expensive
Medical outcomes (x=<patient,treatment>, y=outcome)
Text classificatio
10-601
Machine Learning
Bayesian networks: Inference
Reading: Bishop 8.1 and 8.2.2
d-separation
We will give rules to identify d-connected variables. Variables
that are not d-connected are d-separated.
The following three rules can be used to determine i
10-601
Machine Learning
Inference in HMMs
Hidden Markov models
Model a set of observation with a set of hidden states
- Robot movement
Observations: range sensor, visual sensor
Hidden states: location (on a map)
- Speech processing
Observations: sound si
10-601
Machine Learning
Learning in HMMs
A Hidden Markov model
A set of states cfw_s1 sn
- In each time point we are in exactly one of these states
denoted by qt
i, the probability that we start at state si
A transition probability model, P(qt = si | q
10-601
Machine Learning
Markov decision processes (MDPs)
The weeks ahead
- Applications
of HMM to biology
- Dimensionality reduction
- SVM
- Boosting
- Model and feature selection
Markov decision processes (MDPs)
Whats missing in HMMs
HMMs cannot model im
10-701 Machine Learning (Spring 2012)
Principal Component Analysis
Yang Xu
This note is partly based on Chapter 12.1 in Chris Bishops book on PRML and
the lecture slides on PCA written by Carlos Guestrin in the 10-701 Machine
Learning (fall 2009) course.
10-601
Machine Learning
Support Vector Machine
Types of classifiers
We can divide the large variety of classification approaches into roughly three major
types
1. Instance based classifiers
- Use observation directly (no models)
- e.g. K nearest neighbors
10601
Machine Learning
Model and feature selection
Model selection issues
We have seen some of this before
Selecting features (or basis functions)
Logistic regression
SVMs
Selecting parameter value
Prior strength
Nave Bayes, linear and logistic re