Support Vector Machines
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 18, 2013
Instructor: Arindam Banerjee
Support Vector Machines
Linear SVM
Distance of central hyperplane from origin =
Distance of parallel hyperplanes are
|b 1|
w
a
Perceptrons
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 25, 2013
Instructor: Arindam Banerjee
Perceptrons
Perceptrons
Linear classiers with weight vector w
Prediction on xi is incorrect if yi wT xi < 0.
Let M(w) be the set of points
Neural Networks
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
October 2, 2013
Instructor: Arindam Banerjee
Neural Networks
Layered Linear Classier
Motivation
Linear classiers have limited representation power
Most real problems have non-linear
Linear Models for Regression
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 09, 2013
Instructor: Arindam Banerjee
Linear Models for Regression
Linear Models
Linear models over basis functions
M
wj j (x) = wT (x)
f (x, w) =
j =1
Choice
Linear Discriminants
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 11, 2013
Instructor: Arindam Banerjee
Linear Discriminants
Discriminant Functions
The simplest representation for a 2-class problem
f (x) = wT x + w0
Class assignment
Learning and Bias
Algorithmic learning for prediction tries to solve a big model selection problem. The appropriateness of the model is determined by its predictive accuracy on holdout/future data. The model
selection process is typically broken down into
Kernel Methods
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 23, 2013
Instructor: Arindam Banerjee
Kernel Methods
Non-linear SVMs
All important equations have dot-products
Dual is expressed in terms of xT xj
i
The predictions are in t
Generative and Discriminative Models
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 16, 2013
Instructor: Arindam Banerjee
Generative and Discriminative Models
Generative Models and Bayes Rule
Bayes rule states that
p ( y |x ) =
p (x|y
Boosting
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
October 9, 2013
Instructor: Arindam Banerjee
Boosting
Weak Learning
A weak learner predicts slightly better than random
The PAC setting
Let X be an instance space, c : X cfw_0, 1 be a targe