Support vector machines
Lecture 4
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Vibhav Gogate,
and Carlos Guestrin
-1
0
w.x + b
=
+ C j j
w.x + b
=
+1
w.x + b
=
Allowing for slack: Soft margin SVM
- j
j0
slack variables
What is th
Ensemble learning
Lecture 13
David Sontag
New York University
Slides adapted from Navneet Goyal, Tan, Steinbach,
Kumar, Vibhav Gogate
Ensemble methods
Machine learning competition with a $1 million prize
Bias/Variance Tradeoff
Hastie, Tibshirani, Friedman
Clustering
Lecture 14
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Vibhav Gogate,
Carlos Guestrin, Andrew Moore, Dan Klein
Clustering
Clustering:
Unsupervised learning
Requires data, but no labels
Detect patterns e
Decision Trees
Lecture 12
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Carlos Guestrin,
and Andrew Moore
Machine Learning in the ER
Physician
documentation
Triage Information
(Free text)
MD comments
(free text)
Learning theory
Lecture 8
David Sontag
New York University
Slides adapted from Carlos Guestrin & Luke Zettlemoyer
Whats next
We gave several machine learning algorithms:
Perceptron
Linear support vector machine (SVM)
SVM
Hierarchical Clustering
Lecture 15
David Sontag
New York University
Agglomerative Clustering
Agglomerative clustering:
First merge very similar instances
Incrementally build larger clusters out
of smaller clusters
Algorithm:
Maintain a s
Support Vector Machines & Kernels
Lecture 6
David Sontag
New York University
Slides adapted from Luke Zettlemoyer and Carlos Guestrin,
and Vibhav Gogate
Dual SVM derivation (1) the linearly
separable case
Original optimization problem:
Rewrite
constraints
Support Vector Machines & Kernels
Lecture 5
David Sontag
New York University
Slides adapted from Luke Zettlemoyer and Carlos Guestrin
Support Vector Machines
QP form:
More natural form:
Regularization
term
Equivalent if
Empirical loss
Subgradient method
S
Learning theory
Lecture 9
David Sontag
New York University
Slides adapted from Carlos Guestrin & Luke Zettlemoyer
Introduc8on to probability: events
An event is a subset of the outcome space, e.g.
E=cfw_
,
,
Even die tosses
Nearest neighbor methods
Lecture 11
David Sontag
New York University
Slides adapted from Vibhav Gogate, Carlos Guestrin,
Mehryar Mohri, & Luke Zettlemoyer
Nearest Neighbor Algorithm
Learning Algorithm:
Store training examples
P
Learning theory
Lecture 10
David Sontag
New York University
Slides adapted from Carlos Guestrin & Luke Zettlemoyer
What about con:nuous hypothesis spaces?
Con:nuous hypothesis space:
|H| =
Innite variance?
Only care
Linear classifiers
Lecture 3
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Vibhav Gogate,
and Carlos Guestrin
ML Methodology
Data: labeled instances, e.g. emails marked spam/ham
Training set
Held out set (sometimes call Validati
Introduction to Learning
Lecture 2
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Vibhav Gogate,
Pedro Domingos, and Carlos Guestrin
Hypo. Space: Degree-N Polynomials
1
1
M =0
M =1
1
t
t
1
M =3
t
M =9
t
0
0
0
0
1
1
1
1
0
x
1
0
x
1
Introduction to Machine Learning
(CSCI-UA.0480-002)
David Sontag
New York University
Slides adapted from Luke Zettlemoyer, Pedro Domingos, and
Carlos Guestrin
Logistics
Class webpage:
http:/cs.nyu.edu/~dsontag/courses/ml13/
Sign up for mailing list!
O