Foundations of Machine Learning
Learning with
Infinite Hypothesis Sets
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
With an infinite hypothesis set H, the error bounds
of the previous lecture are not informative.
Is ef
Foundations of Machine Learning
Learning with
Finite Hypothesis Sets
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Some computational learning questions
What can be learned efficiently?
What is inherently hard to learn?
Foundations of Machine Learning
Convex Optimization
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Convex Optimization
Mehryar Mohri - Foudations of Machine Learning
Convexity
N
X
R
Definition:
is said to be convex if for any
two p
Foundations of Machine Learning
Boosting
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Weak Learning
(Kearns and Valiant, 1994)
Definition: concept class C is weakly PAC-learnable
if there exists a (weak) learning algorithm L and
Foundations of Machine Learning
Ranking
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Very large data sets:
too large to display or process.
limited resources, need priorities.
ranking more desirable than classification
Foundations of Machine Learning
On-Line Learning
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
PAC learning:
distribution fixed over time (training and test).
IID assumption.
On-line learning:
no distributional assumpti
Foundations of Machine Learning
Maximum Entropy Models,
Logistic Regression
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Foundations of Machine Learning
page
1
Motivation
Probabilistic models:
density estimation.
classication.
Fo
Foundations of Machine Learning
Lecture 10
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Ranking
Mehryar Mohri - Foundations of Machine Learning
page 2
Motivation
Very large data sets:
too large to display or process.
limited reso