Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Homework assignment 2
Solution (written by Andres Mu oz)
n
Perceptron Algorithm
Download the following data sets from the UC Irvine ML repository:
http:/archive.ics.
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Homework assignment 1
Solution (written by Andres Mu oz)
n
A. Naive Bayes
The objective of this exercise is to apply the Naive Bayes algorithm presented in
class to
Introduction to Machine Learning
Lecture 16
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Ranking
Mehryar Mohri - Introduction to Machine Learning
page 2
Motivation
Very large data sets:
too large to display or process.
limited re
Introduction to Machine Learning
Lecture 15
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Regression
Mehryar Mohri - Introduction to Machine Learning
page 2
Regression Problem
Training data: sample drawn i.i.d. from set X
accordin
Introduction to Machine Learning
Lecture 14
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Density Estimation
Maxent Models
Mehryar Mohri - Introduction to Machine Learning
page 2
Entropy
Denition: the entropy of a random variable
Introduction to Machine Learning
Lecture 13
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Multi-Class Classication
Mehryar Mohri - Introduction to Machine Learning
page 2
Motivation
Real-world problems often have multiple classes:
Introduction to Machine Learning
Lecture 12
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Bagging
Mehryar Mohri - Introduction to Machine Learning
page 2
Ensemble Methods - Classication
Problem: given T binary classication hypothe
Introduction to Machine Learning
Lecture 11
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Boosting
Mehryar Mohri - Introduction to Machine Learning
page 2
Boosting Ideas
Main idea: use weak learner to create strong learner.
Ensemb
Introduction to Machine Learning
Lecture 10
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Decision Trees
Mehryar Mohri - Foundations of Machine Learning
page 2
Supervised Learning Problem
Training data: sample drawn i.i.d. from se
Introduction to Machine Learning
Lecture 9
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Kernel Methods
Motivation
Non-linear decision boundary.
Efcient computation of inner products in high
dimension.
Flexible selection of more c
Introduction to Machine Learning
Lecture 6
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Perceptron and Winnow
This Lecture
On-Line linear classication: two algorithms.
Perceptron algorithm.
Winnow algorithm.
Mehryar Mohri - Intro
Introduction to Machine Learning
Lecture 5
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
On-Line Learning
with Expert Advice
On-Line Learning
No distributional assumption.
Worst-case analysis (adversarial).
Mixed training and test
Introduction to Machine Learning
Lecture 4
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Nearest-Neighbor Algorithms
Nearest Neighbor Algorithms
Denition: x k 1, given a labeled sample
S = (x1 , y1 ), . . . , (xm , ym ) (X cfw_0,
Introduction to Machine Learning
Lecture 3
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Bayesian Learning
Bayes Formula/Rule
Terminology:
likelihood
prior
Pr[X | Y ] Pr[Y ]
Pr[Y | X ] =
.
Pr[X ]
posterior
probability
Mehryar Mohr
Introduction to Machine Learning
Lecture 2
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Basic Probability Notions
Probabilistic Model
Sample space: , set of all outcomes or elementary
events possible in a trial, e.g., casting a d
Introduction to Machine Learning
Lecture 1
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Introduction
Logistics
Prerequisites: basics concepts needed in probability
and statistics will be introduced.
Workload:
5 homework assignmen
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Midterm exam
October 5th, 2011.
A. Perceptron algorithm
In class, we saw that when the training sample S is linearly separable with a maximum margin > 0, then the Pe
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Midterm exam
October 5th, 2011.
A. Perceptron algorithm
In class, we saw that when the training sample S is linearly separable with a maximum margin > 0, then the Pe
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Homework assignment 3
November 05, 2011
Due: November 19, 2011
Support vector machines
1. Download and install the libsvm software library from:
http:/www.csie.ntu.e
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Homework assignment 2
October 10, 2011
Due: October 24, 2011
Perceptron Algorithm
Download the following data sets from the UC Irvine ML repository:
http:/archive.ic
Mehryar Mohri
Introduction to Machine Learning
Courant Institute of Mathematical Sciences
Homework assignment 1
September 19, 2011
Due: September 30, 2011
A. Naive Bayes
The objective of this exercise is to apply the Naive Bayes algorithm presented in
cla