CS-GY 6923: Assignment 1
September 12, 2015
COLLABORATION POLICY: You may discuss general concepts relating
to the homework questions with other students, but you must write up your
solutions on your own, in your own words.
Hard Deadline: Assigment 1 is d

Mehryar Mohri
Foundations of Machine Learning 2014
Courant Institute of Mathematical Sciences
Homework assignment 3
October 28, 2014
Due: November 14, 2014
A. Kernels
1. Show that the kernel K dened by
(x, y) RN RN , K(x, y) =
1
1+
xy
2
2
,
(1)
+ sx s
e e

Foundations of Machine Learning
Multi-Class Classication
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Real-world problems often have multiple classes:
text, speech, image, biological sequences.
Algorithms studied so fa

Foundations of Machine Learning
Kernel Methods
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Efcient computation of inner products in high
dimension.
Non-linear decision boundary.
Learning with non-vectorial inputs.
Fle

Foundations of Machine Learning
Regression
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Regression Problem
Training data: sample drawn i.i.d. from set X
according to some distribution D,
S = (x1 , y1 ), . . . , (xm , ym ) X Y,
wi

Foundations of Machine Learning
Boosting
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Weak Learning
(Kearns and Valiant, 1994)
Denition: concept class C is weakly PAC-learnable
if there exists a (weak) learning algorithm L and >

Foundations of Machine Learning
Ranking
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Very large data sets:
too large to display or process.
limited resources, need priorities.
ranking more desirable than classication.

Foundations of Machine Learning
Learning with
Innite Hypothesis Sets
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
With an innite hypothesis set H, the error bounds
of the previous lecture are not informative.
Is efcien

Foundations of Machine Learning
Learning with
Finite Hypothesis Sets
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Motivation
Some computational learning questions
What can be learned efciently?
What is inherently hard to learn?
A

Foundations of Machine Learning
Support Vector Machines
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Binary Classication Problem
Training data: sample drawn i.i.d. from set X RN
according to some distribution D,
S = (x1 , y1 ), .

Mehryar Mohri
Foundations of Machine Learning 2014
Courant Institute of Mathematical Sciences
Homework assignment 2
October 3, 2014
Due: October 17, 2014
A. VC-dimension of axis-aligned squares or triangles
1. What is the VC-dimension of axis-aligned squa

Mehryar Mohri
Foundations of Machine Learning 2014
Courant Institute of Mathematical Sciences
Homework assignment 2
October 3, 2014
Due: October 17, 2014
A. VC-dimension of axis-aligned squares or triangles
1. What is the VC-dimension of axis-aligned squa

Problem 1:
1 (a)
1
1+ ea 1
=
1+ ea
1+ ea
ea
1
=
= a
= (a)
a
1+ e
e +1
1
Let y = (a) =
1+ ea
1
1+ ea =
y
1
ea = 1
y
1 y
ea =
y
1 y
a = ln
y
y
a = ln
= 1 (y)
1 y
Problem 2:
In the following deduction, for convenience

CS-GY 6923: Assignment 2
COLLABORATION POLICY: You may discuss general concepts relating
to the homework questions with other students, but you must write up your
solutions on your own, in your own words.
Hard Deadline: Assigment 2 is due on October 16.
1

Problem 1
For SGD:
BEGIN:
LOOP 1: Repeat until reach convergence
LOOP2: Repeat until reach convergence
(x (i ) , y(i ) ) = Random(S)
END
h (x) =
j := j + (y(i ) h (x (i ) )x

NOTICE TO ALL STUDENTS PLANNING TO
TAKE CS6923 MACHINE LEARNING:
Machine Learning is more mathematical than most other graduate CS courses.
Students often have diculty with this course (and risk getting a grade of C or
lower) if they know how to program b

Foundations of Machine Learning
Introduction to ML
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Logistics
Prerequisites: basics in linear algebra, probability,
and analysis of algorithms.
Workload: about 3-4 homework assignments

Foundations of Machine Learning - Project Presentation Schedule
Monday Dec 15, WWH Room 109, 5:00PM-7:00PM.
Project Title
Group Members
Predicting visual neuron responses with a convolutional
neural network
Andrew Zaharia
Sample Compression Analysis
Ariel

Mehryar Mohri
Foundations of Machine Learning 2014
Courant Institute of Mathematical Sciences
Homework assignment 1
September 17, 2014
Due: September 30, 2014
A. PAC learning of n-dimensional rectangles
Give a PAC-learning algorithm for C, the set of axis

Mehryar Mohri
Foundations of Machine Learning 2014
Courant Institute of Mathematical Sciences
Homework assignment 3
October 28, 2014
Due: November 14, 2014
A. Kernels
1. Show that the kernel K dened by
(x, y) RN RN , K(x, y) =
1
1+
xy
2
2
,
where > 0 is a

Foundations of Machine Learning
Maximum Entropy Models,
Logistic Regression
Mehryar Mohri
Courant Institute and Google Research
mohri@cims.nyu.edu
Foundations of Machine Learning
page
1
Motivation
Probabilistic models:
density estimation.
classication.
Fo