CS485: Assignment 1 Solutions
1. Error of the Bayes predictor = ErrP (fP )
= ExP [fP (x)(1 P (x) + (1 fP (x)
1 P (x) if P (x) 0.5
= ExP
otherwise
P ( x)
= ExP [mincfw_ P (x), 1 P (x)]
= ExP [h(x) mincfw_
P (x), 1
ExP [h(x) (1
P (x)
P (x)
P (x)]
+ (1 h(x
CS 489/698: Machine Learning,
Winter 2013, Assignment 2
Shai Ben-David
Due date is Thursday Feb. 14, at 4:00pm in class.
Please write clearly (preferably type your assignment), staple the pages together,
and have your name on the top of each page (in case
CS485 - Winter 2013, Assignment 1
Due Jan. 31st 4pm (in class)
Shai Ben-David, University of Waterloo
The rst part of this assignment deals with very basic notions; What is a
learning algorithm? How do we evaluate the success of a learning algorithm?
When
CS485: Assignment 2 Solutions
1. We will prove that the set A = cfw_0, e1 , e2 , ., en can be shattered by HS n , where ei denotes a unit vector in
Rn . In order to do this, we will show that for an arbitrary B A there exists a h HS n that assigns 1 to t
CS485: Assignment 3 Solutions
1.
(a) True. If S is an -net for H w.r.t. P , then for all h H , if P (h) > , then S h = . Since S S , for
every h H , if P (h) > , S h S h = . Therefore S is an -net for H w.r.t. P .
(b) False. Let X = cfw_x1 , x2 , x3 , H =
CS 489/698: Machine Learning,
Winter 2013, Assignment 3
Shai Ben-David
Due date is Thursday March 28, at 4:00pm in class.
Please write clearly (preferably type your assignment), staple the pages together,
and have your name on the top of each page (in cas
CS 685 Machine Learning:
Assignment 1
Due on October 9, 2015
Shai Ben-David
Shan Huang
Student ID: 20488219
1
Shan Huang
CS 685 Machine Learning (Shai Ben-David 5:30):
Problem 1
Listing 1 shows a MATLAB script that was used to generate Figure 1.
Listing 1
CS485/685 - Fall 2015, Assignment 2
Due Monday, Nov 2, at 11:59 am
Shai Ben-David, University of Waterloo
This assignment is about shattering, VC-dimension, Sauers lemma, -nets
and -approximations. The general setup is as follows: We have some class H
of
CS 485/685: Machine Learning, Winter 2015
Assignment 4
Shai Ben-David
Due Friday April 10, at 1:00pm.
Drop your assignment in the assignment boxes in MC on the 4th.
Please write clearly (preferably type your assignment), staple the pages
together, and hav
CS485/685 - Winter 2015
Assignment 3 Solutions
1. (a) Let S := cfw_d(h) : h H. Also, let N = supcfw_|d(h)| : h H. Consider the strings as nodes in
a binary tree. In the general case, we have codes for each node in the tree. Given a tree of
height N , the
CS485/685 - Winter 2015
Assignment 1 Solutions
1. (a) P = Px C where Px is the marginal distribution and C is the conditional probability over
labels. The loss of any deterministic function g is
X
LP (g) = P r [g(x) 6= y] =
P[x] P r [y 6= g(x)|x]
(x,y)D
x
CS 485/685: Machine Learning, Winter 2015
Assignment 3
Shai Ben-David
Due date is Friday March 20, at 1:00pm drop your assignment
in the assignment boxes in MC on the 4th floor or in class.
Please write clearly (preferably type your assignment), staple th
CS 485/685: Machine Learning, Winter 2015
Assignment 2
Shai Ben-David
Due date is Friday, March 6, at 1pm.
Please write clearly (preferably type your assignment), staple the pages together, and have your name on the top of each page (in case they get unst
CS485/685 - Winter 2015
Assignment 2 Solutions
1. (a) Claim: VC-dim(Hkones ) = k. Let A be an arbitrary shattered set. |A| k because the
hypothesis class does not have a member that outputs 1 on more than k points. Therefore,
VC-dim(Hkones ) k. Furthermor
CS485/685 - Winter 2015
Assignment 4 Solutions
1. Consider the procedure F described in the hint. Given S of size m, it divides S into k chunks
(Si , i [k]) of size m1 each and once chunk Sk+1 of size m2 . It runs A on every chunk of size m1
to get hypoth
CS485/685 - Fall 2015, Assignment 2
Due Monday, Nov 2, at 11:59 am
Shai Ben-David, University of Waterloo
This assignment is about shattering, VC-dimension, Sauers lemma, -nets
and -approximations. The general setup is as follows: We have some class H
of
CS 489
June 23, 2009
Shai Ben-
David
We now use the tools of -
approximations and -
nets to prove upper bounds on the sample complexity
of learning.
Theorem:
For any , any of finite VCDim, if is an i.i.d.
CS 489
June 18, 2009
-collection of subsets of .
is the probability distribution over .
is an -net with respect to
if
if
,
,
.
.
Let be a finite subset of
by taking
for every that contains
Example:
to be uniform over we get that if
points of .
=set of all
CS 489/698: Machine Learning,
Winter 2013, Assignment 3
Shai Ben-David
Due date is Thursday March 28, at 4:00pm in class.
Please write clearly (preferably type your assignment), staple the pages together,
and have your name on the top of each page (in cas
9
Linear Predictors
In this chapter we will study the family of linear predictors, one of the most
useful families of hypotheses classes. Many learning algorithms that are being
widely used in practice rely on linear predictors, rst and foremost due to th
Chapter 6
The VC-dimension
In the previous chapter, we decomposed the error of the ERMH rule into approximation error and estimation error. The approximation error depends on the t of
our prior knowledge (as reected by the choice of the hypothesis class H
Machine Learning:
Foundations and Algorithms
Shai Ben-David and Shai Shalev-Shwartz
DRAFT
2
c Shai Ben-David and Shai Shalev-Shwartz.
i
Preface
The term machine learning refers to the automated detection of meaningful patterns in data. In the past couple
8
The Runtime of Learning
So far in the book we studied the statistical perspective of learning, namely, how
many samples are needed for learning. In other words, we focused on the amount
of information learning requires. However, when considering automat
Chapter 1
Minimum Description Length
The notions of PAC learnability discussed so far in the book allow the sample sizes
to depend on the accuracy and condence parameters but they are uniform with
respect to the labeling rule and the underlying data distr
CS 489
June 30, 2009
Online Learning
The student gets examples one at a time and issues a label prediction, then sees what the correct label
is. Repeats. The measure of success is the # of misspredictions (No probability involved).
Student Teacher
How wel
Machine Learning
CS489/698
Lecture 1: Jan 4th, 2017
Pascal Poupart
Associate Professor
David R. Cheriton School of Computer Science
University of Waterloo
CS489/698 (c) 2017 P. Poupart
1
Machine Learning
Arthur Samuel (1959): Machine learning is the fiel