CS446: Pattern Recognition and Machine Learning
Fall 2009
Problem Set 3
Handed Out: September 24, 2009
Due: October 8, 2009
•
Feel free to talk to your classmates about the homework. I am more concerned that you learn how to
solve the problem than that you demonstrate that you solved it entirely on your own. You should,
however, write down your solution yourself. Please try to keep the solution brief and clear.
•
Please, no handwritten solutions. Be sure your name appears on the top of each page.
•
Please present your algorithms in both pseudocode and English. That is, give a precise formulation of
your algorithm as pseudocode and
also
explain in one or two concise paragraphs what your algorithm
does. Be aware that pseudocode is much simpler and more abstract than real code. Take a look at
the textbook pseudocode (e.g. Table 2.5 on page 33) to get an idea about the appropriate level of
abstraction.
•
The homework is due at 4:00 pm on the due date. Email writeup and your code to the TA. Please
do NOT hand in a hard copy of your writeup. Please put “
<
userid
>
CS446 hw3 submission” as the
subject line of the email when you submit your homework to
[email protected]
. Put all of your
files into a
single compressed file
with the file name“
<
userid
>
hw3.tgz”.
1. [Online Algorithm Comparison  100 points]
In this problem set you will implement several online learning algorithms including
Winnow
and
Perceptron
, and experiment with them by comparing their performance
on a synthetic dataset. For this problem set, we recommend using Matlab, given that
we supply Matlab code that can help you run the experiments.
In these experiments you will generate examples that are labeled according to a simple
threshold function
, an
l
of
m
of
n
function.
That is, the function is defined on the
n
dimensional Boolean cube
{
0
,
1
}
n
, and there is a set of
m
attributes such that an
example is positive iff at least
l
of these
m
are active in the example. We call
l
,
m
and
n
concept parameters
, since they specify what exact concept we are talking about.
To make sure you understand the function, try to write it for yourself as a linear
threshold function.
This way, you will make sure that Winnow and Perceptron can
represent this function.
Also, notice that this concept is a generalization of mono
tone conjunctions and of monotone disjunctions, which you get for
l
=
m
and
l
= 1
respectively.
Your algorithm does not know the target function and does not know which are the
relevant attributes or how many are relevant. The goal is to evaluate these algorithms
under several conditions and derive some conclusions on their relative advantages and
disadvantages. The instance space is
{
0
,
1
}
n
(that is, there are
n
boolean features in
the domain). You will run experiments on several values of
l
,
m
, and
n
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 ahuja
 Machine Learning, Perceptron Algorithm

Click to edit the document details