Homework 1, due Monday February 9
COMS 4771 Spring 2015
Problem 1 (Classiers via generative models). Download the OCR image data set mnist.mat
from Courseworks, and load it into MATLAB. The unlabeled training data (i.e., feature vectors)
are contained in
Homework 4, due Wednesday April 15
COMS 4771 Spring 2015
Problem 1 (Anatomy of ordinary least squares). Let X = (X1 , X2 , . . . , Xp ) be a random vector
in Rp , and let Y be a real-valued random variable. Let E denote the expectation operator with
respe
Homework 2, due Wednesday February 25
COMS 4771 Spring 2015
Problem 1 (Bernoulli Na Bayes classier). Download the 20 Newsgroups data set from
ve
Courseworks news.mat, and load it into MATLAB. The training feature vectors/labels and test
feature vectors/la
MACHINE LEARNING COMS 4771, HOMEWORK 4
Assigned November 6, 2014. Due November 20, 2014 before 1.00pm.
1
Problem 1 (10 points): EM Derivation
Consider a random variable x that is categorical with M possible values 1, P. , M . Suppose x is
.
M
represented
MACHINE LEARNING COMS 4771, HOMEWORK 2
Assigned September 30, 2014. Due October 14, 2013 before 1pm.
Please submit separate les for a) write-up, b) Matlab source les and c) gures (if you choose to
include them separately from the writeup). Do not include
MACHINE LEARNING COMS 4771, HOMEWORK 2
Assigned September 30, 2014. Due October 14, 2013 before 1pm.
Please submit separate les for a) write-up, b) Matlab source les and c) gures (if you choose to
include them separately from the writeup). Do not include
COMS 4771 Machine Learning (Spring 2015)
Problem Set #5
Solutions - [email protected]
Discussants: None
May 1, 2015
Problem 1
Let x cfw_0, 1mn denote the matrix of labels provided by the workers.
Let be the current parameters. Dene qi := Pr (Yi
MACHINE LEARNING COMS 4771, HOMEWORK 1
Assigned September 11, 2014. Due September 30, 2014 before 1:00pm.
Submit your work via courseworks.columbia.edu.
Please submit separate les for a) write-up, b) Matlab source les and c) gures (if you choose to
includ
COMS 4771 Machine Learning (Spring 2015)
Problem Set #3
Solutions - [email protected]
Discussants: None
March 28, 2015
Problem 1
The point of this exercise is to carry out the process of evaluating a machine learning algorithm for a given probl
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 14
tructuring Probability Functions for Storage
S
tructuring Probability Functions for Inference
S
asic Graphical Models
B
raphical M
COMS 4771 Machine Learning (Spring 2015)
Problem Set #4
Solutions - [email protected]
Discussants: None
April 14, 2015
Problem 1
(a) Minimize E[(Y w X)2 ] as a function of w. The solution is w = E(XX )1 E(Y X),
so Y = E(Y X) E(XX )1 X.
(b)
E(Y
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 13
xpectation Maximization as Bound Maximization
E
M for Maximum A Posteriori
E
Tony Jebara, Columbia University
EM as Bound Maximizati
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 10
lassification with Gaussians
C
egression with Gaussians
R
rincipal Components Analysis
P
Dimensionality reduction w/ a gaussian
mul
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 12
ixture Models and Hidden Variables
M
lustering
C
-Means
K
xpectation Maximization
E
Last time - a better alternative to maximum li
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 11
aximum Likelihood as Bayesian Inference
M
aximum A Posteriori
M
ayesian Gaussian Estimation
B
Estimation of parameters from data
Gi
Homework 5, due Monday April 27
COMS 4771 Spring 2015
Problem 1 (Expectation-Maximization). Consider the following variant of the MTurk model for
m items and n workers.
Nature picks correct label for item i to be 1 with probability i (and 0 otherwise).
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 9
ontinuous Probability Models
C
aussian Distribution
G
aximum Likelihood Gaussian
M
ampling from a Gaussian
S
Tony Jebara, Columbia
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 15
raphical Models
G
aximum Likelihood for Graphical Models
M
esting for Conditional Independence & D-Separation
T
ayes Ball
B
Expert
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 20
MMs with Evidence
H
MM Collect
H
MM Evaluate
H
MM Distribute
H
MM Decode
H
MM Parameter Learning via JTA & EM
H
Tony Jebara, Col
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 19
idden Markov Models
H
MMs as State Machines & Applications
H
MMs Basic Operations
H
MMs via the Junction Tree Algorithm
H
Tony Jeb
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 16
ndirected Graphs
U
ndirected Separation
U
nferring Marginals & Conditionals
I
oralization
M
unction Trees
J
riangulation
T
Tony
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 18
he Junction Tree Algorithm
T
ollect & Distribute
C
lgorithmic Complexity
A
rgMax Junction Tree Algorithm
A
Tony Jebara, Columbia U
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 12
ixture Models and Hidden Variables
M
lustering
C
-Means
K
xpectation Maximization
E
Tony Jebara, Columbia University
ith mixtures
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 4
utorial: Matlab
T
erceptron, Online & Stochastic Gradient Descent
P
onvergence Guarantee
C
erceptron vs. Linear Regression
P
ulti-
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 9
ontinuous Probability Models
C
aussian Distribution
G
aximum Likelihood Gaussian
M
ampling from a Gaussian
S
Tony Jebara, Columbia
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 8
iscrete Probability Models
D
ndependence
I
ernoulli
B
ext: Nave Bayes
T
ultinomial
M
ext: Bag of Words
T
Tony Jebara, Columbia Un
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 7
nsupervised Learning
U
tatistical Perspective
S
robability Models
P
iscrete & Continuous: Gaussian, Bernoulli, Multinomial
D
aximu
Tony Jebara, Columbia University
Machine Learning
4771
Instructor: Tony Jebara
Tony Jebara, Columbia University
Topic 2
egression
R
mpirical Risk Minimization
E
east Squares
L
igher Order Polynomials
H
nder-fitting / Over-fitting
U
ross-Validation
C