Homework 7: Gaussian Processes & Neural Networks
CSCI 1420 & ENGN 2520, Brown University
Homework due at 11:59pm on November 7, 2013
Question 1:
The rst question explores binary Gaussian process classication on a dataset of images
of pedestrians, and back
Homework 10: EM for Factor Analysis & Regression
CSCI 1420 & ENGN 2520, Brown University
Homework due at 11:59pm on December 5, 2013
Question 1:
The MovieLens dataset (http:/movielens.org) contains ratings for M movies, recorded as
integers between 1 and
CSCI 1950-F Homework 8:
K-Means Clustering & Bernoulli Mixture Models
Brown University, Spring 2012
Homework due at 12:00pm on April 26, 2012
Question 1:
In this question, we use the K-means algorithm to cluster the handwritten digit data. For all
section
CSCI 1950-F Homework 9:
EM for Factor Analysis & Regression
Brown University, Spring 2012
Homework due at 12:00pm on May 3, 2012
Question 1:
The MovieLens dataset (http:/movielens.org) contains ratings for M movies, recorded as
integers between 1 and 5, f
CSCI 1950-F Homework 10: HMMs & Topic Models
Brown University, Spring 2012
EXTRA CREDIT: Homework due at 12:00pm on May 10, 2012
Question 1:
We begin by learning hidden Markov models (HMMs) which describe the statistics of English
text. In this applicatio
CSCI 1950-F Homework 1:
Naive Bayes Spam Classication
Brown University, Spring 2012
Homework due at 12:00pm on February 10, 2012
Hello, I am a prince in desperate need of a personal favor. Ive been receiving a lot
of unwanted emails lately and would reall
ENGN 2520 / CSCI 1950-F Homework 1
Solution Key
Problem 1
The quantity p(apple) represents the marginal probability of selecting an apple, so we can simply sum over
all of the boxes:
p(apple | box)p(box)
p(apple) =
boxes
= p(apple | r) p(r) + p(apple | b)
ENGN 2520 / CSCI 1950-F Homework 2
Due Friday February 15 by 4pm
Problem 1
In this problem we consider binary classication under a non-uniform loss function.
Let X be a nite input space and Y = cfw_0, 1 be two possible labels that can be associated
with a
ENGN 2520 / CSCI 1950-F Homework 2
Solution Key
Problem 1
To make an optimal decision rule for all x, for a given x we can choose the action that minimizes its individual
loss. If we choose c(x) = 0:
Ec(x)=0 [L (c (x) , y | x)] = p (y = 0 | x) L (0, 0) +
ENGN 2520 / CSCI 1950-F Homework 3
Due Friday March 1 by 4pm
Problem 1
Consider binary classication (Y = cfw_0, 1) with logistic regression. In logistic regression we
assume p(y = 1|x) = (wT (x) for a feature vector (x) Rk and parameters w Rk .
The likeli
ENGN 2520 / CSCI 1950-F Homework 3
Solution Key
Problem 1
Part A
We can plug the denition of logistic regression into the denition of the error function:
E(w) = log p(Dy | Dx , w)
n
p(yi | xi , w)
= log
i=1
n
(wT (xi )yi (1 (wT (xi )1yi
= log
i=1
n
yi log
ENGN 2520 / CSCI 1950-F Homework 4
In this assignment you will implement a multiclass SVM to recognize handwritten digits.
You will use the data from Homework 2 that is available on the course website.
A multiclass SVM learns functions from RD to cfw_1, .
ENGN 2520 / CSCI 1950-F Homework 5
Due Friday March 15 by 4pm
Problem 1
Let S = cfw_s1 , . . . , sK be the set of states of a Markov chain. Let be the initial state distribution and M be the transition matrix. Suppose we have a training set T = cfw_x1 ,
ENGN 2520 / CSCI 1950-F Homework 6
Due Tuesday April 16 by 4pm
Students may discuss and work on homework problems in groups. However, each student
must write down their solutions independently.
All of the work submitted should be your own. NO COPYING from
CSCI 1950-F Homework 7:
Gaussian Processes & Laplace Approximations
Brown University, Spring 2012
Homework due at 12:00pm on April 12, 2012
Question 1:
The rst question explores binary Gaussian process classication on a dataset of images
of pedestrians, a
CSCI 1950-F Homework 6: Regularization & Sparsity
Brown University, Spring 2012
Homework due at 12:00pm on April 5, 2012
Question 1:
This problem compares various approaches to regularization and feature selection for binary
classication. Let yi cfw_+1, 1
CSCI 1950-F Homework 5: Logistic Regression
Brown University, Spring 2012
Homework due at 12:00pm on March 12, 2012
Question 1:
In this question, we consider a continuous estimation problem in which the input x and
response variable y are both real number
Homework 9: Hidden Markov Models
CSCI 1420 & ENGN 2520, Brown University
Homework due at 11:59pm on November 21, 2013
Question 1:
We rst examine a simple hidden Markov model (HMM). We observe a sequence of rolls of
a four-sided die at an occasionally dish
Homework 8: K-Means Clustering & Bernoulli Mixtures
CSCI 1420 & ENGN 2520, Brown University
Homework due at 11:59pm on November 14, 2013
Question 1:
We rst use the K-means algorithm to cluster handwritten digit data. We use 1, 000 examples
of each of the
Homework 5: Logistic Regression
CSCI1420 & ENGN2520, Brown University
Homework due at 11:59pm on October 17, 2013
Question 1:
This problem investigates logistic regression classiers. Let y cfw_1, . . . , C denote the discrete class label we want to predic
Homework 6: Regularization & Sparsity
CSCI 1420 & ENGN 2520, Brown University
Homework due at 11:59pm on October 31, 2013
Question 1:
This problem compares various approaches to regularization and feature selection for binary
classication. Let yi cfw_+1,
Homework 1: Naive Bayes Spam Classication
CSCI1420 & ENGN2520, Brown University; revised Sept. 15, 2013
Homework due at 11:59pm on September 19, 2013
Question 1:
Hello, I am a prince in desperate need of a personal favor. Ive been receiving a lot of unwan
Homework 2: ML & Bayesian Estimation
CSCI1420 & ENGN2520, Brown University
Homework due at 11:59pm on September 26, 2013
Question 1:
We begin by considering examples, produced by a sophisticated simulator, of data which
might be collected by a gamma teles
Homework 3: Handwritten Digit Classication
CSCI1420 & ENGN2520, Brown University
Homework due at 11:59pm on October 3, 2013
In this problem set, we consider the problem of handwritten digit recognition. We will use
a subset of the MNIST database, which ha
Homework 4: Linear Regression
CSC1420 & ENGN2520, Brown University
Homework due at 11:59pm on October 10, 2013
In the rst two problems, we study dierent approaches to linear regression using a onedimensional dataset collected from a simulated motorcycle a
ENGN 2520 / CSCI 1950-F Homework 1
Due Friday February 8 by 4pm
Problem 1
Suppose we have three boxes r (red), b (blue), and g (green). Box r contains 3 apples, 4
oranges, and 3 limes, box b contains 1 apple, 1 orange and 0 limes, and box g contains 3
app
CSCI 1950-F Homework 2: ML & Bayesian Estimation
Brown University, Spring 2012
Homework due at 12:00pm on February 16, 2012
We begin by considering examples, produced by a sophisticated simulator, of data which
might be collected by a gamma telescope obse
CSCI 1950-F Homework 3:
Handwritten Digit Classication
Brown University, Spring 2012
Homework due at 12:00pm on February 23, 2012
In this problem set, we consider the problem of handwritten digit recognition. We will
use a subset of the MNIST database, wh
CSCI 1950-F Homework 4: Linear Regression
Brown University, Spring 2012
Homework due at 12:00pm on March 1, 2012
In this problem set, we study dierent approaches to linear regression using a onedimensional dataset collected from a simulated motorcycle acc
ENGN 2520 / CSCI 1950-F Homework 7
Due Monday April 29 by 4pm
In this homework you will implement the EM algorithm for tting mixtures of Gaussians.
As described in class EM is an iterative method with two steps. We start with some
initial parameters and r