CSCI 5525: Machine Learning (Fall16)
Homework 2 Solution
1. (20 points)
(a) (10 points)
K is symmetric:
K(p, q) =
m
X
wj Kj (p, q) =
j=1
m
X
wj Kj (q, p) = K(q, p)
j=1
K is PSD:
m
m
X
X
u Ku = u (
wj
CSCI 5525: Machine Learning (Fall16)
Homework 1 Solution
1.(a) The expected loss is:
Z Z
Z Z
E(x,y) [`(f (x), y)] =
`(f (x), y)p(x, y)dydx =
x
y
`(f (x), y)p(y|x)dy p(x)dx
x
Z Z
=
x
y
(f (x) y)2 p(y|x
PCA vs FA
! PCA
! FA
Project x to z
Combine z to x
z = WT(x !)
x ! = Vz + !
x
z
z
x
E. Alpaydin, Introduction to Machine Learning
Factor Analysis
! Find
a small number of factors z, which when
combine
Kernel Methods
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
October 4, 2016
Instructor: Arindam Banerjee
Kernel Methods
Non-linear SVMs
All important equations have dot-products
Dual is ex
Linear Models for Regression
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 6, 2016
Instructor: Arindam Banerjee
Linear Models for Regression
Linear Models
Linear models over featu
Linear Discriminants
CSci 5525: Machine Learning
Instructor: Arindam Banerjee
September 13, 2016
Instructor: Arindam Banerjee
Linear Discriminants
Discriminant Functions
One of the simplest representa
Problem 1) 1st pic is A 2nd is B and so on, the document is being difficult
Problem 2) I struggled alot with this one, I have attached pictures of all the work I was able to
perform
The first 2 pictur
1. Least squares regression
a.
b.
c. Yes, the error rate over the test set using the 2nd polynomial formula will result in
less errors, in general, because the line is more flexible so the line can fi
1 a)
1 b)
1 c)
1 d)
2 a)
2b)
2c)
Q3
MyLogisticReg2 with Boston50
K
1
2
3
4
5
Mean
Std
.18
.21
.33
.36
.04
.22
.11
MyLogisiticReg2 with Boston75
K
1
2
3
4
5
Mean
STD
.07
.02
.16
.12
.04
.08
.05
Logisti
Nam .
e. Page 1 of 8
CSci 5521: Introduction to Machine Learning
(Spring17)
MidTerm Exam
This is a closed book exam. You are allowed 4 sheets of notes.
Please explain your answers clearly. You will
Linear Models for Regression
CSci 5525: Machine Learning
Instructor: Paul Schrater
Linear Models for Regression
Linear Models
Linear models over feature representations j
f (x, w) =
M
X
wj j (x) = wT
CSCI5525: Machine Learning (Spring 2012)
Clustering
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Mixture of Gaussians
K
p( x | ") = % # iN ( x | i , $i ),
i =1
K
wi
CSCI 5525: Machine Learning (Spring 2012)
Multilayer Perceptron
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Tuning the Network Size
Destructive
Weight decay:
penal
CSCI 5525: Machine Learning (Spring 2012)
Dimension Reduction
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Feature Selection
NP-hard to search through all the combi
CSCI 5525: Machine Learning (Spring 2012)
Dimension Reduction
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Linear Discriminant Analysis
Find
a low-dimensional
spac
CSCI 5525: Machine Learning (Spring 2012)
Course Overview
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Welcome to CSci 5525
Course: Machine Learning
Instructor: Rui
CHAPTER 5:
Multivariate Methods
E. Alpaydin, Introduction to Machine Learning
Multivariate Data
Multiple
measurements (sensors)
d inputs/features/attributes: d-variate
N instances/observations/exam
CSCI 5525: Machine Learning (Spring 2012)
Bayes Decision
Theory and
Parametric Models
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Regression example
Coefficients i
CSCI 5525: Machine Learning (Spring 2012)
Bayes Decision
Theory and
Parametric Models
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Parametric vs Nonparametric
Param
CSCI 5525: Machine Learning (Spring 2012)
Bayes Decision
Theory and
Parametric Models
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Parametric Classification
Discri
CSCI 5525: Machine Learning (Spring 2012)
Supervised Learning
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Supervised Learning
Classification
Regression
Input Fea
CSCI 5525: Machine Learning (Spring 2012)
Supervised Learning
Rui Kuang
Department of Computer Science and Engineering
University of Minnesota
Supervised Learning
Classification
Data:
Regression
X =
CSCI 5521: Introduction to Machine
Learning (Fall 2017)
1
Homework 0
Due date: Monday, Sep 18th, 11:55pm
1. (a) Which of the following courses have you taken?
i. Artificial Intelligence II
ii. Introdu