Homework Set Two
ECE 175
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. Consider the matrix
A=
5
b
b
5
and the function
f (x) = xT Ax
in the range 100 x1 100, 100 x2 100.
a) using MATLAB, make bot
EM Algorithm &
High Dimensional Data
Nuno Vasconcelos
(Ken Kreutz-Delgado)
Kreutz Delgado)
UCSD
Gaussian EM Algorithm
For the Gaussian mixture model, we have
Expectation Step (E-Step):
Maximization Step (M
(M-Step):
Step):
2
EM versus K-means
EM
k-Means
MLE & Regression
Nuno Vasconcelos
(Ken Kreutz-Delgado)
UCSD
Statistical Learning from Data
Goal: Given a relationship between
a feature vector x and a vector y,
and iid data samples (xi,yi), find an
approximating function f (x) y
x
y = f ( x ) y
f ( )
T
Solutions to Homework Set Three
ECE 175
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.a) The constants are found by setting the integral of the PDF equal to 1.
Z
|xaj |
1 =
Kj e bj dx
Z
x
= 2Kj
e bj dx
0
=
2Kj
Least Squares
Nuno Vasconcelos
(Ken Kreutz-Delgado )
UCSD
(Unweighted) Least Squares
Assume linearity in the unknown,
deterministic model parameters
Scalar,
S l additive
dditi noise
i model:
d l
y = f ( x; ) + = ( x )T +
E.g., for a line (f affine in
Maximum Likelihood Estimation
(MLE)
Nuno Vasconcelos
(Ken Kreutz-Delgado)
UCSD
BDR (under 0/1 Loss)
For the zero/one loss, the following three decision rules
are optimal and equivalent
1)
i * ( x ) = arg max PY | X ( i | x )
i
2)
i ( x ) = arg max PX |
Homework Set One
ECE 175
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. a) Consider the matrix
A=
1
0
1
2
0
0
i) what is the column space of A? ii) what is its row space? iii) what is its null spa
Clustering &
Unsupervised Learning
Nuno Vasconcelos
(Ken Kreutz-Delgado)
UCSD
Statistical Learning
Goal: Given a relationship between
a feature vector x and a vector y,
and iid data samples (xi,yyi),
) find an
approximating function f (x) y
x
f ( )
y = f
Homework Set Three
ECE 175
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. Suppose that we have a classification problem with two classes of equal probability, i.e. PY (0) =
PY (1) = 1/2, and class
Solutions to Homework Set One
ECE 175
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1. a)
i) the column space of A is the space spanned by the columns, i.e. the set of vectors of the form
v = a + b
where a = (1, 0