Solutions to Homework Set Two
ECE 271A
Electrical and Computer Engineering
University of California San Diego
1.
a) Start with a vector a and
aT =
ai i .
i
It should not be too dicult to see that this is just
aT = a.
Next, consider a matrix C and
c1

Solutions to Practice Problems
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2010
1. We have seen, in problem set 3 that the least squares problem corresponds to a probabilistic model
of the form
PZ|

Bayesian parameter estimation
Nuno Vasconcelos
UCSD
1
Maximum likelihood
parameter estimation in three steps:
1) choose a parametric model for probabilities
to make this clear we denote the vector of parameters b
e
ector
by
PX ( x; )
note that this mea

ECE 271A
ECE-271A
Statistical Learning I:
Bayesian parameter
estimation
Nuno Vasconcelos
ECE Department, UCSD
Bayesian estimation
last class we considered the Gaussian problem
PX | (x | ) = G (x , , 2 ), 2 k
known
P ( ) = G ( , 0 , 02 )
and showed that
2

Kernel-based density estimation ti ti
Nuno Vasconcelos ECE Department, UCSD p ,
Announcement
last week of classes we will have Cheetah Day (exact day TBA) y( y ) what:
4 teams of 6 people each team will write a report on the 4 cheetah problems each team

Expectation-Maximization
Nuno Vasconcelos
ECE Department, UCSD
p
,
Recall
last class, we will have Cheetah Day
what:
4 teams, average of 6 people
each team will write a report on the 4
p
cheetah problems
each team will give a presentation on one
of the

Expectation-Maximization
Nuno Vasconcelos
ECE Department, UCSD
Expectation-maximization
we have seen that EM is a framework for ML estimation
with missing data
i.e. problems where we have, two types of random
variables
X observed random variable
Z hidde

Mixture density estimation
Nuno Vasconcelos
ECE Department, UCSD
Recall
last class, we will have Cheetah Day
what:
4 teams, average of 6 people
each team will write a report on the 4
cheetah problems
each team will give a presentation on one
of the pro

Mid-term review solutions
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2008
1. a) The posterior is given by
PY |X (1|x)
=
=
=
PX|Y (x|1)PY (1)
PX|Y (x|1)PY (1) + PX|Y (x|0)PY (0)
PX|Y (x|1)
PX|Y (x|

Mid-term review
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2008
1. Consider a classication problem with two Gaussian classes
PX|Y (x|i) = G(x, i , ), i cfw_0, 1
of equal probability
PY (i) = 1/2.

Mixture density estimation
Nuno Vasconcelos
ECE Department, UCSD
p
,
Recall
last class, we will have Cheetah Day
what:
4 teams, average of 6 people
each team will write a report on the 4
p
cheetah problems
each team will give a presentation on one
of t

Midterm Overview
ECE 253 Fall14
UCSD
Summary
Fundamentals (Ch 2)
Spatial Filtering (Ch 3)
Frequency filtering (Ch 4)
Noise (Ch 5)
Morphological processing (Ch 9)
Segmentation (Ch 10)
The following slides DO NOT cover everything that
could be in the exam,

Course Outline
ECE271A Statistical Learning I
Department of Electrical and Computer Engineering
University of California, San Diego
Nuno Vasconcelos
Your responsibilities in this class fall into three main categories:
1. Class participation and homework 3

Solutions to Homework Set Six
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2006
1.
a) This is a standard ML estimation problem. The solution is
=
arg max L()
=
arg max log PX (D; )
=
arg max log
=
1

ECE-271A
Statistical Learning I:
Bayesian parameter
estimation
Nuno Vasconcelos
ECE Department, UCSD
Bayesian parameter estimation
the main difference with respect to ML is that in the
Bayesian case is a random variable
basic concepts
training set D = cf

Bayesian parameter estimation
Nuno Vasconcelos UCSD
1
Bayesian parameter estimation
the main difference with respect to ML is that in the Bayesian case is a random variable basic concepts
training set D = cfw_x1 , ., xn of examples drawn independently p

Solutions to Homework Set One
ECE 271A
Electrical and Computer Engineering
University of California San Diego
1.
a) For this problem, the Bayesian decision rule is to guess heads when
PS|R (heads|heads) >
PR|S (heads|heads)PS (heads) >
(1 1 ) >
>
and tai

Homework 1- (Amey Paranjape
a)
Prior ProbabilitiesP(Cheetah)= 250/(250+1053)=0.1918
P(Grass) = 1- P(Cheetah) =0.8081
b)
Histogram for Background- PX|Y (x|grass)
A53218045)
Histogram for Foreground- PX|Y (x|cheetah)
c) Cheetah mask produced after computati

Solutions to Homework Set Four
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) The main dierence with respect to what we have seen so far is that, in the regression problem
everything is conditioned o

Mid-term solutions
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2016
1. We know that when the classes have equal probability and identity covariance, the boundary between
class i and j is the hyperp

PROBLEMS 65
PROBLEMS
Section 2.1
1. In the two-category case, under the Bayes decision rule the conditional error
is given by Eq. 7. Even if the posterior densities are continuous. this form of
the conditional error virtually always leads to a discont

Bayesian decision theory
Nuno Vasconcelos
ECE Department,
p
, UCSD
Notation
the notation in DHS is quite sloppy
e.g.
e g show that
P (error ) = P(error | z ) P( z )dz
really not clear what this means
we will use the following notation
PX |Y ( x0 | y0 )

Final practice problems
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
Fall 2008
1. Least squares with missing data Consider the least squares problem where we have two random
variables Z an

Solutions to Homework Set Four
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) The main dierence with respect to what we have seen so far is that, in the regression problem
everything is conditioned o

Solutions to Homework Set Three
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2010
1.
a) To minimize f () = |z |2 we compute the gradient
f
= 2T (z )
and set it to zero, which leads to the standard l

Solutions to Homework Set Five
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2006
1. Problem 3.8.38 in DHS
i) Since y = wT x, we have
J1 (w)
=
=
=
(1 2 )2
2
2
1 + 2
wT (1 2 )(1 2 )T w
wT 1 w + wT 2 w

Bayesian decision theory
Nuno Vasconcelos ECE Department, UCSD p ,
Notation
the notation in DHS is quite sloppy
e g show that e.g.
P (error ) = P(error | z ) P( z )dz
really not clear what this means
we will use the following notation
PX |Y ( x0 | y0 )