Solutions to Homework Set Five
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2010
1.
a) In this case the BDR is to say
g(X )
=
arg max log PX|Y (X |i) + log
=
arg max
=
i
i
arg max
i
1
c
log PX|Y (xk
Solutions to Homework Set One
ECE 271A
Electrical and Computer Engineering
University of California San Diego
1.
a) For this problem, the Bayesian decision rule is to guess heads when
PS|R (heads|heads) >
PR|S (heads|heads)PS (heads) >
(1 1 ) >
>
and tai
Homework Set One
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
The purpose of this assignment is to give you experience with Bayesian decision theory. The rst
four problems are of the pen-a
The Gaussian classifier
Nuno Vasconcelos
ECE Department, UCSD
p
,
Bayesian decision theory
recall that we have
Y state of the world
X observations
g(x) decision function
L[g(x),y] loss of predicting y with g(x)
Bayes decision rule is the rule that min
Maximum likelihood estimation
Nuno Vasconcelos
UCSD
Maximum likelihood
parameter estimation in three steps:
1) choose a parametric model for probabilities
to make this clear we denote the vector of parameters b
e
ector
by
PX ( x; )
note that this means
The Gaussian classifier
Nuno Vasconcelos ECE Department, UCSD p ,
Bayesian decision theory
recall that we have
Y state of the world X observations g(x) decision function L[g(x),y] loss of predicting y with g(x)
Bayes decision rule is the rule that minimi
Bayesian decision theory
Nuno Vasconcelos
ECE Department, UCSD
p
,
Bayesian decision theory
recall that we have
Y state of the world
X observations
g(x) decision function
L[g(x),y] loss of predicting y with g(x)
the expected value of the loss is calle
Maximum likelihood estimation
Maximum likelihood estimation
Nuno Vasconcelos
UCSD
Bayesian decision theory
recall that we have
Y state of the world
X observations
g(x) decision function
L[g(x),y] loss of predicting y with g(x)
Bayes decision rule is the
Solutions to Homework Set Two
ECE 271A
Electrical and Computer Engineering
University of California San Diego
1.
a) Start with a vector a and
aT =
a i i .
i
It should not be too dicult to see that this is just
aT = a.
Next, consider a matrix C and
c1
ECE 250 FALL QUARTER 2015
ELECTRICAL & COMPUTER ENGINEERING
Course:
ECE 250 Random Processes
URL:
https:/sites.google.com/a/eng.ucsd.edu/ece-250-f2015/
Text:
H. Stark & J. W. Woods, Probability and Random Processes with
Applications to Signal Processing,
Solutions to Homework Set Three
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) To minimize f () = |z |2 we compute the gradient
f = 2T (z )
and set it to zero, which leads to the standard least squa
Homework Set Four
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. Bayesian regression: in last weeks problem set we showed that various forms of linear regression
by the method of least sq
Solutions to Homework Set Two
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) Start with a vector a and
aT =
ai i .
i
It should not be too dicult to see that this is just
aT = a.
Next, consider a m
Homework Set Two
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. Problem 2.6.26 in Duda, Hart, and Stork (DHS).
2. In this problem we will consider the ML estimate of the parameters of a m
Homework Set Three
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. In this problem we will consider the issue of linear regression and the connections between maximum
likelihood and least
Homework Set Five
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. BDR and nearest neighbors Consider a classication problem with c classes and uniform class
probabilities, i.e. PY (i) = 1/
Final practice problems
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
1. Least squares with missing data Consider the least squares problem where we have two random
variables Z and X, such
Mid-term review solutions
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2014
1. a) The posterior is given by
PY |X (1|x) =
=
=
PX|Y (x|1)PY (1)
PX|Y (x|1)PY (1) + PX|Y (x|0)PY (0)
PX|Y (x|1)
PX|Y (x|
Mid-term review
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1. Consider a classication problem with two Gaussian classes
PX|Y (x|i) = G(x, i , ), i cfw_0, 1
of equal probability
PY (i) = 1/2.
In class,
Solutions to Practice Problems
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1. We have seen, in problem set 3 that the least squares problem corresponds to a probabilistic model
of the form
PZ|X (z|x; )
Solutions to Homework Set One
ECE 271A
Electrical and Computer Engineering
University of California San Diego
1.
a) For this problem, the Bayesian decision rule is to guess heads when
PS|R (heads|heads) >
PR|S (heads|heads)PS (heads) >
(1 1 ) >
>
and tai
Mid-term
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2014
1. John heard on the radio that there is a new bee disease that makes bees suicidal. When they see
water, they try to land on it and drown.
Solutions to Homework Set Three
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) To minimize f () = |z |2 we compute the gradient
f = 2T (z )
and set it to zero, which leads to the standard least squa
Mid-term solutions
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
Fall 2014
1. a) The BDR is to decide for Y = 1 (suicidal) if
PX|Y (x|1)PY (1) PX|Y (x|0)PY (0)
1s
.
PX|Y (x|1)
s
(1)
(2)
This has two poss
Solutions to Homework Set Four
ECE 271A
Electrical and Computer Engineering
University of California San Diego
Nuno Vasconcelos
1.
a) The main dierence with respect to what we have seen so far is that, in the regression problem
everything is conditioned o
ECE 250 SYLLABUS FALL 2015
WEEKS 1, 2 REVIEW OF FUNDAMENTALS
Univariate Distributions and Densities
Multivariate Distributions and Densities
Conditional Distributions and Densities
Characteristic Functions
Moments
Functions of Random Variables
WEEK 2 SOME