EE263
S. Lall
2009.11.12.01.
Midterm exam 0910 Solutions
This is a 24hour takehome exam. You may use any books, lecturenotes or computer programs
that you wish, but you may not discuss this exam with anyone until Saturday Nov 7, after
everyone has taken it.
The only exception is that you can ask the TA’s or Sanjay Lall for
clarification.
Web page for typos and datafiles.
The web page
http://junction.stanford.edu/~lall/ee263/midterm
contains any needed files, and it should always contain all known typos; check this before
contacting one of us for clarification. Please note that we have tried pretty hard to make the
exam unambiguous and clear, so we are unlikely to say much. If you choose to send out an email
for clarification, please use the staff email address
[email protected]
so that you can get reply as soon as possible.
Since you have 24 hours, we expect your solutions to be legible, neat, and clear. Do not hand
in your rough notes, and please try to simplify your solutions as much as possible.
Good Luck!
1.
Minimizing the perpendicular distance
0025
Suppose
c
∈
R
n
and
v
∈
R
n
, and
v
is a unit vector. The set
L
=
{
vz
+
c

z
∈
R
}
is a line in
R
n
. For any point
x
∈
R
n
the distance between
x
and
L
is defined by
d
(
x,L
) = min
y
∈
L
bardbl
x
−
y
bardbl
We have
m
points
x
1
,...,x
m
∈
R
n
, and we would like to find the line
L
that minimizes
the total squared distance
d
total
=
m
summationdisplay
i
=1
d
(
x
i
,L
)
2
This is illustrated below.
x
4
x
3
x
2
x
1
That is, we’d like to fit the points to a straight line, as close as possible in the sense that
d
total
is minimized.
(a) Show that
d
(
x,L
) =
bardbl
(
I
−
vv
T
)(
x
−
c
)
bardbl
(b) Define
Q
=
I
−
vv
T
. Show that
Q
is a projection matrix. What are its eigenvalues?
What is the dimension of null(
Q
)?
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
EE263
S. Lall
2009.11.12.01.
(c) We now proceed to solve the above optimization problem. Show that the optimal
c
must satisfy
Q
(
c
−
μ
) = 0
where we define
μ
to the be centroid of the points, given by
μ
=
1
m
m
summationdisplay
i
=1
x
i
Hence show that one optimal
c
is given by
c
opt
=
μ
Are there any other choices of
c
which are optimal?
(d) With the above choice of
c
, the total distance squared is given by
d
total
=
m
summationdisplay
i
=1
bardbl
(
I
−
vv
T
)(
x
i
−
μ
)
bardbl
2
What is the optimal
v
, in terms of the
x
i
and
μ
?
(e) The file
min
perp
distance.m
contains a matrix
X
∈
R
2
×
m
.
Let
x
i
be the
i
’th
column of
X
.
Find the optimal
c
and
v
, and plot the resulting line and the data
points.
Solution.
(a) We have
d
= min
z
bardbl
vz
+
c
−
x
bardbl
and so the leastsquares solution is
z
= (
v
T
v
)
−
1
v
T
(
x
−
c
)
=
v
T
(
x
−
c
)
which gives the result.
(b) Pick a matrix
U
2
so that
W
=
bracketleftbig
v U
2
bracketrightbig
is orthogonal. Then it is easy to verify that
Q
=
W
bracketleftbigg
0
0
0
I
bracketrightbigg
W
T
and so the eigenvalues of
Q
are 0 and 1.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 BOYD,S
 Singular value decomposition, singular values, Lall, S. Lall

Click to edit the document details