CSC 411 / CSC D11
Quadratics
4
Quadratics
The objective functions used in linear leastsquares and regularized leastsquares are multidimen
sional quadratics. We now analyze multidimensional quadratics further. We will see many more
uses of quadratics further in the course, particularly when dealing with Gaussian distributions.
The general form of a onedimensional quadratic is given by:
f
(
x
) =
w
2
x
2
+
w
1
x
+
w
0
(1)
This can also be written in a slightly different way (called standard form):
f
(
x
) =
a
(
x
−
b
)
2
+
c
(2)
where
a
=
w
2
,b
=
−
w
1
/
(2
w
2
)
,c
=
w
0
−
w
2
1
/
4
w
2
. These two forms are equivalent, and it is
easy to go back and forth between them (e.g., given
a,b,c
, what are
w
0
,w
1
,w
2
?). In the latter
form, it is easy to visualize the shape of the curve: it is a bowl, with minimum (or maximum) at
b
, and the “width” of the bowl is determined by the magnitude of
a
, the sign of
a
tells us which
direction the bowl points (
a
positive means a convex bowl,
a
negative means a concave bowl), and
c
tells us how high or low the bowl goes (at
x
=
b
). We will now generalize these intuitions for
higherdimensional quadratics.
The general form for a 2D quadratic function is:
f
(
x
1
,x
2
) =
w
1
,
1
x
2
1
+
w
1
,
2
x
1
x
2
+
w
2
,
2
x
2
2
+
w
1
x
1
+
w
2
x
2
+
w
0
(3)
and, for an
N
D quadratic, it is:
f
(
x
1
,...x
N
) =
summationdisplay
1
≤
i
≤
N,
1
≤
j
≤
N
w
i,j
x
i
x
j
+
summationdisplay
1
≤
i
≤
N
w
i
x
i
+
w
0
(4)
Note that there are three sets of terms: the quadratic terms (
∑
w
i,j
x
i
x
j
), the linear terms (
∑
w
i
x
i
)
and the constant term (
w
0
).
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 DavidFleet
 Differential Calculus, Machine Learning, Optimization, Quadratic equation, Stationary point

Click to edit the document details