CSC 411 / CSC D11
Quadratics
4
Quadratics
The objective functions used in linear least-squares and regularized least-squares are multidimen-
sional quadratics. We now analyze multidimensional quadratics further. We will see many more
uses of quadratics further in the course, particularly when dealing with Gaussian distributions.
The general form of a one-dimensional quadratic is given by:
f
(
x
) =
w
2
x
2
+
w
1
x
+
w
0
(1)
This can also be written in a slightly different way (called standard form):
f
(
x
) =
a
(
x
−
b
)
2
+
c
(2)
where
a
=
w
2
, b
=
−
w
1
/
(2
w
2
)
, c
=
w
0
−
w
2
1
/
4
w
2
. These two forms are equivalent, and it is
easy to go back and forth between them (e.g., given
a, b, c
, what are
w
0
, w
1
, w
2
?). In the latter
form, it is easy to visualize the shape of the curve: it is a bowl, with minimum (or maximum) at
b
, and the “width” of the bowl is determined by the magnitude of
a
, the sign of
a
tells us which
direction the bowl points (
a
positive means a convex bowl,
a
negative means a concave bowl), and
c
tells us how high or low the bowl goes (at
x
=
b
). We will now generalize these intuitions for
higher-dimensional quadratics.
The general form for a 2D quadratic function is:
f
(
x
1
, x
2
) =
w
1
,
1
x
2
1
+
w
1
,
2
x
1
x
2
+
w
2
,
2
x
2
2
+
w
1
x
1
+
w
2
x
2
+
w
0
(3)
and, for an
N
-D quadratic, it is:
f
(
x
1
, ...x
N
) =
s
1
≤
i
≤
N,
1
≤
j
≤
N
w
i,j
x
i
x
j
+
s
1
≤
i
≤
N
w
i
x
i
+
w
0
(4)
Note that there are three sets of terms: the quadratic terms (
∑
w
i,j
x
i
x
j
), the linear terms (
∑
w
i
x
i
)
and the constant term (