Normal Equations
If
b
is not in the column space of
A
, then
Ax
=
b
has no solution; the system is
inconsistent
. This is typical if
A
is
m
×
n
with
m > n
, which we will assume here.
Let us also assume that
A
has full rank. Since
Ax
=
b
has no solution, one may
reasonably be interested in ﬁnding a vector
x
which minimizes the diﬀerence
between
b
and
Ax
:
min
x
k
Ax

b
k
.
(1)
Equivalently: ﬁnd a vector
y
in the column space of
A
which is closest to
b
(then
x
is the unique solution of the
consistent
system
Ax
=
y
). There are many norms that
we might use in (1), but if we use the norm induced by the dot product, then (1) is
called the
discrete linear least squares problem
:
min
x
k
Ax

b
k
2
.
(2)
Now suppose that we want to ﬁnd an element of
S
≤
R
n
that is closest to some
vector
b
(which is typically not in
S
). Our intuition says that we “drop a
perpendicular” from b to
S
, and that is exactly right: Let
y
∈
S
be such that
b

y
⊥
S
, and consider any vector
w
=
y
+
αz
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '10
 MARK
 Linear Algebra, ax, normal equations, min Ax

Click to edit the document details