This preview shows page 1. Sign up to view the full content.
Normal Equations
If
b
is not in the column space of
A
, then
Ax
=
b
has no solution; the system is
inconsistent
. This is typical if
A
is
m
×
n
with
m > n
, which we will assume here.
Let us also assume that
A
has full rank. Since
Ax
=
b
has no solution, one may
reasonably be interested in ﬁnding a vector
x
which minimizes the diﬀerence
between
b
and
Ax
:
min
x
k
Ax

b
k
.
(1)
Equivalently: ﬁnd a vector
y
in the column space of
A
which is closest to
b
(then
x
is the unique solution of the
consistent
system
Ax
=
y
). There are many norms that
we might use in (1), but if we use the norm induced by the dot product, then (1) is
called the
discrete linear least squares problem
:
min
x
k
Ax

b
k
2
.
(2)
Now suppose that we want to ﬁnd an element of
S
≤
R
n
that is closest to some
vector
b
(which is typically not in
S
). Our intuition says that we “drop a
perpendicular” from b to
S
, and that is exactly right: Let
y
∈
S
be such that
b

y
⊥
S
, and consider any vector
w
=
y
+
αz
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 12/18/2010 for the course PHYS 5073 taught by Professor Mark during the Fall '10 term at Arkansas.
 Fall '10
 MARK

Click to edit the document details