LEAST SQUARES AND DATA
Math 21b, O. Knill
Section 5.4 : 2,10,22,34,40,16*,18*, it is ok to use technology (i.e. Mathematica) for 34 or 40 is
ok but write down the matrices and the steps.
GOAL. The best possible ”solution” of an inconsistent linear systems
Ax
=
b
will be called the
least square
solution
. It is the orthogonal projection of
b
onto the image im(
A
) of
A
. The theory of the kernel and the image
of linear transformations helps to understand this situation and leads to an explicit formula for the least square
fit. Why do we care about nonconsistent systems? Often we have to solve linear systems of equations with more
constraints than variables. An example is when we try to find the best polynomial which passes through a set of
points. This problem is called
data fitting
. If we wanted to accommodate all data, the degree of the polynomial
would become too large, the fit would look too wiggly. Taking a smaller degree polynomial will not only be
more convenient but also give a better picture. Especially important is
regression
, the fitting of data with lines.
The above pictures show 30 data points which are fitted best with polynomials of degree 1, 6, 11 and 16. The
first linear fit maybe tells most about the trend of the data.
THE ORTHOGONAL COMPLEMENT OF
im(
A
). Because a vector is in the kernel of
A
T
if and only if
it is orthogonal to the rows of
A
T
and so to the columns of
A
, the kernel of
A
T
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '03
 JUDSON
 Linear Algebra, Algebra, Matrices, Least Squares, ax, orthogonal projection

Click to edit the document details