LS Regression  1
16  Least Squares Regression
The old adage, KISS (keep it simple stupid) forms the basis for regression analysis. In the
preceding discussion on Lagrangian interpolation, we started with
n
points, and obtained
a polynomial with exactly
n
—1 terms that approximated the
n
points. In regression
analysis, we again start with
n
points, but instead of finding a polynomial with
n
terms,
we fit an arbitrary function (not necessarily a polynomial) with a number of terms that we
are free to select, as long as it is less than
n
—1.
The most wellknown example of regression is the fitting of a straight line (twoterm
polynomial) to a series of
n
data points through a least squares approximation. This is
known as
least squares linear regression.
The following figure shows a series of 100 points.
With linear regression we want to fit a straight line of the form y = mx+b to the series of
data points. In other words, we want to find
m
and
b
.
In order to do so, we recognize that
each data point represents an equation:
...
2
2
1
1
b
mx
y
b
mx
y
+
=
+
=
In matrix form, these relations can be written as:
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
LS Regression  2
=
3
2
1
100
2
1
...
1
...
...
1
1
y
y
y
m
b
x
x
x
Note that it would be trivial to reformulate this problem to fit a parabola or a cubic
equation. The formulation for a cubic fit of the form
y
=
a
0
+
a
1
x
+
a
2
x
2
+
a
3
x
3
would be
=
100
2
1
3
2
1
0
3
100
2
100
100
3
2
2
2
2
3
1
2
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Patzek
 Linear Algebra, Least Squares, Regression Analysis, 2 j, 1 j, 2 j

Click to edit the document details