lesson 16.pdf - 1 / 11 Lesson 16 Least Squares C. L. Cox...

1 / 11
Lesson 16
Least Squares
C. L. Cox
Clemson University
C. L. Cox Lesson 16, Least Squares

2 / 11
MATH 3650-002 Numerical Methods for Engineers, March 27, 2018
Objectives
To explore:
1
Curve fitting by linear least squares regression
2
Curve fitting by transformation to linear least squares
regression
3
Curve fitting by polynomial regression
C. L. Cox Lesson 16, Least Squares

3 / 11
MATH 3650-002 Numerical Methods for Engineers, March 27, 2018
In Lesson 13, we distinguished between interpolation and curve
fitting , saying that:
Curve fitting is a process by which a
function is constructed to best fit , in
some sense, a given set of data
points.
The goal is to capture the trend of the
data, for example the line in the plot
to the right fits the 5 data points.
A fitting curve is normally not an
interpolant.
x
y
( x
5
, y
5
)
•
•
( x
1
, y
1
)
•
( x
2
, y
2
)
•
( x
3
, y
3
)
•
( x
4
, y
4
)
We will quantify the meaning of best fit , with the concept of
least squares regression .
In linear regression , we fit the data (i.e. ( x
i
, y
i
) , i = 1 . . . n )
with a linear polynomial of the form y = a
0
+ a
1
x .
C. L. Cox Lesson 16, Least Squares

4 / 11
MATH 3650-002 Numerical Methods for Engineers, March 27, 2018
In linear least squares regression
(LLSR) the coefficients a
0
and a
1
are
chosen to minimize the sum of the
error at each point, i.e.
e ( a
0
, a
1
) =
n X
i =1
[ y
i
- ( a
0
+ a
1
x
i
)]
2
The residual at the i
th
point is
r
i
= y
i
- ( a
0
+ a
1
x
i
)
x
y
y = a
1
+ a
0
( x
5
, y
5
)
•
| r
5
|
•
| r
1
|
( x
1
, y
1
)
•
( x
2
, y
2
)
| r
2
|
•
( x
3
, y
3
)
| r
3
|
•
( x
4
, y
4
)
| r
4
|
So LLSR minimizes the 2