102
Introduction EGR to
Engineering Modeling
Curve Fitting
Linear Regression
Regression
Chapter 14.1-14.3
Figures from: Applied Numerical Methods with MATLAB, Steven Chapra, McGraw Hill
EGR 102
Lecture 4
1
Objectives
Become familiar with basic statistics and the normal
distribution.
Learn how to compute the slope and intercept of a best-fit
straight line with linear regression.
Learn how to compute and understand the meaning of the
how to compute and understand the meaning of the
coefficient of determination and the standard error of the
estimate.
Learn how to use transformations to linearize nonlinear
Learn how to use transformations to linearize nonlinear
equations so that they can be fit with linear regression.
EGR 102
Lecture 4
2
The Need to Account for
The Need to Account for
Variability in Engineering
Material Properties
Manufacturing
Materials processing
Materials processing,
machining tolerances,
weld and fastener
locations
Frequency
Stiffness, strength,
conductivity, density
Loading
Magnitude, direction,
direction,
distribution
EGR 102
Value
Lecture 4
3
Statistics Review
Statistics Review
Measures of Central Tendency
Arithmetic mean: the sum of
the individual data points (xi)
divided by the number of
points n:
xi
x=
n
Median: the midpoint of a
group of data.
Mode: the value that occurs
most frequently in group of
most frequently in a group of
data.
EGR 102
Lecture 4
4
Statistics Review
Statistics Review
Measures of Spread
Standard deviation:
St
sy =
n 1
where St is the sum of the squares of the data residuals:
St = (yi y )
2
and n -1 is referred to as the degrees of freedom.
is referred to as the degrees of freedom
Variance:
(y y ) = y ( y ) / n
=
2
s
2
y
i
2
i
2
i
n 1
Coefficient of variation: c.v. = s y 100%
y
EGR 102
n 1
Lecture 4
5
Histogram
For large data sets, the histogram can be approximated by a smooth curve
EGR 102
Lecture 4
6
Normal Distribution
EGR 102
Lecture 4
7
Curve Fitting
Method(s) to fit an equation to discrete data points
Two general approaches:
Data exhibit a significant degree of scatter
Derive a single line (curve) that represents the general trend of the data
Data is very precise
Pass single curve through the points (interpolation)
Pass a single curve through the points (interpolation)
Three general applications in engineering:
Trend analysis
Predicting values between data points or beyond data range
values between data points or beyond data range
Hypothesis testing
Comparing existing mathematical model with measured data
Simplified modeling
Developing simple math models to represent complex phenomenon
EGR 102
Lecture 4
8
Curve Fitting
(a)
(b)
(c)
Data with significant error
Polynomial fit oscillating beyond the range of the data
fit oscillating beyond the range of the data
Linear fit
EGR 102
Lecture 4
9
Linear Least-Squares Regression
Linear least-squares regression is a method to determine
the best coefficients in a linear model for given data set.
Best for least-squares regression means minimizing the
sum of the squares of the estimate residuals. For a straight
line model, this gives:
n
n
Sr = e = (yi a0 a1 xi )
2
2
i
i =1
i =1
This method will yield a unique line for a given set of data.
EGR 102
Lecture 4
10
Least-Squares Fit of a Straight Line
Using the model:
y = a0 + a1 x
where: a0 = intercept & a1 = slope,
the slope and intercept producing the best fit can
the slope and intercept producing the best fit can
be found using:
a1 =
n xi yi xi yi
n x
2
i
( x )
2
i
a0 = y a1 x
EGR 102
Lecture 4
11
Quantification of Error
The residual represents the square of the vertical
distance between a data point and the best-fit line:
n
n
Sr = e = (yi a0 a1 xi )
2
i
i =1
EGR 102
Lecture 4
2
i =1
12
Example
a1 =
V
(m/s)
F
(N)
i
xi
yi
(xi)2
xiyi
1
10
25
100
250
2
20
70
400
30
380
900
40
550
1600
50
610
2500
30500
6
60
1220
3600
73200
7
70
830
4900
58100
8
80
1450
6400
116000
360
5135
20400
= 19.47024
22000
5
8(312850 ) (360)(5135 )
11400
4
( x )
=
1400
3
n xi yi xi yi
312850
n x
2
i
2
i
8(20400 ) (360)
2
a0 = y a1 x = 641.875 19.47024 (45) = 234.2857
Fest = 234 .2857 + 19.47024 v
13
Standard Error of the Estimate
Regression data showing (a) the spread of data around the mean of the
dependent data and (b) the spread of the data around the best fit line:
The reduction in spread represents the improvement due to linear
reduction in spread represents the improvement to due linear
regression.
Standard error of the estimate quantifies
spread around the regression line:
around the regression line:
EGR 102
Lecture 4
sy / x =
Sr
n2
14
Coefficient of Determination
The coefficient of determination r 2 is the difference between the sum
of the squares of the data residuals and the sum of the squares of the
estimate residuals, normalized by the sum of the squares of the data
n
residuals:
id
2
St - Sr
r=
St
2
S t = ( yi y )
i =1
n
n
i =1
i =1
S r = ei2 = ( yi a0 a1 xi )
2
r 2 represents the percentage of the original uncertainty explained by
the model.
For a perfect fit, Sr=0 and r 2=1.
If r 2=0, there is no improvement over simply picking the mean.
If
If r 2<0, the model is worse than simply picking the mean!
0, the model is worse
simply picking the mean!
r is called the correlation coefficient
EGR 102
Lecture 4
15
Coefficient of Determination
An alternative formula more convenient for implementation on a computer:
EGR 102
Lecture 4
16
Example
V
(m/s)
i
xi
yi
Fest = 234.2857 + 19.47024 v
F
(N)
St = (yi y ) = 1808297
2
a0+a1xi
(yi- )2 (yi-a0-a1xi)2
1
10
25
-39.58
380535
4171 Sr = (yi a0 a1 xi ) = 216118
2
20
70
155.12
327041
7245
3
30
380
349.82
68579
911
4
40
550
544.52
8441
5
50
610
739.23
1016
6
60
1220
933.93
334229
7
70
830
1128.63
35391
89180
8
80
1450
1323.33
653066
16044
360
5135
1808297
216118
2
sy =
1808297
= 508.26
8 1
216118
= 189.79
82
16699
1808297 216118
2
= 0.8805
81837 r =
1808297
30
sy / x =
88.05% of the original uncertainty
has been explained by the
linear model
17
Nonlinear Relationships
Linear regression is predicated on the fact that the
relationship between the dependent and independent
variables is linear
variables is linear - this is not always the case.
is not always the case
Three common examples are:
exponential :
y = 1e1 x
power :
y = 2 x 2
x
saturation - growth - rate : y = 3
3 + x
EGR 102
Lecture 4
18
Linearization of Nonlinear Relationships
One option for finding the coefficients for a nonlinear fit is to
linearize it. For the three common models, this may involve
taking logarithms or inversion:
taking logarithms or inversion:
Model
Nonlinear
Linearized
exponential :
y = 1e1 x
ln y = ln 1 + 1 x
power :
y = 2 x 2
log y = log 2 + 2 log x
x
saturation - growth - rate : y = 3
3 + x
EGR 102
Lecture 4
1 1 3 1
=+
y 3 3 x
19
Transformation Examples
EGR 102
Lecture 4
20
Least Squares Example
Fit a straight line to the
li
data given below:
xi yi
1 0.5
2 2.5
3 2.0
4 4.0
5 3.5
6 6.0
7 5.5
EGR 102
Lecture 4
21
Least Squares Example
For the data:
th
n = 7 xi = 28 yi = 24
xi yi
1 0.5
24
28
y=
= 3.428571
x=
=4
2 2.5
7
7
3 2.0
2
xi yi = 119.5
xi = 140
4 4.0
5 3.5
6 6.0
7 5.5
EGR 102
Lecture 4
22
Least Squares Example
Calculate the slope and intercept for the line:
a1 =
n xi yi xi yi
n x ( xi )
2
i
2
7(119.5) 28(24)
=
= 0.8392857
2
7(140) ( 28)
a0 = y a1 x = 3.428571 0.8392857(4) = 0.07142857
So, the least-squares fit is:
y = 0.07142857 + 0.8392857 x
EGR 102
Lecture 4
23
Least Squares Example
y = 0.07142857 + 0.8392857 x
r=
n xi yi ( xi )( yi )
n xi ( xi ) 2 n yi ( yi ) 2
2
2
= 0.932
r = 0.868
2
EGR 102
Lecture 4
24
Verification Method
x
y
x*y
x^2
1
0.5
0.5
1
2
2.5
5
4
3
2
6
9
4
4
16
16
5
3.5
17.5
25
6
6
36
36
7
5.5
38.5
49
7
836.5
980
n
nSumxy
nSum x^2
28
Sum x
Sum y
4
3.428571
mean x
mean y
a1
0.839286
a0
EGR 102
24
672
0.071429
Lecture 4
Sumx*Sumy
25
Verification Method
7
To plot in Excel
6
Set x & y values, add titles, etc.
Select a data point
5
4
y
Chart Wizard XY Scatter
y = 0.8393x + 0.0714
R2 = 0.8683
3
2
1
0
Right click
Right click
Add Trendline Linear
Options Display equation on chart
Options Display R-squared value on chart
0
EGR 102
Lecture 4
1
2
3
4
5
6
7
x
26
8
Lecture 04 Quiz
Take out a blank piece of paper
Print your name, lab section # and PID at top
Using only your course text, notes you have taken
& your hand calculator:
Independently, solve the problem on the next slide
When you are finished, place your quiz in the
you
you qu
appropriate envelope at the back of the room &
you may leave
EGR 102
Lecture 4
27
Lecture 04 Quiz
Use Gauss elimination to solve the equations below. Show
all work to receive full credit. Express your answers to 1
decimal place.
2x1 + x2 3x3 = 11
11
4x1 2x2 + 3x3 = 8
-2x1 + 2x2 x3 = -6
EGR 102
Lecture 4
28