to 1
Introduction Regression Analysis
We wish to study the relationship between 2 or more variables
We define:
y = the dependent variable
x = independent (predictor) variable
We wish to predict y on the basis of x.
Example:
y = sales of a product
x = price of the product
Assuming a linear (straight line) relationship between y and x, we wish to find a prediction
equation
Predicted value of y
intercept
y = b 0 + b1 x
slope
Simple Linear Regression
One
Predictor
Variable
Straight Line
Relationship
We use data concerning both x and y to find numerical values for b 0 and b 1 .
2
Multiple Regression
We predict y by using more than one independent (predictor) variable.
Example:
y
x1
x2
x3
=
=
=
=
sales
price
advertising budget
type of advertising ( TV, radio, print, etc.)
The prediction equation might have the following form:
y = b 0 + b1 x1 + b 2 x 2 + b 3 x 3
We find numerical values of b0, b1, b2 , and b3 by using data concerning y, x 1 , x 2 ,
and x 3 .
Alternatively, ( as one possible example) the prediction equation could have the form:
2
y = b 0 + b1 x1 + b 2 x 2 + b 3 x 3 + b 4 x 3 + b 5 x1 x 2
3
Chapter 13: Simple Linear Regression
Section 13.1: The Simple Linear Regression Model and the Least Squares
Estimates
We begin with Example 13.1 on pages 517 520:
We wish to select a site for a Tasty Sub Shop Restaurant based on the Quiznos franchise.
We will base the decision on a prediction of yearly revenue in a particular location.
The dependent variable is
y = yearly revenue (in thousands of dollars)
The independent (predictor) variable is
x = population size in the surrounding area (in thousands of residents)
We gather data concerning 10 existing Tasty Sub Shop restaurants:
as shown in Table 13.1 on page 518
and as graphed in Figure 13.1 (also on page 518).
4
Independent
variable, x
Dependent
variable, y
approximate
straight line
relationship
(linear) positive
slope
We with
a wish to predict y on the basis of x.
The Simple Linear Regression Model
Beta zero
Beta one
Error term
the line of means
= Slope
= y Intercept
= 0 + 1x = mean value of y (represents the effect of x on y)
= the error term (represents the effects of all other factors on y)
See Figure 3.2 on page 519.
5
The slope 1 is the change in mean yearly revenue associated with an increase of 1,000
people living in the area.
does not imply cause and effect
The y intercept 0 is the mean yearly revenue when the population in the area is zero.
This interpretation of 0 is of little or no practical value.
6
Figure 13.3 on page 520 shows a geometrical interpretation of the general simple linear
regression model.
We now see how we estimate the slope
and y-intercept
.
Notation: 0 and 1 are the regression parameters (known by Sir Fisher)
b0 and b1 are the least squares estimates (aka as the regression coefficients)
computed from observed data as best guesses (based on our data) of 0
and 1
7
We compute the least squares estimates as follows: (see the box on page 522)
Example: See the Tasty Sub Shop Case on page 523.
8
The least squares line (prediction equation) is
y = 15.84 .1279x
predicted value of y
9
We now consider the meaning of least squares (see Table 13.2 and Figure 13.5 on page 524):
Predicted Yearly Revenues
Prediction Errors
for the observed data
(called Residuals)
Smaller than the SSE for any other line we can fit to the data.
Blue line
segments
are the
residuals
10
We now use the least squares prediction equation to predict revenue (see page 525).
The point estimate of the mean and the point prediction of a single yearly revenue are the
same because
= 0 + 1x
(the mean value of y)
and
(a single value of y)
only differ by the error term, and we predict the error term to equal zero.