How different is our sample from the true
population due to chance?
Population
Sample
Sampling variability
Sampling variability: The variability among random samples
from the same population
Population
Sample1
Sample2
Sample3
A sampling distribution of a
Polynomial
Regression Models
Polynomial regression models have two basic types of uses:
1)
When the true curvilinear response function is really a polynomial
function.
2)
When the true curvilinear response function is unknown (or
complex) but a polynomial
Multicollinearity
A predictor variable may be redundant, in the sense that it
overlaps with other variables. That is, it can be predicted well by
other predictor variables and doesnt add much new to the model.
For example, suppose we have two species, gia
Model Selection
For a set of p-1 predictors, there are 2p-1 possible regression
models that can be constructed, based on the fact that each
predictor can be either included or excluded from a particular
model.
For example, if there are 4 predictors, then
Multiple Regression
Recall our model for multiple regression is
Yi 0 1 X i1 2 X i 2 p 1 X i , p 1 i
And assuming that Ecfw_i = 0, the regression function is
Ecfw_Y = 0 + 1X1 + 2X2 + .+ p-1Xp-1.
To fit the model, we use the least squares method. The LS est
General Linear
Test Approach
We will explain the general linear test approach in terms of the simple
linear regression model and testing Ho: 1 = 0 vs Ha: 1 0.
The general linear test approach involves three basic steps: the full
model, the reduced model,
Inferences on 1
Recall that
t*
b1 1
~ t (n 2)
scfw_b1
We use this t distribution of our test statistic to do hypothesis tests.
What value do we use for 1 in this statistic and why?
If we want to show there is a linear association between X and Y,
what a
Prediction of mean
of new observations
Occasionally, we may want to predict the mean of m new
observations for a given level of a predictor variable.
For example, suppose the Toluca Company has been asked to
bid on a contract that calls for m=3 production
Multiple Regression
There are many situations were a single predictor variable in the model
would provide an inadequate description since a number of key
variables affect the response variable.
Furthermore, in these situations, predictions of the response
Simple Linear Regression
Yi 0 1 X i i
The response Yi is the sum of two parts: (1) the constant term 0 +
1Xi and (2) the random term i.
Thus Yi is a random variable.
Since Ecfw_i = 0 and 0 + 1Xi is a constant term, we have that
Ecfw_Yi Ecfw_ 0 1 X i i
Considerations in
Applying Regression
The major uses of regression analysis are making inferences about
regression parameters, estimating the mean response for a given X,
and to predict a new observation Y for a given X.
We will now discuss some cautions
Test for
Lack of Fit
We use a lack of fit test to determine whether a specific type of
regression function adequately fits the data.
The test assumes that the observations Y for a given X are
independent and normally distributed and that the distributions
Probability
Exercises
1. Of the last 500 customers entering a supermarket, 50 have
purchased a wireless phone. If the relative frequency
approach for assigning probabilities is used, the probability
that the next customer will purchase a wireless phone is
Standardized
Multiple Regression
Given a fitted multiple regression model, a user may want to
compare the magnitudes of the effects of predictors on the response
variable.
For example, for a model for gas consumption of a car with the car
weight and horse
Extra
Sums of Squares
When we wish to test whether a predictor variable Xk has a linear
effect on Y, we test
Ho: k = 0 vs Ha: k 0
We already know that using a t-test is appropriate for this test.
Equivalently, we can use the general linear test approach,
Point Estimation
We want to estimate parameters from our data. A point estimate is
a single number that is used to estimate the parameter.
An interval estimate is an interval of numbers around the point
estimate, within which the parameter value is believ
Confidence Intervals
for means
Like a confidence interval for a proportion, the confidence interval for a
mean has the form: point estimate margin of error.
The margin of error for is z1-/2* y(bar) , where
y
n
Thus, in the long run
y 1.96 n
will contain
Relations between variables
A functional relationship between two variables is expressed by a
mathematical formula.
For an independent variable X and a dependent variable Y, a
functional relationship has the form
Y= f(X)
For example, if we sell widgets fo
Significance Tests
Suppose we have
Ho: It is not raining
vs
Ha: It is raining
Now suppose everyone is soak and wet.
We then conclude that it is extremely unlikely that, if the null were
true and it was not raining, everyone would be soak and wet.
Thus we
Estimation of
regression function
The Gauss-Markov theorem states that the least squares estimators
b0 and b1 are unbiased an have minimum variance among all
unbiased linear estimators.
Thus Ecfw_b0= 0 and Ecfw_b1= 1
It can be shown that b0 and b1 are lin
Maximum Likelihood
Estimation
The maximum likelihood estimate (mle) of a parameter is the
parameter value for which the probability of observing our data is
maximized.
For instance, say weights follow a normal distribution and the average
weight of a samp
Inferences on 1
We are assuming the normal error regression model
Yi 0 1 X i i
where
0 and 1 are parameters
Xi are known constants
i are independent N(0,2)
Frequently, we are interested in drawing inferences about 1, the
slope of the regression line.
Infe
Diagnostics
Whenever data are obtained in a time sequence or some other type of
sequence, such as for adjacent geographic areas, it is a good idea to
prepare a plot of the residuals against time, called a sequence plot of
the residuals.
The purpose of plo
Estimation of Ecfw_Yh
A common objective is to estimate the mean for one or more
conditional probability distributions of Y. That is, to estimate the
mean of Y for a given value of X.
For example, the Toluca Company was interested in the mean
number of wo
Breakdown of
Degrees of Freedom
Corresponding to the partitioning of the total sums of squares
SSTO, there is a partitioning of the associated degrees of freedom
(df).
There are n-1 df associated with SSTO. We lose a degree
because we are using the sample
Simultaneous
Inferences
Suppose we run a regression analysis and we want to create 95%
confidence intervals for both 0 and 1.
If we were to create two separate confidence intervals, they would not
provide 95% confidence that both 0 and 1 were in their res
General Linear Model
in Matrix Terms
In matrix terms, the general linear regression model is* (the
dimensions in the book are wrong)
Y
=
X
+
nx1
where
Y1
Y
Y 2
Yn
1 X 11
1 X 21
X
1 X n1
nxp px1
X 12 X1,p-1
X 22 X 2,p-1
X n2 X n, p-1
nx1
0
Remedial measures
for lack of fit
1)
2)
If a simple linear regression model is not appropriate for the data,
there are two basic choices
Use a more appropriate model
Use a transformation on the data so that the linear regression model
is appropriate for t
Significance test
for a mean
Suppose now our alternative hypothesis is that the average weight
of UT students is LESS than 250. We sample 11 people and the
average weight is 225 and the standard deviation is 70.
We are going to test Ho:=250 vs Ha: < 250 (