Stat 231 Handout on Regression Diagnostics
There are various kinds of residuals, ways of plotting them and measures of "in°uence" on a
regression that are meant to help in the black art of model building.
We have already alluded to
the fact that under the MLR model, we expect ordinary residuals
e
i
=
y
i
°
b
y
i
to look like mean
0
normal random noise and that standardized or studentized residuals
e
°
i
=
e
i
standard error of
e
i
should like standard normal random noise.
Deleted Residuals and the PRESS Statistic
There is also the notion of deleted residuals.
These are built on the idea that a model should not
be terribly sensitive to individual data points used to ±t it, or equivalently that one ought to be
able to predict a response even without using that response to ±t the model.
Beginning with a
particular form for a MLR model and
n
data points, let
b
y
(
i
)
=
the value of
y
i
predicted by a model ±t to the other
(
n
°
1)
data points
(note that this is not necessarily
b
y
i
).
The
i
th deleted residual is
e
(
i
)
=
y
i
°
b
y
(
i
)
and the hope that if a model is a good one and not overly sensitive to the exact data vectors used
to ±t it, these shouldn²t be ridiculously larger in magnitude than the regular residuals,
e
i
.
The
"prediction sum of squares" is a single number summary of these
PRESS
=
n
X
i
=1
(
y
i
°
b
y
(
i
)
)
2
and one wants small values of this.
(Note that
PRESS
±
SSE
, but one hopes that it is not too
much larger.)
This does not exhaust the ways in which people have suggested using the residual idea.
It is
possible to invent standardized/Studentized deleted residuals
e
°
(
i
)
=
e
(
i
)
standard error of
e
(
i
)
and there are yet other possibilities.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Staff
 Regression Analysis, Yi, residuals, xj

Click to edit the document details