553 EPI/STA Principles of Statistical Inference II Fall 2006
Multiple regression -- the overall hypothesis
October 17, 2006
Model The data consist of an independent random sample yi = o + 1 xi1 + L + k xik + i for k fixed values xi1 ,L, xik , i = 1,K, n , and deviation from mean i ~ N 0, 2 . So the mean of y is a linear function of the independent variables, i.e., yi | xi1 ,K, xik = o + 1 xi1 + L + k xik . The k partial slope coefficients have
the interpretation of mean changes in the dependent variable for a unit change in the respective independent variable, holding the others constant. Note that the simple linear regression model has k = 1 .
(
)
Estimating parameters The regression parameters can be estimated according to the principle of least squares, i.e., minimizing ^ ^ ^ ^ ^ ^ the sum of square errors (residuals) SSE = ei2 = ( yi - yi )2 , where the yi = o + 1 xi1 + L + k xik are the ^ ^ ^ estimated mean values The least squares estimators , ,K, are weighted linear combinations of
o, 1 k
the yi , with weights determined by the independent values. Under the standard model assumptions, the
2 estimators are normally distributed, unbiased random variables, with sampling variances proportional ^
j
to the error variance . They also have minimum variance over all unbiased estimators (BLUE). The actual formulas are best derived and written using techniques of linear (matrix) algebra.
2
Testing the overall hypothesis overall The hypothesis of interest is that there is a such a regression relation, i.e., that not all of the slope coefficients are zero. This can be tested using an analysis of variance, in which the total variation in y is decomposed into two parts, one due to error and the other due to the regression, namely
^ ^ ( yi - y ) = ( y i - yi ) + ( yi - y )
2 2 2
, or SST = SSE + SSR . The ANOVA table has the following
form: ANOVA for multiple regression Source df SS regression error total k n - k -1 n -1 SSR = SSE = ^ ( yi - yi )
2
MS MSR = SSR k MSE = SSE (n - k - 1)
F MSR SSR k = MSE SSE (n-k-1)
^ ( yi - yi ) 2 SST = ( yi - y )
2
The hypothesis of no relation, and its alternative, can be written: H 0: 1 = 2 = K = k = 0 H1: at least one j 0
Test statistic and decision rule The test statistic F is the quotient of two mean squares. Under the null hypoth...