of Equivalence model comparison and Cb F tests
Continuation of the Storage time example from Part 9. Data: Storage Time 3 months 6 months Storage Temperature 20 C 30 C 2 5 6 6 7 7 9 12 15 16
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
1/1
Summary of results from Cb estimates and tests
Quantity Time main effect Temp main effect Interaction These correspond to tests of: Time Ho: Temp Ho: 3,20 + 3,30 = 6,20 + 6,30 3,20 + 6,20 = 3,30 + 6,30 Estimate 3.5 9.0 1.0 F stat 6.00 39.71 0.12 p value 0.049 0.00074 0.74
Interaction Ho: 3,20 - 3,30 = 6,20 - 6,30 No question about these Ho.
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
2/1
Model comparisons
If you fit two specific models and compare residual SS, no question what models being compared. Interaction: In R: lm(y time+temp+time:temp) SSE = 23.50 with 6 d.f. lm(y time+temp) SSE = 23.98 with 7 d.f. In SAS: model y=time temp time*temp; SSE = 23.50 with 6 d.f. model y=time temp; SSE = 23.98 with 7 d.f. F= (23.98 - 23.50)/(7 - 6) = 0.48/3.917 = 0.122 23.50/6
Same as Cb test
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
3/1
Model comparisons
Time main effect: In R: lm(y time+temp+time:temp) SSE = 23.50 with 6 d.f. lm(y temp+time:temp) SSE = 23.50 with 6 d.f. In SAS: model y=time temp time*temp; SSE = 23.50 with 6 d.f. model y=temp time*temp; SSE = 23.50 with 6 d.f. We seem to have fit the same model each time Clue is model matrix associated with lm(y temp+time:temp) Has two columns representing interactions
X matrix with time:temp is rank 4 X matrix with intercept + temp is rank 2 The comparison between this and time:temp has 2 df The time:temp has "captured" the SS and df associated with time Residual df and SSE same for both full and reduced model
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
4/1
Think about the two models using a non-full rank representation
time, temp, time*temp Yijk = + i + j + ij + time, time*temp Yijk = + i + ij + Column space for (time,time*temp) is the same as column space for (time, temp, time*temp)
ijk
ijk
Not easy to get model comparison F statistics for main effects in models with interactions. However, model comparison provides a framework for understanding the relationship between three potential tests
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
5/1
Three potential tests of Time main effect Model comparison full: + i red: full: + i + j red: + j full: + i + j + ij red: + j + ij C (0.411 + 0.612 ) - (0.821 + 0.222 ) (0.6411 + 0.3612 ) - (0.6421 + 0.3622 ) (0.511 + 0.512 ) - (0.521 + 0.522 ) F 0.0255 6.005 6.005 p 0.88 0.0498 0.0498
Major difference between the third pair of models and the other two.
Third pair: always equal for all levels of "other" factor, e.g. temp First and second pair: coeff. depend on # obs in each cell
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
6/1
Where do the coefficients come from? First pair of models: # observations in each cell divided by row total: 2/5, 3/5, -4/5, that -1/5
Notice 'Temp' differences influence this comparison Time 3 is 40% temp 20 but time 6 is 80% temp 20 General consensus: not a valid evaluation of time effects
Last pairs of models: equal weight on each level of temp. Equiv. to an equally weighted average of simple effects 0.5(11 - 21 ) + 0.5(12 - 22 )
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
7/1
Second pair of models Also an average of the simple effects 0.64(11 - 21 ) + 0.36(12 - 22 ) So no confounding influence of temperature (good) But why more weight on 11 - 21 ? In this model, there is no interaction.
Not "we choose to ignore the interaction" or "the interaction is not significant" The model says the simple effect of time at 20 degrees 11 - 21 is exactly the same as the simple effect of time at 30 degrees 12 - 22
We have two estimates of that single effect: 11 - 21 and 12 - 22 ^ ^ ^ ^ If design not balanced, Var 11 - 21 = Var 12 - 22 ^ ^ ^ ^ Logical to average those two estimates. The most precise average is a weighted average with inverse variance weights.
Statistics 511 8/1
Copyright c 2011 Dept. of Statistics (Iowa State University)
Var 11 - 21 = 2 ^ ^ Var 12 - 22 = 2 ^ ^
1 2 1 3
+ +
1 4 1 1
= 0.75 2 = 1.3 2 3
weight for 11 - 21 is ^ ^ 1/0.75 2 = 0.64 1/0.75 2 + 1/1.3 2 3 weight for 12 - 22 is ^ ^ 1/1.3 2 3 = 0.36 1/0.75 2 + 1/1.3 2 3
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
9/1
Names
First pair of models produces sequential SS
Each term compared to reduced model with all previous terms Sometimes called Type I SS (SAS name, common in US, not in rest of world)
Second pair of models produces type II SS (SAS name, no other name)
To evaluate time, use models with all main effects and interactions except interations containing time Assumes no interactions including time. Gives more precise est. of time effect if model true (no interaction)
Third pair of models produce partial SS
Each term evaluated by dropping it from a full model with all terms Same as partial effects in multiple regression Called Type III SS (SAS name, common in US). Equal weights on all simple effects
Copyright c 2011 Dept. of Statistics (Iowa State University)
Statistics 511
10 / 1
Use and good practice
If sample sizes are the same for all cells, all types of SS are the same
Coefficients for C are same for all three types Columns in X matrix are orthogonal, so partial and simple regression coefficients are equal
If data unbalanced:
Type I used only when need to add components to construct a test Some folks argue for type II (more precise) The ISU / NCSU / Cornell school strongly favors type III SS Most common approach in US I favor type III because: The hypothesis doesn't depend on arbitrary sample sizes The test makes sense even if there is an interaction more robust, less dependent on assumption of no interaction Increase in precision over type II usually small.
If you use an additive model (no interaction), you're getting type II estimates (even if labelled type III on output)
Statistics 511 11 / 1
Copyright c 2011 Dept. of Statistics (Iowa State University)