Barron ANOVA
David Jesus College
David Barron
ANOVA
1
Introduction
ANOVAissimilartomethodforcomparing twomeans,butcomparesseveralmeans simultaneously Theheartofthemethodisasignificance test,usingtheFdistribution OnewayANOVAsometimescalled completelyrandomizeddesignmodel
David Barron ANOVA 2
Applications
Manyindustrialapplicationsinvolveexperimentsin whichthegroupsorlevelspertainingtoonlyonefactor ofinterestareconsidered.Afactorsuchasbaking temperaturemayhaveseveralnumericallevelsora factorsuchaspreferredplacementforaproductina supermarketmayhaveseveralcategoricallevels(front ofaisle,middleofaisle,etc.).Theseareexamplesof onefactororcompletelyrandomizeddesigns. Whenthenumericalmeasurementsacrossthese groupsarecontinuous,andcertainassumptionsare met,ANOVAcanbeusedtocomparemeansof groups.
David Barron ANOVA 3
Methods summary
Outcomevariable Categorical, 2groups Categorical, Diff.of proportions 2groups Categorical, Contingency > 2groups table Numerical [ Logistic regression] Categorical,> 2groups Contingency tables Contingency table [ Logistic regression] Numerical Diff.ofmeans ANOVA Regression
Grouping variable
David Barron
ANOVA
4
Basic model
In the following formulas, g denotes the number of groups. The means of the response variable for the g populations are 1 , 2 ,, g. The ANOVA is an F test of H0: 1 = 2 = = g against the alternative hypothesis Ha: at least two means are unequal. We denote the group sample sizes by n1, n2,,ng and the total sample size by N = n1 + n2 + + ng. The sample means are Y1 , Y2 , , Yg , and the sample standard deviations are s1, s2,,sg.
David Barron
ANOVA
5
Assumptions
Thepopulationdistributionsonthe responsevariablefortheggroupsare normal. Thestandarddeviationsofthepopulation distributionsfortheggroupsareequal, denotedby. Independentrandomsamplesare selectedfromthegpopulations.
David Barron ANOVA 6
Two assumptions
Three normally distributed populations with different means and equal standard deviations.
David Barron ANOVA 7
Example data
210 -3.5 2 -4.8 -2.1 -4 -2.48 7.237 2.690167 Line Speed (in BPM) 240 270 300 3.4 -1.4 4.3 -2.1 3.2 3.3 0.6 -1.2 2 -4.5 2.7 -0.8 -1.6 0.9 2.5 -0.84 0.84 2.26 8.903 4.553 3.683 2.98379 2.133776 1.919114
Mean Variance Std. Dev.
Deviationsfromspecifiedtargetfillheights(inmm)ofsoft drinkbottlesforfourdifferentspeedsoftheproductionline.
David Barron ANOVA 8
Example
5 4 Deviation from target (in mm) 3 2 1 0 -1 -2 -3 -4 -5 200 220 240 260 280 300 320 Line speed (in bpm) Mean group 1 Mean group 2 Mean group 3 Grand mean Mean group 4
David Barron
ANOVA
9
Basic principle
ThebasicprincipleoftheANOVAistocomputetwo differentestimatesofthepopulationvariance.Oneof theseestimateshasgoodpropertieswhetherornotthe nullhypothesisistrue.Theotherestimateonlyhasgood propertiesifthenullhypothesisittrue.Inotherwords, whenthenullhypothesisistrue,thetwoestimatesshould beveryclosetoeachother.Whenitisfalse,theyshould diverge.Theessenceoftheprocedureistodecidewhether anydifferenceintheratioofthetwoestimatesislarge enoughtopersuadeustoconcludethatthenullhypothesis isfalse.
David Barron ANOVA 10
Within-groups estimate
Thewithingroupsestimate(orwithinestimate)pools togetherthesumsofsquaresoftheobservationsabout theirmeansanddividesbyadegreesoffreedomterm.
WSS =
i =1 j =1
g ni
(Yij Yi ) 2
Thisiscalledthewithingroupsumofsquares.Anestimate ofthepopulationvarianceisobtainedbydividingWSSbythe numberofdegreesoffreedom:Ng.
David Barron ANOVA 11
Example
210 1.0404 20.0704 5.3824 0.1444 2.3104 28.948 97.504 16 6.094 240 17.9776 1.5876 2.0736 13.3956 0.5776 35.612 5.0176 270 5.5696 4.1616 3.4596 0.0036 18.212 300 4.1616 1.0816 0.0676 9.3636 0.0576 14.732
Total WSS df Mean Square
David Barron
ANOVA
12
Between-groups estimate
Theotherpossibleestimatorofthepopulationvariance usesthepooleddataset.Thesumofsquaresiscalculated withreferencetothegrandmean,thatis,themeanof alltheobservationspooledtogether.Clearly,thisisonly agoodestimateofthesamplevarianceifthenull hypothesisistrue,becauseonlythenwillthegrandmean beagoodestimateofmeanofeachgroup.Ifitisnota goodestimate,thentheWSScannotbebasisofagood estimateofthepopulationvariance.
BSS =
David Barron
i =1
ni (Yi Y )
ANOVA 13
g
Example
Grand mean -0.055
210 240 270 300 29.403125 3.081125 4.005125 26.79613 BSS 63.2855 df 3 Mean square 21.09516667
David Barron
ANOVA
14
F test statistic
TheANOVAFteststatisticistheratioofthebetween estimateandthewithinestimate:
Between estimate BSS /( g 1) F= = Within estimate WSS /( N g )
Whenthenullhypothesisisfalse,thebetweenestimate tendstooverestimatethepopulationvariance,soittends tobelargerthanthewithinestimate.Then,theFtest statistictendstobeconsiderablylargerthan1.
David Barron ANOVA 15
Sampling distribution
TheFteststatistichasanFprobabilitydistributionfunction asitssamplingdistribution.Ithastwodegreesoffreedom thatdetermineitsexactshape.
df1 = g 1 df 2 = N g.
TheFdistributionlooksratherlikethechisquare distribution,exceptthatitisboundedontheleftby1 ratherthan0.Therefore,only1tailedtestsare defined.
David Barron ANOVA 16
Example ANOVA table
ANOVA Source of Variation Between Groups Within Groups
SS 63.2855 97.504
df
MS F P-value F crit 3 21.09517 3.461629 0.041366 3.238867 16 6.094
21.09517 6.094
David Barron
ANOVA
17
One-Way ANOVA F Test Solution
H0: 1= 2= 3= 4 H1:NotAllEqual =.05 df1=3df2=16 CriticalValue(s):
= 0.05
Test Statistic:
21.09517 F= = = 3.46 MSW 6. 094
Decision: Reject at = 0.05 Conclusion: There is evidence that at least one i differs from the rest.
ANOVA 18
MSB
0
David Barron
3.24
F
Confidence intervals
Wecanconstructconfidenceintervalsforeachmeanor differencesbetweenpairsofmeans. Confidenceintervalforeachmean, i
Yi t , ni
where
= WS...