13: CHAPTER INTRODUCTION TO ANALYSIS OF VARIANCE (ANOVA)
Analysis of variance (ANOVA) is a hypothesis testing procedure that is used to
test mean differences between groups.
How is this different from an independent-measures t test?
To answer this questions, we should review some basic information and
introduce some terms
Remember independent variables? These will now be called _________.
______________ of a factor refer to the individual treatment conditions that make
up a factor.
For example, we might be interested in seeing how effective a new drug is at
treating depression. However, instead of just looking at drug vs. no drug
(control), we want to look at how effective varying doses of the drug are at
treating depression. So, we administer the following doses to a group of
willing participants: 0mg, 2mg, 5mg, 10mg. Our independent variable, or
______________, is drug dose. This factor has four ______________ (0, 2, 5,
and 10).
So, with ANOVA, we are not limited to looking at factors with only ____ levels,
as we were with the t test.
A study that involves only one factor is called a _________________________.
This will be our focus in this chapter. However, we are not limited to a single
factor in ANOVA, as we will see in later chapters. A study with more than one
factor is called a __________________. This ability to examine more than one
factor is another advantage of ANOVA as compared to a t test.
Hypotheses using ANOVA
To help illustrate ANOVA, we will use the following example from the book:
Suppose a psychologist examined learning performance under three
temperature conditions: 50, 70, and 90. Three samples of subjects are
selected, one sample for each treatment. The purpose of the study is to see if
temperature affects learning performance.
Our null hypothesis is that temperature has no effect on performance. More
specifically, we state:
H0:
In other words, on average, performance is the same in all three conditions. The
alternative hypothesis would be that performance is not the same in all three
conditions.
H1:
Not that we are not saying which population mean is different well get to that
later when we talk about post-hoc tests.
Now, when we calculate a t score, we use the using the following:
t = (obtained difference between sample means) / (error)
If we have exactly two groups, this generally gets at what were interested in
testing, but it is limited to testing differences between two groups. In this example,
we have three groups how do you calculate differences with three groups? If we
tried to do so, it would be difficult to interpret.
With ANOVA, we can test more than two groups using a more complicated test
statistic:
F = (variance between sample means) / (error)
Whats the major difference between the t- statistic and the F-ratio?
Basically, instead of sample mean difference, for F-ratio we look at the variance
and try to determine how much variance is due to the ______________, and how
much is just due to ______________. Well now go a little more thoroughly
through the logic of ANOVA, then through the calculations.
THE LOGIC OF ANOVA
The math is complicated so lets start by reviewing the basic logic. As we go
through this, remember that the goal of ANOVA is to: measure the amount of
variability in our data and explain where it came from.
This is a big conceptual distinction between ANOVA and t-tests. t-tests deal
with differences (ex., is X 1 X 2 greater than 0?). In ANOVA, the questions
(i.e. hypotheses) are also set up in terms of differences (as well see shortly),
but the calculations dont deal with differences directly they deal with
______________.
Below is some data for the temperature example:
TREATMENT 1
50
(SAMPLE 1)
0
1
3
1
0
TREATMENT 2
70
(SAMPLE 2)
4
3
6
3
4
TREATMENT 3
90
(SAMPLE 3)
1
2
2
0
0
X1 = 1
X2 = 4
X3 = 1
The first step is to determine the total variability for the entire set of data. To do
this, we just treat all scores as though they were from the same sample,
calculate the overall mean (G), and the overall variance.
Once we have the overall variance, we want to start to break it apart into two
separate components: _____________________ and _____________________.
Between-treatments variance: how much difference exists ______________
treatment conditions.
In general, differences between treatments can be due to the following:
1. Treatment effect: The differences are caused by the treatment (manipulation).
2. Chance: The differences are simply due to chance
a. Individual differences- differences due to individuality of participants
b. Experimental error- differences to due faulty measurement
c. Other unmeasured confounds
Within-treatments variance: How much do scores differ among subjects in the
same treatment?
Because some subjects were in exactly the same treatment condition as
others, the variability in their scores tells us how much difference is
reasonable to expect by chance alone.
- ex., Not all the people who were taught in 50 conditions have the
same score as each other.
The F-ratio: Comparing Between- and Within-Treatment Variance
Once the variability has been separated into components, it is possible to compute
the F-ratio, which involves comparing between- and within-treatment variance
F=
variance between treatments
variance within treatments
What happens to this ratio when
The treatment has no effect?
The treatment has a small effect?
The treatment has a large effect?
Variability due to the denominator of the F-ratio is called ______________
because it is not under the experimenters control. Thus, the denominator of the Fration is known as the error term. The error term provides a measure of the
variance due to chance. When the treatment effect is zero (H0 is true), the error
measures the same source of variance as the numerator of the F-ratio variance
between treatments is simply due to chance, just as is the variability within
treatments so the value of the F-ratio is expected to be nearly equal to_____when
the treatment does not have an effect. When the treatment does have an effect, the
F-ratio will be larger than ______.
ANOVA VOCABULARY, NOTATION, AND FORMULAS
Notation:
k = the number of treatment conditions (e.g., the number of levels).
n = the number of scores in each treatment
N = the total number of scores in the entire study (kn)
T = X = total for each treatment condition
G = T = sum of all scores in the study
The following is a data set from an experiment examining learning performance
under three temperature conditions.
TEMPERATURE CONDITIONS
1
2
3
50
70
90
0
4
1
1
3
2
3
6
2
1
3
0
0
4
0
T1 = 5
T2 = 20
T3 = 5
SS1 = 6
SS2 = 6
SS3 = 4
n1 = 5
n2 = 5
n3 = 5
X1 = 1
X2 = 4
X 2 = 106
G = 30
N = 15
k=3
X3 = 1
Sum of Squares (SS)
Type
GeneralFormula
AppliedtoTemperature
Example
Total
Withintreatments
Betweentreatments
SStotal = X 2
G2
N
SS within = SSinside each treatment
SSbetween
T 2 G2
=
n
N
SS total =
SS within =
SS between =
Degrees of Freedom (df)
Type
GeneralFormula
AppliedtoTemperature
Example
df total = N 1
Total
Withintreatments
Betweentreatments
df within = df inside each treatment
df between = k 1
df total =
df within =
df between =
Calculation of Variances (MS) and the F-ratio
The general formula for mean squares (variance): MS =
Type
GeneralFormula
SS
df
AppliedtoTemperature
Example
Withintreatments
Betweentreatments
MS within =
SS within
df within
SS
= between
df between
MS within =
MSbetween
The formula for the F-ratio is: F =
MS between =
MSbetween
MS within
Applying the F-ratio to our example, we get F =
It is useful to organize the resulting information into a summary table, called an
ANOVA source table:
SOURCE
Between treatments
Within treatments
Total
SS
30
16
46
df
2
12
14
MS
15
1.33
F = 11.28
THE DISTRIBUTION OF F-RATIOS
Remember,
F=
treatment effect + difference s due to chance
difference s due to chance
F=
variance between treatments
variance within treatments
F=
MS between
MS within
What does an F-ratio of >1 mean?
What does an F-ratio of <1 mean?
What does an F-ratio of <0 mean?
Notice a couple things:
1.
2.
F-ratios are calculated from two variances (which are ALWAYS positive)
and will therefore always be ______________ .
When H0 is true, the numerator and the denominator of the F-ratio are
measuring the same variance and the F-ratio will be ~1.00. In other words,
the distribution of the F-ratio should pile up around 1.00.
What F-ratio do we expect when H0 is true?
What F-ratio do we expect when H0 is not true?
How do we determine whether to reject or fail to reject H0?
More tables! What information do we need to determine the critical region? We
need the almost same information we needed with a t statistic we need and
df.
What information did we need for the t tables that is no longer relevant?
How many df values do we need?
We dont need the __________________anymore, but we do need TWO df values:
dfbetween and dfwithin. The dfbetween are the degrees for the ______________ and
dfwithin are the degrees of freedom for the ______________.
Look at the F-distribution table (Table B.4 in the appendix- page 705) and
determine whether the F value we obtained above, F(2,12) = 11.28, is significant.
df: Denominator
11
12
13
1
4.84
9.65
4.75
9.33
4.67
9.07
df: Numerator
2
3.98
7.20
3.88
6.93
3.80
6.70
3
3.59
6.22
3.49
5.95
3.41
5.74
Light-face = .05, bolded = .01
The critical F-value for df = 2,12 and =.05 is 3.88. To reject Ho, the F-value
must be greater than 3.88.
Fcritical= 3.88 (=.05)
Fobtained= 11.28
Since Fcritical <Fobtained, ______________ we H0 and conclude that temperature does
have a significant effect on learning performance.
EXAMPLES OF HYPOTHESIS TESTING WITH ANOVA
Here are the basic steps (more detail will follow): Sound familiar?
1. State the hypothesis
2. Locate the critical region
3. Compute the test statistic
4. Make a decision
OK, now were going to go through each step in more detail
The following table depicts data from an independentmeasures experiment
designed to measure the effectiveness of three pain relievers (A, B, and C).
A fourth group, which received a sugar pill (placebo), was also tested.
PLACEBO DRUG A
0
0
0
1
3
2
T1 =
T2=
SS1 =
SS2 =
DRUG B
3
4
5
T3 =
SS3 =
DRUG C
8
5
5
T4 =
SS4 =
N=
G=
X 2 =
K=
PREP WORK: Compute summary statistics: treatment totals (T), SS values, N, G,
and X2.
STEP ONE: State the hypothesis
H0:
H1:
STEPTWO:Locatethecriticalregion
What is ? .05
What is dfbetween?
What is dfwithin?
df between = k 1 =
df within = (n 1) = N K =
Using Table B.4 on pg. 705, what is the critical value?
df: Denominator
1
5.32
11.26
5.12
10.56
4.96
10.04
8
9
10
df: Numerator
2
4.46
8.65
4.26
8.02
4.10
7.56
3
4.07
7.59
3.86
6.99
3.71
6.55
The critical F-value for df = 3,8 and =.05 is 4.07. To reject Ho, the F-value must
be greater than 4.07.
STEP THREE: Compute the test statistic (the F-ratio)
What is SSbetween?
SSbetween
T 2 G2
=
n N=
SSwithin?
Using the SS values and the df values, what is MSbetween?
SS within = SST =
MSbetween =
What is MSwithin?
MS within =
SSbetween
=
df between
SS within
=
df within
Using the MS values, what is the F-ratio?
F=
MSbetween
=
MS within
STEP FOUR: Make a decision
Does the value of the F-ratio fall in the critical region?
If yes, we reject the null hypothesis, meaning our evidence suggests there is
a treatment effect.
If no, we fail to reject the null hypothesis, meaning our evidence does not
suggest there is a treatment effect.
SUMMARY WORK: Make a source table
SOURCE
Between treatments
Within treatments
Total
SS
54
16
70
df
3
8
11
MS
18
2
F=9
The critical F-value for df = 2,9 and =.05 is 4.26. To reject Ho, the F-value
must be greater than 4.26.
Fcritical= 4.26 (=.05)
Fobtained= 9
Since Fcritical <Fobtained, we ______________ H0 and conclude that treatment has an
effect on pain relief.
APA STYLE: REPORTING THE RESULTS OF ANOVA
The means and standard deviations are presented in Table 1. The analysis of
variance revealed a significant difference, F(3,8) = 9.00, p<.05.
What is the value of df between? df within? 3,8
What is k? df between = k 1 = 3
What is N?
What is the value of the F-ratio? 9.00
What is the statistical decision? Reject null hypothesis
df within = N k = N 4 = 8
EXAMPLE:
Were interested in the effects of background noise on the # of words recalled in a
memory test.
IV (factor): noise level, with 3 levels: silent (level 1), background conversation
(level 2), and construction (level 3).
DV: number of words recalled from a list
Level 1
10
13
14
13
15
T1 =65
SS1 =14
Level 2 Level 3
10
2
8
3
7
7
5
2
10
6
T2=40
T3 =20
SS2 =18 SS3 =22
N = n = 5 + 5 + 5 = 15
G = T = 65 + 40 + 20 = 125
X2 = 1299
STEP ONE: State the hypothesis
H0:
H1:
STEP TWO: Locate the critical region
a)
b)
What is ? .05
What is dfbetween?
N =15
G =125
X2 =1299
K=3
df between = k 1 =
c)
What is dfwithin?
d)
Using Table B.4 on pg. 705, what is the critical value?
df within = (n 1) = N k =
df: Denominator
1
4.84
9.65
4.75
9.33
4.67
9.07
11
12
13
df: Numerator
2
3.98
7.20
3.88
6.93
3.80
6.70
3
3.59
6.22
3.49
5.95
3.41
5.74
The critical F-value for df = 2,12 and =.05 is 3.88. To reject Ho, the F-value must
be greater than 3.88.
STEP THREE: Compute the test statistic (the F-ratio)
a) What is SSbetween?
T 2
SS between =
n
G2
N=
b)
SSwithin?
c)
Using the SS values and the df values, what is MSbetween?
SS within = SST =
MS between =
d)
What is MSwithin?
MS within =
e)
SS between
=
df between
SS within
=
df within
Using the MS values, what is the F-ratio?
F=
MS between
=
MS within
STEP FOUR: Make a decision
SUMMARY WORK: Make a source table
SOURCE
Between treatments
Within treatments
Total
SS
203.33
54
257.33
df
2
12
14
MS
101.67
4.5
F = 22.59
The critical F-value for df = 2,12 and =.05 is 3.88. To reject Ho, the F-value
must be greater than 3.88.
Fcritical= 3.88 (=.05)
Fobtained= 22.59
Since Fcritical <Fobtained, we ______________ H0 and we can conclude that at least 1
of our means is different: noise level has some effect on the # of words recalled.
We do not know which means are different from each other.
Post Hoc Tests
Once we find a significant ANOVA, indicating that at least one
treatment condition is different from the others, how do we figure out
which condition is different?
We could simply run the appropriate t-tests on all combinations of 2
groups. However, this would greatly increase our chances of
committing a Type I error.
Why?
Because every time we conduct an additional hypothesis we have
an additional chance of committing a Type I error. For example, if we
conducted 20 separate t-tests at a .05 alpha level, we would expect to
make at least ONE Type I error, just by chance!
Terms:
___________________ The overall probability of a Type I error that is
accumulated over a series of separate hypothesis tests. This is always
greater than ___________________ (which is simply the alpha level you
select for an individual hypothesis test)
SOLUTION:
Post hoc tests allow us to make ___________________between groups
while ATTEMPTING to control for the ___________________. These
should ONLY be used AFTER you have obtained a significant ANOVA
indicating that differences do exist because otherwise you increase your
chances of making a Type I error.
2 Major Types of Post Hoc Tests:
1. Tukeys Honestly Significant Difference Test (HSD)
2. The Scheffe Test
Tukeys HSD
Allows you to compute a single value that determines the minimum
difference between treatment means that is necessary for significance
This value, called the __________________________________________
is then used to compare any two treatment conditions.
If mean differences exceed the computed HSD, you conclude that
there is a significant difference between the treatments.
You can ONLY use this test when ALL treatment conditions have the
SAME sample size (n).
HSD = q
MS within
n
You find q in the appendix. q is the standardized range statistic
found in the q table. To do this you need to know the number of
treatment conditions (k) and the df for the error term (i.e. dfwithin).
Table B.5, p. 708
You then just compare the mean differences between your groups
to the HSD value. If the mean differences exceed the HSD value
then you conclude that a significant difference exists between the
two groups you were comparing.
The Scheffe Test
This test is more conservative than the HSD.
o This means that you are less likely to commit a Type I error with
this test, but more likely to commit a Type II error.
o Requires that a ___________________ meet the same criteria as the
overall ANOVA in order to conclude a significant difference.
o An F-ratio is calculated between 2 treatment conditions using the
same denominator value (for MSwithin).
o You use the same df for the numerator (dfbetween= k 1) and the
same critical F-value as for the overall ANOVA.
Scheffe Test =
MSbetween
MS within
NOTICE: this is the same form of equation for the F-ratio that you used
to conduct an ANOVA.
THE DIFFERENCES:
You compute your MSbetween on ONLY the two groups you want to
compare (instead of on all groups as you did when you were
conducting the ANOVA)
To conduct this hypothesis test you use the ___________________
Fcritical value that you computed while conducting the ANOVA.
If the obtained F-ratio exceeds the Fcritical you conclude that there are
significant differences between the two groups you were comparing.
THE RELALATIONSHIP BETWEEN ANOVA AND t TESTS
General observation
F = t2
If you squared all the values in the t distribution, it looks like the F
distribution.
Use: When do you use a t test and when do you perform an ANOVA? It depends
on the __________________________. If more than 2, use ANOVA.
One more thing: df for the t statistic = n 1. This is equal to dfwithin when k = 2
Assumptions
The independent-measures ANOVA requires the same three assumptions that
were necessary for the independent-measures t-test:
1. The observations within each sample must be independent.
2. The populations from which the samples are selected must be normal.
3. The populations from which the samples are selected must have equal
variances (homogeneity of variance.)
Similar to Question 22 from Chapter 13:
A pharmaceutical company has developed a drug that is expected to reduce
hunger. To test the drug three samples of rats are selected with n = 12 in
each sample. The first sample receives the drug every day. The second
sample is given the drug once a week, and the third sample receives no drug
at all. The DV is the amount of food eaten by each rat over a 1-month period.
These data are analyzed by an ANOVA, and the results are reported in the
following summary table. Fill in all missing values in the table (Hint: Start
with the df column).
Source
Between
SS
Within
df
99
F = 6.00
Total
dftotal = N 1 = 12*3 1 = 36 1 = 35
dfbetween= k -1 = 3 1 = 2
dfwithin = N k = 36 3 = 33
MSwithin=
F=
MS
SS within 99
=
=
df within 33
MS between
MSbetween
= 6.00 =
MS within
99
MSbetween =
SSbetween
df between
SStotal = SSbetween + SSwithin