Multiple testing / Multiple comparison Overview
From Glantz, Primer of Biostatistics, Chapter 4
Suppose that we perform 5 ttests, each with alpha = 0.05.
What is the probability that we will get at least one false positive result?
P(at least one false positive result) = 1  P(zero false positive results)
The probability of getting a false positive result for a single test is alpha = 0.05.
So the probability of not getting a false positive result for a single test is
1 – alpha = 1  0.05 = 0.95.
If we do k = 5 ttests, the probability of getting no false positives on any test is 0.95^k
= 0.95^5
P(at least one false positive result)
= 1  P(zero false positive results)
= 1 – (1  .05) ^ k
= 1 – (1  .05) ^ 5
= 1 – (.95) ^ 5
= 0.226
If we do 10 ttests with alpha = 0.05, then
P(at least one false positive result)
= 1 – (.95) ^ 10
= 0.40
If we do 20 ttests with alpha = 0.05, then
P(at least one false positive result) =
= 1 – (.95) ^ 20
= 0.64
What if we do 100 ttests with alpha = 0.05, for 100 genes?
Then P(at least one false positive result)
= 1 – (.95) ^ 100
= 0.994
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
We will often be interested not just in the probability of one error, but in the expected
total number of errors. The expected number of false positives is simply alpha
multiplied by the number of tests:
For k=100 independent ttests with alpha = 0.1, the expected number of false positives
is 100 * 0.10 = 10 false positives.
Familywise error rate
The probability that we will get at least one false positive result, P (at least one false
positive result), is called the Familywise error rate (FWER).
Protecting against false positive results
Many statistical methods have been developed to protect against making a false
positive conclusion. We’ll examine the Bonferroni correction and Holm’s test.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Summer '08
 Staff
 Statistics, Null hypothesis, Hypothesis testing, Statistical hypothesis testing, Type I and type II errors

Click to edit the document details