13%20Multiple%20Testing%202_28_08 - Possible Errors in...

1
1
Introduction to Multiple Testing
Method for the Analysis of
Microarray Data
Peng Liu
2/28/2008
2
Possible Errors in Testing ONE Hypothesis
Type I Error : false positives
Type II Error: false negatives (1-power)
Power : true positives
correct (Power) Type II Error False Null
Type I Error correct True Null
Reject Null Accept Null Hypothesis
3
The Multiple Testing Problem
Suppose one test of interest has been
conducted for each of the m genes in a
microarray experiment.
Often, the test of interest is that for each gene,
whether it is differentially expressed between
different treatments or not.
Let H
01
, H
02
, ... , H
0m
denote the null hypotheses
(often interpreted as non-differential expression)
corresponding to the m tests (genes).
4
The Multiple Testing Problem (continued)
Suppose
m
0
of the null hypotheses are true
(corresponding to non-differentially expressed
genes)
m
1
of the null hypotheses are false
(corresponding to differentially expressed
genes).

2
5
The Multiple Testing Problem (continued)
If we set level 5% for each test , the number of
type 1 error (false positives) is expected to be
5% of m
0
.
Considering the high dimensionality of
microarray data, the number of errors is
expected to be really big if we only control error
at the level of individual test.
We need procedure to control multiple testing
error.
6
The Multiple Testing Problem (continued)
Let p
1
, p
2
, ... , p
m
denote the p -values corresponding to
the m tests ( each test corresponding to one gene ).
Let c denote a value between 0 and 1 that will serve as a
cutoff for significance :
- Reject H
0i
if p
i
≤ c (declare significant)
- Fail to reject (or accept) H
0i
if p
i
> c
(declare non-significant)
7
Table of Outcomes
Accept Null Reject Null
Declare Non-Sig. Declare Sig.
No Discovery Declare Discovery
Negative Result Positive Result
True Nulls U V m
0
False Nulls T S m
1
Total W R m
This is the slide that you need for question 4 of homework 3.
8
Accept Null Reject Null
Declare Non-Sig. Declare Sig.
No Discovery Declare Discovery
Negative Result Positive Result
True Nulls U V m
0
False Nulls T S m
1
Total W R m
U=number of true negatives
Table of Outcomes

3
9
Accept Null Reject Null
Declare Non-Sig. Declare Sig.
No Discovery Declare Discovery
Negative Result Positive Result
True Nulls U V m
0
False Nulls T S m
1
Total W R m
V=number of false positives
=number of false discoveries
=number of type 1 errors
Table of Outcomes
10
Accept Null Reject Null
Declare Non-Sig. Declare Sig.
No Discovery Declare Discovery
Negative Result Positive Result
True Nulls U V m
0
False Nulls T S m
1
Total W R m
T=number of false negatives
=number of type 2 errors
Table of Outcomes
11
Accept Null Reject Null
Declare Non-Sig. Declare Sig.
No Discovery Declare Discovery
Negative Result Positive Result
True Nulls U V m
0
False Nulls T S m
1
Total W R m
S=number of true positives
=number of true discoveries
Table of Outcomes
12
Accept Null Reject Null
Declare Non-Sig. Declare Sig.