PLoS Medicine

www.plosmedicine.org
0696
Essay
Open access, freely available online
August 2005

Volume 2

Issue 8

e124
P
ublished research f
ndings are
sometimes reFuted by subsequent
evidence, with ensuing conFusion
and disappointment. ReFutation and
controversy is seen across the range oF
research designs, From clinical trials
and traditional epidemiological studies
[1–3] to the most modern molecular
research [4,5]. There is increasing
concern that in modern research, False
f
ndings may be the majority or even
the vast majority oF published research
claims [6–8]. However, this should
not be surprising. It can be proven
that most claimed research f
ndings
are False. Here I will examine the key
Factors that inﬂ
uence this problem and
some corollaries thereoF.
Modeling the Framework for False
Positive Findings
Several methodologists have
pointed out [9–11] that the high
rate oF nonreplication (lack oF
conf
rmation) oF research discoveries
is a consequence oF the convenient,
yet illFounded strategy oF claiming
conclusive research f
ndings solely on
the basis oF a single study assessed by
Formal statistical signif
cance, typically
For a
p
value less than 0.05. Research
is not most appropriately represented
and summarized by
p
values, but,
unFortunately, there is a widespread
notion that medical research articles
should be interpreted based only on
p
values. Research f
ndings are def
ned
here as any relationship reaching
Formal statistical signif
cance, e.g.,
eFFective interventions, inFormative
predictors, risk Factors, or associations.
“Negative” research is also very useFul.
“Negative” is actually a misnomer, and
the misinterpretation is widespread.
However, here we will target
relationships that investigators claim
exist, rather than null f
ndings.
As has been shown previously, the
probability that a research f
nding
is indeed true depends on the prior
probability oF it being true (beFore
doing the study), the statistical power
oF the study, and the level oF statistical
signif
cance [10,11]. Consider a 2 × 2
table in which research f
ndings are
compared against the gold standard
oF true relationships in a scientif
c
f
eld. In a research f
eld both true and
False hypotheses can be made about
the presence oF relationships. Let
R
be the ratio oF the number oF “true
relationships” to “no relationships”
among those tested in the f
eld.
R
is characteristic oF the f
eld and can
vary a lot depending on whether the
f
eld targets highly likely relationships
or searches For only one or a Few
true relationships among thousands
and millions oF hypotheses that may
be postulated. Let us also consider,
For computational simplicity,
circumscribed f
elds where either there
is only one true relationship (among
many that can be hypothesized) or
the power is similar to f
nd any oF the
several existing true relationships. The
prestudy probability oF a relationship
being true is
R
⁄(
R
+ 1). The probability
oF a study f
nding a true relationship
reﬂ
ects the power 1 − β (one minus
the Type II error rate). The probability