Experimental Design, I
9.07
3/18/2004
Goal of experiments (and thus
experimental design)
•
–
–
•
•
minimum
•
•
Determine whether a relationship is likely to exist between
One or more independent variables (factors) and a dependent
variable, or
Two or more dependent variables
Minimize the possibility that the results you get might be
due to a hidden confounding factor
Maximize the power of your test for this relationship,
while keeping the probability of a Type I error to a
Quantify your uncertainty in the results
Wide range of applicability of the results
Minimizing confounding factors
•
•
•
Power
•
fact exists.
•
–
α
–
replication
)
–
1
–m
2
), or
decreasing the irrelevant
variability
.
–
Confounding = a difference between the
“treatment” and comparison groups, other than the
“treatment”, which affects the responses under
study (the dependent variables).
From homework: does dressing well cause you to
do better on the SAT?
Confounding factor =
income.
We’ll talk about experimental designs that are
better or worse as far as confounding factors.
Probability of detecting a relationship when one in
Increasing power:
Increase
(tradeoff between Type I & Type II errors)
Increase n (
Increase the “signaltonoise ratio,” i.e. if possible,
increase the size of the effect by either increasing the
raw effect size (m
Do a better statistical test.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document•
–
j
about the variability in the responses, and thus can’t
•
–
to conditions.
–
results
•
takes on wide range of values, think twice
•
factors might interact (
Quantifying uncertainty
Use replication
If one sub ect does a task only once, you have no idea
quantify uncertainty
Use a proper form of randomization
Our models make strong assumptions about, e.g. how
subjects were chosen from the population and assigned
If these assumptions are not correct, we can’t accurately
quantify our uncertainty.
Wide range of applicability of the
If in the real world the independent variable
about only testing a small range
If there are a number of factors that might
affect the results, understand how those
factorial designs)
Minimizing the possibility of
•
–
–
–
control/comparison group
–
–
–
•
Controlled experiments vs.
observational studies
•
j
–
smoking vs. not) or by chance (exposure to radiation
leak or not)
confounding factors
Much of experimental design is aimed, at least in
part, at this problem
Observational studies vs. controlled experiments
Contemporaneous vs. historical controls
Other issues with choosing the proper treatment and
Use of placebos
Doubleblind experiments
The Hawthorne effect
We’ll talk about these design issues, as well as
about Simpson’s Paradox, which is related
•I
n
a
controlled experiment
, the experimenter
assigns
individuals to a group and decides upon
the value of the independent variable (the
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '04
 RuthRosenholtz
 natural treat

Click to edit the document details