Lecture11-Feb+11th-Internal+validity

Lecture11-Feb+11th-Internal+validity -  ...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview:   Exercise
3
due
Today
   Files
not
dropped
in
drop‐box
before
1.40pm
will
 be
considered
late
   Grader
is
LAURA.
Address
any
questions/concerns
 to
her
   Midterm
2
next
Tuesday
   Chapter
3
(reliability
&
validity),
4,
5,
6,
8
 (assigned
pages),
12
(assigned
pages)
   Lectures
7,8,9,
10,
&
11
   Two
review
sessions:
   Monday,
Feb
15th,
7.00‐9.00pm
(TAs)
   Tuesday,
Feb
16th,
11.00‐12.30
(Instructor)
   As
usual…
   Bring
#2
pencil
   Bring
Scantron
(Davis
200)
   Be
On
Time!
   Limitations
   Direction
of
causality
is
unknown
   Third
variable
problem
 ▪  Relationship
between
variables
is
spurious
   How
can
we
tease
apart
these
different
 possibilities?
 Type
of
design
 Typical
studies
 Descriptive
 Observational
studies
 Case
studies
 Surveys
 Predictive
 Correlational
studies
 Quasi‐Experiments
 Explanatory
 Experiments
   In
an
experiment,
at
least
one
variable
is
 manipulated
(i.e.,
systematically
varied)
by
 the
researcher
in
order
to
study
its
effects
on
 another
variable.
   Features
of
an
experiment
 (a)
At
least
one
variable
is
manipulated
or
varied
by
 the
experimenter:
independent
variable
(IV)
 (b)
The
variable
presumably
affected
by
the
 manipulation
is
called
the
dependent
variable
 (DV)
 (c)
random
assignment
to
conditions
 IV DV Independent Variable: Watching violent TV Dependent Variable: Aggressive behavior Levels: Number of times the child punches his or her peers on the playground (a)  view an episode of the Sopranos (b)  view an episode of the Sopranos in which the violent scenes have been edited   Between‐
and
within‐subjects
designs
   between‐subjects:
different
people
are
exposed
 to
each
level
of
the
IV
   within‐subjects:
the
same
people
exposed
to
 each
level
of
the
IV
   Allows
to
establish
cause‐effect
relationships,
 BUT…
   To
be
able
to
do
that
we
need
to
ensure
that
 our
study
has…
   INTERNAL
VALIDITY
   Note
that
we
can
NEVER
prove
causality!
We
can
 only
show
to
what
degree
it
is
PROBABLE!

 Establishing
Internal
Validity
   What
it
means
to
establishing
internal
validity?
   Threats
to
internal
validity
(Between
SS
 designs)
   Simple
experiment
   Pretest‐posttest
design
 1.  2.  Covariation
 Temporal
precedence
 3.  Eliminate
spuriousness
 Measure
other
variables
of
concern
   Ensuring
extraneous
variables
don’t
turn
into
 CONFOUNDS
     Confound
(or
confounding
variable)
   Variable
that
influences
the
dependent
variable
 and
is
associated
with
the
independent
variable
   Prevents
us
from
making
strong
inferences
about
 causality!
 IV DV Independent Variable: Watching violent TV Dependent Variable: Aggressive behavior Levels: Number of times the child punches his or her peers on the playground (a)  view an episode of the Sopranos (b)  view an episode of the Sopranos in which the violent scenes have been edited   Non‐confounding
variable:
   A
variable
that
has
an
effect
on
the
dependent
 variable
but
that
is
uncorrelated
with
the
 independent
variable.
 Watching violent TV + Acting violent + Living in a violent family Acting violent + Living in a violent family + Watching violent TV   What
it
means
to
establishing
internal
validity?
   Threats
to
internal
validity
   Simple
experiment
   Pretest‐posttest
design
   AKA
posttest
only
control
group
design
   Two
groups:
control
vs.
experimental

 Practice Group 1: questions experimental M Noroup 2: G practice qcontrol uestions   Non‐equivalent
groups
 Whole Score on sample: exam measure outcome   Pretest‐Posttest
with
Control
design
 WScore on hole sample: measure before exam manipulation Practice Group 1: questions experimental No roup 2: G practice questions control M Whole Score on sample: exam measure outcome   Measurements
before
AND
after
manipulation
   Groups
+
time
   Nonequivalent
groups
   Selection
Bias
 ▪  Participants
are
self‐assigned
to
groups
   Applying
arbitrary
rules
   Matching
   Constructing
each
group
so
they
have
 identical
characteristics
   Find
a
“match”
for
every
subject
 ▪  T1DM
vs.
“matched”
controls
 ▪  Match
age,
gender,
IQ,
SES,
mood
on
experiment
day…
 Our
sample:
 a
D
b
Z
E
X
R
T
L u o q c
p N v h
Y
S
M
F i w
j Matching 
 
Experimental


























Control










 X L Y j q Do N Rc wh XL Yj q Do N Rc w h   Matching
   Balancing
   Balance
characteristics
of
group
(not
 individual
subject)
 Our
sample:
 a
D
b
Z
E
X
R
T
L u o q c
p N v h
Y
S
M
F i w
j Balancing 

 
Experimental







 XYZ abc LMN opq 
 
Control










 RST hIj DEF uvw   Matching
   Balancing
   Random
assignment
   Individuals
have
equal
probability
of
being
 assigned
to
one
of
the
groups
   Differences
are
“averaged
out”
   Not
good
with
small
samples!!!!!!
 Our
sample:

 q
B
g
k
p
X
L
n
m
Q
G
x
M
b
K
P
 Random Assignment 

Experimental




 






 
 LGpKXBnb LGPKXBQM gkpLnmGx 
Control










 PkMmxgQq qgkpnmxb qBXQbKPM   Matching
   Balancing
   Random
assignment
   Limited
population
   Restrict
population
on
specific
characteristic
 ▪  Male,
right‐handed,
native
English
speaker
 ▪  If
we
restrict
too
much
we
lose
EXTERNAL
 VALIDITY!
 The
whole
population:
 a
D
b
Z
E
X
R
T
L u o q c
p N v h
Y
S
M
F i w
j…
 Limiting population Our
Sample:
X Y Z R S T 

 
 
 
Experimental













Control









 XYZ RST Method Pros Cons Matching Keeps matched variable constant between conditions • Finding matching participants is difficult • Selection bias (by another variable) Balancing Prevents confounding by the variable • Selection bias (by another variable) Random Produces equivalent Assignment groups Works poorly with small samples Limiting Eliminates some Reduces external validity Populations extraneous variables   Matching
   Balancing
   Random
assignment
   Limited
population
   Adding
pretest
 Whole sample: presetting on the measure   Before
manipulation
 Whole sample: presetting on the measure Group 1: experimental Group 2: control M Whole sample: measure outcome   Regression
towards
the
mean
   Extreme
scores
are
more
likely
to
be
closer
to
the
 mean
when
measured
again
 ▪  High
scoring
individuals
are
likely
to
do
worse
on
a
retest
 ▪  Low
scoring
individuals
are
likely
to
do
better
on
a
retest
   Regression
towards
the
mean
   Attrition
   Loss
of
participants
from
pretest
to
posttest
   Affects
external
validity
as
well
   Regression
towards
the
mean
   Attrition
   Maturation
   History
   Change
in
environment
   Regression
towards
the
mean
   Attrition
   Maturation
   History
   Testing
effects
   Get
more
practice
on
the
same
measure
   Regression
towards
the
mean
   Attrition
   Maturation
   History
   Testing
effects
   Instrumentation
effect
   Measurement
has
changed
 Nonequivalent groups Subjects may be divided to groups in a biased fashion. History Events may occur between multiple observations. Maturation Participants may become ‘older’ or fatigued. Regression to the mean Subjects may be selected based on extreme scores. Attrition Differential loss of subjects from groups in a study may occur. Testing Taking a pretest can affect results of a later test. Instrumentation Changes in instrument ‘calibration’ or observers may change results.   Other
threats?
   Diffusion
of
treatment
 ▪  Pp
already
have
information
about
study
   Participant
and
experimenter
effects
 ▪  Single
and
double‐blind
experiments
   Sensitivity
of
measure
 ▪  Avoid
floor
and
ceiling
effects
 ▪  Variability
between
scores
necessary
to
detect
 difference!
   Make
sure
groups
are
equal
before
 manipulation
   Balancing,
matching,
etc.
   Make
sure
groups
are
equal
before
 manipulation
   Make
sure
manipulation
actually
works
   Use
a
Manipulation
Check
 ▪  Explicit
measure
of
the

 independent
variable
 ▪  Embedded
questions
   Make
sure
groups
are
equal
before
 manipulation
   Make
sure
manipulation
actually
works
   Make
sure
to
use
a
good
control
group
   No‐treatment
control
vs.
Placebo
control
   Make
sure
groups
are
equal
before
 manipulation
   Make
sure
manipulation
actually
works
   Make
sure
to
use
a
good
control
group
   Make
sure
to
control
for
PP
and
experimenter
 effects
   Single
and
double‐blind
   Make
sure
groups
are
equal
before
 manipulation
   Make
sure
manipulation
actually
works
   Make
sure
to
use
a
good
control
group
   Make
sure
you
control
for
PP
and
experimenter
 effects
   Make
sure
you
use
a
sensitive
measure
   Check
for
floor
and
ceiling
effects
 ...
View Full Document

Ask a homework question - tutors are online