Lecture6-January+21st-sampling%2C+reliability%2C+validity

Lecture6-January+21st-sampling%2C+reliability%2C+validity -...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview:   Midterm
1
next
TUESDAY
   Lectures
2,
3,
4,
5,
6
   Chapters

1,
2,
3,
4
(pp.
94‐97
on
sampling)
   Midterm
1
next
TUESDAY
   Bring
#2
pencil
+
Blue
SCANTRON
   Be
on
time!!!!
   You
won’t
be
allowed
to
take
the
test
if
someone
 has
already
turn
in
one!
   Review
sessions
time
   Monday,
6.30‐8.30
pm
(TAs)
   Tuesday,
11.30‐1.00pm
(Instructor)
   How
to
access
Review
sessions
   Log
into
smartsite,
choose
the
chat
tab
   Specific
chat
rooms
will
be
set
for
the
reviews
   Exercise
2
is
due
on
TUESDAY
before
class
   Allison
will
be
your
grader
   Late
assignment
are
deducted
5
pts
for
each
day
 late
   Generating
Hypothesis
   Descriptive
vs.
causal
   Directional
vs.
non‐directional
   Identifying
and
defining
variables
   Setting
up
a
study
   Choosing
study
type
   Choosing
measures
   Choosing
participants
   Depends
on
the
characteristics
of
our
 hypothesis!
   Descriptive:
Formal
statement
of
a
predicted
 observation
 ▪  Correlational
   Causal:
Explains
possible
cause
for
the
 pattern
stated
in
the
descriptive
hypothesis. 
 ▪  Experimental
   Generating
Hypothesis
   Descriptive
vs.
causal
   Directional
vs.
non‐directional
   Identifying
and
defining
variables
   Setting
up
a
study
   Choosing
study
type
   Choosing
measures
   Choosing
participants
   How
do
we
measure
variables?
   Measurement
Types
   Measurement
scales
   Nominal
Scale
   Numbers
represent
groups
or
categories

   Qualities
NOT
quantities
 ▪  Relationship
with
partner:

 ▪  1=friend,
2=sexual
bond,
3=
emotional
bond
   Only
allows
us
to
test
whether
there
is
a
 difference
between
categories,
not
if
one
is
 more
than
the
other!
   Ordinal
Scale
   Values
of
variable
are
ranked
according
to
 quantity
 ▪  Reading
abilities
 ▪  1=low,
2=medium,
3=high
   Interval
between
groups
may
differ!
 ▪  
Know
which
is
greater,
but
do
not
know
how
much
greater
   Interval
Scale
   Numeric
values
represent
equal
distance
   Intervals
are
all
same
size!
 ▪  Celsius
or
Fahrenheit
scale
of
temperature
 ▪  IQ
test
score
 ▪  the
difference
b/w
2
people
with
a
120
score
and
a
 score
of
100
is
the
same
as
the
difference
b/w
 people
with
scores
of
80

and
60
(i.e.,
20
points)
   No
true
zero
point=>
values
can
be
negative
   Ratio
Scale
   Same
as
interval
but
with
a
true
ZERO
point
 ▪  Number
of
relationships
 ▪  0
quite
literally
means
none
 ▪  a
person
who
has
had
4
relationships
has
had
 twice
as
many
as
someone
who
has
had
2
   Can
determine
relative
magnitude
 ▪  Age,
height,
weight,
#
errors
on
a
test…
   Depends
on
your
question
   Do
two
group
differ?
 Nominal   Does
one
group
have
more?
 Ordinal   How
much
more?
 Interval   Three
times
as
much?
 Ratio   Depends
on
your
planned
statistical
analyses
   Nominal
and
ordinal
data
=>
nonparametric
 statistical
tests

   Interval
and
ratio
data
=>
parametric
statistical
 tests 

   Likert
Scales
   Are
actually
NOMINAL
scales
   In
practice
used
as
interval
scales
 Strongly Moderately Slightly Neutral Slightly disagree disagree disagree agree 1 2 3 4 5 Moderately Strongly agree agree 6 7   Generating
Hypothesis
   Descriptive
vs.
causal
   Directional
vs.
non‐directional
   Identifying
and
defining
variables
   Setting
up
a
study
   Choosing
study
type
   Choosing
measures
   Choosing
participants
   Population
   Large
group
including
all
potential
subjects
   The
group
of
interest
   Sample
   Small
subgroup
of
subjects
chosen
from
the
 population
   Need
to
be
representative
 ▪  External
validity
   Random
sampling
   All
members
of
population
of
 interest
have
equal
and
independent
 chance
of
being
selected
   Simple
   Systematic

   Random
sample
   Stratified
 ▪  Randomly
pick
from
subgroups

 ▪  match
to
proportions
in
the
population
   Cluster
 ▪  Randomly
pick
the
subgroups
 ▪  All
people
in
the
group
are
selected
   Nonrandom
sample
   Convenience
   Quota
 ▪  Identify
subgroups
 ▪  Establish
quotas

 ▪  how
many
to
sample
from
each
subgroup
 ▪  Use
convenience
sampling
   Nonrandom
sample
   Convenience
   Quota
   Snowball
 ▪  ‘Word
of
mouth’
technique
   Random
Sampling
   Random
Assignment
   The
problem
of
artifacts
(systematic
errors)
   Reliability
   Validity
   The
problem
of
artifacts
   Uncontrolled
human
aspects
of
the
research
 situation
that
CONFOUND
researcher’s
 conclusions
   Participants
related
artifacts
 (aka
Demand
Characteristics)
   Cooperative
 ▪  Tries
to
give
the
‘best
performance’
that
matches
the
 presumed
hypothesis
   Non‐Cooperative
 ▪  Doesn’t
care
about
study
or
tries
to
sabotage
results
   Defensive
 ▪  Wants
to
be
portrayed
in
good
light
   Researchers
related
artifacts
   Observer
bias
 ▪  Over‐
or
under‐estimation
of
what
was
observed
   Expectancy
bias
 ▪  ‘Self‐fulfilling
prophecy’
   Blind
experiments
   Deception
   ‘Double
blind’
   Automation
(Standardization)
   Computers
   Recording
instructions
   
Question
participants
   The
problem
of
artifacts
   Reliability
   Are
our
measurements
precise?
   Validity
   Are
we
really
measuring
what
we
think
we
are
 measuring?
   Extent
to
which
measurements
are
free
 of
random
errors
   Random
error:
nonsystematic
mistakes
in
 measurement
 ▪  misreading
a
questionnaire
item
 ▪  observer
looks
away
when
coding
behavior
 ▪  nonsystematic
misinterpretations
of
a
behavior
   What
are
the
implications
of
random
 measurement
errors
for
the
quality
of
our
 measurements?
  O
=
T
+
E
+
S
 O
=
a
measured
score
(e.g.,
performance
on
an
exam)
 T
=
true
score
(e.g.,
the
value
we
want)
 E
=
random
error
 S
=
systematic
error

  O
=
T
+
E
 (we’ll
ignore
S
for
now,
but
we’ll
return
to
it
later)
   O
=
T
+
E
   The
error
becomes
a
part
of
what
we’re
 measuring!
   Do
random
errors
accumulate?
   Answer:
No.

If
E
is
truly
random,
we
are
just
as
 likely
to
overestimate
T
as
we
are
to
 underestimate
T.
   important
way
to
reduce
the
influence
of
 random
errors
of
measurement
is
to
use
 multiple
measurements.
   Operationally
define
latent
variables
via
multiple
 indicators
   Use
more
than
one
observer
when
quantifying
 behaviors
   Multiple
observations
   How
do
we
assess
reliability?
 (a)
test‐retest
reliability
 (b)
alternate‐forms
reliability
 (c)
internal
consistency
reliability
   Test‐retest
reliability:

   measure
something
at
least
twice
at
different
 time
points.
   If
errors
of
measurement
are
truly
random,
then
 the
same
errors
are
unlikely
to
be
made
more
 than
once.


   If
two
measurements
of
the
same
thing
agree,
it
is
 unlikely
that
they
contain
random
error.
   Test‐retest
reliability:

   IMPORTANT:
we
are
assuming
that
what
we
are
 measuring
does
NOT
vary
over
time!!!!!!
   Sometimes
people
remember
previous
answers
 ▪  INFLATED
reliability
coefficient
   Other
methods?
   Alternate‐forms
reliability

   Use
two
equivalent
tests
   Correlation
should
be
high
   Internal
consistency
   Extent
to
which
items
in
questionnaire
correlate
 with
each
other
   If
measuring
same
thing,
correlation
should
be
 high
 Split‐half:
based
on
an
arbitrary
split
(e.g,
 comparing
odd
and
even,
first
half
and
second
 half)
 Cronbach’s
alpha
(α):
based
on
the
average
of
all
 possible
split‐halves
   Inter‐rater
reliability
   Percentage
of
time
agreed
   Correlation
ratings
   Kappa
Coefficient
   ASSUMPTION
   The
entity
being
measured
is
not
changing.
   IMPLICATIONS
   
As
you
increase
the
number
of
indicators,
the
amount
of
 random
error
in
the
averaged
measurement
decreases.

   NOTE

   Common
indices
of
reliability
range
from
0
to
1;
higher
 numbers
indicate
better
reliability
(i.e.,
less
random
error).
   The
problem
of
artifacts
   Reliability
   Are
our
measurements
precise?
   Validity
   Are
we
really
measuring
what
we
think
we
are
 measuring?
  O
=
T
+
E
+
S
 O
=
a
measured
score
(e.g.,
performance
on
an
exam)
 T
=
true
score
(e.g.,
the
value
we
want)
 E
=
random
error
 S
=
systematic
error

   Validity
   Degree
to
which
measurements
are
free
of
both
 random
error,
E,
and
systematic
error,
S.
   Systematic
errors
reflect
the
influence
of
any
 non‐random
factor
beyond
what
we’re
 attempting
to
measure.
   Do
systematic
errors
accumulate?
   YES!
Systematic
errors
exert
a
constant
source
of
 influence
on
measurements.

   
We
will
always
overestimate
(or
underestimate)
T
 if
systematic
error
is
present!
   Are
we
measuring
what
we
think
we’re
 measuring?

   Construct
Validity
   Is
the
cause‐effect
relationship
really
there?

   Internal
Validity
   Are
our
results
generalizable?

   External
Validity
   How
well
we
measured
what
we
intended
to
   Can
be
established
in
a
variety
of
ways
   Face
validity
   Content
validity
   Convergent
validity
   Discriminant
validity
   Concerns
cause
and
effect
relationship
   Low
internal
validity
for
predictive
designs
   Correlation
does
not
imply
causation
   High
internal
validity
for
explanatory
designs
   Manipulation
changes
outcome
   Need
to
rule
out
effect
of
extraneous
variables
 ▪  Extraneous
vs.
Confounding
variable
   Do
finding
generalize?
   Representative
Sample
   Report
sample
characteristics
   Representative
Setting
   Difficult
in
experimental
designs
   Report
setting
characteristics
 High
 Internal
Validity
 High
 External
Validity
 Causality
 Generalizablilty
 Explanatory

 designs
 Predictive
 designs
 Low
External
Validity

 Low
Internal
Validity

   Cannot
have
validity
if
there
is
no
 reliability
   Reliability
does
NOT
guarantee
 validity
 ...
View Full Document

This note was uploaded on 06/21/2011 for the course PSYCHOLOGY Psych 41 taught by Professor Castelli during the Winter '10 term at UC Davis.

Ask a homework question - tutors are online