Lecture11-Feb+11th-Internal+validity

# Lecture11-Feb+11th-Internal+validity -  ...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview:   Exercise 3 due Today    Files not dropped in drop‐box before 1.40pm will  be considered late    Grader is LAURA. Address any questions/concerns  to her    Midterm 2 next Tuesday    Chapter 3 (reliability & validity), 4, 5, 6, 8  (assigned pages), 12 (assigned pages)    Lectures 7,8,9, 10, & 11    Two review sessions:    Monday, Feb 15th, 7.00‐9.00pm (TAs)    Tuesday, Feb 16th, 11.00‐12.30 (Instructor)    As usual…    Bring #2 pencil    Bring Scantron (Davis 200)    Be On Time!    Limitations    Direction of causality is unknown    Third variable problem  ▪  Relationship between variables is spurious    How can we tease apart these diﬀerent  possibilities?  Type of design  Typical studies  Descriptive  Observational studies  Case studies  Surveys  Predictive  Correlational studies  Quasi‐Experiments  Explanatory  Experiments    In an experiment, at least one variable is  manipulated (i.e., systematically varied) by  the researcher in order to study its eﬀects on  another variable.    Features of an experiment  (a) At least one variable is manipulated or varied by  the experimenter: independent variable (IV)  (b) The variable presumably aﬀected by the  manipulation is called the dependent variable  (DV)  (c) random assignment to conditions  IV DV Independent Variable: Watching violent TV Dependent Variable: Aggressive behavior Levels: Number of times the child punches his or her peers on the playground (a)  view an episode of the Sopranos (b)  view an episode of the Sopranos in which the violent scenes have been edited   Between‐ and within‐subjects designs    between‐subjects: diﬀerent people are exposed  to each level of the IV    within‐subjects: the same people exposed to  each level of the IV    Allows to establish cause‐eﬀect relationships,  BUT…    To be able to do that we need to ensure that  our study has…    INTERNAL VALIDITY    Note that we can NEVER prove causality! We can  only show to what degree it is PROBABLE!   Establishing Internal Validity    What it means to establishing internal validity?    Threats to internal validity (Between SS  designs)    Simple experiment    Pretest‐posttest design  1.  2.  Covariation  Temporal precedence  3.  Eliminate spuriousness  Measure other variables of concern    Ensuring extraneous variables don’t turn into  CONFOUNDS      Confound (or confounding variable)    Variable that inﬂuences the dependent variable  and is associated with the independent variable    Prevents us from making strong inferences about  causality!  IV DV Independent Variable: Watching violent TV Dependent Variable: Aggressive behavior Levels: Number of times the child punches his or her peers on the playground (a)  view an episode of the Sopranos (b)  view an episode of the Sopranos in which the violent scenes have been edited   Non‐confounding variable:    A variable that has an eﬀect on the dependent  variable but that is uncorrelated with the  independent variable.  Watching violent TV + Acting violent + Living in a violent family Acting violent + Living in a violent family + Watching violent TV   What it means to establishing internal validity?    Threats to internal validity    Simple experiment    Pretest‐posttest design    AKA posttest only control group design    Two groups: control vs. experimental   Practice Group 1: questions experimental M Noroup 2: G practice qcontrol uestions   Non‐equivalent groups  Whole Score on sample: exam measure outcome   Pretest‐Posttest with Control design  WScore on hole sample: measure before exam manipulation Practice Group 1: questions experimental No roup 2: G practice questions control M Whole Score on sample: exam measure outcome   Measurements before AND after manipulation    Groups + time    Nonequivalent groups    Selection Bias  ▪  Participants are self‐assigned to groups    Applying arbitrary rules    Matching    Constructing each group so they have  identical characteristics    Find a “match” for every subject  ▪  T1DM vs. “matched” controls  ▪  Match age, gender, IQ, SES, mood on experiment day…  Our sample:  a D b Z E X R T L u o q c p N v h Y S M F i w j Matching    Experimental                           Control            X L Y j q Do N Rc wh XL Yj q Do N Rc w h   Matching    Balancing    Balance characteristics of group (not  individual subject)  Our sample:  a D b Z E X R T L u o q c p N v h Y S M F i w j Balancing     Experimental         XYZ abc LMN opq    Control            RST hIj DEF uvw   Matching    Balancing    Random assignment    Individuals have equal probability of being  assigned to one of the groups    Diﬀerences are “averaged out”    Not good with small samples!!!!!!  Our sample:   q B g k p X L n m Q G x M b K P  Random Assignment   Experimental                LGpKXBnb LGPKXBQM gkpLnmGx  Control            PkMmxgQq qgkpnmxb qBXQbKPM   Matching    Balancing    Random assignment    Limited population    Restrict population on speciﬁc characteristic  ▪  Male, right‐handed, native English speaker  ▪  If we restrict too much we lose EXTERNAL  VALIDITY!  The whole population:  a D b Z E X R T L u o q c p N v h Y S M F i w j…  Limiting population Our Sample: X Y Z R S T         Experimental              Control           XYZ RST Method Pros Cons Matching Keeps matched variable constant between conditions • Finding matching participants is difficult • Selection bias (by another variable) Balancing Prevents confounding by the variable • Selection bias (by another variable) Random Produces equivalent Assignment groups Works poorly with small samples Limiting Eliminates some Reduces external validity Populations extraneous variables   Matching    Balancing    Random assignment    Limited population    Adding pretest  Whole sample: presetting on the measure   Before manipulation  Whole sample: presetting on the measure Group 1: experimental Group 2: control M Whole sample: measure outcome   Regression towards the mean    Extreme scores are more likely to be closer to the  mean when measured again  ▪  High scoring individuals are likely to do worse on a retest  ▪  Low scoring individuals are likely to do better on a retest    Regression towards the mean    Attrition    Loss of participants from pretest to posttest    Aﬀects external validity as well    Regression towards the mean    Attrition    Maturation    History    Change in environment    Regression towards the mean    Attrition    Maturation    History    Testing eﬀects    Get more practice on the same measure    Regression towards the mean    Attrition    Maturation    History    Testing eﬀects    Instrumentation eﬀect    Measurement has changed  Nonequivalent groups Subjects may be divided to groups in a biased fashion. History Events may occur between multiple observations. Maturation Participants may become ‘older’ or fatigued. Regression to the mean Subjects may be selected based on extreme scores. Attrition Differential loss of subjects from groups in a study may occur. Testing Taking a pretest can affect results of a later test. Instrumentation Changes in instrument ‘calibration’ or observers may change results.   Other threats?    Diﬀusion of treatment  ▪  Pp already have information about study    Participant and experimenter eﬀects  ▪  Single and double‐blind experiments    Sensitivity of measure  ▪  Avoid ﬂoor and ceiling eﬀects  ▪  Variability between scores necessary to detect  diﬀerence!    Make sure groups are equal before  manipulation    Balancing, matching, etc.    Make sure groups are equal before  manipulation    Make sure manipulation actually works    Use a Manipulation Check  ▪  Explicit measure of the   independent variable  ▪  Embedded questions    Make sure groups are equal before  manipulation    Make sure manipulation actually works    Make sure to use a good control group    No‐treatment control vs. Placebo control    Make sure groups are equal before  manipulation    Make sure manipulation actually works    Make sure to use a good control group    Make sure to control for PP and experimenter  eﬀects    Single and double‐blind    Make sure groups are equal before  manipulation    Make sure manipulation actually works    Make sure to use a good control group    Make sure you control for PP and experimenter  eﬀects    Make sure you use a sensitive measure    Check for ﬂoor and ceiling eﬀects  ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online