Psych 7 - Establishing a Cause-Effect Relationship...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Establishing a Cause-Effect Relationship Relationship Internal Validity Internal Is the relationship causal between... Is causal • The “treatment” and the “outcomes” • The independent and dependent variables. Alternative cause Alternative cause Treatment What you do What do Outcomes Alternative cause What you see What see Alternative cause Observation In this study In this Establishing Cause and Effect Establishing Temporal precedence Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time It can get complicated through: -sloppiness (campaign contributions - Chicken and egg cyclical functions Chicken (democracy and GDP) (democracy Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time Covariation of cause and effect Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time Covariation of cause and effect if X, then Y iif not X, then not Y f not not if treatment given, then outcome observed (usually) iif program not given, then outcome not observed f not not Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time Covariation of cause and effect if X, then Y iif not X, then not Y f not not if program given, then outcome observed iif program not given, then outcome not observed f not not Dosage effects or comparative statics: If more of treatment, then more of outcome observed If more more iif less of treatment given, then less of outcome observed f less less Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time Covariation of cause and effect No alternative No explanations explanations if X, then Y iif not X, then not Y f not not Treatment Micromediation Outcome Establishing Cause and Effect Establishing Temporal precedence Cause then Effect Time Covariation of cause and effect No alternative No explanations explanations if X, then Y iif not X, then not Y f not not Alternative cause (substantive) Alternative Alternative cause Treatment Micromediation Alternative cause Outcome Alternative cause (nuisance) (nuisance) In Lab or Field Experiments… In Temporal Temporal precedence precedence q Is taken care of because you Is intervene before you measure outcome outcome Covariation of q Is measured by comparing Covariation Is cause and effect treated and untreated groups cause treated No alternative No explanations explanations q Is the central issue of internal Is validity -- usually taken care of through random assignment through Single-Group Threats to Internal Validity Validity The Single Group Case The Two designs: The Single Group Case The Two designs: Administer program Measure outcomes X - X is the treatment is - O is the observation is O “Post-test only single-group design” The Single Group Case The Two designs: “pre-test, post-test single-group design” or “interrupted time-series” Measure baseline Administer program Measure outcomes O X O The Single Group Case The Two designs: Administer program Alternative explanations Measure outcomes X Measure baseline Administer program O X Alternative explanations Measure outcomes O O Alternative explanations Example Example q q q After the 2003 recall election, did After Democrats in the California Assembly move to the center? move California ran a full legislative “season” California before the October, 2003 election, then ran another “season” afterward. ran We can look at roll call vote behavior Example: What Kind of Design? Example: History Threat History Pretest Program Program Posttest O q q X O q Any other event that occurs between pretest Any and posttest and Perhaps the nation was just shifting to the Perhaps center at this time. center How might we rule it out? How Maturation Threat Maturation Pretest Program Posttest O q X O q Normal growth between pretest and Normal posttest. posttest. Coming into an election year, state Coming legislators always shift to the center. legislators Ruling Out a Maturation Threat Ruling Testing Threat Testing Pretest Program Program Posttest O q X O q The effect on the posttest of taking the The pretest pretest Legislators may have learned that the state Legislators was watching them. When real tests are given, this is a big problem. Instrumentation Threat Instrumentation Pretest Program Posttest O q X O q Any change in the test from pretest and Any posttest posttest A different test may have been used if a different different roll call estimation technology used. different Mortality Threat Mortality Pretest Program Posttest O q X O q Nonrandom dropout between pretest and Nonrandom posttest posttest If some legislators had been recalled along If with Gray Davis, this would be a problem. with Regression Threat Regression Pretest Program Program Posttest O q X O q Group is a nonrandom subgroup of Group population. population. The 2003 session was particularly extreme, The any other session would look more centrist. Multiple-Group Threats to Internal Validity Validity The Central Issue The q q q When you move from single to multiple When group research the big concern is whether the groups are comparable. comparable Usually this has to do with how you Usually assign units (for example, persons) to the groups (or select them into groups). select If you are not careful, may mistake a If selection effect for a treatment effect. for The Multiple Group Case The Alternative explanations Measure baseline Administer treatment Measure outcomes O O Measure baseline X Do not Do not administer administer treatment O O Measure outcomes Alternative explanations Example Example q q q Suppose USAID looked before and Suppose after at countries where it did and didn’t run governance programs in the last decade decade Pre-post program-comparison group Pre-post design design Measures (O) are all of the things Clark Measures hates, but let’s set that aside for now. hates, Selection Threats Selection O O q X O O q Any factor other than the program that leads to posttest differences between groups. groups. USAID did not randomly select the USAID countries in which it ran programs, and sent aid to those with the lowest-rated governments governments Selection-History Threat Selection-History O O q X O O q Any other event that occurs between Any pretest and posttest that the groups experience differently. experience For example, countries that begin with For more stable democracies faced fewer challenges in the past decade. challenges Selection-Maturation Threat Selection-Maturation O O q X O O q Differential rates of normal growth Differential between pretest and posttest for the groups. groups. It is easier to move from a semidemocracy to a full democracy than it democracy is to move from a non-democracy to a semi-democracy semi-democracy Selection-Testing Threat Selection-Testing O O q X O O q Differential effect on the posttest of taking the pretest. taking At least these measures are At “unobtrusive,” so this probably is not a grave threat grave Selection-Instrumentation Threat Selection-Instrumentation O O q X O O q Any differential change in the test used Any differential for each group from pretest and posttest posttest For example, the Polity measures may For give some countries credit for having a USAID program USAID Selection-Mortality Threat Selection-Mortality O O q X O O q Differential nonrandom dropout between pretest and posttest. between Perhaps the countries with weak Perhaps governments are more likely to cease being a country over the past decade. Selection-Regression Threat Selection-Regression O O q X O O q Different rates of regression to the Different mean because groups differ in extremity. extremity. For example, the countries that USAID For chooses may have nowhere to go but up. up. “Social Interaction” Threats to Social Internal Validity Internal What Are “Social” Threats? What • All are related to social pressures in the research context, which can lead to posttest differences that are not directly caused by the treatment itself. • Most of these can be minimized by isolating the two groups from each other, but this leads to other problems (for example, hard to randomly assign and then isolate, or may reduce generalizability). Types of Designs Types Types of Designs Types Random assignment? Types of Designs Types Random assignment? Yes Types of Designs Types Random assignment? Yes Randomized or true experiment? Types of Designs Types Random assignment? Yes No Randomized or true experiment? Types of Designs Types Random assignment? Yes No Control group or multiple measures? Randomized or true experiment? Types of Designs Types Random assignment? Yes No Control group or multiple measures? Yes Randomized or true experiment? Types of Designs Types Random assignment? Yes No Control group or multiple measures? Yes Randomized or true experiment? Quasi-experiment Types of Designs Types Random assignment? Yes No Control group or multiple measures? Yes No Randomized or true experiment? Quasi-experiment Types of Designs Types Random assignment? Yes No Control group or multiple measures? Yes No Randomized or true experiment? Quasi-experiment Nonexperiment Design Notation Example Design R R O O X O O Os indicate different waves of measurement. Elements of a Design Elements q q q q q Observations and measures Treatments Groups Assignment to group Time Design Notation Example Design Vertical alignment of Os shows that pretest and posttest are measured at same time. R R O O X O O Design Notation Example Design X is the treatment. R R O O X O O Design Notation Example Design There are two lines, one for each group. R R O O X O O Design Notation Example Design R R R iindicates ndicates the groups are randomly assigned. assigned. O O X O O Design Notation Example Design R R O1 X O1 O1, 2 O1, 2 Subscripts indicate subsets of measures. Design Notation Example Design R R O O X O O Pretest-posttest (before-after) Treatment versus comparison group Randomized experimental design Design Example Design Posttest Only Randomized Experiment Design Example Design Posttest Only Randomized Experiment R R X O O Design Example Design Pretest-Posttest Nonequivalent Groups Quasi-Experiment Design Example Design Pretest-Posttest Nonequivalent Groups Quasi-Experiment (note multiple groups or multiple observations are REQUIRED to have a quasi-experiment) N N O O X O O Design Example Design Posttest Only Nonexperiment Design Example Design Posttest Only Nonexperiment X O ...
View Full Document

Ask a homework question - tutors are online