Unformatted text preview: Experimental Design Overview
q Kinds of designs Experimental--direct manipulation Quasi-Experimental--measurement of independent variables Non-experimental--no independent variables q Statistics Map onto the design Experimental Design
q Today: Elements of experimental design Begin examination of different kinds of designs Basics of experimental design
q Goal: Manipulate independent variable to affect dependent variable Want manipulation to be significant Want to rule out other possible explanations (threats to validity) q Do this be comparing differences between groups differing on the independent variable How can you do this:
q q Assume probabalistic equivalence! What this means: Will always be some difference between our experimental group and our control group before the experimental manipulation But these differences are due to chance What Is Probabilistic Equivalence? Pretest Population Experimental Group Control Group What Is Probabilistic Equivalence? q Pretest differences: Hope When we observe a pretest difference, it will be due to chance because we assigned by chance In other words, a nonsignificant difference between experimental and control group What Is Probabilistic Equivalence?
q q q q For post-test? Want the differences between experimental and control group to be greater than chance Presumably due to your manipulation. Still possible that the difference it's due to chance, but very small (p < .05). What Is Probabilistic Equivalence?
q q q Because we assigned randomly, we could observe differences on either the pretest or posttest, but these might be due to chance (or the luck of the draw). We hope that when we've manipulated something, that the differences in the groups at post-test will be beyond that due to chance Why we need statistics!!! Steps in an experimental design
q q q q Figure out your design first! Sampling, measurement Assignment of sample into groups (IV) Assessment of effect of group membership on outcome (DV) Terminology
q Independent variable: What you are manipulating Also called a FACTOR Level: The values your independent level can take q Effects: Null, Main, Interaction Example: Post-test only design R R X O O Factor 1: Time in beh Instruction Level 1: 4 hour per week Level 2: 0 hours per week Dependent Variable: ADHD symptomtology Example Time in Instruction A Simple Example Time in Instruction Factor: Major independent variable A Simple Example Time in Instruction
4 hour/week 0 hour/week A Simple Example Time in Instruction
4 hours/week 0 hour/week Levels: subdivisions of factors A Simple Example Time in Instruction
4 hours/week 0 hour/week Group 1 average Group 2 average Usually, averages are in the cells. The Null Case
4 hr Time 0 hr 5 5 The lines in the graphs below overlap each other. time 8 7 6 5 4 3 2 4hr 0hr Main Effects
4 hr Time 0 hrs 5 7 Main Effect time 8 7 6 5 4 3 2 4hr ohr Factorial Design
Design and Effects Last time
q q Simple 2 group 1 factor design Today Factorial designs What is an experiment? A factorial design means
q q ONE dependent variable TWO or MORE independent variables Example R R R R X11 X12 X21 X22 O O O O Factor 1: Time in Instruction Level 1: 1 hour per week Level 2: 4 hours per week Factor 2: Setting Level 1: In-class Level 2: Pull-out Dependent Variable: ADHD symptomtology Example Time in Instruction
1 hour/week In-class 4 hours/week Setting A 2(rows) x 2 (columns) design Pull-out Example Time in Instruction
1 hour/week In-class 4 hours/week Setting Group 1 average Group 2 average Group 3 average Group 4 average Usually, averages are in the cells. Pull-out The Null Case
1 hr Time 4 hrs Setting Out In 5 5 5
6 5 4 3 2 1 0 5 5 5 5 5 The lines in the graphs below overlap each other. Null Case Score Pull-out In-class 1-hr Time 4-hr A Main Effect
q q A consistent difference between levels of a factor For instance, we would say there's a main effect for time if we find a statistical difference between the averages for the 1 hour and 4 hour groups Main Effects
1 hr Time 4 hrs Setting Out In 5 5 5 7 7 7 6 6 Main Effect of Time
Main Effect- Time Case 8 Score 6 4 2 0 1-hr Time 4-hr Pull-out In-class Main Effects
1 hr Time 4 hrs Setting Out In 5 7 6 5 7 6
8 Score 6 4 2 0 5 7 Main Effect of Setting
Main Effect- Setting Case Pull-out In-class 1-hr Time 4-hr Main Effects
1 hr Time 4 hrs Setting Out In 5 7 6 7 9 8
10 8 Score 6 4 2 0 6 8 Main Effects of Time and Setting Main Effect- Time & Setting Case Pull-out In-class 1-hr Time 4-hr An Interaction Effect
q q q When differences on one factor depend on the level you are on on another factor An interaction is between factors (not levels) You know there's an interaction when can't talk about effect on one factor without mentioning the other factor The lines are not parallel when you plot them Interaction Effects
1 hr Time 4 hrs Setting Out In 5 5 5 5 7 6 5 6 The in-class, 4-hour per week group differs from all the others.
Interaction Effect 8 6 Score 4 2 0 1-hr Time 4-hr Pull-out In-class Interaction Effects
1 hr Time 4 hrs Out In 7 5 6 5 7 6 6 6 The 1-hour amount works well with pull-outs while the 4 hour works as well with in class.
Interaction Effect Setting 8 Score 6 4 2 0 1-hr Time 4-hr Pull-out In-class Advantages of Factorial Designs
q q q Offers great flexibility for exploring or enhancing the "signal" (treatment) Makes it possible to study interactions Combines multiple studies into one ...
View Full Document
This note was uploaded on 08/26/2009 for the course BB H 310W taught by Professor Saltsman,brian during the Spring '07 term at Penn State.
- Spring '07