Unformatted text preview: SE 10 Research Design
Lecture 6 Learning from the midterm Dealing with threats to internal validity External validity
1 Variables From Lecture 1 A variable can be measured or recorded Variables have a set of possible values or attributes Variables are measured in specific units Gender: male or female Years, feet, index score, etc. 2 Independent and Dependent Variables From Lecture 1, slide 9 Dependent Variable: the outcome of interest Independent Variable(s): Variables you think affect or cause the dependent variable X > Y 3 From the midterm A researcher performs a study to determine if being a victim of a property crime causes residents to put locks on their doors or get a security system. In this study, the independent variable is: a. gender b. the victims of property crime c. home security behaviors d. property crime victimization
4 Types of research Inductive Research
Research Question Observations From lecture 1 Deductive Research
Theory Hypothesis Evaluate Question Develop Theory Observations Confirm, refute, revise theory 5 Both inductive and deductive methods are part of the research process From lecture 1 Inductive Research Theory Deductive Research 6 From the midterm You wonder if female labor force participation and childhood obesity are related. You decide to measure these two variables and see what the results are. This is an example of a. conceptualizing your variables b. deductive research c. inductive research d. measurement reliability
7 Levels of measurement From Lecture 2 Nominal: categories, no logical numerical value, measures type or quality Ordinal: categories that can be logically ordered but are not inherently numerical Interval: numbers are meaningful, equal distance between each rank Ratio: similar to interval, but must have a meaningful "0" value
8 Dichotomous: only two categories From the midterm You need to devise a way to measure the quality of a child's upbringing on a scale from 1 to 7, with 1 being a very poor upbringing and 7 being an excellent upbringing. The level of measurement that would best describe this variable is:
a. Interval b. Ordinal c. Ratio d. Nominal 9 Types of validity From lecture 2 Criterion validity how does the measure compare to other standard and accepted measures of the same thing? Face/Content validity does the measure make sense? Construct validity does the performance of the measure fit with existing theories?
10 Criterion validity From lecture 2 Can formally test the relationship between a new measure and an old measure of the same concept Only works when there is a previously existing accepted measure Types: Predictive, concurrent, and postdictive validity
11 Face or Content validity From lecture 2 Subjective assessment of how well the measure captures the concept Should always be considered, but especially when no criterion to use Panel of experts, logic, focus groups, previous research used to determine content
12 Construct validity From lecture 2 Asks how well the measure reflects the concept by looking at relationship with other variables Is it positively associated with concepts as indicated by theory: convergent validity Is it not associated with concepts not indicated by theory: discriminant validity
13 From the midterm You want to study the relationship between delinquency and self esteem. You know a lot about delinquency, but not that much about self esteem. So, before you start, you ask a few psychologist friends to look over your measure of self esteem to make sure it is reasonable and complete. For this measurement you are testing the: a. content or face validity b. criterion validity c. reliability d. construct validity
14 Internal Validity From lecture 3 How sure are you that the results of your study are due to a true causal relationship? Can you draw conclusions about the relationship between the independent variable ("cause") and the dependent variable (outcome)?
Threats to Internal Validity From lecture 3 Time Threats: Group Threat Maturation History Instrumentation Test Reactivity Selection Group and Time Threats Mortality/Attrition Selection by Time Selective attrition Regression Towards the Mean
16 History From lecture 3 Time threat: something happens between measurements An event occurs between measurements that changes the score Natural disasters, political changes, major case decisions The major event causes the change in measurement, not what you are studying
17 Measurement error and regression toward the mean From lecture 3 For our high risk subject there is likely to be high measurement error Extreme scores have more error If measurement error is random a second measurement will likely have less error Makes it look like subject's score went down, when really all that changed was amount of measurement error
18 From the midterm A researcher conducts a study of an after school program using boys with high levels of reported delinquency as subjects. The researcher finds that after only two weeks of the program the boys' levels of reported delinquency has gone down dramatically. What is a likely explanation for this? a. Regression toward the median b. Regression toward the mean c. A history threat to internal validity d. None of the above 19 Requirements for Causation From lecture 3 Covariation (correlation) Proper time order Just because A and B are correlated does not mean they are causally related, but correlation is required to prove causation A must occur in time before B Ruling out alternative hypotheses Other reasons for the relationship between A and B
20 From the midterm List and describe the three things needed to establish that a causal relationship exists. What would you say about the claim that because being involved in violence and being male are related, being involved in violence causes a person to be male? 21 From lecture 3 CORRELATION DOES NOT MEAN CAUSATION
22 Reverse causation A B A B From lecture 3 Both examples of direct causation The reverse causation explains correlation between A and B without A causing B
23 Reciprocal Causation A B From lecture 3 Feedback loop: A causes B and B causes A "the chicken or the egg?"
24 Spurious Causation C is a "confounding variable" From lecture 3 A B C C causes A and C causes B, explaining the correlation between A and B C must occur in time before A and before B
25 Measurement Association From lecture 3 L is a latent variable or "construct" A and B measure L (remember the conceptualizing process) L B A A and B only related because of L
26 Indirect Causation A I B From lecture 3 I is an "Intervening Variable" or "Mediating Variable" A causes I and I causes B, explaining the correlation between A and B
27 From the midterm A researcher demonstrates that babies born prematurely are more likely to use drugs at age 15 than babies who were carried full term. The researcher thinks this is a remarkable finding and suggest that it is more important than ever to make sure women get proper prenatal care because being born prematurely can cause later drug use. What do you think of this claim? Diagram a possible causal relationship other than the one the researcher suggests.
28 Back to the regularly scheduled programming The quality of a research design Will we be able to detect an effect? Will we be sure the effect is related to the independent variable? Are the findings generalizable can we use them in other situations?
29 Remember Type I and II errors How do we address threats with experimental design? Random assignment helps with Groups are equivalent in all respects (ideally) with the exception of the experimental treatment
30 History Maturation Selection Regression toward the mean Random Assignment/Equivalent Groups Post test only or pre/post test w/ random assignment A time threat would affect both groups O1 O1 X1 O2 O2
31 R Dealing with instrumentation and test reactivity Post test only or Solomon 4 group design O1 X1 X1 O2 O2 O2 O2
32 R O1 Attrition and selective attrition Hard to control for with experimental design alone Random assignment helps if the attrition is related to a preexisting characteristic Random assignment doesn't help if attrition is related to an experimental condition
33 You can work to minimize them If you can't use random assignment, can you still control threats?
Use pretests to check for group characteristics Select what you think is a similar comparison group Even with random assignment, groups can still be different
34 Random assignment ISN'T a cure all External Validity Can you generalize your results to other Can you replicate your findings? Populations Places Times 35 What affects external validity? Who you study The setting A specific type of subject or all possible subjects Highly controlled lab environment, a natural setting, or somewhere in between Are you able to measure the actual phenomenon you are interested in or do you have to use a proxy? The measures 36 Sample and external validity The results from your study (based on a sample) might be used to draw conclusions about a population Is your sample a good representation of your population? Are you only talking about a specific subset of your population? What would happen if you used a sample with more variability? 37 Setting and external validity Research usually done to find out information about how people act in a natural environment Natural environments have lots of uncontrolled factors Moving into the lab increases control, but reduces realism 38 Measurement and external validity We want to make conclusions about things that really happen We use variables to measure concepts Some measurements might be more realistic than others The most realistic measurement might not be possible
39 We've seen that there are numerous ways to measure the same concept Researcher's dilemma Internal validity: how certain are you that the results are caused by independent variable External validity: how generalizable are the results Control the experimental conditions Make experiment as similar to real world as possible less control
40 Can you always control for weaknesses in a study? No all research has flaws Some threats will be more important to some projects Decide where you are on the researcher's dilemma scale do you want to take something that occurs in the field into a lab to test it?
41 Focus on those ...
View Full Document
This note was uploaded on 09/14/2008 for the course SOCECOL 10 taught by Professor Pazzani-raitt during the Summer '08 term at UC Irvine.
- Summer '08