Lecture 08 - PAM 3300 Research Designs and Causal Claims...

Info icon This preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
PAM 3300: Research Designs and Causal Claims
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Research Design For our purposes, research design refers to the set of analyses that will be implemented to assess the causal effect of X on Y. The design describes how the constructs of interest are operationalized & measured. Most importantly, the research design clarifies how the outcomes of observations with different levels of X will be compared to one another to relate X to Y. The design used is determined by: How data was collected - in other words, the reasons for variation in X in the data. Availability of data on control groups and availability of data from pre- and post-treatment outcomes. The availability of other important information about observations.
Image of page 2
Research Design: some options 1. Randomized control trials Units are randomly chosen to be given the treatment/program/drug, etc. The gold standard, but not perfect. 2. Post Treatment w/Comparison Group Compare post-treatment outcomes for people who did and did not receive the treatment. 3. Pre/Post Treatment (reflexive controls) Compare outcomes of a a group before and after receiving a treatment. 4. Pre/Post Treatment with Comparison Group Compare outcomes of two groups before and after treatment for a group that does and a group that does not receive the treatment. 5. Interrupted time-series. Similar to pre-post design, only with many years of ‘pre-’ and many years of ‘post-’ Note: All of these designs may or may not use regression to control for other variables.
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Research Design: more options 6. Matching Outcomes for people in the treatment group are compared to someone in the control group who looks most like a match in terms of their characteristics. 7. Natural Experiments: difference-in-differences Utilize a change in policy or some natural event that approximate random assignment of a treatment. 8. Instrumental Variables Find another variable that is uncorrelated with things in the error term, but is correlated with whether one receives the treatment. Analyze the impact of changes in that ‘instrumental variable’ to get at the true effect of the treatment on the outcome. 9. Regression Discontinuity The gold standard of non-experimental methods.
Image of page 4
Validity The research design determines the “validity” of causal claims. Validity is simply the extent to which we believe a claim - a measure of its “truthiness.” Kinds of validity: Construct validity Statistical conclusion validity Internal validity External validity
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Construct Validity Do the variables in the data set adequately capture the concepts/constructs we are trying to learn about? Ex: Does having more ‘social capital’ improve health? Examine by regressing county-level disease prevalence rates on number of churches.
Image of page 6
Statistical Conclusion Validity Are the relations between variables statistically significant, or are they the result of chance?
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern