qualitative_research_methods_in_program_evaluation.pdf -...

This preview shows page 1 - 6 out of 46 pages.

We have textbook solutions for you!
The document you are viewing contains questions related to this textbook.
Understanding Statistics in the Behavioral Sciences
The document you are viewing contains questions related to this textbook.
Chapter 18 / Exercise 3
Understanding Statistics in the Behavioral Sciences
Pagano
Expert Verified
QUALITATIVE RESEARCH METHODS IN PROGRAM EVALUATION: CONSIDERATIONS FOR FEDERAL STAFF Office of Data, Analysis, Research & Evaluation Administration on Children, Youth & Families May 2016
We have textbook solutions for you!
The document you are viewing contains questions related to this textbook.
Understanding Statistics in the Behavioral Sciences
The document you are viewing contains questions related to this textbook.
Chapter 18 / Exercise 3
Understanding Statistics in the Behavioral Sciences
Pagano
Expert Verified
Table of Contents EXECUTIVE SUMMARY ............................................................................................................. i INTRODUCTION .......................................................................................................................... 1 PART ONE: DECIDING WHEN TO USE QUALITATIVE METHODS .................................... 3 What are Qualitative Methods? .................................................................................................. 3 The Role of Qualitative Methods in Program Evaluation .......................................................... 4 TEXT BOX 1: The Role of Qualitative Methods in Program Evaluation .............................. 6 Summary ..................................................................................................................................... 7 PART TWO: GETTING STARTED ............................................................................................. 8 Researcher Experience ................................................................................................................ 8 Questions to ask about researcher credibility ......................................................................... 9 Budgeting and Time Planning .................................................................................................... 9 Questions to ask about time and cost estimates .................................................................... 11 Summary ................................................................................................................................... 11 PART THREE: A BRIEF PRIMER ON QUALITATIVE METHODS ..................................... 12 Research Design: The Structure of the Study .......................................................................... 12 Questions to ask about research design ................................................................................ 13 Sampling: What Gets Measured ............................................................................................... 14 Sampling Strategies in Qualitative Research ........................................................................ 14 Sample Size ........................................................................................................................... 15 Questions to ask about sampling ........................................................................................... 16 Qualitative Data Collection: Methods and Other Considerations ............................................. 16 Interviews and Focus Groups ................................................................................................ 17 Asking Open-Ended Questions ............................................................................................. 18 Capturing Interview Data ...................................................................................................... 19 Qualitative Observation ........................................................................................................ 19 Document Review ................................................................................................................. 20 On-Site Data Collection and Researcher Bias ...................................................................... 20 A Few Words About Data Collection By Program Staff ..................................................... 22 Questions to ask about data collection .................................................................................. 23 Qualitative Data Analysis ......................................................................................................... 24 TEXT BOX 4 Overview of the Qualitative Data Analysis Procedures ............................... 25 Inter-Rater Reliability ........................................................................................................... 26 Questions to ask about qualitative data analysis ................................................................... 26
Summary ................................................................................................................................... 27 PART FOUR: THE CREDIBILITY OF QUALITATIVE FINDINGS ....................................... 28 Internal Validity: Ruling Out Alternative Explanations .......................................................... 28 Rival Conclusions ................................................................................................................. 29 Contrasting and Disconfirming/Negative Cases ................................................................... 30 Triangulation ......................................................................................................................... 31 Getting Feedback from Participants/Informants ................................................................... 32 Questions to ask about internal validity .................................................................................... 32 External Validity: Generalizing the Findings .......................................................................... 33 Sample-to-population extrapolation ...................................................................................... 34 Case-to-case transfer ............................................................................................................. 35 Analytic generalization ......................................................................................................... 35 Summary ................................................................................................................................... 36 Questions to ask about generalizability ................................................................................ 36 CONCLUSION ............................................................................................................................. 37 REFERENCES ............................................................................................................................. 38
EXECUTIVE SUMMARY Qualitative research methods can play a powerful role in program evaluation, but they frequently are misunderstood and poorly implemented, giving rise to the idea that they are just not as rigorous and credible as quantitative methods. That is not true, but qualitative methods are less standardized than are quantitative methods, and that makes determining their appropriate use and assessing their quality more difficult for federal staff overseeing program evaluation projects that include a qualitative component. This document was written to support federal program officers, particularly those who have not had formal research training, as they develop and oversee projects that include program evaluations that use qualitative methods. Divided into four parts, this paper begins with the decision to include qualitative methods (or not) in a contemplated project. This first section defines qualitative methods, distinguishes them from quantitative research methods, and outlines the roles qualitative approaches can play in program evaluation. This portion of the document will be particularly useful as a project or a funding announcement is being developed.

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture