Unformatted text preview: JS202: Research Design, Methods and Evaluation Week 2 Introduction to evaluative research Social programs should have demonstrable benefits How do we know?
Study what the Px actually does and whether it's doing what it was meant to do Social contract it's our responsibility to know if it's working Evaluation research A heterogeneous mix of procedures Cost benefit analysis; meta-analysis; survey; randomized experiments Relies on the scientific method Unbiased evaluation Multidisciplinary a lot can be learned from different fields of study Includes
Design of social Px Ongoing monitoring of how well programs are functioning Assessment of Px impact Analysis of Px benefits relative to their costs Types of evaluation research Meta-analysis Estimates Px impact by combining findings from a number of distinct evaluations advice to stakeholders about how to analyze and construct social policies Policy analysis Offers Types of evaluation research Formative evaluation Cook tastes the soup tastes the soup Summative evaluation Guest Program effectiveness Whether or not the Pxs are achieving their goals Cost benefit outcomes per $$ Performance auditors Staffing, equipment, management, money, efficiency, effectiveness, Purposes of evaluation Provide unbiased information about what is happening (on the ground) Evidence of short and long term effects Do variations in activity provide differential outcomes Postpone decisions, good/bad PR, duck responsibility Might become part of a larger political game Can the same results be achieved for less money Purposes of evaluation Midcourse corrections Find out what's wrong, make changes Is the Px achieving its goals Series of demonstration Pxs Continue, expand, or end the Px Test a new idea Choose the best of several alternatives Which option has the best outcome For agencies, or individual Pxs Decide whether to continue funding What to focus on Vision statement Performance indicators Who is the recipient/customer Mission Goal Activity Outputs/outcome indicators Basic steps of an evaluation Have some idea of what you're supposed to be doing in the long and short term Have a short and long term plan for how to do it Develop some way to gauge and report progress Differences between academic and evaluation research
Academic Longstanding interest in theories Usually published Exploration of something unknown Description scope of the problem, counting or documenting behavior Explanation more powerful why does it happen? Application when it matters "pure" Evaluative Interest in particular problem RQ comes from policy and Px concerns Practical Can be political Compares what is with what should be Obligations to the organization "Lower order" Criminal justice and social inquiry Two realities Experiential reality Knowledge from direct experience, childhood from general agreement, because we've been told it is so. Agreement reality Knowledge What does science mean to you? Science: a systematic search for the most accurate and complete description and exploration of events, relying on the principles of empiricism, objectivity, relativism, skepticism, ethical neutrality, parsimony and at least a modified determinism as guides Can CJ and Crim'y be scientific? Empirical research Production of knowledge based on experience or observation Confirms assertions Links agreement reality with experiential reality There is empirical support for the argument that ice cream sales rise when crime rates rise... but is this a logical link? Empirical research should be: Objective Ethically neutral Skeptical Parsimonious deterministic Human inquiry Knowledge without science comes from Tradition Authority Errors in human inquiry Inaccurate observation Overgeneralization Selective observation Ex post facto hypothesizing Illogical reasoning Ideology and politics Premature closure of inquiry Foundations of social science Logic or rationality Observation Theory Data collection Data analysis Attributes Variables Using variables to explain behavior ...
View Full Document
- Evaluation methods, Evaluation Research, social Px Ongoing, Px impact Analysis, Unbiased evaluation Multidisciplinary