Unformatted text preview: JS 202: Research Design, Methods and Evaluation Week 3 General issues in evaluation and research design Key concepts and terminology Determinism and the social sciences Units of analysis Definition of evaluation The systematic assessment of the operation and/or the outcomes of a program or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy Marvelously adaptable Might include social audits, system reviews, performance measurement Policy evaluation An officially accepted statement of objectives tied to a set of activities that are intended to realize the objectives in a particular jurisdiction. operations, national programs, local projects, the actual workings of a policy Program evaluation Local Determinism and the social sciences Primary question Theoretical frameworks Determinism Free will Explaining behavior Correlation/association Causality Types of causation (necessary v sufficient cause) Primary question Do we control our own behavior? Are we driven by outside influences? Basically, things are caused by other things But, what is the underlying basis for these causes? Theoretical frameworks Determinism Social disorganization theory Social process theories External factors influence our actions Free will Rational choice theory Routine activity theory We're in control of our actions Explaining behavior Correlation/association Association between two or more variables that is not causal (shoe size and IQ) Ice cream and crime both high when it's hot As a child's shoe size increases, so does their IQ Causality Influence of one variable causes a change in another variable Criteria for causation Cause must precede the effect A change in the first variable must cause a change in the second variable These changes make sense (they are empirically related) The relationship cannot be explained away by the presence of a third variable Causal effects cannot be directly observed, they must be inferred Types of causation Necessary cause Condition must be present in order for effect to occur that pretty much guarantees and effect our field, it is unlikely that we ever uncover one of these relationships really clearly Sufficient cause Condition In Threats to validity Construct validity/generalization External validity Ecological fallacy Units of analysis We need to be really clear about what we're measuring Individuals Groups Organizations Social artifacts The time dimension Looking forward v looking back Prospective Retrospective Types of studies Cross sectional Census eg. Longitudinal Panel Trends Cohort studies Research designs should... MAXIMIZE the variance in the DV due to the variable or variables of the substantive research hypothesis CONTROL the variance in the DV due to extraneous or unwanted variables that may have an effect on the outcome MINIMIZE the variance in the DV due to random error How to MAX CON MIN MAXIMIZE variance Keep conditions between control and experiment as distnict and different as possible How to MAX CON MIN CONTROL variance Randomize conditions Build the extraneous variable into the design as another variable Match subjects to achieve equivalence across groups How to MAX CON MIN MINIMIZE variance Carefully control the condition of the research Clear instructions, no distractions Use reliable measurement instruments (that don't contain a lot of random error) ...
View Full Document
- Causality, CONTROL variance Randomize, variable Causal effects, MAX CON MIN, variable Match subjects