Criminal justice it suggests that sys tematic reviews

This preview shows page 18 - 19 out of 21 pages.

criminal justice, it suggests that sys- tematic reviews of what works in criminal justice may be strongly biased when including nonrandomized studies. In efforts such as those being developed by the Campbell Collaboration, such poten- tial biases should be taken into account in coming to conclusions about the effects of interventions. Notes 1. Statistical adjustments for random group differences are sometimes employed in experimental studies as well. 2. We should note that we have assumed so far that external validity (the degree to which it can be inferred that outcomes apply to the populations that are the focus of treat- ment) is held constant in these comparisons. Some scholars argue that experimental stud- ies are likely to have lower external validity because it is often difficult to identify institu- tions that are willing to randomize partici- pants. Clearly, where randomized designs have lower external validity, the assumption that they are to be preferred to nonrandomized studies is challenged. 3. Kunz and Oxman (1998) not only com- pared randomized and nonrandomized stud- ies but also adequately and inadequately con- cealed randomized trials and high-quality versus low-quality studies. Generally, high- quality randomized studies included ade- quately concealed allocation, while lower- quality randomized trails were inadequately concealed.In addition,the general terms high- quality trials and low-quality trials indicate a difference where “the specific effect of random- ization or allocation concealment could not be separated from the effect of other methodologi- cal manoeuvres such as double blinding” (Kunz and Oxman 1998, 1185). 4. Moreover, it may be that the finding of higher standardized effects sizes for random- ized studies in this review was due to school- level as opposed to individual-level assign- ment. When only those studies that include a delinquency outcome are examined, a larger effect is found when school rather than stu- dent is the unit of analysis (Denise Gott- fredson, personal communication, 2001). 5. As the following Scientific Methods Scale illustrates, the lowest acceptable type of evaluation for inclusion in the Maryland Re- port is a simple correlation between a crime prevention program and a measure of crime or crime risk factors. Thus studies that were de- scriptive or contained only process measures were excluded. 6. There were also (although rarely) stud- ies in the Maryland Report that reported two findings in opposite directions.For instance,in Sherman and colleagues’ (1997) section on specific deterrence (8.18-8.19), studies of ar- rest for domestic violence had positive results for employed offenders and backfire results for nonemployed offenders. In these isolated cases,the study was coded twice with the same scientific methods scores and each of the in- vestigator-reported result scores (of 1 and –1) separately.

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture