Course 1 Past h.pdf - The author(s shown below used Federal funds provided by the U.S Department of Justice and prepared the following final report

Course 1 Past h.pdf - The author(s shown below used Federal...

This preview shows page 1 out of 397 pages.

Unformatted text preview: The author(s) shown below used Federal funds provided by the U.S. Department of Justice and prepared the following final report: Document Title: Training Evaluation Model: Evaluating and Improving Criminal Justice Training Author(s): Kelly Bradley, Edward Connors Document No.: 244478 Date Received: December 2013 Award Number: 2003-DD-BX-K101 This report has not been published by the U.S. Department of Justice. To provide better customer service, NCJRS has made this Federallyfunded grant report available electronically. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Institute for Law and Justice Alexandria, Virginia Training Evaluation Model: Evaluating and Improving Criminal Justice Training Final Report September 2007 Submitted to National Institute of Justice Prepared by Kelly Bradley Edward Connors Institute for Law and Justice Acknowledgements The authors are deeply appreciative of the cooperation and support of all the individuals who made this project possible. We are indebted to Mark Gage, Marcia Williams, and Jim Foley of the National White Collar Crime Center; William Brunson of the National Judicial College and Nancy Yeend and John Paul Jones of the John Paul Jones Group; Liebe Geft, Sunny Lee, and Mark Katrikh of the Simon Wiesenthal Center Museum of Tolerance; and Steve Morrison, Michael Logsdon, Randall Milks, Regina Patrosi, and Michael Hummel of the National Corrections and Law Enforcement Training and Technology Center. These individuals showed the courage to open their programs to outside evaluators. We are also truly grateful for the cooperation of the training participants who agreed to be interviewed multiple times and complete surveys for this evaluation. We are also genuinely indebted to Detective Dave D’Amico of the Monmouth County, New Jersey Prosecutor’s Office, and Detective Alix Olson of the Madison, Wisconsin, Police Department, for allowing us to learn more about their efforts to combat crimes of hate and terror. A study of this depth and length does not thrive without strong support from the Office of Justice Programs' staff who provided guidance and oversight to the project, including Dr. Edwin Zedlewski, Dr. Katherine Browning and Maggie Heisler from NIJ, and Todd Brighton and Elaine Snyder from BJA. Additionally, we would like to thank Dr. James Wells, Dr. Kevin Minor, and Dr. Gary Cordner of Eastern Kentucky University for conducting and writing the evaluation of the NCLETTC training. They also contributed significantly to the development of the training evaluation model. Finally, we would like to thank the ILJ staff who worked so hard on this project, including Chera Baity, Tiffaney Hall, and Joan Peterschmidt for assistance in data collection; Deborah Spence for assistance with the literature review; and Dr. Thomas McEwen, Barbara Webster, and Laurie Samuel for their contributions in writing this report. This project was supported by grant number 2003-DD-BX-K101 awarded to the Institute for Law and Justice by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. Points of view in this document are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice Table of Contents Chapter 1: Project Background and Report Overview Project Summary..............................................................................................................................1 Need for a Criminal Justice Training Evaluation Model .................................................................2 Overview of the Report....................................................................................................................3 Chapter 2: Planning for Evaluations Types of Evaluations........................................................................................................................6 Evaluation Planning Steps ...............................................................................................................9 Identify Program Goals, Objectives, and Evaluation Questions..............................................10 Develop Conceptual Framework and Logic Model.................................................................10 Design Evaluation Methodology .............................................................................................11 Conduct the Evaluation............................................................................................................15 Analyze and Communicate Evaluation Results.......................................................................18 Evaluating Criminal Justice Training Programs ............................................................................20 Opportunities for Control and Comparison Groups.................................................................20 Challenges in Evaluating Criminal Justice Training ...............................................................20 Chapter 3: Factors That Contribute to Successful Practitioner Training Outcomes Training Objectives........................................................................................................................22 Adult Learning Concepts ...............................................................................................................23 Instructional Methods ....................................................................................................................25 Practical Training Matters..............................................................................................................28 Facilitation Skills ...........................................................................................................................28 Communication Skills..............................................................................................................29 Active Listening.......................................................................................................................31 Body Language ........................................................................................................................32 Sensitivity to Adult Students' Cultural Diversity.....................................................................32 Chapter 4: Criminal Justice Training Evaluation Model Kirkpatrick’s Training Evaluation Model......................................................................................34 Customizing and Expanding on Kirkpatrick’s Evaluation Model for Criminal Justice Training .37 Conduct Needs Assessment .....................................................................................................40 Design Training Plan ...............................................................................................................41 Develop and Test the Curriculum ............................................................................................43 Deliver the Curriculum ............................................................................................................47 Evaluate the Training and Trainers and Revise .......................................................................48 This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Chapter 5: Project Methodology Key Decision Processes for Site Selection ....................................................................................65 Overview of Methods ....................................................................................................................68 Chapter 6: Cross-site Comparisons and Findings Summary of the Training Evaluation Model's Applications .........................................................71 Needs Assessment....................................................................................................................71 Training Plan............................................................................................................................73 Develop and Test Curriculum..................................................................................................75 Pilot Test ..................................................................................................................................77 Trainer Selection......................................................................................................................78 Training Course Evaluation .....................................................................................................79 Conclusions....................................................................................................................................85 Recommendations and Lessons Learned: Tips for Evaluating and Improving Criminal Justice Training...............................................................................................................86 Costs of Training......................................................................................................................96 Chapter 7: National White Collar Crime Center's Foundations of Intelligence Analysis Training The National White Collar Crime Center ......................................................................................99 History and Background ..........................................................................................................99 Center Services ......................................................................................................................100 Review of the Intelligence Literature ..........................................................................................103 Intelligence-led Policing ........................................................................................................104 National Intelligence Plan......................................................................................................105 Core Standards .......................................................................................................................106 Foundations of Intelligence Analysis Training............................................................................107 Program Overview .................................................................................................................108 Evaluation Methodology..............................................................................................................109 Evaluation Questions .............................................................................................................110 Data Collection Methods and Framework .............................................................................111 Study Strengths and Weaknesses...........................................................................................114 Evaluation Findings .....................................................................................................................114 Participant Reaction ...............................................................................................................115 Knowledge and Skills Gained................................................................................................119 Behavior Changes ..................................................................................................................121 Discussion ....................................................................................................................................127 Strengths of the Course..........................................................................................................127 Recommendations for Change...............................................................................................128 APPENDIX 7-A: FIAT Development SME Participants...........................................................131 APPENDIX 7-B: NW3C FIAT Course Training Evaluation Materials.....................................131 APPENDIX 7-C: Pre-Post FIAT Participant Self-assessment of Course Comfort Level ..........145 APPENDIX 7-D: Matched Pairs T-test Results of Pre/Post FIAT Course Comfort Level........146 This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Chapter 8: Simon Wiesenthal Center's National Institutes Against Hate Crimes and Terrorism Training Simon Wiesenthal Center ............................................................................................................148 Review of Hate Crimes and Terrorism Literature .......................................................................148 Hate Crime Defined ...............................................................................................................148 Statistics .................................................................................................................................151 Terrorism................................................................................................................................152 Training..................................................................................................................................153 Teaching Tools for Tolerance......................................................................................................154 Tools for Tolerance National Institutes Against Hate Crimes and Terrorism .......................154 Evaluation Methodology..............................................................................................................157 Evaluation Questions .............................................................................................................157 Data Collection Methods and Tools ......................................................................................158 Strengths and Weaknesses .....................................................................................................167 Evaluation Findings .....................................................................................................................168 Participant Reaction ...............................................................................................................169 Learning/Knowledge Gained .................................................................................................178 Attitude and Behavior Changes .............................................................................................182 Organizational Impact............................................................................................................194 Discussion ....................................................................................................................................195 APPENDIX 8-A: SWC Training Evaluation Materials..............................................................201 APPENDIX 8-B: Case Study of Monmouth County, New Jersey .............................................216 APPENDIX 8-C: Case Study of Madison, Wisconsin ...............................................................229 Chapter 9: NCLETTC Advanced Leadership Techniques Training for First Responders, Corrections, and Security Officers Introduction..................................................................................................................................240 Context for Evaluation...........................................................................................................240 Overview of Literature Relevant to the Training...................................................................242 Research Questions................................................................................................................245 Method .........................................................................................................................................245 Trainings and Participants......................................................................................................246 Design, Instrumentation, and Data Collection.......................................................................252 Results..........................................................................................................................................258 Level 1 Reaction Results .......................................................................................................258 Level 2 Knowledge Results ...................................................................................................266 Level 3 Behavior Change Results..........................................................................................268 Level 4 Organizational Impact Results..................................................................................279 Discussion ....................................................................................................................................286 Main Findings ........................................................................................................................286 Strengths, Limitations, and Recommendations .....................................................................291 This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. APPENDIX 9-A: NCLETTC Study Timeline, Milestone, and Workplan Chart .......................300 APPENDIX 9-B: NCLETTC Training Evaluation Materials ....................................................305 Chapter 10: National Judicial College's Civil Mediation Training National Judicial College.............................................................................................................330 Literature Review.........................................................................................................................331 Civil Mediation Training .............................................................................................................333 Program Overview .................................................................................................................333 Evaluation Methodology..............................................................................................................335 Evaluation Questions .............................................................................................................335 Data Collection Methods and Framework .............................................................................336 Evaluation Findings .....................................................................................................................340 Participant Reaction ...............................................................................................................340 Knowledge and Skills Gained................................................................................................342 Behavior Change....................................................................................................................345 Discussion ....................................................................................................................................345 APPENDIX 10-A: NJC Training...
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture