This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Figure 1. From professional development to student achievement We predict that student achievement will improve by providing teachers with improved mathematics content and pedagogy. These treatments are applied to teachers in grades 4-8. Modeling student achievement in K-12 math and science classrooms: An experimental approach M.J. Bryant, R.A. Cardullo, K. Bocian, K.A. Hammond. University of California, Riverside, CA 92521 Figure 2. Comparison of possible experimental designs The treatment ( ) can be applied to either entire schools or to individual classrooms within a school. Schools or classrooms that do not receive the treatment serve as controls ( ). Ignoring issues of human subjects, the treatment would be applied to half of the schools or classrooms. The bottom two panels illustrate and example of increasing the proportion of treatment to control groups which is desirable in an educational experiment. Year 1 Year 2 Year 3 Year 4 Year 1 Year 2 Year 3 Year 4 Treatment Applied to Schools Treatment Applied to Classrooms 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Treatment:Control Constant (School) Treatment:Control Increases 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0.00 0.05 0.10 0.15 0.20 0.0 0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0 Alpha (< 0.05 considered significant) Probability of P < Alpha Treatment Applied to: Schools Classrooms Classroom design has the most statistical power School design has the most statistical power The experimental design with the highest statistical power is dependent on where the variation is greatest. Variation greatest among schools Variation greatest among classrooms 0.0 Figure 3. Monte Carlo based power analyses inform which design has the highest statistical power. The choice between schools or classrooms as the experimental unit is subject to many logistical considerations. The design impacts statistical power (the probability that a significant difference is found when a difference exists). Computer simulations reveal that statistical power is optimized as a function of where random variation lies. For example if all classrooms in a school are equal but variation exists among schools, randomly assigning treatment to classrooms has greater statistical power. Figure 4. Assignment of treatment and control groups as a stratified random design. In educational settings, demographic factors (i.e. socio-economic status or SES) are known to have a great impact on student performance, but random assignment of treatment is essential. Random treatment assignment may lead to unintended pooling on either end of an SES continuum. By selecting schools at random from...
View Full Document
- Winter '08