Study The of Promising After-School Programs: Examination of Longer Term Outcomes After Two Years of Program Experiences
August 8, 2006
Deborah Lowe Vandell Elizabeth R. Reisner Kim M. Pierce B. Bradford Brown Dale Lee Daniel Bolt Ellen M. Pechman
Vandell, D. L., Reisner, E. R., Pierce, K. M., Brown, B. B., Lee, D., Bolt, D., & Pechman, E. M. (2006). The Study of Promising After-School Programs: Examination of longer term outcomes after two years of program experiences. Report to the Charles Stewart Mott Foundation.
i Table of Contents Chapter 1: Introduction Study Design Theory of Change (Revised) Previous Analyses and Findings Chapter 2: Sample Description After-School Programs Student Participants Chapter 3: Measures of After-School Experiences Program Attendance Other After-School Experiences Chapter 4: Measures of Child and Youth Functioning Teacher Measures Student Measures Parent Measure Chapter 5: Findings: After-School Experience Clusters Creation of After-School Clusters Demographic Characteristics of the After-School Clusters Longer Term Links Between the After-School Clusters and Student Functioning Chapter 6: Conclusions References 1 2 4 5 9 9 12 16 16 18 21 21 23 25 27 27 31 31 43 48
ii List of Tables Table 2.1 Table 2.2 Table 2.3 Table 2.4 Table 3.1 Table 3.2 Table 3.3 Table 3.4 Table 4.1 Table 5.1 Table 5.2 Table 5.3 Table 5.4 Table 5.5 Table 5.6 Table 5.7 Table 5.8 Table 5.9 Observer Ratings of Key Process Features in the Programs Percentage Distribution of Observer Ratings in the Promising Programs Sample Characteristics at Recruitment and Year 2 Follow-Up Comparison of Retained and Lost Subjects on Demographic Variables Program Attendance Descriptive Statistics Program Intensity Across Two School Years (2003-2005) Two-Year Participation in Structured and Unstructured After-School Experiences Correlations of Program Attendance and Child-Reported Activity Participation Scores Across Time Descriptive Statistics for Outcome Measures Means (Standard Deviations) and Percentages on Clustering Variables, Elementary Clusters Means (Standard Deviations) and Percentages on Clustering Variables, Middle School Clusters Percentage Distribution of Elementary School Clusters on Background Variables Percentage Distribution of Middle School Clusters on Background Variables HLM Analyses of Relative Change from Baseline to Second-Year Follow-Up, Student Reports HLM Analyses of Relative Change from Baseline to Second-Year Follow-Up, Elementary Teacher Reports HLM Analyses of Relative Change from Baseline to Second-Year Follow-Up, Middle School Teacher Reports HLM Analyses of Relative Change from Baseline to Second-Year Follow-Up, Parent Reports Effect Sizes for the Cluster Comparisons 10 11 13 14 17 18 19 20 26 29 30 32 33 36 37 38 39 42
iii List of Figures Figure 1 Revised Theory of Change for the Study of Promising After-School Programs 5
1 Chapter 1 Introduction Because educational achievement remains a primary pathway out of poverty for youth from economically disadvantaged circumstances (Behrman, 1997; Dryfoos, 1998; Hechinger, 1992), increasing attention is being directed to both in-school and out-of-school activities that support educational achievement and set young people on a positive trajectory toward adulthood (Larson, 2000; Lerner, 1998). Recently, scholars have placed special emphasis on how young people spend their time in the after-school hours (Eccles & Gootman, 2002; Mahoney, Lord, & Carryl, 2005). They have been concerned with how youth involved in structured, adultsupervised activities (e.g., after-school programs, school extracurricular activities, and community-based activities such as lessons or sports teams) compare to peers who are supervised at home by parents or other adults, or who spend time in unsupervised settings after school. This research provides insights into the benefits and drawbacks of different after-school arrangements, although it often has failed to capture the full experience of children and adolescents, many of whom encounter a variety of after-school settings over the course of an academic year (Capizzano, Tout, & Adams, 2000; Lareau, 2000; Vandell & Shumow, 1999). One insight gained from other studies of after-school settings, based on a deficit model that underlies a considerable amount of the prevention and intervention literature, is that youth who are not involved in adult-supervised settings after school are at greater risk for problem behavior than youth who spend their after-school hours in the care of adults (e.g., Mahoney & Stattin, 2000; Vandell, Pierce, & Dadisman, 2005). From this perspective, the objective of afterschool programs and other structured activities is to keep children out of trouble, to avoid negative academic, emotional, and behavioral outcomes. A second insight is derived from a positive youth developmental perspective, whose focus is on enhancing young peoples potential
2 for healthy adaptation and prosocial behavior. Researchers have reported that programs and activities with particular features are most likely to foster healthy adaptation among children and youth. For example, Eccles and Gootman (2002) identified eight key features of successful youth programs: physical and psychological safety; appropriate structure; supportive social relationships; opportunities for belonging; positive social norms; support for efficacy and mattering; opportunities for skill building; and integration of family, school, and community efforts. The current study was undertaken, under the sponsorship of the Charles Stewart Mott Foundation, to assess the developmental benefits associated with youth participation in afterschool programs that met certain research-based quality criteria. As this report explains, we broadened our research focus as the study progressed, to include a more comprehensive analysis of the range of settings in which sampled children and youth spent the after-school hours. Nevertheless, subjects participation in high-quality after-school programs and their development of protective factors associated with that participation remained central foci of the research. Study Design The Study of Promising After-School Programs was conducted in three phases. The first phase, reported in Vandell et al. (2004), involved a comprehensive, nationwide search for school-based or school-affiliated programs that manifested the characteristics of high-quality after-school environments. Based on recommendations from experts in the field and a careful search of public records (published information and websites), we identified 116 programs that merited closer attention. Seventy-five program directors consented to a telephone interview from which we identified programs that served a predominantly economically disadvantaged population, operated at least 3 days a week throughout most of the academic year, served a large enough clientele to allow for statistical analyses, and showed promise that they would remain
3 viable for the 3 years of our study. From the programs that met these criteria and based on considerations of geographic diversity and accessibility of the program locations to our research staff, we selected 57 programs for school visits in Fall 2002 and Spring 2003 that allowed us to evaluate, firsthand, the features of each program and its appropriateness on other study criteria. Based on the school visits, we invited 20 programs serving elementary school youth and 18 programs serving middle school youth to participate in the next two phases of the study. Four elementary programs and five middle school programs subsequently were withdrawn from the study due to loss of funding, changes in administrative personnel, or school district concerns. During early Fall 2003, we screened and selected replacement programs that were in the pool of 57 programs we observed in the 2002-03 school year or were part of the same umbrella organization as other selected programs, resulting in a final program sample of 19 elementary and 16 middle school programs located in 14 cities in 8 states (Los Angeles, CA; Oakland, CA; San Diego, CA; San Ysidro, CA; Seaside, CA; Aurora, CO; Denver, CO; Bridgeport, CT; Baldwin, MI; Missoula, MT; New York, NY; Salem, OR; Pawtucket, RI; Central Falls, RI). Phase 2, reported in Vandell, Reisner, et al. (2005), began by gathering baseline data in Fall 2003 on a sample of 1,796 elementary school and 1,118 middle school youth. About 57% of the elementary participants and 48% of the middle school participants were enrolled in the selected after-school programs during the 200...
PP Examination in Year 3