You've reached the end of your free preview.
Want to read all 16 pages?
Unformatted text preview: Journal of Economic Perspectives—Volume 24, Number 3—Summer 2010—Pages 151–166 Teachers’ Views on No Child Left
Behind: Support for the Principles,
Concerns about the Practices
Richard J. Murnane and John P. Papay “ W hat can we do to make Adequate Yearly Progress?” “What will happen
if we fail to make AYP again?” These questions are dominant topics of
conversation in teacher lunchrooms across the nation. They illustrate
the influence that the No Child Left Behind legislation (NCLB) has had on the
work of millions of public school teachers in the United States. The legislation, which
took effect in 2002, requires that each state administer annual tests in mathematics
and English Language Arts in grades 3 through 8 and once again in high school.
(Science testing requirements took effect in 2008–09.) The legislation mandates that
all students be “proficient,” as defined by the state, by 2014 and that every school must
make Adequate Yearly Progress towards meeting this goal, not only overall, but for
a number of demographic subgroups within each school. Schools that do not make
Adequate Yearly Progress for several years face increasingly stringent sanctions. In
addition, schools must comply with several other requirements, including the provision that all teachers must be “highly qualified,” as defined by the state.
In this article, we describe teachers’ views of the behavioral responses the No
Child Left Behind legislation has elicited and the extent to which research reveals
evidence of these responses and their effects on the distribution of student achievement. We begin with two broad warnings about any attempts to characterize the
views of teachers as a group: No Child Left Behind has played out very differently in
different states, and studies of teacher opinions typically suffer from selection bias.
We then focus on teachers’ reactions to three aspects of NCLB that are particularly
Richard J. Murnane is the Juliana W. and William Foss Thompson Professor of Education and Society and John P. Papay is an advanced doctoral student, both at the Harvard
University Graduate School of Education, Cambridge, Massachusetts. Their e-mail addresses
are 〈[email protected]〉 and 〈 [email protected]〉.
■ doi=10.1257/jep.24.3.151 152 Journal of Economic Perspectives relevant to them: 1) the testing requirements and the rules determining Adequate
Yearly Progress; 2) the sanctions imposed on schools that fail to meet AYP; and
3) the requirement that all teachers of core academic subjects be “highly qualified”
in their areas of teaching assignment.
Overall, we find that teachers overwhelmingly support the principles underlying the No Child Left Behind legislation, including that schools should be held
accountable for educating all children well. However, teachers are concerned
that the incentives created by some provisions of the law have elicited unintended
responses that reduce the quality of education provided to at least some children.
These behavioral responses vary widely across settings and depend on the quality of
a state’s standards and assessments, the amount of support school districts provide
for instructional improvement, and the quality of leadership in schools. Difficulties in Determining Teachers’ Views of No Child Left Behind
Any attempt to characterize the beliefs of teachers as a group about the No Child
Left Behind legislation faces several difficulties. First, there is substantial heterogeneity
across states in their existing test-based accountability policies and their implementation of the law. Within states, there is also a great deal of variation in school policies
and resources, which may lead teachers to experience the legislation in different
ways. Second, many efforts to evaluate teachers’ opinions suffer from various kinds of
sample selection bias—the schools and teachers who participate in these studies are
not necessarily representative of schools and teachers across the country.
Variation across and within States
The No Child Left Behind legislation overlaid federal law and regulations
on state standards-based accountability systems. Some states, including Texas,
Massachusetts, Kentucky, and Florida, had developed comprehensive test-based
accountability systems well before the passage of NCLB. In fact, the Texas school
accountability program served as one model for NCLB legislation. Other states
began the process of developing these systems as a result of the federal mandate.
None of the teachers in our focus groups could differentiate between NCLB and
their state accountability system, and almost all attributed their concerns about
test-based accountability to NCLB. However, in many states, concerns about
dysfunctional responses to test-based accountability antedate NCLB and would still
be present if NCLB were repealed.
The federal legislation provides considerable discretion to states in implementing
their accountability provisions. In particular, states can develop their own academic
content standards, choose the tests they will administer, and specify the minimum
scores students must obtain to be declared “proficient.” Some states, such as Massachusetts, have invested in the development of quite rigorous examinations that are
closely aligned with detailed and demanding academic standards. Other states have
chosen off-the-shelf tests that are only loosely aligned with academic standards, or Richard J. Murnane and John P. Papay 153 have specified relatively low requirements for students to be deemed “proficient.”
The variation in proficiency requirements is illustrated in Figure 1, which shows
the mathematics proficiency standards across states, as measured on the scale used
to assess skill on the nationally representative National Assessment of Educational
Progress (NAEP) mathematics examination (methods for these comparisons are
described in Bandeira de Mello, Blankenship, and McLaughlin, 2009).
As Figure 1 shows, the proficiency standards in some states are higher than the
score of 299 that the National Assessment of Educational Progress designates as
proficient for grade 8, while the proficiency standards in other states are considerably
below the NAEP standard. For example, the Massachusetts standard is at 302 points
on the NAEP scale, but the Tennessee standard is 234 points. This pattern reflects
both the rigor of standards and the degree of score inflation over time on the state
test (which will be discussed below). This variation across states in content standards,
assessments, and proficiency standards is problematic. Many teachers are critical
of pressure to improve students’ scores on tests they feel are not well aligned with
academic standards respected by the teachers and that do not reflect mastery of skills
students need to thrive in twenty-first century America.
Within states, the challenges that school districts and particular schools face
in meeting the Adequate Yearly Progress provisions of the No Child Left Behind
legislation also vary widely, even among schools facing the same academic standards.
Schools serving significant numbers of economically disadvantaged children, children with limited English proficiency, and those with special needs find it especially
difficult to make AYP. The human and financial resources available to meet the
accountability challenges posed by NCLB vary widely across schools and districts.
For teachers in some schools, AYP is a challenge they have, to date, been able to
meet with consistently good teaching. For teachers in other schools, even Herculean
efforts to improve teaching will not produce test scores that satisfy AYP requirements.
For teachers in the first group of schools, NCLB may be an annoyance, but no more.
For teachers in the second group of schools, especially those teaching grades 3–8 (in
which student testing is mandatory), NCLB may threaten their jobs.
Finally, teachers’ views of No Child Left Behind are sensitive to the school context
and, in particular, to the quality of school-site leadership. In fact, some teachers in
schools serving large numbers of disadvantaged students expressed concerns about
pressure from administrators to raise student test scores without clear guidance
about how to do so in a way that benefits students. Other teachers, though, reported
that external pressures were not a serious burden because teachers in their school
already held each other accountable for student performance (Szczesiul, 2009).
Thus, differences in the challenges that teachers face and the resources available to
them contribute to the variation in teachers’ views of NCLB.
Self-Selection of Respondents
We collected evidence of teachers’ views about the No Child Left Behind legislation in three ways, each of which is informative, but each of which also has distinct
limitations. First, we gathered information from past surveys of teachers drawn 154 Journal of Economic Perspectives Figure 1
National Assessment of Educational Progress (NAEP) Scale Equivalent Scores
for Grades 4 and 8 Mathematics Standards for Proficient Performance,
by State: 2007
Grade 4 Grade 8 NAEP Basic (214)
NAEP Proficient (249)
District of Columbia –
180 NAEP Basic (262)
NAEP Proficient (299) South Carolina
District of Columbia
NAEP Scale Equivalents 312
– 180 230
NAEP Scale Equivalents Source: Reproduced from Bandeira de Mello, Blankenship, and McLaughlin (2009). Data from U.S.
Department of Education, Institute of Education Sciences, National Center for Education Statistics,
National Assessment of Educational Progress (NAEP), 2007 Mathematics Assessments, U.S. Department
of Education, Office of Planning, Evaluation, and Policy Development, EDFocus SY 2006–07, Washington,
DC, 2008. The National Longitudinal School-Level State Assessment Score Database (NLSLSASD) 2008.
– State assessment data not available.
* Relative error greater than .5. Teachers’ Views on No Child Left Behind 155 from well-specified sampling frames. Most of these surveys focused on teachers’
views of test-based accountability, rather than specifically on NCLB. An example is
the National Survey on State Testing Programs (Pedulla, Abrams, Madaus, Russell,
Ramos, and Miao, 2003). One strength of this approach is that, in principle, it is
possible to generalize the results to a broader population. However, this strength is
undercut by the relatively low response rate in most national studies: for example,
only 35 percent of teachers surveyed responded to the National Survey on State
Testing Programs. Other surveys have higher response rates, but they typically focus
on individual districts or states as part of a larger, in-depth analysis. As a result, the
districts that agree to participate in such studies may not be representative.
Studies in which researchers interviewed in-depth samples of teachers provide
a second source of data. For example, RAND examined the experiences of teachers
with No Child Left Behind in California, Georgia, and Pennsylvania (Hamilton et
al., 2007). An advantage of data drawn from studies of this type is that they typically
provide a great deal of information about the context in which the reporting teachers
work. However, these studies also tend to suffer from self-selection, in that teachers
with strong views about NCLB are more likely to take the time for in-depth interviews.
Finally, we conducted focus groups in which we asked teachers to share their
views of the No Child Left Behind legislation. We conducted four focus groups
with approximately 25 participants. One strength of this type of data collection is
that participants often disagreed, and we could probe the sources of the divergent
views. Again, however, self-selection is a problem in that teachers with strong views
about NCLB were the most likely to volunteer to participate in the focus groups.
We included teachers at schools across the country, most of whom had experience
in urban education. However, our respondents were either current teachers in
Massachusetts school districts, teachers attending summer professional development at the Harvard Graduate School of Education, or current graduate students
at Harvard who had taught in U.S. public schools in the previous year. Thus, our
results do not generalize to the full teaching population. Testing and Accountability
Many teachers believe that No Child Left Behind has increased pressure
on states to develop rigorous content standards and curriculum along with tests
that align with those standards. They applaud this response. For example, expert
teachers interviewed by Barnett Berry (2007) “believe that NCLB and its accountability measures have set clearer expectations for what students need to learn and
what teachers need to teach.” One large national survey of teachers conducted
by MetLife found that 53 percent of teachers in 2008 reported that their school
had “excellent” academic standards, up from just 26 percent in 1984 (Markow and
Cooper, 2008). In a study of educators in California, Georgia, and Pennsylvania,
RAND researchers found that “teachers reported an increased focus on student
achievement in their schools as a result of NCLB, as well as increased curriculum 156 Journal of Economic Perspectives coordination and increased rigor of the school’s curriculum” (Hamilton et al.,
2007). Thus, teachers credit NCLB with sparking the development of academic
content standards and focusing attention on academic achievement for all students.
Another benefit of the No Child Left Behind legislation that many teachers
report is greater attention to identifying the specific skill deficiencies of lowachieving children in mathematics and English Language Arts and to developing
strategies to enhance these skills. One of the most common strategies has been
to increase the amount of time in the school day devoted to English Language
Arts and mathematics instruction. For example, according to teacher reports in
the Schools and Staffing Survey (a nationally representative survey carried out
periodically by the U.S. Department of Education), the amount of time devoted to
teaching mathematics in U.S. elementary schools increased by 40 percent between
2000 and 2003 (Hannaway and Hamilton, 2008).
There is some evidence that students’ performances on national tests have
improved as a result of this focus, although the extent to which the score increases
can be attributed to No Child Left Behind is less clear. Dee and Jacob (2009) find
that NCLB led to a modest improvement in scores of eighth graders in all parts of
the achievement distribution on the National Assessment of Educational Progress
mathematics examination, but no improvement in scores in any part of the achievement distribution on the National Assessment of Educational Progress eighth
grade examination of reading skills.
Score Infl ation
While teachers applaud the attention that No Child Left Behind has brought
to the importance of developing all students’ skills in English Language Arts and
mathematics, they often express concern about pressures to focus instructional time
on preparation for the state tests, or “test prep.” Many teachers express the belief
that while test prep may increase students’ scores on these tests, it does not provide
students with skills and knowledge that they can apply to other tasks. Teachers report
that score inflation—an improvement in students’ test scores with no improvement
in their underlying proficiency—is rampant, a consequence of focusing instruction
on preparation for a particular test. According to a national survey, 40 percent of all
teachers “reported that they had found ways to raise state test scores without really
improving learning” (Abrams, 2004). Many teachers resent taking time away from
activities they believe are valuable to students to focus attention so directly on a
single test. This belief contributes to teachers’ perceptions that increases in student
scores on the state tests do not reflect actual increases in students’ human capital. For
example, in a survey of Texas reading teachers, half of the respondents reported that
the state’s rapid improvement on the Texas reading test did not “reflect increased
learning and higher quality teaching” (Hoffman, Assaf, and Paris, 2001).
A growing body of evidence supports these teacher perceptions. For example,
Jacob (2007) examined four states and found much faster growth trends on the state
tests than on the National Assessment of Educational Progress (NAEP), even in cases
where the state tests were supposed to reflect the same level of difficulty from year to Richard J. Murnane and John P. Papay 157 year. Other researchers have found similar results in individual states (for example,
Klein, Hamilton, McCaffrey, and Stecher, 2000; Koretz and Barron, 1998). Some
of this pattern stems from a focus on test preparation. Some of it also stems from
strategic but dysfunctional responses to the pressure to increase the percentage of
test-takers whose scores meet the proficiency standard. For example, Figlio (2006)
found that schools in Florida disciplined students differently based on their likely
performance on an upcoming state test; students with lower test scores were much
more likely to be suspended from school than their peers with higher test scores
around the time of the year when the state administered the accountability test. The
evidence that increases in scores on high-stakes tests do not reflect increases in skills
and knowledge that students can demonstrate on other assessments is, in our view,
the most compelling evidence supporting teachers’ criticisms of test-based accountability. Koretz (2008) offers a clear statement of the test score inflation argument.
Focusing on “Bubble Kids”
Another concern stems from the Adequate Yearly Progress formula, which is
based on the percentage of students in each subgroup who meet the proficiency
standard, not on the amount of academic progress students make during a school
year. Teachers report pressure to focus attention on “bubble kids,” those close to
the proficiency threshold, at the expense of the most academically able students
who will meet the proficiency standard in any case and the especially low-achieving
students who are unlikely to meet the proficiency standard even if given a great
amount of attention (Booher-Jennings, 2005).
Several studies provide evidence showing that the incentives implicit in the
Adequate Yearly Progress formula did change the shape of the distribution of student
achievement as measured by scores on particular tests. For examp...
View Full Document
- Spring '14
- No child left behind Act, Child Left, NCLB