Rothstein wasn't convinced. Looking at a similar set of data on test scores and incomes from North Carolina, he observed the same correlation between students' teachers, their test scores and how well they turn out later in life. Yet he found that scores seemed to be changing throughout the school in some cases, suggesting that certain teachers were looking for an opportunity to teach better
students, maybe by moving to schools in gentrifying neighborhoods. When a teacher whose students do well on tests moves to a school where test scores were improving the previous year, and average scores continue improving after that teacher arrives, it is hard to know how much of that continued improvement is due to the new teacher and how much to other factors. Chetty's group responded to Rothstein's analysis, saying that they are pleased that the the test scores from North Carolina show the same connections they found, even though they disagree with Rothstein on how much that relationship means. "Everybody has the same findings in all the data sets, which is actually quite rare in economics. That's a good starting point," Chetty said, adding that another group of researchers is working on a similar paper that confirms these results. With regard to Rothstein's criticism, they argue that what appear to be larger trends in test scores across a school are in fact due to statistical quirks and noise in the data. For example, test scores for a school's students in particular subjects can vary widely from year to year, and an usually good year for fourth-grade math could distort the results for fifth-grade math the next year. This dispute is just one example of the mathematical acrobatics required to isolate the effect of one teacher on
their students' test scores, when so many other factors inside and outside the school's walls affect how students perform. School districts around the country have already implemented what are known as "value-added measures" to comply with what is effectively the administration's requirement that teachers be evaluated at least partly on the basis of student achievement. Still, economists agree that some important issues remain unresolved. Chetty said that whether value-added scores are the best way to assess teachers is still an open question. His group's paper doesn't examine alternative methods, such as observations by other faculty members. Chetty added that basing teacher's salaries or tenure on value-added measures could have unforeseen consequences if teachers don't try to develop important qualities in their students beyond academics. "Once you start using value-added measures in practice, their signal quality might get eroded," he said. "People might start teaching to the test." He's optimistic that teachers wouldn't make this mistake, but doesn't yet have the data to support that hunch.
You've reached the end of your free preview.
Want to read all 826 pages?
- Fall '09
- American Schools