This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Lavy: “Performance Pay and Teachers’ Productivity” Policy guestion: should we use teacher bonuses to increase student performance?
mg: creates greater incentives for teachers to work hard
bonuses for individuals will reduce “free riding”
(a problem with group bonuses)
con: makes pay riskier, which employees don’t like, so pay rate will have to go up
employees will emphasize only the behaviors that generate bonuses
(e.g., "teaching to the test”)
much work is complex, requires a lot of teamwork/cooperation,
group can monitor behavior of individuals,(so use group bonuses?) Use tournaments (based on rank order of finish, not absolute amount)?
announce prizes in advance (easy to stay within budget)
each employee has incentive to work incredibly hard
(if employees are basically identical, huge incentive to exert greater effort)
problem: sabotage? Empirical guestion: will bonuses to teacherincrease student performance? Background: high schools assigned to an individualperformance bonus experiment
(bonus to a teacher based on ranking of his/her students’ test scores) assignment of school based on the school’s “matriculation rate”
(“matriculation rate” = percent passing national exams) school assigned to the experiment if its matriculation rate was below 45% BUT: “official” matriculation rate S was based on incomplete data
“true” matriculation rate 3* only became known
after schools were selected for the experiment
S = S* + a, where a is measurement error key insight: conditional on S*, a is random!
thus, conditional on 8*, selection of schools and students
into the experiment is random
thus, control for S* and pick schools with S < 45 _r_19_t in the experiment
these schools form a “control group” that can be compared with
schools with the same 8* that were in the experiment (“experimentals”) effectively, we have random selection of experimentals and controls
(conditional on S*, the “true” matriculation score) basic model (data for two years: preexperiment in 2000, experiment in 2001) Yijt=a+xijtl3+zjtV+5Tjt+¢j+ rI Dt+8ijt where Yijt = score of student i in school j at time t Xijt = characteristics of student I in school j at time t th = characteristics of school j at time t Tjt = interaction: dummy variable for “treated school”
dummy variable for “year is 2001” (equals 1 if a treated school in 2001, equals 0 otherwise)
4)th = set of dummy variables for school (j = 1, 2., ...)
Dt = dummy variable for “year is 2001” (= year of experiment) )The estimated coefficient on T, namely 5, is the
“differenceindifference” estimate
of the effect of the experiment on student scores. First consider the difference for the experimental schools
between 2000 (no experiment) and 2001 (with experiment):
improvement could have been due to unmeasured things
that changed between 2000 and 2001,
rather than due to the experiment itself. (Beware the “post hoc” fallacy — “post hoc, ergo propter hoc.”) Likewise, consider the difference for the control schools
between 2000 and 2001:
again, any change could have been due to unmeasured things that changed between 2000 and 2001,
rather than due to the experiment itself. But why is 6 an estimate of the effect of the experiment? To simplify, assume that there is only one experimental school
and one control school. (Argument doesn’t depend on this —
allowing for multiple schools just makes the math more messy.)
Likewise, ignore differences in X, Z, etc. (Again, argument doesn’t
depend on this.) So the equation is now Yijt = a + 6 Tjt + (b, + n D, + 8m where Tjt = 1 for the experimental school in 2001, and 0 otherwise;
<>j = 1 for the experimental school, 0 for the control
Dt = 1 for the year 2001, and 0 for 2000 For the experimental school, the difference between 2001 and 2000 is Y'11=a+5"1+¢1+nx1+3i11 Yi10=a+6x0+¢1+nx0+8i10 Y'11:: _Yi10 = 5 + n "' 3m ' 3i1o Likewise, for the control school, the 20012000 difference is
Yi11=a"'5"0""l>o"'r"1"":io1 Yi1o=a+5x0+¢o+rlx0+8ioo Yi01x " i00 = '1 + 3m ' £i00 So, finally, we have: Yi11  m, = 6 + n + am  cm, experimental difference Yi01 "" Yioo = n + Sim " Sioo COI’ItI’OI difference Provided both types of schools have the same trends over time,
the n‘s will cancel. Provided the differences in the 52’s are the same,
the terms involving the s’s will also cancel. In this case, the difference between the two differences
gives the effect of the experiment — and this is equal to 6! ”H1 — Yi1o]  [Yi01  Yioo] = 5 ( = experimental — control difference)
Thus, the difference between the two differences provides an estimate of the effect of the experiment. (Note that this estimate avoids the
“post hoc” fallacy.) Lavy’s main results for the treatment effectI 6 (Table 4, p. 1992) math (all guartiles) limited full
outcome control control
testing rate 0.046 0.041
(0.027) (0.021)
control group mean 0.802
pass rate 0.110 0.087
(0.036) (0.028)
control group mean 0.637
average score 5.469 5.307
(2.292) (1.950)
control group mean 55.046 Standard errors are in parentheses,
underneath the coefficient estimates En lish all uartiles limited full
control control
0.040 0.033
(0.017) (0.013)
0.865
0.047 0.039
(0.022) (0.020)
0.795
3.240 2.527
(1.666) (1.452)
59.496 Table 5: Ignoring dummy variables for “school” produces estimates that are
less statistically significant Further looks at Table 4:
* The program helped students atlbelow median the most
(if highlevel student, might pass anyway?)
* Check to see how the program worked:
improved scores of students who took the exam
got more students to take the exam Tables 67: regressiondiscontinuity estimates (probability of being treated
drops sharply if matriculation rate goes above 45%) — results similar to the basic results Table 8: effect of experiment on teaching methods, teaching effort
effects evident mostly for English teaching,
mostly via extra teaching time Table 9: did the program cause teachers to inflate their grades?
(part of the overall score was based on local test results)
answer — apparently not (sanctions for discrepancies between
local and national test scores!) ...
View
Full Document
 Fall '11
 killingsworth

Click to edit the document details