New evidence indicates that paying teachers bonuses raises student performance by a small amount

One of the big policy debates in education is whether paying teachers bonuses increases student achievement along with teachers’ bank accounts. A new federal report puts a fresh twist on this old debate: does it matter if something works if the benefit is very tiny? And its findings also shed light on why even the very best research doesn’t tell us much about how merit pay works and whether it’s worthwhile.

First, some background. The federal government has given out more than $1.8 billion in taxpayer money to school districts to reward their best teachers. The payouts began in 2006 under the George W. Bush Administration when policymakers believed that the same kinds of pay incentives that motivate salesmen in the private sector might also improve the teaching force. Initial studies of these bonus schemes for teachers weren’t promising. Researchers who studied the federally funded efforts found no benefits to students in New York City or Nashville. Many, including myself, concluded that merit pay doesn’t work because teachers, unlike salesmen, aren’t terribly motivated by money.

However, other research complicates the picture. More than 30 studies have analyzed merit pay for teachers in the United States. Most, like the New York City and Nashville studies, showed that student achievement was the same regardless of whether teachers had the opportunity to earn a bonus. But in other cities, such as Little Rock, Ark., students sometimes posted higher test scores when their schools gave teachers bonuses. Last year, a group of Vanderbilt University researchers totaled up all the conflicting results. In their meta-analysis, they found that performance bonuses for teachers were a tiny net positive for students, producing achievement gains equal to three extra weeks of school.

That’s the same number that the federal government report arrived at. In what is arguably the largest and best-designed study of teacher merit pay, researchers studied 10 school districts. The cities were not identified, but the report characterizes them as large, mostly urban and predominantly Southern, where more than 60 percent of the students were not white and there were high concentrations of poverty. The 10 districts agreed to adopt other changes too, such as measuring teacher effectiveness and providing more on-the-job training for teachers. Researchers randomly assigned half the elementary and middle schools in these districts to a teacher bonus program. Then, over the next four years — from the fall of 2011 to the spring of 2015 — the researchers compared the student test scores in the 65 schools that gave out performance bonuses with the 66 schools that didn’t. They found that students in the bonus schools posted test scores that were slightly higher by 1 to 2 percentile points.* That test score bump is equivalent to three to four weeks of school.

“I leave it to the educators to decide if three to four weeks is important,” said Allison Wellington, who was the project director of the federal study, conducted by Mathematica Policy Research, a research firm in Princeton, N.J..

When you read the 350-page report, released in December, several odd things emerge about the way that teacher bonus programs actually played out in the real world.

The first one is that many teachers said they were unaware of the bonus program. Even in the fourth year, 40 percent of teachers at the bonus schools said they didn’t know about the opportunity to earn extra pay for good performance. For merit pay to work, teachers ought to know about it and what they need to do to earn it. Teachers should take it as a signal that if they don’t receive a bonus, there might be some things they could do to improve their teaching. However, that signal was not heard by many.  The ignorance wasn’t just at one school or school district, but across 65 bonus schools in 10 districts.

The researchers also found that 70 percent of the teaching faculty earned bonuses, averaging $2,000 each. That means even some of the teachers who claimed they were unaware of the bonuses had, in fact, received them. But a bigger problem is that bonuses were given out to so many teachers, even to those who performed below average. To be sure, some teachers earned larger bonuses than others, exceeding $9,000 at some schools. Still, if merit bonuses are supposed to work by signaling who the good teachers are, fewer people ought to get them.

“Implementation was not everything we thought it might be,” said Wellington.

It was unclear why school systems tended to spread out bonuses thinly among so many people, whether it was political pressure or poor design of bonus systems. Each district had the freedom to structure the bonuses as they saw fit as long as some teachers got bigger bonuses than others. In some districts, teachers could earn separate merit bonuses for hitting different goals. Some teachers got them, as you might expect, when their students produced higher than expected test scores. But there were also bonuses for good ratings on observations of classroom teaching. Some got bonuses if their school’s overall test scores increased regardless of what happened in their individual classrooms.

A third strange finding is that the small improvement in student test scores occurred rather quickly at the bonus schools, within the first year or two. Afterward, students didn’t rack up additional gains in years three and four. The way merit pay is supposed to work is that schools offering bonuses should have more luck recruiting and retaining the best teachers. And the effective teaching practices of the good teachers would be emulated by others. You might expect zero or a small bump in student test scores initially, but then over the years, student performance at the bonus schools ought to gather steam. But the way it played out in the real world, it was as if the teachers in the bonus schools were magically more effective as soon as they were offered a chance to earn bonuses. That’s it. The teaching quality of the faculty didn’t keep improving.

So what caused the initial bump in student performance? Mathematica’s Wellington admits it’s a mystery. All the teachers were observed and rated on their actual performance in the classroom. Over time, the teachers in the bonus schools earned higher performance ratings. However, there was no connection between teacher ratings and their students’ test scores. Some teachers with high ratings had terrible test scores in their classrooms. And some teachers with poor ratings had brilliant test scores in their classrooms.

That’s frustrating. You’d hope to get some insight into what works so that you could train and coach teachers to do better.

At the end of the four years, fewer than half the districts in the study planned to continue the bonus program after the federal grant ran out. For them, three weeks wasn’t a big enough bang for the buck. Meanwhile, the federal government continues to dole out new grants for teacher bonuses.  More than $88 million was spent in fiscal 2017.

For those who are unimpressed with three weeks, it can seem like funding teacher bonuses is a waste of money. Still, the federal study is a vindication to others that teacher bonuses can work, even if we’re a long way off from figuring out how to structure and implement them properly.

* Some schools had higher achieving students than others before the experiment began. The federal report’s authors factored in prior academic performance before calculating test score increases.