One theory for how to improve schools begins not with teachers in the classroom but with the principals who hire and oversee them. To that end, the Wallace Foundation spent $85 million on a five-year project to improve school principals in six cities and large urban counties, from New York to Denver, beginning in 2011. (The Wallace Foundation is also among the funders of the The Hechinger Report.) Now, an analysis the foundation commissioned has found that these wide-ranging reforms in training, hiring, mentoring and reviewing the performance of principals tended to boost student achievement.
Reading achievement gains outpaced expectations in five of the six school districts that implemented the Wallace program. Math was better in three of the six districts. But in one district, which was not identified, the math performance of students deteriorated compared to similar schools in the state. Overall, the new principals in these districts were less likely to quit but retention varied a lot, from 100% to 63% after two years on the job.
The six districts were New York City; Denver; Charlotte-Mecklenburg, N.C.; Prince George’s County, Maryland, outside of Washington D.C.; Gwinnett County, Georgia, outside of Atlanta, and Hillsborough County, which includes Tampa, Florida. All six are among the 50 largest school districts in the country and students of color were the majority in each place.
The final report, “Principal Pipelines: A Feasible, Affordable, and Effective Way for Districts to Improve Schools,” was released on April 8, 2019* and conducted by the RAND Corporation, a research firm that the foundation paid to do the research. (RAND provided The Hechinger Report with an advance copy for review.)
“For the first time, we know that there are benefits for kids,” said Jody Spiro, director of education leadership at the Wallace Foundation. “That has been the missing piece.”
Related: The big jobs of small-town principals
Drawing clear lessons that are useful to school leaders is challenging but this study could be a potentially influential document in persuading school districts across the country to spend money on this “principal pipeline initiative” and enable them to pay for it with federal funds earmarked for poor students. That’s why it’s important to understand exactly what this research did and didn’t find.
First, the numbers. More than half of the schools in the six districts hired a new principal between 2012 and 2016, evidence of problematic high principal turnover around the country. RAND researchers tracked student test scores in more than 1,000 schools for three years and compared the annual changes with similar schools elsewhere in each state that had also hired new principals (their districts weren’t participating in the Wallace principal pipeline program).
For example, an elementary school in New York City, which was part of the intervention, might be compared with two different elementary schools in other cities with similar poverty rates, demographics and student achievement levels. On average, across all six Wallace districts, student reading scores were 6 percentile points higher than in schools where new principals didn’t participate in the intervention. That means if the comparison school scored at the 50th percentile — in the middle — then the students in the principal pipeline school would have been likely to score at 56th percentile. In math, it was a much smaller 3 percentile point improvement, on average.
On the face of it, that’s an impressive boost for student achievement, especially for something that indirectly trickles down to students. For context, RAND found less than half these reading gains but slightly larger math gains when it evaluated another principal training program, New Leaders Aspiring Principals Program, in February 2019.
One big problem with this Wallace study is that the student gains materialized throughout the districts, not just in schools that got new principals. In other words, all the students in these six districts tended to outpace comparison students elsewhere in their respective states. That’s a sign that the six districts in the study were doing other things that could have caused these gains and had nothing to do with the principal pipeline reforms.
However, the researchers could think of no other things that all six cities and counties were doing that the control group wasn’t doing. RAND senior economist Susan Gates, the lead researcher on the study, speculated that there might have been “spillover effects” from some elements of the principal pipeline initiative that changed how even veteran principals did their jobs. For example, all principals, not just new hires, were evaluated in new ways. Still, most of the pipeline elements were targeted to rookies and new hires.
The multi-faceted reforms are largely procedural and include such tactics as keeping track of aspiring principals, deciding what skills principals ought to have, matching them to schools where they would be most suited and coaching them on the job. One of the things that the Wallace initiative emphasized was that district leaders should stop hiring principals based on personal connections and instinct but use more objective criteria and rate applicants on how they handled simulated faculty and parent meetings.
“These pipeline activities are things that all districts do to some extent,” said Gates. “But they do not do these things in a strategic way and, in many cases, not effectively.”
Gates didn’t find that any one aspect of the program drove the positive results and so it is impossible to know exactly which things made a difference for student learning.
Related: Why do more than half of principals quit after five years?
The six districts implemented each aspect of the pipeline differently. Each district set its own standards and qualifications that teachers ought to have. For example, New York City decided principals should have seven years of prior instructional experience. The other districts didn’t demand nearly as much. Some districts partnered with universities to create training programs. Other districts built in-house training programs. Some districts had future principals participate in training residencies at schools, like doctors do in hospitals. Others didn’t. Some built mentorship programs to help new principals in their early years. Others relied on supervisors who were supposed to coach new principals in addition to being their bosses. Plainly, as the report states, “there is no ‘recipe’ for other districts to follow.”
Unfortunately, the study didn’t disclose which school districts produced the best or worst student results so there is no way to figure out which implementation choices worked best. Jason Grissom, an expert in principal effectiveness at Vanderbilt University, said that’s what schools and districts would want to know from a study like this. “These are big investments,” said Grissom, who did not have an advance copy of the report. “You need the evidence to convince districts it’s worth making.”
The changes can be expensive. RAND calculated it costs $42 per student a year, which adds up to $31,000 per principal annually. The large districts in this study spent an average of $5.6 million per year. That far exceeded the Wallace grant; districts shared the cost of the program during the experiment.
Gates said the biggest factor in how much bang a district got for its buck depended upon how much work it had previously done in improving school leadership. Education reformers were familiar with the ideas that the Wallace Foundation assembled and some districts had already implemented many aspects of the Wallace program before receiving the grant in 2011. They saw smaller student gains. Districts that needed to make more changes to implement the initiative saw the largest gains.
There were some hints in the study that residency training programs, in which aspiring principals start practicing their duties inside schools under supervision, were not as effective. But RAND said they did not have enough data to confirm this.
One of the greatest mysteries of the study is that the test score gains kicked in rather quickly, during the second year of a new principal’s tenure, and then stayed steady in subsequent years. I would think it would take years for a new principal to replace teachers and make curriculum changes that would eventually trickle down to students and grow over time. More research is needed to understand what things new principals are doing immediately that boost learning throughout the building. Perhaps it’s cultivating a happier atmosphere or drafting a better schedule. RAND didn’t analyze individual principals so the study doesn’t really shed light on the effectiveness of particular practices.
Another mystery is why the lowest performing schools, ranked among the bottom quarter in each state, produced the largest test score gains. On one hand, it shows that the most vulnerable students might have the most to gain from a system that steers better principals to them. However, fewer than a fifth of the schools in the study fell into this bottom category. More than half of the schools in the study were slightly higher performing but still in the lower half of academic achievement and their performance was the weakest in the study. These schools showed no gains in math and far more modest gains in reading.
Nonetheless, Wallace’s Spiro is calling for districts around the country, especially large urban ones, to “seriously consider” adopting “all the components” of this multi-faceted program to identify leadership talent and nurture it. Wallace is no longer footing the bill for pipelines but it commissioned another research firm, Abt Associates, to put its stamp of approval on the evidence. Spiro explained this would allow school districts to start tapping into a giant $16 billion bucket of Title I funds from the U.S. Department of Education, which are intended to supplement education spending for schools with high rates of student poverty.
Perhaps the strongest seal of approval is from the six districts themselves. A separate February 2019 study by Policy Studies Associates, another research firm that Wallace commissioned, found that the districts were continuing to spend their own money on major aspects of the principal pipeline program two years after the Wallace funding ran out in 2016.
This story about the Wallace principal pipeline was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.
*This story has been updated with the correct year that the RAND report was published. It is 2019.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.