Last week, U.S. News & World Report released its first-ever rankings of public elementary and middle schools in the United States. Relying solely on aggregated state standardized test scores and demographic characteristics of students attending a given school, the periodical ranked elementary schools and middle schools within states and school districts. Schools in the bottom quarter of all schools in a given jurisdiction were not ranked, but simply identified as being in the bottom quarter.
Although USN&WR has proclaimed itself “the global authority in education rankings,” a healthy degree of skepticism is appropriate. Even “global authorities” can screw up! We already have good evidence that school and college rankings can distort normal educational processes, reinforcing social hierarchies that govern who enrolls in a school, how those students are treated and what happens to them thereafter.
At the K-12 level, we have seen how school ratings can boost or depress property values and shift who seeks to enroll in a given school.
The GreatSchools.org ratings attached to real estate websites such as Zillow, realtor.com and Redfin often are lower for schools serving low-income students and higher concentrations of Black and Hispanic students. If parents vote with their feet, the ratings may independently contribute to even greater levels of segregation across schools.
In truth, we know that, on average, schools serving lower-income children and children from historically marginalized racial and ethnic groups will inevitably receive lower ratings from schemes—like the new USN&WR rankings—that rely on the percentage of students who are deemed proficient in English and math.
The USN&WR ratings seek to blunt such criticisms by claiming that they take into account not only raw proficiency rates, but also rates that are adjusted for the mix of students in a given school. By blending these two ways of thinking about a school’s performance—the raw performance level of a school’s students, and an adjustment that seeks to identify schools that are doing better than other schools serving similar students—and then ranking schools on this blend, USN&WR intends to factor in both a school’s overall academic performance and its relative performance compared to similar schools.
At the K-12 level, we have seen how school ratings can boost or depress property values and shift who seeks to enroll in a given school.
The trouble is, it doesn’t work. At least that is the conclusion I reached after looking at data on more than 400 traditional public middle schools in New York City, where the rankings are dominated by students’ absolute proficiency levels. Moreover, those proficiency levels are closely tied to students’ family economic status and their entering levels of proficiency in English and math.
What the USN&WR rankings do, essentially, is identify schools that have been successful in enrolling higher-achieving, more affluent students. The focus is rather problematically on inputs rather than outputs. Shouldn’t we be far more interested in outputs, such as how schools contribute (or not) to student learning and development?
Related: Do U.S. colleges reinforce or reduce inequality?
I chose New York City middle schools to illustrate this point because New York publishes a lot of data on its public schools. All districts and states do; the 2015 Every Student Succeeds Act requires states and school districts to publish report cards that include information on student performance, accountability ratings, per-pupil expenditures, and other information that parents and school officials deem relevant.
The USN&WR percentile rating for New York City public middle schools is predicted almost perfectly by the average percentage of students judged proficient in English and math on New York’s state assessments for grades 6 through 8.
The correlation between the two is r = .95, where 1.0 represents a perfect correlation. The USN&WR effort to introduce an adjustment for the kinds of students that a given school enrolls seems to be of little consequence. All schools in the bottom quartile of the USN&WR rankings have an average proficiency rate of 28 percent or lower.
School and college rankings can distort normal educational processes, reinforcing social hierarchies that govern who enrolls in a school, how those students are treated and what happens to them thereafter.
There is also strong evidence that the USN&WR rankings reflect the composition of the students attending schools, much more so than anything that schools actually do. Schools with concentrations of low-income students, based on New York City’s economic need index, are much more likely to have low USN&WR rankings than schools with economically advantaged students. The correlation is r = -.68, a strong association between student needs and a school’s ranking: the more students in poverty, the lower the ranking.
Finally, USN&WR ranking percentiles for New York City middle schools are tightly linked to students’ academic performance at the time they entered middle school. The correlation of r = .88 indicates that we can predict a middle school’s USN&WR rating very, very well simply by knowing how its students did in fifth grade — before entering middle school.
Is there a better way to rank schools? Some questions answer themselves. The right question to start with is: What problems are school rankings intended to solve?
Related: Should parents value academic achievement or academic growth in a school?
Are the USN&WR rankings intended to provide parents with information that can help them choose a school that is right for their child? Ranking schemes such as USN&WR’s do nothing to inform parents about what kind of experience their child might have in a school. Moreover, not all families have the opportunity to choose a desirable elementary or middle school without moving to a new neighborhood. As recently as five years ago, the parents of only 41 percent of students in grades 1-12 reported that public school choice was available to them.
Are the USN&WR rankings designed to hold schools accountable for their performance? If so, we would want a robust public debate about which outcomes matter most.
I guarantee you it wouldn’t boil down to scores on standardized tests. Moreover, policy entrepreneurs now realize that there is as much, if not more, interest in a school’s contribution to a child’s growth than in a snapshot of where children are at a particular moment in time.
Are the USN&WR rankings intended to be an accessible tool for parents to compare schools to one another? If so, what’s the point? State and local school report cards already do this.
It’s hard not to conclude this without a joke about where I would rank the new USN&WR rankings among all school rating and ranking schemes. As a matter of courtesy, I’ll simply say that they are somewhere in the bottom quarter of all ranking schemes.
Aaron Pallas is the Arthur I. Gates Professor of Sociology and Education and chair of the Department of Education Policy and Social Analysis at Teachers College, Columbia University. The Hechinger Report is an independently funded unit of Teachers College.
This story about school ratings was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for Hechinger’s newsletter.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.