Getting more bright, low-income students into top schools is a laudable goal toward helping more people rise out of poverty. Dozens of elite college prep programs, from Leadership Enterprise for a Diverse America to the University of Chicago’s Collegiate Scholars Program, have been established across the United States in the last 20 years, each with heartwarming stories of college success.
But a newly published French study of one of that country’s most elite college prep programs for disadvantaged youth concluded that it harmed half the high school students who participated and left them worse off. It’s a cautionary tale about efforts to do good.
“The French and American contexts are so different but the finding that this work potentially did damage to students causes some pause and should be explored,” said Jason Klugman, director of the Princeton University’s Preparation Program.
The French two-year program of extra tutoring and mentoring, called a cordée de la réussite or “team for success” in French, was run by École Normale Supérieure (ENS) of Paris, which is one of the most prestigious and selective schools of higher education in the country. I think of ENS as something like the Harvard of France.
The program worked brilliantly for high school students who were already very high achieving to start — those with ninth grade test scores that placed them in the top 20 percent of their high schools. They posted higher high school exam scores, were admitted to rigorous preparation programs for the elite grande ecoles (akin to the Ivy League) at higher rates and persisted in their studies more than similar students who also wanted to join the ENS program but weren’t selected to participate.
By contrast, middle-to-high achieving students — those with ninth grade test scores that fell between the 30th and 70th percentiles in their high schools — ended up with lower high school test scores and fewer offers for the prestigious grandes ecoles preparation programs than similar students who didn’t participate in the program.
The problem: the additional lessons and college prep were a time-consuming distraction from students’ main high school studies. The very best students managed the extra workload but the rest struggled.
“Our paper shows that the main mechanism behind the negative impact is that ‘mid-high’ ability students (good but not top performers) will spend less time reviewing their courses and doing homework and more time preparing for higher education in top institutions, which can be harmful,” explained Arnaud Peich, one of the study’s three authors, by email.
The researchers found that students of all abilities enjoyed the program. In surveys of the students five years later, even the students whom the program harmed said they liked the tutoring sessions and making friends with other students in the program. It’s an example of how positive survey results aren’t evidence of effectiveness.
The paper, “A Pleasure That Hurts: The Ambiguous Effects of Elite Tutoring on Underprivileged High School Students,” was published online in the April 2020 issue of the Journal of Labor Economics.
What sets this study apart from previous evaluations of college access programs is that the French economists were able to set up a randomized controlled experiment back in 2010 and 2011 by working with principals in 12 French high schools serving underprivileged students. Six schools were in low-income Paris neighborhoods and six were in the suburbs, such as the immigrant “banlieues” of Noisy le Sec and Aubervilliers. Principals invited good students who were deemed to be of college caliber to apply for ENS’s free college prep program. More than 550 students applied over two years. About 300 students were randomly selected to participate and given slots at the end of 10th grade. The remaining unselected 250 students became a comparison group and the researchers tracked their progress too.
The program began with a week-long summer session at ENS designed to boost study and writing skills, followed by Saturday sessions throughout 11th and 12th grades with ENS students who had volunteered to be tutors. Students and tutors selected advanced topics that were absent from the curricula of the low-income high schools.
The economists waited years for the high school and post-high school results to trickle in. Even though there were so many success stories among program participants, the data seemed to indicate that the college prep program had no effect at all. Academic achievement and post-high school options, on average, were the same for the control group of students who didn’t participate.
The economists decided to divide the students in half and looked at them separately. That’s how they discovered that the top-achieving students had done well by the program but the bottom half had fared poorly. Lumped together, the results washed out to zero. Separately, the students’ futures diverged starkly.
“You can’t look at all these students as one big blob. They are different people and they have different needs,” said Catherine Millett, a researcher at ETS who has been evaluating similar programs in the United States, the United Kingdom and South Africa since 2000.
“We might need to tweak the programs slightly to differentiate them for different students. This is something we need to look at,” she said.
U.S. programs haven’t been evaluated as rigorously as the ENS one. No one I talked to was aware of U.S. studies with control groups so it’s impossible to say how American students might have fared otherwise without these programs. Millett said that’s because “the idea of denying a student an opportunity is very hard” but necessary to create a comparison group. She said she is just now starting to receive good data on students in order to create good comparison groups for future analysis. In 2018, Millet compared students who participated in Princeton University’s Preparation Program with similar students but the data she had for matching students was limited. For example, she did not have access to students’ grades.
Instead, we have many evaluations that tout how many high school students are getting into “selective” colleges. While researching this story, I learned that the definition of selective can be very broad, including hundreds of institutions that are easier to get into than the Ivy League and their equivalents.
Tatiana Poladko, the founder of TeenSHARP, an elite college prep program in Delaware and the Philadelphia area, doesn’t believe the lessons of the French study would be applicable here because American colleges tend to look at applicants more holistically and admittance doesn’t depend on an exam score as it does in France. Extracurricular activities and sports, for example, aren’t considered in France’s college admissions process and students specialize in academic fields while still in high school.
“Our kids are scoring average and below average [on the SAT] and yet they end up in one of the top 200 colleges in the country,” said Poladko. Instead of high test scores and grades, Poladko admits students to her program who demonstrate that they are “coachable”: motivated to learn, open to criticism and willing to revise.
Many American programs invest considerably more than the 1,500 euros (about $1,600) per student that the ENS program spent a year and include social workers and other services to address disadvantaged students’ social and emotional needs. TeenSHARP, for example, said it spends $22,000 on each of its students over the course of four years rather than just two.
Klugman’s Princeton program similarly has a broad measure of success, not just admittance to an Ivy League school, and provides students with a lot of non-academic support. He says that this approach can work well for a large range of disadvantaged students, not only the very brightest of the bright.
The French economists told me that after they discovered the ENS program backfired for many students, ENS began to restrict this college prep program to only the very best students. The school also experimented with a different program design for slightly lower achieving students. But those changes haven’t been evaluated.
This story about elite college prep was written by Jill Barshay and produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.