assessment

Why the nation’s K-12 accountability and assessment system doesn’t make the grade

By:

Mar 5, 2019

Two new studies lay bare how the assessment and accountability systems in use throughout the United States shortchange teachers and schools by not presenting accurately what their students know, can do, and have learned in the last year.

The recently released studies concern the efficacy of Teach to One: Math, an innovative blended-learning model that Heather Staker and I discussed in our bookBlended: Using Disruptive Innovation to Improve Schools. The studies highlight not only the profound impact blended-learning models can have on student outcomes, but also raise deeper questions about whether our country’s current assessment and accountability system may be undermining the ability for schools to deliver on those outcomes.

As background, Teach to One: Math is a model that personalizes learning. Developed by the non-profit New Classrooms Innovation Partners, it emerged from an initiative within the New York City Department of Education called School of One, which Time named as one the year’s Best Inventions in 2009. Thirty-nine schools across 11 states now use Teach to One: Math.

Partner schools that adopt Teach to One: Math typically replace their traditional textbook-based math program with this new approach. Students learn through a variety of modalitiesfrom teachers, in collaboration with peers, and independently. Each day students take an online assessment that helps determine what they will learn the next day. Teach to One generates about 10,000 unique daily student schedules that include what skill a student will work on, the material through they will learn it, where in the room the learning takes place, and which teachers are assigned to which groups. The approach is designed to enable personalization for students while fostering a more collaborative and sustainable role for teachers.

The first study was a rigorous, federally-funded evaluation by Doug Ready at Teachers College and the Consortium for Policy Research in Education (CPRE) that focused on the implementation of Teach to One: Math at five schools in Elizabeth, New Jersey. The study focused on comparing student performance on state tests each year of the program. Although it could not discern impact in the five schools that implemented the program, the study could not form any generalizable conclusions either.

The education trade reports and headline writers raced predictably to offer banners of failure while ignoring the most useful part of the evaluation itself—what actually happened in the schools themselves and what it teaches us about the impact of well-intentioned education policies that assume that annual grade-level instruction for all students is the most efficient path to college and career readiness.

Teach to One is designed to meet students where they are and enable them to accelerate  toward college and career readiness as efficiently as possible.  For many students, that may mean having to go back and address unfinished learning from prior years in service of getting students back to grade level. This is especially critical in math, where skills build upon one another each year.

As sensible as that approach might seem, the problem is that every day a student spends working below their grade level by age is one less day they have to cover grade-level material. And the test that matters (both for state accountability purposes and for the federal evaluation) is one that’s focused only on grade-level content.

Even though it might make pedagogical sense to go back and fill gaps, that’s not what today’s policies and tests signal teachers should do.

This challenge is by no means unique to Teach to One. Speak with most middle grade math teachers and you would hear a similar predicament. What’s different with Teach to One is that schools must grapple explicitly with this tension in how they configure the program itself so that those choices can be embedded into student learning progressions. In schools that don’t use models like Teach to One, teachers are expected to somehow figure it all out on their own.

The federal study describes in great detail what happened in Elizabeth when schools confronted this hard choice. According to the study, schools acted in vastly different ways. Some required that Teach to One include academic “floors” and “ceilings” in order to prioritize exposing students to grade-level content and required all 8th graders to learn Algebra regardless of their academic readiness. The strategies varied by school and also changed continually throughout the implementation.

These shifts were far more profound than the typical implementation ebbs and flows that most studies include. It fundamentally caused the program students experienced to fall out of alignment with what was being assessed. As a result, the study made clear that the results shouldn’t be used to draw any generalized conclusions about the program itself—a key point sadly omitted from most of the press coverage.

The second study, which the Bill and Melinda Gates Foundation funded and MarGrady Research conducted, focused on the entire set of schools implementing Teach to One over three years (not just those in Elizabeth) and on results from NWEA’s MAP assessment, a widely-used adaptive test that is more effective at measuring student learning growth than state tests because it includes items from across multiple grade levels. For example, when a 7th-grade student who arrives to school at a 4th-grade level grows to a 6th-grade level in the course of one year, those gains would more likely get picked up by the MAP than by the state test. Which, when you think about it, is absurd that schools and teachers wouldn’t be rewarded for helping a student make two full years of academic gains in just one year.

MarGrady found that students enrolled continuously in Teach to One saw 23% greater gains than students nationally, and, more salient to prove that the nation’s assessment systems shortchange individual students’ growth, that students in schools with accountability systems aligned fully to the program’s intent—meaning no floors or ceilings—saw gains that were 53% above the national average. Students in the latter group grew an impressive 38 percentile points from the time they came into 6th grade to the time they left 8th grade.

New Classrooms should probably have thought twice before signing up for a federal evaluation oriented around proficiency gains on annual standardized assessments given that the program is oriented around individualized learning pathways in service of high school readiness.

But those who take the time to read the federal study, not just the press coverage, will come to understand the key lesson: Be sure to align the instructional approach with the measure of success from day one. Otherwise, there isn’t much to be learned. 

And those who read the the MarGrady study will see that Teach to One seems to be having its strongest impact in schools that allow for meeting each student’s needs regardless of her assigned grade level.

We need more research to deepen our understanding of both Teach to One and personalizing learning more broadly. But in the interim, the more meaningful contribution of these two studies may actually be in the additional question they inherently raise: Is an assessment and accountability system that signals an exclusive instructional focus on grade-level material in the best interest of all students? 

Michael is a co-founder and distinguished fellow at the Clayton Christensen Institute. He currently serves as Chairman of the Clayton Christensen Institute and works as a senior strategist at Guild Education.