Want to remake assessment? Think disruptively.

By:

Apr 13, 2021

The K–12 world is in the midst of another heated debate over testing. Last spring, states were given a reprieve from implementing federally-mandated standardized tests due to COVID. But this year, the Biden administration has instructed states to proceed with testing because policymakers need the data to identify inequity and direct resources accordingly. 

In protest of the move, over 500 education researchers wrote to Secretary of Education Miguel Cardona arguing that while accounting for school quality is critical, administering tests this year will produce flawed data and exacerbate inequalities. Instead, the scholars urge the administration to invest in “new measures” for school quality and educational opportunity.

In a similar vein, a group of school and intermediary partners called the Assessment for Learning Project is calling for educators to reimagine assessment from the classroom all the way up to state and federal levels. In the group’s perspective, traditional assessments of learning do little more than rank and sort students, alienating them in the process. Assessments like these can expose inequity, but do little to address it. 

New assessments for learning, on the other hand, could convert an assessment experience into a learning opportunity, help students build evidence of their learning over time, and actively drive towards equity. 

In February 2020, I attended the group’s first conference, called the Assessment for Learning Conference (AFL). (The second annual conference, virtual this time, will take place in May.) As I immersed myself in conversations with educators, administrators, and researchers about what assessment could become, it became clear that innovation is critical both to develop reimagined assessments and to scale them into the mainstream.

But not all innovation strategies lead to the same result. Professor Clayton Christensen broke ground by distinguishing “sustaining innovations,” which improve upon existing products to make them better, from disruptive innovations, which are completely new or redesigned products that expand access to and affordability of a product and eventually redefine what “good quality” means for that product. Both forms of innovation have value, but purely sustaining innovations are unlikely to unlock student-centered assessments for learning at scale. It’s time for assessment innovators to think disruptively.

First, a look at sustaining innovations in assessment

Let’s consider several common criteria that most assessments are expected to meet. Quality traditional assessments are supposed to:

  • Determine whether students have learned what the teacher intended to teach
  • Objectively and efficiently measure individual student learning at the time of the exam
  • Result in a final score that can be used to compare student achievement 
  • Judge a student’s readiness for advancement to the next level

Making assessments accomplish these criteria even better would be a “sustaining innovation” strategy. For example, new computer-based exams incorporate questions that adapt based on students’ previous answers, pinpointing a student’s level of understanding more accurately. Some exams incorporate performance tasks such as an essay, where students’ abilities to argue a point using evidence are judged according to a rubric—expanding, to a degree, the kinds of skills and knowledge the exam is capable of capturing. 

Both of these innovations introduce improvements in test quality but do not fundamentally change the criteria on which that quality is judged. They can squarely be categorized as sustaining innovations. 

How disruptive innovations in assessment will be different

On the other hand, disruptive innovations should play a role if the goal is to transform assessment systems to prioritize a different definition of quality—such as being a positive learning experience for students or offering evidence for collaboration skills. Disruptive approaches to assessments will share these three features: 

1. New assessments won’t be better than tests—they’ll be different.

Disruptive innovations start as rudimentary from the perspective of incumbent companies and their core consumers. While these innovations don’t perform well according to the traditional criteria for quality, they do offer different advantages for users who aren’t so demanding. Over time, as disruptive innovations improve, they nail those new, desirable advantages and get good enough at the old criteria, showing core consumers that different is actually valuable. 

Likewise, new assessments on a disruptive trajectory won’t succeed by competing directly with tests on factors like efficiency and comparability of final scores. Instead, they’ll compete on a set of new metrics, like student experience. 

For example, the AFL community describes key shifts away from traditional assessment. Rather than prioritizing objective evaluation of a student’s learning, assessments for learning will prioritize reflection and feedback to support the student’s further development. And instead of optimizing for a numeric score, they will produce a varied body of evidence that demonstrates student mastery.

2. New assessments will target nonconsumers and first take root outside of the core of the school system.

Disruptive innovations start among users whose alternative to the product is nothing at all, or for whom the traditional product is too complicated or expensive to use. Since incumbent companies don’t see these users as valuable customers, they initially ignore the disruptive innovation.

Similarly, disruptive assessment strategies will take root outside of core subjects, where they are exempt from the pressures of the existing assessment paradigm. In one session at the AFL conference, researchers from the Playful Journey Lab at MIT discussed how some maker learning programs are developing “embedded assessments,” where assessment and reflection are part and parcel with the learning experience. Their Beyond Rubrics toolkit, in partnership with MakerEd, offers embedded assessment tools for educators to try out in their own maker education programs—which can often be found in alternative school models, libraries, and other more informal learning spaces. It’s characteristic of a disruptive approach to see new models emerging from outside the core, where they have time and space to develop and mature.

3. New assessments will have a technological enabler to scale.

Disruptive innovations always have a technological enabler that makes it possible for the innovation to improve and scale over time. 

So too will disruptive assessment strategies. Introducing technology in conversations about remaking assessment can be polarizing, since many people may associate technology with sustaining innovations on traditional assessments, like Scantron. But any innovative approach with a chance to disrupt our test-dominated assessment landscape must also take advantage of a technological enabler.

On this front, several potential areas for investigation include:

  • Game-based assessments that use data from gameplay to suggest evidence of learning outcomes. (Some conference participants, such as the Playful Journey Lab, are engaged in research to advance these nascent tools.) 
  • Collaborative, multi-assessor feedback and grading tools that streamline assessment of student work products, and may even expand access to peer and expert assessors both inside and outside of schools. 
  • Portfolio platforms designed for school use, where students can collect, curate, and share evidence of their learning. 
  • Instructional models harnessing technology to expand teacher capacity, allowing teachers’ focus to shift from delivering instruction towards diagnosing individual student needs and offering individualized higher-order feedback. 

The future of assessment presents opportunities to both improve existing strategies through sustaining innovations, and to redefine the purpose of assessments altogether by thinking disruptively. Recognizing the difference will help innovators create new assessment models in spaces where they are likeliest to succeed, paving the way for assessments that don’t just measure learning, but also help it flourish.

Chelsea is a research fellow at the Institute, where she analyzes how innovation theory can inform the evolution of student-centered learning and the advancement of student agency. As part of this work, she leads the Canopy project, a collaborative effort to build better collective knowledge about school innovation.