Editor’s note: This story led off this week’s Future of Learning newsletter, which is delivered free to subscribers’ inboxes every Wednesday with trends and top stories about education innovation. Subscribe today!
Robert Mawhinney’s math students each sat at their own computers, solving equations. As he looked around the room, he saw some students had smiley faces floating above their heads, some had question marks or exclamation points, and a few had Zzzs. He tapped the air in front of him and up popped a view of a student’s computer screen.
Mawhinney is not an actor in a science fiction movie. He is a teacher at Hopewell Junior High School in Aliquippa, Pennsylvania, near Pittsburgh. A pair of augmented-reality glasses offer him those visual cues and the power to conjure up more information with an outstretched hand.
The glasses, called Lumilo, are powered by an app that works with “cognitive tutor” programs, which use artificial intelligence to offer students feedback as they attempt problems, hints when they struggle and an adaptive set of questions based on their performance.
When Mawhinney tested out the glasses, he could scan the room and the icons would tell him who was on track, who was stuck and who seemed to have abandoned the assignment altogether. If a student had an exclamation point above his head, the app had reason to believe that student might be abusing the program’s “hints” function.
Related: How one high-poverty district is adding virtual reality to its classrooms
Mawhinney could also look at the classroom wall and see what portion of students had mastered a given topic. If the floating percentage indicated few had succeeded with a particular skill, Mawhinney could stop everyone and offer a group lesson. If it suggested several students were struggling with the same skill, he could pull those students aside while everyone else continued to work on their own computers.
“There were times I wouldn’t even have to walk up to the student,” Mawhinney said. “I’d say, ‘You’re not using the negative. Make sure you do that to both sides.’ I could help them from across the room.”
“There were times I wouldn’t even have to walk up to the student. I’d say, ‘You’re not using the negative. Make sure you do that to both sides.’ I could help them from across the room.”
Lumilo is the brainchild of a team at Carnegie Mellon University. Ken Holstein, a doctoral candidate at the university, designed the app with significant input from teachers like Mawhinney who use cognitive tutors in their classrooms. The project treads new ground for the use of artificial intelligence in schools.
“The focus has been on how can we make personal tutors to enhance student learning,” Holstein said. “A major stakeholder that has been left out of the equation was the teacher.”
Holstein started by talking to teachers who were early adopters of artificial intelligence software. They liked that it let students move at their own pace through certain content, and they were in a position to talk about areas for improvement.
Common complaints were about the analytics that teachers could get from these programs. The built-in dashboards offered useful information, but it wasn’t always easy to access or interpret. Teachers said some of the most useful feedback came from students’ faces or body language. But if teachers missed these cues, they might not get another timely signal of a student’s confusion.
“There’s generally an assumption that students who need help the most are least likely to raise their hand,” Holstein said.
Teachers told him they liked the idea of getting real-time information about student performance while still keeping their heads up and being able to look around the classroom.
Enter Lumilo, augmented reality glasses that make those analytics appear in the air in front of them.
The glasses aren’t ready for market – Mawhinney says they’re still too heavy to want to wear for an extended period of time – but preliminary research suggests students using cognitive tutors end up learning more overall when teachers use the glasses. Those results are driving industry interest that could bring Lumilo to classrooms in just a few years.
This story about augmented reality was produced by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education. Sign up for the Hechinger newsletter.
At The Hechinger Report, we publish thoughtful letters from readers that contribute to the ongoing discussion about the education topics we cover. Please read our guidelines for more information. We will not consider letters that do not contain a full name and valid email address. You may submit news tips or ideas here without a full name, but not letters.
By submitting your name, you grant us permission to publish it with your letter. We will never publish your email address. You must fill out all fields to submit a letter.