U.S. ranks No. 13 in new collaborative problem-solving test

The United States may be known for its rugged individualism. But it turns out American teens are, surprisingly, much better at group collaboration than at individual academic work. That’s according to a new, unusual version of the Program for International Student Assessment, or PISA, which tested collaborative problem-solving skills among 15-year-olds in more than 50 countries and regions around the world in 2015. Those results were released last week.

The PISA is known for its testing of high school students around the world, especially in math and reading. In general, nations with high math and reading scores also tended to do well on this new collaboration test. Singapore, Japan, Hong Kong and South Korea topped the new social skills ranking (see chart below), and they’re also among the top 10 for individual student achievement.

But for some countries, there was a big deviation. For example, the United States ranked 39th in math on the 2015 PISA test. But in collaborative problem-solving, the U.S. ranked 13th. For China, it was the opposite. Four regions in mainland China, including Beijing and Shanghai, collectively ranked 6th in math and in 2015.  But these Chinese regions ranked 26th in collaborative problem-solving.

The Organization for Economic Cooperation and Development (OECD), which administers the PISA tests, is assessing group collaboration because it believes that’s what employers will want more of from workers. However, PISA officials found that very few students in 2015 could collaborate with sophistication. Only 8 percent of students tested around the world could handle problem-solving tasks that require them to maintain awareness of group dynamics, take the initiative to overcome obstacles, and resolve disagreements. Even in Singapore, the highest-scoring nation, just one in five students could do this.

“Strong academic skills will not automatically lead to strong social skills,” Andreas Schleicher, who oversees the PISA tests, wrote in an editorial note accompanying the score report. “As workplaces around the globe are demanding – and paying higher wages for – people with well-honed social skills, schools need to do more to help their students develop these skills.”

Schleicher suggested more organized sports, as well as creating classrooms that “celebrate diversity” and encourage students to share their ideas.

Worldwide, girls were much better at collaboration than boys.

Students who said they play video games scored lower than students who said they don’t. Even among students of the same gender, income levels and academic backgrounds, those who played video games didn’t collaborate as well. For example, between two boys with the same math scores, the boy who didn’t play video games tended to score higher on the collaboration test.

Measuring collaboration is tricky. You can’t just put a bunch of 15-year-olds in a room and ask them to work together. Observers might have their own biased or subjective judgments about who listened and performed best. And each student’s performance would depend on the peers who happened to be in the room with him.

So the OECD turned to educational-software developers, led by Arthur Graesser at the University of Memphis, to create simulated computer agents to pose as humans. Students took the test individually in front of a computer and were guided through scenarios where they had to interact with imaginary “peers,” reading their comments on chat sessions and deciding on the best course of action. All questions in the assessment either were multiple-choice or involved moving icons into a slot. There were no free-response questions. In some cases, a test taker isn’t given enough information and has to pool information from his “peers” to solve a problem.  In other cases, a student must show that he has considered the arguments of others in arriving at a decision. Sometimes, students are guided through a negotiation exercise, where they must navigate through competing individual goals and come up with the best outcome.

One might question whether students respond the same with unseen computerized agents as they would face-to-face with real humans. But the OECD argues that more and more workplace collaboration takes place in virtual settings over the internet, as people increasingly work with others in different locations and companies. However, what employers probably value the most is the ability to produce a group project. And this PISA test didn’t ask students to produce anything.

Below is a chart showing the top 20 nations and regions in collaborative problem-solving, compared with their 2015 math score ranking:

The Hechinger Report provides in-depth, fact-based, unbiased reporting on education that is free to all readers. But that doesn't mean it's free to produce. Our work keeps educators and the public informed about pressing issues at schools and on campuses throughout the country. We tell the whole story, even when the details are inconvenient. Help us keep doing that.

Join us today.