In a world where artificial intelligence chatbots often exude an air of infallibility, a new educational tool is turning the tables. University of Washington researchers have developed AI Puzzlers, a game designed to highlight an area where AI systems still struggle: solving certain reasoning puzzles. This initiative aims to empower children to critically evaluate AI responses, a skill that even adults, including professionals like lawyers, sometimes lack.
The game allows users to tackle ‘ARC’ puzzles, short for Abstraction and Reasoning Corpus, by completing patterns of colored blocks. Players can then challenge various AI chatbots to solve the same puzzles and explain their solutions—an exercise that frequently exposes the systems’ limitations. This innovative approach was presented on June 25 at the Interaction Design and Children 2025 conference in Reykjavik, Iceland.
Understanding the Game’s Mechanics
ARC puzzles, developed in 2019, are intentionally designed to be challenging for computers yet accessible to humans. They require abstraction, a cognitive process of recognizing patterns and applying them to new scenarios. Despite advancements, even cutting-edge AI models have not fully mastered these puzzles.
AI Puzzlers includes 12 ARC puzzles that children can solve and compare with AI-generated solutions. An “Ask AI to Explain” feature provides a text explanation of the AI’s attempt, often revealing inaccuracies. The game also offers an “Assist Mode,” allowing children to guide the AI toward a correct solution.
Lead author Aayushi Dangol, a doctoral student in human-centered design and engineering at UW, emphasized the universal appeal of ARC puzzles. “Kids naturally loved ARC puzzles and they’re not specific to any language or culture,” she said. “Even kids that can’t read yet can play and learn.”
Testing and Insights
The researchers tested AI Puzzlers at the UW College of Engineering’s Discovery Days with over 100 children from grades 3 to 8. Additionally, two sessions were conducted with KidsTeam UW, a collaborative technology design project involving children aged 6-11. These sessions provided valuable feedback, leading to the development of the Assist Mode feature.
Jason Yip, co-senior author and UW associate professor in the Information School, noted the children’s ability to critically assess AI. “The kids in KidsTeam are used to giving advice on how to make a piece of technology better,” he said. “We hadn’t really thought about adding the Assist Mode feature, but during these co-design sessions, we were talking with the kids about how we might help AI solve the puzzles and the idea came from that.”
“This is the internet’s mind,” one child remarked. “It’s trying to solve it based only on the internet, but the human brain is creative.”
Implications for AI Education
The findings from AI Puzzlers suggest that children are capable of discerning the differences between human and AI cognition. As they engaged with the game, they learned to use AI as a tool requiring guidance rather than a definitive source of answers.
Julie Kientz, co-senior author and UW professor in human-centered design and engineering, highlighted the potential of young learners. “Kids are smart and capable,” she said. “We need to give them opportunities to make up their own minds about what AI is and isn’t, because they’re actually really capable of recognizing it. And they can be bigger skeptics than adults.”
This research was supported by The National Science Foundation, the Institute of Education Sciences, and the Jacobs Foundation’s CERES Network. Co-authors include doctoral students Runhua Zhao and Robert Wolfe, and master’s student Trushaa Ramanan.
For further information, interested parties can contact Aayushi Dangol at [email protected], Jason Yip at [email protected], and Julie Kientz at [email protected].