What happened
The Hechinger Report reported on a draft working paper, “Making AI Tutoring Productive,” by researchers from the University of Toronto and the University of Pennsylvania’s Wharton School. In a randomized experiment with more than 6,000 Tennessee middle schoolers practicing fractions for one 50-minute class period, students using an AI tutor plus a requirement to answer three same-skill questions correctly after a mistake scored about 3 percentage points higher on a 15-minute retention test a week later than students using conventional computerized instruction.
Why it matters
The result points to a narrow design lesson: the reported gain came from AI tutoring combined with repetition, not AI alone. The AI tutor, Numi, responded to student work and guided error review, while the mastery rule slowed practice. But the advantage was small and appeared mainly on easier fraction questions similar to practice items, not on more challenging problems or transferable understanding.
What to do next
Treat this as early evidence, not a procurement mandate. The paper had not yet been peer reviewed, and Oreopoulos cautioned that the study tested only four combinations. Pilot error-review and mastery features on specific skills, compare them with current software, monitor time on task and post-error accuracy, and avoid claiming broader math gains without longer-term evidence.
About this briefing
Reviewed by TutorFlow Editorial. We link the primary source, preserve its publication date, and distinguish reported claims from TutorFlow analysis. Our commentary focuses on practical decisions for educators and training teams.
Discussion (0)
Open to everyone