Student Opinion Scale
Category: Student Learning
Abstract: Generative AI has been promoted as the technology that could transform education by providing every student a personal tutor. We provide some of the first large-scale experimental evidence, from a two-year cluster randomized trial in 18 Tennessee middle schools in which randomly assigned students used Khan Academy with its AI tutor, Khanmigo, configured to coach rather than give answers, during existing daily remedial mathematics sessions. Assignment raises math achievement by 1.3 national percentile ranks per term, or about 0.06 to 0.08 standard deviations over a school year; the implied effect of a full year of active participation reaches 0.14 standard deviations. These gains resemble those from Khan Academy practice without AI assistance. One explanation is that students used the tutor infrequently and, when they did, rarely engaged it in substantive mathematical dialogue: 96 percent of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced, and in only 17 percent of the exercise sessions in which they made a mistake. Messages that students did send were mostly bare answers or clicks on suggested prompts. The binding constraint appears to be engagement: realizing the promise of AI tutoring will require getting students to use it, not just giving them access.
A Proof Points column on a two-year study of the Khanmigo AI tutor in Tennessee middle schools. It describes the draft paper by Philip Oreopoulos and Nina Low, which found students largely stopped using the tutor when it withheld answers.
A news article weighing evidence on AI tutors versus human tutoring. It describes the Khanmigo experiment by Philip Oreopoulos and Nina Low in Tennessee middle schools, which found thin engagement and no learning gains.
Chalkbeat story built around a two-year experiment finding middle schoolers rarely engaged with Khan Academy's Khanmigo AI tutor. It links and credits EdWorkingPaper 26-1551 ("One Click Away") and a companion EdWorkingPaper by Philip Oreopoulos and colleagues on a mastery-based AI math program.
Khan Academy founder Sal Khan reviews Philip Oreopoulos and Nina Low's randomized trial of Khan Academy and its Khanmigo AI tutor in 18 Tennessee middle schools. He highlights a 0.14 standard deviation math gain for students with sustained access, while noting that students rarely used the AI tutor itself.