Get stories like this delivered straight to your inbox.Sign up for The 74 Newsletter

An AI-powered tutor, paired with a human helper and individual-level data on a student’s proficiency, can outperform a human alone, with near-flawless results, a new study suggests.

The results could open a new front in the evolving discussion over how to use AI in schools — and how closely humans must watch it when it’s interacting with kids.

In a randomized controlled trial involving 165 British secondary school students, ages 13–15, the ed-tech startup Eedi.com put a small group of expert human tutors in charge of a large language modelor LLM, offered by Google’s LearnLM. As it tutored students on math problems via Eedi’s platform, it drafted replies when students needed help. Before the messages went out, the human tutors got a chance to revise each one to the point where they’d feel comfortable sending it themselves.

a و the و human – تفاصيل مهمة

Students didn’t know whether they were talking to a human or a chatbot, but they had longer conversations, on average, with the “supervised” AI/human combination than simply with a human tutor, said Bibi Groot, Eedi’s chief impact officer.

In the end, students using the supervised AI tutor performed slightly better than those who chatted online via text with human tutors — they were able to solve new kinds of problems on subsequent topics successfully 66.2% of the time, compared to 60.7% with human tutors.

The AI, researchers concluded, was “a reliable source” of instruction. Human tutors approved about three out of four drafted messages with few to no edits.

Students who got both human and AI tutoring were able to correct misconceptions and offer correct answers over 90% of the time, compared to just 65% of the time when they got a “static, pre-written” response to their questions.

to و of و the – تفاصيل مهمة

And the AI only “hallucinated,” or offered factual errors, 0.1% of the time — in 3,617 messages, that amounted to just five hallucinations. It didn’t produce any messages that gave the tutors pause over safety.

The results suggest that “pedagogically fine-tuned” AI could play a role in delivering effective, individualized tutoring at scale, researchers said. Interestingly, students who received support from the AI were likely to solve new kinds of problems on subsequent topics.

The key to the AI’s success, said Groot, was that researchers gave it access to detailed, “extremely personalized” information about what topics students had covered over the previous 20 weeks. That included the topics they’d struggled with and those they’d mastered.

the و AI و to – تفاصيل مهمة

“We know what topics they’re covering in the next 20 weeks — we know the curriculum. We know the other students in the classroom.

We know whether they’re putting effort into their questions. We know whether they’re watching videos or not — we know so much about the student without passing any personally identifiable information to the AI.”

That guided the AI’s strategy about whether students needed an extra push or just support — something an “out-of-the-box, vanilla LLM” can’t do, she said.

“They don’t know anything about what the teacher is teaching in the classroom,” Groot said. “They don’t know what misconceptions or what topics the students are struggling with and what they’ve already mastered, so they’re not able to dynamically change how they address the topic, as a human tutor would.”

the و know و what – تفاصيل مهمة

Human tutors, she said, generally have “a really good sense of where the student struggles, because they have some sort of ongoing relation with a student most of the time. An LLM tutor generally doesn’t.”

All the same, even master tutors typically don’t go into a session knowing a student’s comprehensive history in a course, including their misconceptions about the material. “All of that is too much information for a human tutor to read up on and deal with while they’re having one conversation” with a student, Groot said.

And they’re under pressure to respond quickly “so that the student is not left waiting. And that’s quite an intensive experience for tutors that leads to a bit of cognitive overload,” she said.

The AI doesn’t suffer from that. It needs less than a millisecond to read all of those contexts and come up with that first question.”

a و that و to – تفاصيل مهمة

Even with their personal connection to students, human tutors can’t be available 24/7. Groot said Eedi employs about 25 tutors across several time zones who are available to students from 9 a.m.

to 10 p.m. every day, but to give students broader access would require hiring “an army of tutors,” she said.

The new findings could encourage schools to use AI as a kind of “front line” tutor, with humans intervening when a student is “derailing the conversation, or they have such a persistent misconception that the AI can’t deal with it,” said Groot. “We think that would be an interesting way to collaborate between the AI and the human, because there is still a really important role for a human tutor. But our human tutors just cannot have conversations with thousands of students at once.”

to و a و with – تفاصيل مهمة

The new study, published last week on Eedi’s site and scheduled to appear in a peer-reviewed journal next year, differed in one important way from recent studies that looked at AI tutoring. Researchers at Stanford University in October 2024 examined AI-assisted human tutoring, in which tutors primarily drove the conversation.

But in that case, the AI acted as a kind of assistant, providing suggestions behind the scenes. In the Eedi study, it was the other way around, with AI driving the conversation and humans overseeing it.

Robin Lake, director of the Center on Reinventing Public Education at Arizona State University, said the study is important in and of itself, but also in the context of broader findings elsewhere suggesting that, with proper training and guidance, “AI can be an incredibly powerful tool — and certainly has a potential to take tutoring to scale in ways that we’ve never seen before.”

the و in و of – تفاصيل مهمة

Under controlled circumstances, she said, it’s also “outperforming humans — that’s really important.”

Lake noted a June study from Harvard researchers that examined results from 194 undergraduates in a large physics class. They presented identical material in class and via an AI tutor and found that students learned “significantly in less time” using the tutor. They also felt engaged and motivated about the material.

Liz Cohen, vice president of policy for 50CAN and author of the recent bookThe Future of Tutoring: Lessons from 10,000 School District Tutoring Initiativessaid the study provides “valuable evidence” about new kinds of tutoring.

and و in و the – تفاصيل مهمة

But one of its limitations, she said, is that it relied on 13-to-15-year-olds. “So immediately I have a lot of questions about if the findings are applicable for younger students, especially using a chat based model,” which may not be a good one for such students.

She also noted that there are many questions around student persistence with AI tutors, including what happens when students get frustrated or aren’t sufficiently engaged in the work?

“I still mostly think that entirely AI tutoring programs are biased towards students who want to do the work or are interested in learning,” Cohen said, “and it’s pretty easy to see that students who aren’t bought in or are frustrated are going to give up readily with an AI tutor.”

are و that و AI – تفاصيل مهمة

She noted that her 12-year-old daughter has experienced problems persisting in an AI-powered math tutoring program. “She gets frustrated if she can’t get the answer and then she doesn’t want to do it any , so I think we need to figure out that piece of it.”

Did you use this article in your work?

We’d love to hear how The 74’s reporting is helping educators, researchers, and policymakers.Tell us how

to و that و in – تفاصيل مهمة