AI tutoring matched human tutors in the lesson, but not on the test: the Eedi and Google DeepMind trial

Eedi and Google DeepMind have published the full findings of a randomised controlled trial involving 1,179 secondary pupils across 10 UK state schools. Released on 29 October 2026, it is one of the larger and more carefully designed studies of AI tutoring in real classrooms so far, and the results are more nuanced than the usual headlines about AI in education.

TLDR: AI tutoring helped pupils put mistakes right in the moment just as well as expert human tutors did. But after ten weeks of lessons, neither kind of tutoring produced a clear gain on an independent maths test.

What the trial tested

Pupils spent one lesson a week practising maths on Eedi's platform with their teacher present. Whenever a pupil answered a question incorrectly, they received one of four kinds of support, and kept the same support for the whole term:

• Eedi's standard static learning support, such as explanations and hints;
• a typed chat with an expert human tutor;
• a typed chat with an AI tutor, built on a version of Google's Gemini model fine-tuned for teaching and working inside Eedi's tightly constrained design;
• the same AI tutor, given additional context about each pupil's past performance and misconceptions.

Every message drafted by the AI tutor was reviewed by a human expert before it reached a pupil.

The headline findings

In the lesson, tutoring worked. After AI or human tutoring, pupils answered a similar question correctly 56 to 58% of the time, compared with 43% after static support. There was no clear difference between AI and human tutors.

On the independent test, there was no clear gain. Pupils sat Star Maths, a standardised assessment, before and after the trial. Those who had AI tutoring scored about the same as those who had static support. Those with human tutors scored slightly lower, although the researchers note that gap could be down to chance. One likely factor is that tutored pupils covered less material, in more depth.

Extra personal context added little. Giving the AI a summary of each pupil's learning history did not clearly improve outcomes, probably because both AI versions already knew the misconception behind each wrong answer.

Human review did not catch everything. Reviewers approved around 19 in 20 AI drafts without changes. Nevertheless, 11 AI messages contained a false mathematical statement, and seven of these reached pupils. For balance, the checks also found seven genuine maths errors in the human tutors' own messages.

The averages hide differences. Exploratory analysis suggests tutoring of either kind worked better for girls than for boys, and roughly one in five pupils never replied when a tutor appeared. The researchers describe these findings as tentative.

What is the applicability across the sector?

The participants were secondary aged, so the findings should not be transferred directly to a primary classroom. As research evidence, however, this study is very valuable.

Its most important contribution is the distinction it draws between immediate task success and lasting learning. A tool can look highly effective in the moment, with pupils getting the next question right and engaging with the conversation, without that translating into what they can do independently weeks later. This is the same concern that sits behind our earlier article, Is AI doing the thinking your pupils need to do?, and it applies to any AI product a school might consider.

It is also a useful reminder about accuracy. Even with an expert checking every single message, incorrect mathematics still reached pupils. Most products that schools are offered will not have that level of oversight.

My view

I find this genuinely interesting, and I will continue to keep an eye on this kind of research. Ultimately for those of us either struggling to recruit support staff or struggling to pay for staff (or both) the idea that an AI tutor can get broadly equivalent results for a significantly cheaper cost and better availability is at the very least an interesting idea. Further to that the difference between boys and girls warrants further consideration.

In Eedi's own earlier two-year study, clear gains on Star Maths only emerged after a year or more. Eedi began a year-long trial in September 2026 involving more than 15,000 pupils in 26 UK schools, and those results will be worth watching closely.

How you could approach trialling this:

1. Measure what children can do independently afterwards. When evaluating AI tutoring, look at what pupils can subsequently do on their own, rather than relying on activity completion, time on task or engagement.
2. Ask suppliers harder questions. Ask what evidence they have beyond usage data, whether outcomes were checked with an independent assessment, and what error rates they have found in their own testing.
3. Look beyond the average. Monitor which pupils engage and which do not, and whether particular groups appear to benefit less.

Sources: Eedi: full findings · Google and Eedi: technical research report (PDF)