Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs

Q: What differentiated tutoring policy does the study evaluate?

The policy assigns human tutors to proactively support lower-performing students, while higher-performing students receive reactive, on-demand support.

Q: How is the study designed to estimate the policy’s effects?

It uses a difference-in-discontinuities (DiDC) quasi-experimental design comparing fall (AI-only tutoring for all) to spring (proactive-reactive human-AI tutoring).

Q: What outcomes improve with human-AI tutoring compared with AI-only baseline?

The study finds significant overall improvements, including increases in time on task, skill proficiency, and academic growth measured by standardized MAP tests.

Oct 15, 2026

Uploaded by Aria Callaghan

Hybrid human-AI tutoring, combining technology and human facilitation, can outperform AI-only tutoring, yet evidence suggests lower-performing students benefit more. This study tests a differentiated policy: human tutors proactively support students below within-grade median test scores, while higher-performing students receive reactive on-demand help. In a quasi-experimental difference-in-discontinuities design comparing fall (AI-only) and spring (proactive-reactive) outcomes, results show significant overall gains in time on task, skill proficiency, and academic growth.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Abstract

1. Introduction

Human tutoring evidence and scalability challenges

Automated tutoring systems and differential effects

Hybrid human-AI tutoring and differentiated policies

Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs Ashish Gurung Carnegie Mellon University [agurung@cmu.edu](agurung@cmu.edu)[ ](agurung@cmu.edu)Danielle R. Thomas Carnegie Mellon University [drthomas@cmu.edu](drthomas@cmu.edu)[ ](drthomas@cmu.edu)Emma Brunskill Stanford University [ebrun@cs.stanford.edu](ebrun@cs.stanford.edu) Ge Gao Stanford University [gegao@stanford.edu](gegao@stanford.edu)[ ](gegao@stanford.edu)Shivang Gupta Carnegie Mellon University [shivang@cmu.edu](shivang@cmu.edu)[ ](shivang@cmu.edu)Vincent Aleven Carnegie Mellon University [aleven@cs.cmu.edu](aleven@cs.cmu.edu) Jordan Gutterman Carnegie Mellon University [jgutterm@andrew.cmu.edu](jgutterm@andrew.cmu.edu)[ ](jgutterm@andrew.cmu.edu)Lee Branstetter Carnegie Mellon University [branstet@andrew.cmu.edu](branstet@andrew.cmu.edu)[ ](branstet@andrew.cmu.edu)Kenneth R. Koedinger Carnegie Mellon University [koedinger@cmu.edu](koedinger@cmu.edu) ABSTRACT Hybrid human-AI tutoring, where technology and humans jointly facilitate student learning, can be more beneficial than AI-only tutoring. However, preliminary evidence suggests that lower-performing students derive greater benefit from human-AI tutoring than higher-performing students. As such, this study evaluates whether a differentiated tutoring policy can effectively support both groups: human tutors initiate support for lower-performing students, while higher-performing students receive reactive, on-demand support. Using their within-grade median state test scores, we assigned 635 students (grades 5 − 8) to receive proactive (< median) or reactive (≥ median) tutoring. Using a difference-in-discontinuities (DiDC) design, we compare outcomes across two time periods: fall (AI-only tutoring for all) and spring (proactive-reactive human-AI tutoring) . This quasi-experimental design isolates the effects of proactive-reactive tutoring approaches by comparing the discontinuity in spring outcomes to the fall baseline, where no such discontinuity existed. Using data around the cutoff (Imbens-Kalyanaraman criterion), we find significant overall improvements from human-AI tutoring compared to AI-only baseline: 25% increase in time on task, 36% in skill proficiency, and 61% in academic growth (standardized MAP test) . Between proactive and reactive tutoring, we find comparable improvements in time-on-task and skill proficiency (i.e., no significant difference) . We also did not find a significant difference in MAP growth at the cutoff itself, however, proactive tutoring, on average, showed marginally higher MAP growth (75%, p = .065) than reactive tutoring, i.e. , proactive tutoring was more beneficial to students farther below the cutoff and helped narrow achievement gaps. Our findings provide evidence that differentiated human-AI tuAshish Gurung, Ge Gao, Boyuan Guo, Alex Houk, Erin Gatz, Danielle R Thomas, Shivang Gupta, Lee Branstetter, Emma Brunskill, Vincent Aleven, Kenneth Koedinger, and Jordan Gutterman. Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs. In Anthony Botelho, Maria Mercedes T. Rodrigo, Adish Singla, Hiroaki Ogata, Hyojeong So, and Young Hoan Cho (eds.) Proceedings of the 19th International Conference on Educational Data Mining, Seoul, Republic of Korea, June, 2026, pp. 130–142. International Educational Data Mining Society (2026) .© 2026 Copyright is held by the author(s) . This work is distributed under the Creative Commons Attribution NonCommercial NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license. [https://doi.org/10.5281/zenodo.21039782](https://doi.org/10.5281/zenodo.21039782) toring addresses the needs of both groups, offering a practical and cost-effective strategy for scaling hybrid instruction. Keywords Human-AI Tutoring, Academic Growth, Remote Tutoring, Difference-in-Discontinuities 1. INTRODUCTION The efficacy of human tutoring is well esta

FAQs

What differentiated tutoring policy does the study evaluate?

The policy assigns human tutors to proactively support lower-performing students, while higher-performing students receive reactive, on-demand support.

How is the study designed to estimate the policy’s effects?

It uses a difference-in-discontinuities (DiDC) quasi-experimental design comparing fall (AI-only tutoring for all) to spring (proactive-reactive human-AI tutoring).

What outcomes improve with human-AI tutoring compared with AI-only baseline?

The study finds significant overall improvements, including increases in time on task, skill proficiency, and academic growth measured by standardized MAP tests.

Oct 15, 2026

Uploaded by Aria Callaghan