Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs

Ashish Gurung Ge Gao Jordan Gutterman, Danielle R. Thomas Shivang Gupta Lee Branstetter, Emma Brunskill Vincent Aleven Kenneth R. Koedinger · 2026

Read Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs by Ashish Gurung Ge Gao Jordan Gutterman, Danielle R. Thomas Shivang Gupta Lee Branstetter, Emma Brunskill Vincent Aleven Kenneth R. Koedinger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11155.txt.

Introduction

Ashish Gurung and eight colleagues test whether differentiating human tutor roles by student need improves hybrid human-AI tutoring. Hybrid tutoring can beat AI-only tutoring, but early evidence suggests lower-performing students benefit more, raising the question of how to support both groups.

Using within-grade median state test scores, 635 students in grades 5 through 8 received proactive human-initiated support if below median or reactive on-demand support if at or above. A difference-in-discontinuities design compares fall AI-only tutoring against spring proactive-reactive tutoring, isolating the policy effect around the cutoff. Results show human-AI tutoring beats AI alone by a clear overall margin. Tutoring designers get evidence that differentiated human roles beat one-size-fits-all support.

Catalog record

CREATORAshish Gurung Ge Gao Jordan Gutterman, Danielle R. Thomas Shivang Gupta Lee Branstetter, Emma Brunskill Vincent Aleven Kenneth R. Koedinger
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54020
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11155

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving Hybrid Human-AI Tutoring by Differentiating Human Tutor Roles Based on Student Needs Ashish Gurung Ge Gao Jordan Gutterman, Danielle R. Thomas Shivang Gupta Lee Branstetter, Emma Brunskill Vincent Aleven Kenneth R. Koedinger Abstract Hybrid human-AI tutoring, where technology and humans jointly facilitate student learning, can be more beneficial than AI-only tutoring. However, preliminary evidence suggests that lower-performing students derive greater benefit from human-AI tutoring than higher-performing students. As such, this study evaluates whether a differentiated tutoring policy can effectively support both groups: human tutors initiate support for lower-performing students, while higher-performing students receive reactive, on-demand support. Using their within-grade median state test scores, we assigned 635 students (grades 5 − 8 5-8 ) to receive proactive ( < m ​ e ​ d ​ i ​ a ​ n <median ) or reactive ( ≥ m ​ e ​ d ​ i ​ a ​ n \geq median ) tutoring. Using a difference-in-discontinuities (DiDC) design, we compare outcomes across two time periods: fall (AI-only tutoring for all) and spring (proactive-reactive human-AI tutoring). This quasi-experimental design isolates the effects of proactive-reactive tutoring approaches by comparing the discontinuity in spring outcomes to the fall baseline, where no such discontinuity existed. Using data around the cutoff (Imbens-Kalyanaraman criterion), we find significant overall improvements from human-AI tutoring compared to AI-only baseline: 25% increase in time on task, 36% in skill proficiency, and 61% in academic growth (standardized MAP test). Between proactive and reactive tutoring, we find comparable improvements in time-on-task and skill proficiency (i.e., no significant difference). We also did not find a significant difference in MAP growth at the cutoff itself, however, proactive tutoring, on average, showed marginally higher MAP growth ( 75 % , p = .065 75\%,\,p=.065 ) than reactive tutoring, i.e., proactive tutoring was more beneficial to students farther below the cutoff and helped narrow achievement gaps. Our findings provide evidence that differentiated human-AI tutoring addresses the needs of both groups, offering a practical and cost-effective strategy for scaling hybrid instruction. \additionalauthors Additional authors: Alex Houk ( [email protected] ), Erin Gatz ( [email protected] ), and Boyuan (Bill) Guo ( [email protected] ). keywords Human-AI Tutoring, Academic Growth, Remote Tutoring, Difference-in-Discontinuities 1 Introduction The efficacy of human tutoring is well established in educational research [ 43 ] , with meta-analyses showing it to be particularly beneficial for lower-performing and socioeconomically disadvantaged students [ 19 , 44 ] . However, scaling human tutoring remains a persistent challenge due to cost and logistical constraints. Automated alternatives that replicate key elements of human tutoring, such as Intelligent Tutoring Systems (ITS) and Adaptive Tutoring Systems (ATS), have also proven effective [ 20 , 36 , 56 ] . At the same time, Steenbergen-Hu and Cooper’s [ 54 ] meta-analysis found that automated systems were less effective for lower-performing students than for higher-performing peers, suggesting that some learner needs go beyond the affordances of automated systems (e.g., accountability and motivation). Recently, hybrid human-AI tutoring models have emerged as a potential cost-effective alternative to high-impact human tutoring [ 1 , 11 , 14 , 26 , 55 ] . This hybrid approach combines the continuous adaptive support of AI tutors with the supplemental motivational and conceptual support from human tutors [ 1 ] . Initial evaluations by Chine et al. [ 14 ] reported nearly double the learning gains among students receiving human-AI tutoring compared to a business-as-usual control group. Similar evaluations by Gurung et al. [ 26 ] found that human-AI tutoring was more beneficial for lower-performing students than their higher-performing peers, suggesting that the benefits of human support vary by performance level. One possible explanation for this differential benefit is that higher-performing students are better equipped to overcome challenges and utilize the support provided by tutoring systems [ 36 ] , whereas lower-performing students may benefit from additional motivational or conceptual support beyond the affordances of a tutoring system. Findings from Holstein et al. [ 31 ] support this interpretation: strategically directing teacher time and effort to struggling students increases overall learning gains for the entire group compared to students in a business-as-usual control. If tutor attention in hybrid settings is especially effective for lower-performing students, then decision policies to optimize tutor time and effort are critical. Yet there is limited evidence on how effective such policies are in real classrooms or what forms they should take (e.g., should tutors proactively initiate support for struggling students or provide reactive, on-demand assistance?). Developing effective policies is particularly important in hybrid settings where a single tutor supports multiple students during a session [ 14 , 26 ] . Educational data mining is well-positioned to address this gap, as it has demonstrated success with similar decision policies in peer tutoring [ 59 , 58 ] . In short, there exists a research gap in studying and evaluating effective and adaptive forms of tutor-student pairing in hybrid tutoring. We address this gap by examining a proactive-reactive tutoring policy in which tutors initiate support for lower-performing students, while higher-performing students work independently and receive reactive, on-demand support. To evaluate the policy’s effects on learning, we implement a Difference-in-Discontinuities design (DiDC [ 46 ] ) using prior-year state test performance as the running variable. 2 Background This section reviews the current evidence on the effectiveness of human tutoring, AI tutors, and human-AI tutoring. We also examine how researchers and practitioners have sought to scale these interventions. 2.1 Human Tutoring Bloom’s seminal 2-sigma effect [ 9 ] highlighted the impressive learning benefits of one-on-one human tutoring (under ideal conditions) compared to traditional classroom instruction. While such strong gains remain difficult to replicate in real-world settings, efforts to implement human tutoring at scale have consistently shown positive results. For instance, several meta-analyses have reported an effect size of 0.36 SD for human tutoring [ 19 , 44 ] , with Nickow et al. [ 44 ] finding tutoring most effective when delivered by certified teachers (0.59 SD). Encouragingly, support from non-professional tutors also resulted in meaningful learning gains (0.21 SD). The benefits of human tutoring are generally attributed to the personalized motivational and conceptual support provided by tutors [ 19 , 44 , 56 , 9 , 28 ] . VanLehn’s meta-analysis (see [ 56 ] ) provides a comprehensive review of human tutoring. He attributes tutor success to their ability to closely monitor learning processes, identify misunderstandings, and offer immediate feedback and scaffolding support. Immediate feedback helps tutors address uncertainty in student responses, correct errors, and provide clarification [ 23 , 22 ] . Scaffolding, in turn, enables tutors to deliver structured support that strengthens reasoning and helps learners grasp complex concepts [ 17 , 13 ] . Together, these strategies create a highly personalized learning experience tailored to the student’s task-specific needs. 2.2 AI Tutors Current human-AI tutoring research refers to automated tutoring systems as AI tutors [ 26 , 55 ] to highlight their AI use while contrasting them with their human counterparts. For instance, platforms such as MATHia (formerly Cognitive Tutor [ 5 ] ) and Lynnette [ 57 ] incorporate Symbolic AI and Bayesian Knowledge Tracing (BKT [ 16 ] ) to estimate student mastery and personalize their math practice. Others, such as W-Pal [ 50 ] and

Read the complete text