AI Knows Best? The Paradox of Expertise, AI-Reliance, and Performance in Educational Tutoring Decision-Making Tasks

Eason Chen, Jeffrey Li, Scarlett Huang, Xinyi Tang, Jionghao Lin, Paulo Carvalho, Kenneth Koedinger · 2025

Read AI Knows Best? The Paradox of Expertise, AI-Reliance, and Performance in Educational Tutoring Decision-Making Tasks by Eason Chen, Jeffrey Li, Scarlett Huang, Xinyi Tang, Jionghao Lin, Paulo Carvalho, Kenneth Koedinger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16772.txt.

Introduction

Eason Chen and co-authors study tutors judging AI praise in a 2025 education paper. Tutors must decide if a praise reply to a student is apt. AI help can show text reasons or highlight parts inline. The team test both styles with seasoned tutors and with novices.

Neither style raised scores in a clear way. Both styles cost users more time instead of saving it. Skill level shaped how people leaned on the AI. The results expose a tension between expertise, design, and speed. Schools buying AI tools should weigh help that slows experts down. Good design must aid judgment without taxing time.

Catalog record

CREATOREason Chen, Jeffrey Li, Scarlett Huang, Xinyi Tang, Jionghao Lin, Paulo Carvalho, Kenneth Koedinger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-16772

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AI Knows Best? The Paradox of Expertise, AI-Reliance, and Performance in Educational Tutoring Decision-Making Tasks Eason Chen, Jeffrey Li, Scarlett Huang, Xinyi Tang, Jionghao Lin, Paulo Carvalho, Kenneth Koedinger Abstract We present an empirical study of how both experienced tutors and non-tutors judge the correctness of tutor praise responses under different Artificial Intelligence (AI)-assisted interfaces types of explanation (textual explanations vs. inline highlighting). We first fine-tuned several Large Language Models (LLMs) to produce binary correctness labels and explanations, achieving up to 88% accuracy and 0.92 F1 score with GPT-4. We then let the GPT-4 models assist 95 participants in tutoring decision-making tasks by offering different types of explanations. Our findings show that although human-AI collaboration outperforms humans alone in evaluating tutor responses, it remains less accurate than AI alone. Moreover, we find that non-tutors tend to follow the AI’s advice more consistently, which boosts their overall accuracy on the task—especially when the AI is correct. In contrast, experienced tutors often override the AI’s correct suggestions and thus miss out on potential gains from the AI’s generally high baseline accuracy. Further analysis reveals that explanations in text reasoning will increase over-reliance and reduce underreliance, while inline highlighting does not. Moreover, neither explanation style actually has a significant effect on performance and costs participants more time to complete the task instead of saving time. Our findings reveal a tension between expertise, explanation design, and efficiency in AI-assisted decision-making, highlighting the need for balanced approaches that foster more effective human-AI collaboration. 1 Introduction Many educational tasks—such as grading [ 10 , 11 ] , assessment [ 1 ] , and providing feedback [ 5 , 7 , 12 ] —require educators to make critical judgments about student performance and progress. These tasks are inherently decision-making processes: teachers assess student outputs against learning objectives, deciding how best to score, comment on, or otherwise respond to student work. As educational settings become more complex and class sizes grow, educators often face increasing time and cognitive demands to maintain fair and consistent evaluations [ 10 , 11 , 8 ] . In the present era, Artificial Intelligence (AI) significantly aids human decision-making. Studies show that AI recommendations can improve speed and performance as well as introduce reliance in tasks such as question-answering [ 18 ] , logical reasoning [ 2 , 16 ] , sentiment analysis [ 2 ] , and image annotation [ 13 ] . Nevertheless, prior work predominantly employed traditional machine-learning methods for non-educational tasks. Little research has examined how the latest Large Language Models (LLMs) could support decision-making in educational contexts, specifically in tutoring. Unlike these tasks, tutoring requires accountability and specialized pedagogical knowledge (e.g., how to give constructive feedback). Given the context-dependent nature of AI’s impact, applying findings from non-educational domains directly to tutoring settings with LLMs poses inherent risks. These circumstances create a strong incentive to incorporate AI tools that reduce educators’ cognitive load and help them make effective decisions or feedback [ 12 ] . For instance, AI-driven feedback could help educators quickly assess how well they are motivating or guiding students. Yet, as discussed above, research on human-AI decision-making in education, especially in tutoring scenarios, remains limited. This paper seeks to close this gap by investigating how LLMs can support decision-making in judging effective tutor responses, focusing on deciding whether certain praise or feedback is appropriate. Specifically, we address the following R esearch Q uestions: RQ1 : How accurately can an LLM evaluate the appropriateness of tutor-praising responses, and how does its performance compare to (a) humans working alone and (b) human-AI collaboration? RQ2 : How do differing levels of tutor expertise and various AI explanation styles (e.g., textual reasoning vs. inline highlighting) influence human performance—specifically in terms of accuracy, time efficiency, and reliance on AI? Our Contribution. In response to those research questions, we find that an LLM can achieve up to 88% accuracy and 0.92 F1 score in evaluating tutor-praising responses, surpassing both humans alone and human-AI collaboration. We also developed two explanation interfaces and observed that textual reasoning fosters greater reliance on AI; further, humans spend more time when collaborating with AI. Finally, while novices boost their overall accuracy by trusting the AI’s generally correct suggestions (yet over-rely when it is wrong), experienced tutors often override correct AI advice, resulting in under-reliance. 2 Related Work Prior research on AI-assisted decision-making has produced mixed results, underscoring that its benefits depend on contextual factors. On the one hand, studies indicated that humans are sometimes misled by AI when it makes errors, especially if explanations are overly complex or fail to clarify uncertainties [ 18 , 4 , 6 ] . On the other hand, well-designed explanations can bolster users’ ability to judge the correctness of AI outputs. For instance, [ 13 ] found that causal explanations help participants recognize AI unreliability when the AI is wrong, reducing blind reliance. However, if the AI’s decision is correct but the explanation is flawed, most participants do not immediately doubt the overall judgment; rather, they attempt to reconcile the erroneous explanation. Notably, visually misleading cues often carry greater persuasive power than textual errors, and combining incorrect visual and textual explanations can create a “double misdirection” effect [ 13 ] . If, however, visual and textual explanations conflict, participants are more likely to question the AI’s decision. Despite these challenges, well-designed AI systems can complement human decision-makers, achieving performance that neither humans nor AI alone can match [ 2 ] . Yet, research also shows that humans frequently either over-rely on AI even when it is wrong or under-rely on it even when AI is correct, often missing the “appropriate reliance” that leverages AI’s suggestion when it is correct and rejecting it when it errs [ 19 , 18 , 4 , 6 ] . Even when human-AI teams outperform humans working by themselves, they often fail to exceed AI-only performance. One key to boosting human-AI collaboration performance is enhancing AI explainability. For example, explanation designs that integrate the AI’s reasoning within the question interface (e.g., inline highlights) reduce cognitive load and help users better assess correctness [ 2 , 18 ] . Additionally, users’ reliance on AI may vary according to their task expertise. For instance, [ 14 ] find that non-experts who are less equipped to gauge correctness are more susceptible to over-reliance. In this study, we aim to explore how different explanation methods, particularly highlighting and text explanations, can help improve human-AI performance in evaluating the effectiveness of tutoring feedback. 2.1 Evaluate Tutor Giving Effective Praise with AI According to prior work [ 17 ] , giving effective praise involves genuine and timely acknowledgment of a student’s specific strengths, avoiding overused expressions that lose impact. Rather than saying “good job,” effective praise highlights the student’s accomplishment by connecting it to effort rather than inherent ability. Clear, truthful, and immediate praise fosters greater motivation, resilience, and engagement. Prior work has employed AI to classify tutor responses like using a BERT-based Named Entity Recognition (NER) model to classify the effective praise [ 12 ] and obtained up to 73% accuracy and 0.81 in F1 score. They found that the model worked well for effort-related praise but it struggled with outcomes-based

Read the complete text