SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs
Sihang (Nagi) Zhao, Kangrui Yu Youliang Yuan, Pinjia He, Hongyi Wen · 2026
Read SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs by Sihang (Nagi) Zhao, Kangrui Yu Youliang Yuan, Pinjia He, Hongyi Wen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22134.txt.
Introduction
Zhao and four colleagues name a weakness in tutoring models: pedagogical jailbreaks, where students use answer-pulling prompts to get solutions instead of guided teaching. To study it, they join safe, helpful, and pedagogical behavior with a knowledge-mastery graph and build SHAPE, a benchmark of 9,087 student-question pairs. They also offer a graph-guided tutoring pipeline that finds missing prerequisites, spots mastery gaps, and switches between teaching and solving. Tests over several models show much better safety under two jailbreak settings, with helpfulness near its ceiling.
Catalog record
| CREATOR | Sihang (Nagi) Zhao, Kangrui Yu Youliang Yuan, Pinjia He, Hongyi Wen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64820 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.22134
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SHAPE: Unifying Safety, Helpfulness and Pedagogy for Educational LLMs Sihang (Nagi) Zhao, Kangrui Yu Youliang Yuan, Pinjia He, Hongyi Wen Abstract Large Language Models (LLMs) have been widely explored in educational scenarios. We identify a critical vulnerability in current educational LLMs, pedagogical jailbreaks , where students use answer-inducing prompts to elicit solutions rather than scaffolded instructions. To enable systematic study, we unify and formalize safe, helpful, and pedagogical behaviors with a knowledge-mastery graph and introduce SHAPE , a benchmark of 9,087 student-question pairs for evaluating tutoring behavior under adversarial pressure. We propose a graph-augmented tutoring pipeline that infers prerequisite concepts from queries, identifies mastery gaps, and routes generation between instructing and problem-solving via explicit gating. Experiments across multiple LLMs show that our method yields significantly improved safety under two pedagogical jailbreak settings, while maintaining near-ceiling helpfulness under the same evaluation protocol. Our code and data are available at https://github.com/MAPS-research/SHaPE . † † footnotetext: Correspondence to <[email protected]> Figure 1: Desired educational LLM tutoring under mastery-awareness and jailbreak pressure. We illustrate six representative interactions conditioned on the student’s mastery state (bottom). When the student has not mastered prerequisite concepts, the tutor should withhold direct answers and provide guided, concept-targeted instruction (a), while remaining robust to answer-inducing prompts (b–c). When mastery is demonstrated, the tutor should permit efficient direct answers (f) and avoid redundant Socratic dialogue on already-mastered material (d, e). 1 Introduction LLMs have rapidly reshaped AI in Education (AIED) Baidoo-Anu and Ansah 2023 since the release of ChatGPT Achiam et al. 2023 . LLM-based systems are used in classrooms and learning workflows – for example, to prepare instructional materials Balaji et al. 2025 , support collaborative teaching Zhang et al. 2025 , and provide tutoring assistance to students Dinucu-Jianu et al. 2025 ; LearnLM et al. 2024 . Like other LLM deployments, educational use inherits well-known misuse risks (e.g., privacy leakage, toxic content, misinformation), motivating safeguards that constrain outputs to a “safe” range; attempts to bypass such safeguards are commonly referred to as jailbreaks Ramaswamy et al. 2020 ; Fasching and Lelkes 2025 ; Huang et al. 2025 ; Wei et al. 2023 . However, educational LLMs introduce a distinct and under-explored risk profile because of their inherently different objectives Delikoura et al. 2025 . While general LLMs are primarily optimized to solve users’ problems, an educational LLM should scaffold reasoning and support durable learning rather than maximize immediate task completion. This objective mismatch creates a new class of restricted behaviors: safe actions in general settings (e.g., directly giving final answers or completing code) can be harmful in learning contexts by eroding students’ cognitive skills Stadler et al. 2024 ; Bastani et al. 2025 or fostering unwarranted confidence Lehmann et al. 2024 . Prior work attempts to mitigate this tension by using pedagogical fine-tuning Yuan et al. 2025 , pedagogical chain-of-thought prompts OpenAI 2025a ; Google 2025 often apply Socratic questioning Liu et al. 2024 to discourage models from giving direct answers, instead guiding students through the underlying ideas. Yet these approaches can fail in two intertwined ways . First, they are insufficiently personalized: indiscriminately applying instructive, question-led tutoring to content a student has already mastered can be counterproductive Pachman et al. 2013 , increasing frustration and wasting time and attention MathAcademy 2025a . In practice, an educational LLM should dynamically balance between an instructing mode (scaffolding and probing) and a problem-solving mode (providing more direct help when appropriate). Second, even when the instructing objective is clear, students may still prefer direct answers and actively pressure models into providing them Lehmann et al. 2024 . Consequently, existing “pedagogical safeguards” are vulnerable to adversarial manipulation for cheating: models that reliably reject conventional harmful content (e.g., hate or criminal instructions) can nonetheless be turned into uncritical solution engines for homework and exams through simple prompt-based jailbreak attacks (e.g., emotional blackmail or forbidding the model from asking clarifying questions). These observations surface three concrete challenges for deploying educational LLMs responsibly. (1) Specification: there is a lack of rigorous, education-grounded definitions of safe, helpful, and pedagogical behaviors, as well as metrics derived from those definitions. Therefore, (2) Evaluation: there is no systematic assessment of how educational LLMs withstand adversarial attacks across different strengths and threat models. (3) Adaptation: practical deployment often requires a training-free method to adapt existing LLMs into real-world educational systems that can strike the instructing/problem-solving balance. To address this gap, we first formally define safe, helpful, and pedagogical behaviors of personalized educational LLMs, as well as the related jailbreaks. We then introduce SHAPE , a dataset of 9,087 student-question pairs with varied knowledge backgrounds in linear algebra. Using SHAPE together with our selected and designed jailbreak methods, we evaluate advanced open-source QwenTeam 2025 and closed-source models OpenAI 2025b ; Team et al. 2023 ; Anthropic 2025 with educational system prompt or study mode, as well as models built on pedagogical chain-of-thought or fine-tuning Dan et al. 2024 ; Liu et al. 2024 . We find that most models cannot withstand even simple prompt-based jailbreaks: they either suffer a substantial drop in safety (directly give the answer) or fail to maintain helpfulness and pedagogy while remaining safe. To fix this, we design a simple yet effective graph-augmented pipeline on knowledge mastery graph MathAcademy 2025a . Our experiments show that AI tutors built under our pipeline maintain strong safety and consistent pedagogical behavior against answer-inducing jailbreak attacks while keeping helpfulness. Our contributions can be summarized as follows: 1. We provide a unified definition for personalized educational LLMs together with their safe, helpful, pedagogical behaviors based on the knowledge mastery graph. 2. We introduce SHAPE, a benchmark designed to evaluate the safety, helpfulness, and pedagogical capabilities of educational models. Through SHAPE, we find that conventional models relying on fine-tuning or system prompts are not only susceptible to jailbreak attacks but also significantly compromise helpfulness and pedagogical quality in their pursuit of safety. 3. We propose a graph-augmented pedagogical pipeline where a parsing agent aligns queries with a knowledge graph to identify knowledge gaps. This gating mechanism allows the model to balance safety and helpfulness. 2 Related Work 2.1 Personalization in Educational LLMs While AIED predates the current generative era Stamper et al. 2024 , the emergence of LLMs has revolutionized the field. Recent work focuses on educational LLMs Dan et al. 2024 ; LearnLM et al. 2024 , which function as LLM-driven Intelligent Tutoring Systems (ITS) emphasizing personalization. However, due to the lack of formal definitions, current personalization efforts often superficially target learning styles Wan et al. 2025 , hobbies Do et al. 2025 , or coarse-grained demographics OpenAI 2025a . These approaches are limited: the validity of learning styles is widely contested Kirschner 2017 , and complex cognitive states (e.g., concentration, working memory) are difficult to simulate accurately Finn et al. 2014 . Even agent-based approaches like SocraticLM Liu et al. 2024 , which introduces a dean agent, rely on fixed response templates and fail to model the student’s actual knowledge mastery. Knowledge Tracing (KT) Piech et al. 2015 is a technique to dynamically evaluate students’ knowledge mastery state and predict their responses. Although LLM-based KT has surpassed non-LLM methods across numerous metrics and demonstrated robust capabilities in predicting student learning trajectories Neshaei et