Ceci n’est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems
Ben Knight, Wm. Matthew Kennedy, James Edgell · 2026
Read Ceci n’est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems by Ben Knight, Wm. Matthew Kennedy, James Edgell (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26145.txt.
Introduction
Knight and four colleagues examine how language-learning feedback can fail in ways learners and teachers miss. Such failures can reinforce misconceptions over long use. Drawing on part of a benchmark for language education, they analyze failures across six dimensions of good feedback. These run from diagnostic accuracy and awareness of appropriacy to prioritization, guidance, and support for self-control. Failures that look helpful but rest on flawed explanations, which they call explainability pitfalls, risk attainment, human-AI interaction, and socioaffective harms. The language-learning context amplifies those risks.
Catalog record
| CREATOR | Ben Knight, Wm. Matthew Kennedy, James Edgell |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20279 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26145
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ceci n’est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systems Ben Knight, Wm. Matthew Kennedy, James Edgell Abstract . AI-powered language learning tools increasingly provide instant, personalised feedback to millions of learners worldwide. However, this feedback can fail in ways that are difficult for learners—and even teachers—to detect, potentially reinforcing misconceptions and eroding learning outcomes over extended use. We present a portion of L2-Bench, a benchmark for evaluating AI systems in language education that includes (but is not limited to) six critical dimensions of effective feedback—diagnostic accuracy, awareness of appropriacy, causes of error, prioritisation, guidance for improvement, and supporting self-regulation. We analyse how AI systems can fail with respect to these dimensions. These failures, which we argue are conducive to "explainability pitfalls", are AI-generated explanations that appear helpful on the surface but are fundamentally flawed, increasing the risk of attainment, human-AI interaction, and socioaffective harms. We discuss how the specific context of language learning amplifies these risks and outline open questions we believe merit more attention when designing evaluation frameworks specifically. Our analysis aims to expand the community’s understanding of both the typology of explainability pitfalls and the contextual dynamics in which they may occur in order to encourage AI developers to better design safe, trustworthy, and effective AI explanations. † † footnotetext: Accepted to Misleading Impacts Resulting from AI Generated Explanations (MIRAGE) Workshop @ IUI 2026 1. Introduction Feedback in the form of explanations is a critical part of any learning system. In AI systems design, explanations have been proposed as a key mechanism through which more explainable, transparent, and ultimately accountable AI systems can be developed. If an AI system’s actions can be explained to a user, that user may, in theory, be more empowered to steer the system towards more beneficial outcomes ( 37 ) . Elsewhere, in educational spaces, explanations provided as feedback in learning contexts supports both knowledge and cognitive process dimensions of individual learning ( 3 ; 38 ) . Clear feedback improves learners’ ability to identify areas of weakness and understand target behaviours ( 36 ; 18 ; 42 ) . As it is typically delivered by trusted instructors or peers, it also facilitates social processes that are conducive to knowledge formation ( 4 ; 6 ) . In this vein, feedback is more than information transmission; it engages and motivates learners, provided it is constructive, specific, and timely ( 40 ) . These human-computer interaction and pedagogical considerations intersect in the development of digital learning systems. Digital learning system design is aware of the value and function of feedback in digital pedagogy ( 39 ; 44 ; 32 ) . Some of the first Intelligent Tutoring Systems viewed appropriate feedback as the core function of the system itself ( 30 ) . Developers of new digital learning systems—those powered by generative AI technologies—maintain this emphasis on feedback as essential to effective AI pedagogy ( 19 ) . Many indeed perceive a new opportunity: recent AI systems may be especially good at delivering constructive, specific, and timely feedback, given their reliance on powerful large language models or multimodal models that perform well on tasks critical to human interaction, such as natural language processing or image recognition ( 17 ; 43 ; 31 ; 19 ; 27 ; 26 ) . Although little evidence on the efficacy of such systems exists, many efforts to understand the real-world capabilities of AI learning systems are ongoing ( 25 ) . At the same time that we await more evidence that AI powered learning systems are indeed capable of supporting learning, and, therefore, providing effective feedback in educational contexts, several researchers and educational stakeholders have warned that AI systems may be prone to failures that can have substantial effects on digital learning ( 16 ) . Specifically, although AI tools can generate instant and apparently personalized feedback at scale, the interaction design that structures how feedback is delivered can mask flaws in its quality. These flaws are not the result any intent on the part of system designers to deceive learners (or other educational stakeholders); They occur despite considerable system post-training and fine-tuning for pedagogically appropriate subject matter, interaction design (e.g. optimizing for long conversations), and instructional goals ( 22 ) . Given that many of these failures present themselves most clearly in feedback—for example, in conversational turns that follow student completion of a learning task—they can also be exceedingly difficult to detect as those most directly exposed are the least likely to be sufficiently expert to perceive them ( 15 ; 22 ) . Failures can therefore compound over time. For all these reasons, we believe they may best be approached as “explainability pitfalls” ( 9 ) . Problematizing these kinds of failures in AI for education systems in this way has implications both for system designers hoping to improve their systems through monitoring usage data and for learners and instructors who may find that such failures mistakenly reinforce misunderstandings or weaken trust between students and instructors. Understanding when these kinds of failures may occur, with what variety, and to what effect is crucial to advancing any AI for education system design. This need is particularly pronounced in AI for language learning ( 36 ) . To this end, our paper discusses our efforts to design L2-Bench ( 7 ) , an evaluation benchmark to assess the efficacy of AI systems in language learning design, which decomposes the tasks that underpin a ‘learning experience designer’ into a hierarchical taxonomy of competencies . In this workshop paper, we focus specifically on the portion of our recently proposed taxonomy of competencies that concern the application of “giving feedback” effectively. In the first section of the paper, for each of these six areas, we briefly outline the core criteria for good feedback in language learning and comment on how AI systems may fail in each dimension. In the second section, we enumerate how AI failures discussed above may produce conditions conducive to explainability pitfalls. We also note how the specific context of language learning can create especially pronounced attainment, human-AI interaction, and other socioaffective risks that are specific to learning contexts and particularly intensified in language-learning contexts ( 28 ) . Ultimately, we aim to expand our understanding of both the typology of explainability pitfalls, and the contextual dynamics in which they may occur, lessons we believe can be generally useful to all AI system designers who are interested in safe, trustworthy, and effective AI explanations. 2. Giving Effective Feedback: a Review of Core Properties Research on effective feedback identifies several key properties that distinguish pedagogically sound feedback from superficial or harmful responses ( 36 ; 14 ) . Within L2-Bench, we operationalise the competency of "giving feedback" through a structured evaluation framework examining six critical dimensions. These extend beyond simple error correction to encompass the full range of considerations essential to effective language teaching feedback. 2.1. Diagnostic Accuracy Does the AI correctly identify the type and locus of a problem (grammar vs. lexis vs. discourse) and infer intended meaning? Effective feedback begins with accurate diagnosis—understanding not just that an error occurred, but precisely where and what kind ( 36 ) . Hallucinations and