Interpretability Can Be Actionable
Hadas Orgad, Fazl Barez, Tal Haklay, Isabelle Lee, Marius Mosbach, Anja Reusch, Naomi Saphra, Byron C Wallace, Sarah Wiegreffe, Eric Wong, Ian Tenney, Mor Geva · 2026
Read Interpretability Can Be Actionable by Hadas Orgad, Fazl Barez, Tal Haklay, Isabelle Lee, Marius Mosbach, Anja Reusch, Naomi Saphra, Byron C Wallace, Sarah Wiegreffe, Eric Wong, Ian Tenney, Mor Geva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11161.txt.
Introduction
Hadas Orgad and eleven others argue that interpretability research should be judged by actionability: whether its insights enable concrete decisions and acts beyond the field itself. Despite fast growth, work in this field has had weak real-world impact, raising doubts about its relevance.
This position paper defines actionable work through concreteness and validation, maps barriers blocking real-world impact, and names five domains where the field offers unique leverage. A framework with evaluation criteria tied to practical outcomes follows, aiming to make actionability a core goal without downplaying exploratory research. Researchers get a checklist for work that changes decisions, not just understanding.
Catalog record
| CREATOR | Hadas Orgad, Fazl Barez, Tal Haklay, Isabelle Lee, Marius Mosbach, Anja Reusch, Naomi Saphra, Byron C Wallace, Sarah Wiegreffe, Eric Wong, Ian Tenney, Mor Geva |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44020 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11161
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interpretability Can Be Actionable Hadas Orgad, Fazl Barez, Tal Haklay, Isabelle Lee, Marius Mosbach, Anja Reusch, Naomi Saphra, Byron C Wallace, Sarah Wiegreffe, Eric Wong, Ian Tenney, Mor Geva Abstract Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not translated into practical impact, raising questions about its relevance and utility. This position paper argues that the central missing ingredient is not new methods, but evaluation criteria: interpretability should be evaluated by actionability —the extent to which insights enable concrete decisions and interventions beyond interpretability research itself. We define actionable interpretability along two dimensions—concreteness and validation—and analyze the barriers currently preventing real-world impact. To address these barriers, we identify five domains where interpretability offers unique leverage and present a framework for actionable interpretability with evaluation criteria aligned with practical outcomes. Our goal is not to downplay exploratory research, but to establish actionability as a core objective of interpretability research. † † affiliationnotice: Equal contribution Figure 1: Actionability checklist for interpretability research. 1 Introduction Interpretability research seeks to explain modern machine learning systems. In recent years, it has grown into a large and active research area ( 90 ; 86 ) , driven by the intuition that understanding models should help make them more reliable, efficient, safer, and aligned with human values ( 21 ) . Despite its growth, interpretability work is often seen as lacking practical impact such as informing changes to models, training practices, deployment decisions, or policy ( 73 ; 49 ; 103 ) , motivating calls to focus on clearly demonstrable outcomes beyond “understanding” itself ( 53 ; 121 ; 93 ; 18 ) . Our framing draws in part on discussions from the ICML 2025 workshop on actionable interpretability, which aimed to foster dialogue on leveraging interpretability insights to drive tangible advancements in AI. In this paper, we argue that interpretability research should be evaluated not only by how well it explains models, but by what those explanations enable us to do. That is, interpretability should be held to a standard of actionability . We contend that the field’s impact will be strengthened if it explicitly tracks not only what we understand, but what that understanding enables us to do. We do not, however, argue that all interpretability work must immediately yield actionable outcomes, nor that purely exploratory contributions lack value. Indeed, methodological novelty and demonstrated applications are not at odds—grounding findings in real-world actions holds methods to a higher standard, providing evidence that insights reflect genuine model behavior rather than artifacts of a particular analysis. What is missing in interpretability research is not methods , but evaluation criteria : a shared framework for determining when interpretability research is successful from a practical, decision-oriented perspective. We therefore advance a framework for actionable interpretability: analyzing current limitations, identifying opportunities for impact, and suggesting practical tools to increase actionability. Figure 1 translates our central thesis into concrete steps for researchers. Scope. We consider interpretability in modern machine learning, focusing on deep learning and foundation models. Although we draw many examples from LLMs, our arguments apply broadly to any deep neural networks domain which may require explanation. This is a position paper: rather than an exhaustive survey, we propose actionability as a unifying lens for evaluating interpretability work. The paper is organized as follows: Section 2 defines actionable interpretability. Section 3 diagnoses the barriers that currently prevent interpretability from achieving real-world impact. The rest of the paper paves the way towards more actionable interpretability. Section 4 identifies opportunities for actionability. Section 5 presents a framework for categorizing actions and Section 6 discusses evaluation criteria aligned with actionability. Section 7 addresses counter-arguments. Section 8 reviews related work. Section 9 concludes with an actionable checklist for researchers, summarized in Figure 1 . 2 Defining Actionable Interpretability We consider a work 1 1 1 By “work”, we refer broadly to research or engineering contributions, including methods, models, analyses, benchmarks, and empirical studies. to be interpretability-oriented if it aims to explain or analyze an AI model—for example, works that analyze model representations, explain specific behaviors or capabilities, or discover internal mechanisms. Having this distinction, we provide the following definition: Actionable Interpretability An interpretability-oriented work is actionable if it produces insights about an AI model that inform or guide actions toward non-interpretability objectives. Insights are outputs of interpretability work: findings about how models represent or process inputs, explanations of internal mechanisms, or methods that clarify behavior. Actions (toward non-interpretability objectives) are decisions made by humans in response to interpretability insights that would not have been taken otherwise. These fall outside the scope of interpretability itself and ideally lead to concrete improvements such as enhanced performance, better-calibrated trust, or improved safety. 2.1 Dimensions of Actionability In practice, actionability is more fine-grained and not binary. Interpretability-oriented work can support different levels of actionability, which we characterize along two key dimensions: concreteness and validation . Concreteness captures how precisely an action is articulated. At the low end are vague suggestions (“could inform safety research”) or no suggestions at all; at the high end are exact specifications with implementation details. Validation captures empirical support for an action’s utility. At the low end, actions are untested hypotheses; at the high end, they are systematically evaluated with quantitative or qualitative evidence of meaningful outcomes beyond interpretability research itself. Together, these dimensions span a space for situating interpretability work (illustrated in Figure 3 in the Appendix): Low concreteness, low validation : Work in this region recommends no specific actions to validate. The insights from this work may, however, inform future work by providing a starting point that others can build upon and test. For example, 45 ’s key-value memory view of MLPs directionally motivated subsequent work on knowledge localization and model editing. 124 , 34 and others laid groundwork for circuit-based analysis. While not the emphasis of this paper, such exploratory work is imperative to drive the field forward. High concreteness, low validation : Concrete actions proposed but not empirically validated—e.g., approaches for verifying scientific models to build trust in their predictions ( 69 ; 77 ; 41 ) or optimizing model deployment and training ( 135 ; 32 ) . High concreteness, high validation. Precise specifications with demonstrated utility, informed by interpretability insights drawn either from the work itself or prior work. Examples include model editing methods leveraging the MLP key-value store view ( 87 ; 124 ; 11 ; 40 ) , are based on sparse-auto-encoders ( 51 ; 15 ) or insights into the role of cross-attention layers ( 100 ; 43 ) . Representation finetuning ( 127 ) , an alternative to LoRA-based methods, was inspired by interpretability findings. 110 use concept vectors to uncover novel chess concepts transferable to human players. 9 analyzed internal activations during a safety audit of Claude. 3 Why Interpretability Isn’t (Yet) Actionable Despite growing interest, several barriers limit interpretability’s real-world impact: misaligned incentives, methodological limitations, and deployment challenges. These reinforce a cycle where actionability is not prioritized, methods lack validation, and deployment yields little feedback. The rest of the paper discusses how to advance actionable interpetability despite these limitations. 3.1 Misaligned Incentives The interpretability community does not