Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety
Seongmin Lee, Aeree Cho, Grace C. Kim, ShengYun Peng, Mansi Phute, Duen Horng Chau · 2025
Read Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety by Seongmin Lee, Aeree Cho, Grace C. Kim, ShengYun Peng, Mansi Phute, Duen Horng Chau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05451.txt.
Introduction
Reading a language model's mind can make it safer, Seongmin Lee and co-authors survey in a 2025 paper. Interpretation tools expose why a model produced an unsafe answer. Past surveys treated interpretation and safety apart, leaving the bridge unmapped.
Their framework links interpretation methods to the safety fixes they inform and the tools that run them. About seventy works are sorted by stage of the model's workflow. Open problems close the survey. Builders who can see a failure's cause inside the model can patch the cause, not just filter the output.
Catalog record
| CREATOR | Seongmin Lee, Aeree Cho, Grace C. Kim, ShengYun Peng, Mansi Phute, Duen Horng Chau |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-05451
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety Seongmin Lee, Aeree Cho, Grace C. Kim, ShengYun Peng, Mansi Phute, Duen Horng Chau Abstract As large language models (LLMs) see wider real-world use, understanding and mitigating their unsafe behaviors is critical. Interpretation techniques can reveal causes of unsafe outputs and guide safety, but such connections with safety are often overlooked in prior surveys. We present the first survey that bridges this gap, introducing a unified framework that connects safety-focused interpretation methods, the safety enhancements they inform, and the tools that operationalize them. Our novel taxonomy, organized by LLM workflow stages, summarizes nearly 70 works at their intersections. We conclude with open challenges and future directions. This timely survey helps researchers and practitioners navigate key advancements for safer, more interpretable LLMs. Figure 1: Visual overview our survey’s unified framework bridging LLM interpretation and safety, summarizing the connections between safety-focused interpretation methods ( §3 ), the safety enhancements they inform ( §4 ), and the tools that operationalize them ( §5 ). We organize surveyed research using a novel taxonomy based on LLM workflow: training, input tokens, inference, and generation. 1 Introduction Large language models (LLMs) have shown remarkable capabilities across many domains ( Yu et al. 2022 ; Singhal et al. 2023 ; Sadybekov and Katritch 2023 ; Wang et al. 2024e ) , but their outputs can be unsafe, posing significant challenges for real-world use ( Tang and Li 2025 ) . In response, researchers have developed interpretation methods and tools to better understand the mechanisms behind unsafe generation and to improve safety ( McGrath and Jonker 2024 ; Ajwani et al. 2024 ) . Bridging Interpretation and Safety. As interest in both LLM interpretation and safety grows, a unifying survey that bridges the two becomes essential. Existing surveys largely focus on either interpretation Zhao et al. 2024b ; Ferrando et al. 2024 ; Zhao et al. 2024c ; Calderon and Reichart 2025 or safety Huang et al. 2023b ; Ayyamperumal and Ge 2024 ; Shi et al. 2024b ; Chua et al. 2024 ; Ma et al. 2025 , without addressing how interpretation can enhance safety or inform users to operationalize such enhancements. Yet, both safety and human understanding are core motivations for interpretation research Ferrando et al. 2024 . Some works suggest safety as a downstream application of interpretation Wu et al. 2024b or explore only limited intersections between interpretation and safety Bereska and Gavves 2024 . Moreover, emerging directions like self-reasoning interpretation, where LLMs explain their own behaviors, remain underexplored Zhao et al. 2024b ; Singh et al. 2024a ; Calderon and Reichart 2025 . Our survey fills these critical gaps by contributing: • The first survey bridging LLM interpretation and safety ( Fig. 1 ). Our timely survey introduces a unified framework for summarizing safety-focused interpretation methods ( §3 ), the safety enhancement strategies they inform ( §4 ), and the practical tools that operationalize such enhancements ( §5 ). Although improving safety and human understanding is often cited as a motivation for interpretation research Doshi-Velez and Kim 2017 ; Madsen et al. 2022 ; Zhao et al. 2024b ; Zhao et al. 2024c ; Ferrando et al. 2024 , this connection has not been systematically surveyed until now. • Novel taxonomy of LLM interpretation methods organized by LLM workflow focus ( Fig. 1 ): training process ( §3.1 ), input prompts ( §3.2 ), inference ( §3.3 ), and generation for self-reasoning ( §3.4 ). These taxonomy categories anchor connections to six safety enhancement strategies ( §4 ) and four tool types ( §5 ), summarizing nearly 70 works at their intersections ( Table 1 ), 1 1 1 Table 2 and Table 3 in the appendix extends Table 1 to include safety-oriented interpretation methods not yet leveraged for safety enhancements or tool use. including emerging areas like self-reasoning for interpretation not covered in prior surveys. • Distill open problems and challenges to guide future NLP research and raise awareness of unresolved safety issues ( §6 ). These include defending against interpretation-informed attacks, evaluating interpretation, using training attribution for safety, designing user-centered presentation of interpretation, and refining safety dimensions. 2 Survey Methodology We focus on interpretation methods addressing safety issues in autoregressive Transformer-based generative LLMs Vaswani et al. 2017 ; Brown et al. 2020 , among the most widely used and studied architectures Huang et al. 2023c ; Veeramachaneni 2025 . We adopt the established definition of interpretation as extracting knowledge from an LLM to explain its behaviors in human-understandable terms Doshi-Velez and Kim 2017 ; Räuker et al. 2023 ; Hsieh et al. 2024 ; Singh et al. 2024a . We exclude methods requiring major architectural changes, as they hinder practical integration Ludan et al. 2023 ; Tan et al. 2024 ; Sun et al. 2025 . We focus on four major safety concerns commonly studied in interpretation research Qian et al. 2024 : 2 2 2 We focus on risks of direct harm or misuse, excluding general performance issues like out-of-distribution robustness. hallucination , jailbreaks and harmfulness 3 3 3 We do not consider user intent (malicious or benign) as interpretation methods reveal mechanisms behind unsafe outputs regardless of intent. , bias , and privacy leakage . We curated nearly 70 works from top venues in machine learning, natural language processing, human-computer interaction, and visualization, with a focus on understanding, enhancing, and communicating LLM safety through interpretation ( Table 1 ). Table 1: Overview of representative works at the intersections of safety-focused interpretation ( §3 ), safety enhancements they inform ( §4 ), and tools operationalizing them ( §5 ). Each row is one work; each column corresponds to a technique or tool. Safety issues, techniques, and tools addressed by a work are indicated by a colored cell. SAFETY TYPE § 3 INTERPRET FOR SAFETY § 4 ENHANCE SAFETY § 5 PRACTICAL TOOLS VENUE Work Hallucination Jailbreak & Harm Bias Privacy Leakage 3.1 Training Attrib. 3.2 Input Token 3.3.1 Probe Latent 3.3.2 Perturb Comp 3.3.3 Decipher Latent 3.4 Self-Reason 4.1 Attn. to Rel. Token 4.2.1 Steer Latent Vec 4.2.2 Modulate Neuron 4.2.3 Edit Model 4.3 Verify & Output 4.4 Output w. Reason Ease Impl. 5.1 TDA Vis 5.2 Token Vis 5.3 Latent Vec Vis 5.4 Neuron Vis Publication Lee et al. 2025a AAAI Hazra et al. 2024 EMNLP Zhao et al. 2024e ArXiv Qian et al. 2024 ACL Sarti et al. 2023 ACL Mishra et al. 2025 TVCG Vig 2019 ACL Wang et al. 2025b ArXiv Dale et al. 2023 ACL Chuang et al. 2024 EMNLP Pan et al. 2025a ArXiv Li et al. 2023b ArXiv Tenney et al. 2020 EMNLP Zhang et al. 2024b ICLR Zhu et al. 2024 ArXiv Duan et al. 2024 ArXiv Ball et al. 2024 ArXiv Li et al. 2025c COLING Wang et al. 2024a ArXiv Yang et al. 2024a ACL Bhattacharjee et al. 2024 SafeGenAI Chu et al. 2024 CCS Rimsky et al. 2024 ACL Singh et al. 2024b ICML Zhang et al. 2024c ACL Li et al. 2023a NeurIPS Turner et al. 2023 ArXiv Gao et al. 2024 ArXiv Shen et al. 2024 ICLR Han et al. 2025 ArXiv Hernandez et