From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models
Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo · 2026
Read From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models by Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25167.txt.
Introduction
Shi and nine colleagues propose Interpretability-Guided Data Selection, a framework that turns interpretability insights into training actions for large language models. It first finds causal task features through frequency recall and interventional filtering. It then selects feature-resonant data that maximally activates those features for fine-tuning. On math, summarization and translation tasks in Gemma-2, LLaMA-3.1 and Qwen3 models, the method beat full-dataset fine-tuning on math by 17.4 percent on Gemma-2-2B while using only half the data. Analysis links stronger feature amplification to better task performance.
Catalog record
| CREATOR | Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45231 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models Ling Shi, Xinwei Wu, Xiaohu Zhao, Hao Wang, Heng Liu, Yangyang Liu, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo Abstract While mechanistic interpretability tools like Sparse Autoencoders (SAEs) can uncover meaningful features within Large Language Models (LLMs), a critical gap remains in transforming these insights into practical actions for model optimization. We bridge this gap with the hypothesis that data selection guided by a model’s internal task features is a effective training strategy. Inspired by this, we propose Interpretability-Guided Data Selection (IGDS), a framework that first identifies these causal task features through frequency recall and interventional filtering, then selects “Feature-Resonant Data” that maximally activates task features for fine-tuning. We validate IGDS on mathematical reasoning, summarization, and translation tasks within Gemma-2, LLaMA-3.1, and Qwen3 models. Our experiments demonstrate exceptional data efficiency: on the Math task, IGDS surpasses full-dataset fine-tuning by a remarkable 17.4% on Gemma-2-2B while using only 50% of the data, and outperforms established baselines focused on data quality and diversity. Analysis confirms a strong positive correlation between feature amplification and task performance improvement. IGDS thus provides a direct and effective framework to enhance LLMs by leveraging their internal mechanisms, validating our core hypothesis. 1 Introduction Large Language models (LLMs) have demonstrated increasingly superior performance across diverse downstream tasks ( 26 ; 46 ; 51 ; 24 ; 31 ; 35 ) . Recent research in mechanistic interpretability has revealed that LLMs are not entirely black boxes; instead, they contain disentangled, human-understandable components ( 14 ; 1 ) . Discoveries such as steering vectors for factual knowledge ( 12 ) and sparse features for cross-modal entities ( 27 ) have provided invaluable insights into model mechanisms. However, while these insights are powerful for analysis, a critical gap remains in translating them into practical actions for model optimization ( 32 ; 34 ) . We bridge this gap with the hypothesis that data selection guided by a model’s internal, causally-validated task features is a highly effective training strategy. To operationalize this, we propose Interpretability-Guided Data Selection (IGDS) , a framework that transforms interpretability insights into a tangible optimization pipeline. This conceptual loop, which we term Insight2Action , is visualized in Figure 1 . Our core idea is to first identify the model’s beneficial internal mechanisms and then select data that maximally activates them, which is called “Feature-Resonant Data”, to reinforce these mechanisms through fine-tuning. Figure 1: Conceptual illustration of the IGDS paradigm. The diagram depicts the closed loop from internal insight to optimization action, showing how model features are leveraged to guide data selection. Figure 2: An overview of our Interpretability-Guided Data Selection (IGDS) framework. Specifically, IGDS operationalizes this vision through a two-stage process. The first stage, Task Feature Identification , moves from a broad high-frequency recall of candidate features using an SAE to a rigorous causal intervention filtering step, which isolates the potent subset directly impacting task performance. The second stage, Feature-Based Data Scoring , then uses this validated set of causal features to construct a highly effective utility function, selecting data that “resonates” with and reinforces the model’s internal problem-solving structure. We validate IGDS on mathematical reasoning, summarization, and translation tasks across Gemma-2, LLaMA-3.1, and Qwen3 models. Our experiments show that our method achieves the highest data efficiency across all settings. Notably, on the Math task, IGDS surpasses full-dataset fine-tuning by a remarkable 17.4% on the Gemma-2-2B model while using only 50% of the data, and consistently outperforms established baselines focused on data quality and diversity. Furthermore, our analysis confirms a strong positive correlation between the targeted amplification of these internal features and the improvements in downstream task performance, providing strong mechanistic evidence for our method’s success. In summary, our key contributions are: • We propose a novel optimization strategy that leverages a model’s causally validated internal features to guide data selection, offering a direct path to efficient capability enhancement. • We introduce IGDS, a general and practical framework that transforms descriptive interpretability insights into a prescriptive pipeline for identifying and selecting high-utility training data, effectively closing the loop from analysis to optimization. • We provide extensive empirical validation across multiple models and tasks, demonstrating that IGDS surpasses both competitive baselines and full-dataset fine-tuning while utilizing only a fraction of the data. 2 Related Work Mechanistic Interpretability Mechanistic interpretability (MI) aims to reverse-engineer Large Language Models (LLMs) to uncover the causal mechanisms behind their behaviors ( 8 ; 34 ) . Recent breakthroughs have revealed that fine-grained semantic information is encoded within model representations, from neurons corresponding to specific concepts ( 29 ; 10 ) , to steering vectors that control high-level attributes like factuality and safety ( 12 ; 48 ) , and sparse features that relevant to downstream tasks ( 28 ; 36 ) . However, most MI research currently stops at the explanatory level, further exploration into how these insights can proactively build better models is often neglected ( 32 ; 34 ) . While prior research has begun to explore applications, such as controlling model behavior via inference-time interventions ( 42 ; 16 ) or refining reward models specifically for safety alignment ( 52 ; 25 ) , a general pipeline for leveraging these insights to guide the broader training process remains absent. This work aims to bridge this gap, establishing a framework that turns descriptive interpretability findings into a prescriptive guide for model optimization. Sparse Autoencoders Sparse Autoencoders (SAEs) have emerged as a pivotal tool in MI for addressing the superposition hypothesis ( 34 ) . This premise suggests that densely parameterized models frequently superimpose multiple independent semantic signals onto individual neuronal activations. Such polysemanticity obscures interpretability at both the single-unit and embedding-vector scales ( 9 ; 17 ) . By untangling these entangled representations into a sparse basis of semantically coherent dimensions, SAEs markedly improve architectural transparency ( 5 ; 13 ) . Current applications of SAEs typically focus on identifying task-specific features to enable precise, post-hoc interventions on model behavior ( 41 ; 11 ; 43 ) . While effective for changing outputs, such interventional approaches can suffer from high latency and instability in practical applications. Departing from this paradigm, we leverage SAE-identified features to guide the data selection process, establishing a new pathway from internal insight to optimization action. Data Selection Training data remains a critical determinant of model efficacy ( 37 ; 47 ; 30 ) . The paradigm for post-training data selection has shifted from prioritizing quantity to emphasizing quality ( 53 ) , giving rise to automated methods that score data based on data quality using external models ( 3 ) or self-assessed metrics ( 21 ; 44 ) . Beyond quality, data diversity is recognized as crucial for model robustness, with frameworks like ZIP proposed to balance this trade-off ( 2 ; 49 ) . However, the efficacy of these methods is scrutinized at scale, as recent studies suggest they often yield negligible gains over simple random selection ( 45 ) . Departing from prior approaches that rely on external signals and treat the model as a black box, we posit that a direct and more potent signal for data utility