SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing
Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun · 2026
Read SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing by Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10831.txt.
Introduction
Mingxu Zhang and five colleagues present SLIM, a framework that steers language-model molecule editors toward chosen properties without retraining them. Molecular editing tweaks a starting compound to improve traits like solubility while keeping its core, yet property signals tangle inside dense hidden states, so many edits fail or backfire.
SLIM splits hidden states into sparse, property-aligned features with a sparse autoencoder and gated importance, then steers only the features tied to the target trait. On the MolEditRL benchmark across four model designs and eight properties, gains reach more than forty points over baselines. Drug and materials teams get interpretable control knobs for edits that previously ran blind.
Catalog record
| CREATOR | Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61766 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10831
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun Abstract Large language models possess strong chemical reasoning capabilities, making them effective molecular editors. However, property-relevant information is implicitly entangled across their dense hidden states, providing no explicit handle for property control: a substantial fraction of edits fail to improve or even degrade target properties. To address these issues, we propose SLIM (Sparse Latent Interpretable Molecular editing), a plug-and-play framework that decomposes the editor’s hidden states into sparse, property-aligned features via a Sparse Autoencoder with learnable importance gates. Steering in this sparse feature space precisely activates property-relevant dimensions, improving editing success rate without modifying model parameters. The same sparse basis further supports interpretable analysis of editing behavior. Experiments on the MolEditRL benchmark across four model architectures and eight molecular properties show consistent gains over baselines, with improvements of up to 42.4 points. 1 Introduction Molecular editing, the task of modifying a given molecule to optimize specific properties while preserving its core structure, is a fundamental step in drug discovery and materials science ( 10 ; 13 ) . In real-world pharmaceutical development, a promising lead compound often requires iterative structural refinements to improve druglikeness, synthetic accessibility, binding affinity, or other pharmacological properties. The ability to perform such targeted modifications efficiently and reliably holds great potential for accelerating the drug design pipeline. Early computational approaches to molecular optimization have relied on reinforcement learning ( 19 ; 33 ) and diffusion models ( 16 ; 11 ) . Recently, large language models (LLMs) have demonstrated strong potential for molecular editing ( 30 ; 4 ; 7 ; 21 ) . LLMs acquire rich chemical priors, including structure–activity relationships, functional group chemistry, and pharmacophore patterns, through pretraining on massive corpora of chemical literature ( 32 ; 31 ) . Post training methods then bridge this chemical knowledge with SMILES-level structural transformations, positioning LLM-based editors to produce chemically meaningful modifications grounded in domain knowledge. Despite these advances, existing LLM-based editors provide no explicit handle for property control. Property-relevant information is implicitly entangled across the model’s dense hidden states, making it impossible to selectively amplify a target property without retraining. As a result, supervised fine-tuning offers only coarse control: a substantial fraction of generated edits fail to improve, or even degrade, the target property. Improving a specific property currently requires collecting new paired data and retraining the model, a costly process that scales poorly across properties and models. To address this, we propose SLIM, a plug-and-play framework that decomposes the editor’s hidden states into sparse, property-aligned features via a task-oriented Sparse Autoencoder (SAE) ( 3 ; 2 ) . Unlike standard SAEs trained for reconstruction alone, our SAE is jointly trained with contrastive, predictive, and gradient-alignment objectives that shape the sparse basis to be property-aware. Per-property Importance Gates then learn to select, from this shared basis, the subset of features most relevant to each target property. To steer the model, we project a gradient-derived causal direction through the top- k selected features of the SAE, producing a sparse steering vector that precisely activates property-relevant dimensions at inference time without modifying model parameters. The sparse basis further supports interpretable analysis, as individual SAE features encode recognizable chemical semantics. In summary, our contributions are as follows: • We propose SLIM, a plug-and-play framework that decomposes dense LLM hidden states into sparse, property-aligned features for property-directed steering, requiring no retraining of the base molecular editor. • We introduce task-oriented SAE with contrastive, predictive, and gradient-alignment objectives and per-property Importance Gates, ensuring that the learned features are causally aligned with target properties. • Experimental results on the MolEditRL benchmark across four model architectures and eight molecular properties show consistent improvements of up to 42.4%. Case studies further show that individual SAE features encode interpretable chemical semantics. 2 Related Work Molecular Editing. Molecular editing aims to modify a given molecule to optimize target properties while preserving structural similarity. Early approaches use reinforcement learning with scoring functions ( 19 ) or combine discrete diffusion with RL ( 33 ) . More recently, LLM-based methods have shown strong results. DrugAssist ( 30 ) fine-tunes LLaMA-2 ( 28 ) on paired molecular data, framing editing as instruction following. GeLLM3O ( 4 ) extends this to LLaMA-3 ( 18 ) and Mistral ( 12 ) , demonstrating cross-model transferability. MolGen ( 7 ) adopts a BART-based ( 17 ) encoder-decoder trained on SELFIES ( 14 ) with chemical feedback. These methods optimize molecular properties through training but offer no mechanism to interpret or steer the learned representations at inference time. Molecular Representation Learning. Understanding how models represent molecular structure and properties is central to interpretable editing. ChemATP ( 32 ) introduces a training-free chemical reasoning framework that leverages LLM internal representations. AtomDisc ( 31 ) proposes atom-level tokenization that reveals interpretable structure–property associations within LLM hidden states. More broadly, the linear representation hypothesis ( 20 ) suggests that high-level concepts are encoded as directions in activation space, motivating methods that extract and manipulate these directions for controllable generation. Sparse Autoencoders and Activation Steering. Neural networks represent more features than they have dimensions through superposition ( 5 ) . Sparse autoencoders (SAEs) recover these latent features by learning an overcomplete basis ( 25 ; 3 ; 2 ) . 27 show that SAE features scale to large models and capture semantically meaningful concepts. Gated SAEs ( 22 ) improve feature quality by separating gating from magnitude estimation, and a recent survey ( 8 ) covers SAE architectures comprehensively. On the steering side, activation addition ( 29 ) modifies model behavior by adding direction vectors to hidden states during inference. Contrastive activation addition ( 23 ) computes directions as mean differences between positive and negative activations, and representation engineering ( 34 ) extends this with a top-down approach. However, existing SAEs are trained with reconstruction objectives alone and used only for post-hoc analysis, while steering methods rely on mean-difference directions that we show fail for molecular property control (§ 4.3 ). SLIM bridges these two lines by integrating task-oriented SAE training with gradient-based steering, producing sparse property-aligned features that enable precise, training-free property control. 3 Method SLIM operates as a plug-and-play module on top of any molecular editing LLM. The pipeline consists of three stages: (1) a layer scan to identify the optimal intervention point, (2) task-oriented SAE training with gradient alignment, and (3) inference-time activation steering via interpretable sparse directions. Figure 1 illustrates the overall framework. Figure 1: Overview of the SLIM framework. Stage 1 : Ridge probes scan all layers to identify the optimal intervention point l^{*} . Stage 2 : A task-oriented SAE is trained at layer l^{*} with four objectives: (A) sparse reconstruction, (B) supervised property prediction via per-property Importance Gates, (C) contrastive alignment of importance-gated sparse codes, and (D) gradient alignment to ensure the SAE basis faithfully represents causal steering directions. Stage 3 : At inference time, a sparse steering vector is added to the residual stream at layer l^{*} , directing the model toward improved molecular properties without modifying model parameters. 3.1 Layer Scan As shown in Figure 1 (Stage 1), different layers encode different levels of