Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping
Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong & Jundong Li · 2026
Read Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping by Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong & Jundong Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06923.txt.
Introduction
Lei and colleagues introduce Reasoning Editing for targeted changing of reasoning patterns in LLMs while sparing other pathways. Older reasoning fixes treat reasoning as one general skill, using broad training that wastes effort and misses target errors. Each edit trades Generality, working across tasks sharing a pattern, against Locality, keeping other reasoning skills intact. Their study finds the Circuit-Interference Law: interference grows with circuit overlap. Their REdit framework reshapes circuits before editing, tuning interference between patterns.
Catalog record
| CREATOR | Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong & Jundong Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55078 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06923
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong & Jundong Li Abstract Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability. Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors. We introduce Reasoning Editing , a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents a fundamental trade-off between Generality , the ability of an edit to generalize across different tasks sharing the same reasoning pattern, and Locality , the ability to preserve other reasoning capabilities. Through systematic investigation, we uncover the Circuit-Interference Law : Edit interference between reasoning patterns is proportional to the overlap of their neural circuits. Guided by this principle, we propose REdit , the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping , which directly addresses the generality-locality trade-off by disentangling overlapping circuits; (ii) Meta-Contrastive Learning , which extends transferability to novel reasoning patterns; and (iii) Dual-Level Protection , which preserves preexisting abilities by constraining reshaping update directions and regularizing task-level predictions. Extensive experiments with Qwen-2.5-3B on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consistently achieves superior generality and locality compared to baselines, with additional validation in mathematics showing broader potential. Our code is available at https://github.com/LzyFischer/REdit . 1 Introduction Large language models (LLMs) achieve state-of-the-art performance across various domains such as mathematics ( 40 ; 38 ) , law ( 7 ; 61 ) , and medicine ( 89 ; 14 ) . The success arises from their exceptional reasoning ability when executing complex instructions ( 42 ; 64 ; 68 ) . Despite this success, LLMs often produce incorrect or misleading responses ( 53 ; 27 ) driven by spurious reasoning processes, which significantly undermines their reliability and safety. For example, an LLM may correctly encode the fact that “ if a brain aneurysm is present, a CT scan will show bleeding or swelling (A \to B OR A \to C) ”, but still wrongly infer “ no bleeding implies no aneurysm ( \neg B \to \neg A) ”, risking harmful medical consequences ( 57 ) . Addressing such gaps remains a critical challenge for researchers and practitioners alike. To strengthen reasoning, researchers typically view it as one general, monolithic skill that calls for broad enhancement ( 66 ; 52 ; 65 ) . Standard approaches include fine-tuning on large reasoning corpora ( 80 ; 31 ) , reinforcement learning from human feedback (RLHF) ( 16 ; 78 ) , and sophisticated test-time prompting ( 2 ; 83 ) . However, treating the LLM’s reasoning as a monolithic ability has several drawbacks. First, overall reasoning enhancement can be difficult and expensive, demanding extensive human annotation and huge computational budgets ( 43 ; 32 ) . Second, growing evidence indicates that LLMs’ reasoning is not monolithic but can be decomposed into separable patterns ( 82 ; 28 ; 87 ; 56 ) . Indiscriminately training over every reasoning pattern fails to distinguish between those the model already handles well and those it struggles with, thus leading to inefficient use of resources and suboptimal correction of specific reasoning errors. Therefore, recent approaches have shifted towards enhancement at the level of specific reasoning trajectories or intermediate steps, which involve only a handful of reasoning patterns ( 8 ; 17 ) . However, these methods heavily rely on the model’s own self‐verification often without the model truly mastering the correct reasoning patterns, thus failing to reliably remedy reasoning errors. As a result, how to correct erroneous and inject new reasoning patterns without retraining on the whole reasoning datasets still remains an open problem. Recent work has demonstrated that specific reasoning patterns are encoded in localized parameters or neural circuits within LLMs ( 21 ; 30 ) , mirroring the way factual knowledge is stored in model weights ( 46 ; 76 ; 88 ) . Given the success of parameter-based methods for editing piecewise knowledge in LLMs ( 10 ; 47 ) , we propose a natural extension: If knowledge can be edited through parameter modification, can we analogously edit LLMs to correct flawed reasoning patterns or inject new ones? In this paper, we take an initial step toward reasoning editing, defined as the selective modification of a certain LLM’s reasoning pattern while preserving its factual knowledge and other reasoning pathways. To establish a rigorous foundation for this investigation, we focus on propositional logic (PL), where reasoning patterns can be precisely defined and systematically evaluated. Although structurally simple, reasoning editing in PL remains challenging due to two fundamental desiderata ( 25 ; 60 ) : (1) Generality , edits applied to one instance should consistently generalize to all instances with the same reasoning pattern across domains, rather than memorizing surface semantics. For example, editing the transitive rule “ A \to B,B \to C \Rightarrow A \to C ” in math should also hold in medicine. (2) Locality , edits must remain narrowly scoped, correcting the targeted inference rule without impairing the LLMs’ performance on other reasoning patterns it already handles correctly. For example, editing the spurious rule “ \neg B \to \neg A \Rightarrow A \to B ” should not affect modus tollens “ (A \to B, \neg B) \Rightarrow \neg A ”.The two desiderata constitute a trade-off as shown in Section 2.2 , whereby enhancing one dimension typically diminishes the other, thus presenting a significant dilemma. To tackle this trade-off, we first probe the mechanism underlying reasoning edits. Motivated by evidence that reasoning mechanism can be faithfully revealed by neural circuits, we conduct a systematic investigation into the relationship of edit effects and the circuit of reasoning pattern. Through this analysis, we discover a fundamental principle we term the Circuit-Interference Law : the degree to which an edit to one reasoning pattern affects another is directly proportional to the overlap between their respective neural circuits. Guided by this observation, we introduce REdit , the first framework to actively reshape circuits prior to reasoning editing, enabling controlled modulation of interference among reasoning patterns. REdit employs three key components: At its core, (1) Contrastive Circuit Reshaping directly addresses the generality–locality trade-off by disentangling overlapping circuits to reduce cross-reasoning pattern interference which improves locality while consolidating pattern-specific circuits to promote within-reasoning pattern generality. Building upon this foundation, (2) Meta-Contrastive Learning enhances transfer to broader reasoning patterns beyond those observed during reshaping and (3) Dual-Level Protection safeguards preexisting reasoning abilities by constraining reshaping update directions via soft null-space projection and regularizing prediction distributions of reasoning tasks. After reshaping, widely used LoRA-based editing ( 13 ) suffices to achieve the desired generality and locality. We conduct extensive experiments on Qwen-2.5-3B across three propositional-logic difficulty levels, showing that REdit consistently enhances generality while reinforcing locality , surpassing strong baselines. Furthermore, additional evaluations in the mathematics domain highlight REdit’s potential to generalize effectively to broader reasoning scenarios. Our contributions can be summarized as follows: • Reasoning Editing