Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs
Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li · 2025
Read Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs by Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21044.txt.
Description
Reinforcement learning fine-tuning makes language models better at reasoning than supervised fine-tuning alone, but the internal reason was unclear. Honglin Zhang and three co-authors look inside the models in a 2025 study. They borrow a technique called edge attribution patching to map the models' internal circuitry.
Across several model families and mathematics datasets, they find two robust effects of reinforcement learning tuning. Internal activations grow stronger, and the circuitry involved becomes more diverse. Supervised tuning does not produce the same pattern. The findings suggest reinforcement learning does not merely polish outputs; it recruits a broader, more active set of internal pathways. That picture helps explain why the method generalizes across models with different starting characteristics.
Catalog record
| CREATOR | Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21044