Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs

Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li · 2025

Read Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMs by Honglin Zhang, Qianyue Hao, Fengli Xu, Yong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21044.txt.

Description

Reinforcement learning fine-tuning makes language models better at reasoning than supervised fine-tuning alone, but the internal reason was unclear. Honglin Zhang and three co-authors look inside the models in a 2025 study. They borrow a technique called edge attribution patching to map the models' internal circuitry.

Across several model families and mathematics datasets, they find two robust effects of reinforcement learning tuning. Internal activations grow stronger, and the circuitry involved becomes more diverse. Supervised tuning does not produce the same pattern. The findings suggest reinforcement learning does not merely polish outputs; it recruits a broader, more active set of internal pathways. That picture helps explain why the method generalizes across models with different starting characteristics.

Catalog record

CREATORHonglin Zhang, Qianyue Hao, Fengli Xu, Yong Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-21044

Browse the full library · About the Tower of Babel