REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing

Haitian Zhong, Yuhuan Liu, Ziyang Xu, Guofan Liu, Qiang Liu, Shu Wu, Zhe Zhao, Liang Wang, Tieniu Tan · 2025

Read REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing by Haitian Zhong, Yuhuan Liu, Ziyang Xu, Guofan Liu, Qiang Liu, Shu Wu, Zhe Zhao, Liang Wang, Tieniu Tan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18933.txt.

Introduction

Editing a fact inside a large language model often overshoots: the model starts forcing the new fact into places it does not belong. Haitian Zhong and co-authors propose REACT to keep edits on a leash. First, tailored prompts draw out the model's internal representation of the fact, and a principal component step turns it into a belief-shift vector. Then the method nudges hidden states along that vector, with a trained gate deciding whether the context calls for the edit at all.

On editing benchmarks, REACT cut overfitting on nearly every measure. It kept standard editing quality, balancing reliability, locality, and generality across varied scenarios.

Catalog record

CREATORHaitian Zhong, Yuhuan Liu, Ziyang Xu, Guofan Liu, Qiang Liu, Shu Wu, Zhe Zhao, Liang Wang, Tieniu Tan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18933

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing Haitian Zhong, Yuhuan Liu, Ziyang Xu, Guofan Liu, Qiang Liu, Shu Wu, Zhe Zhao, Liang Wang, Tieniu Tan Abstract Large language model editing methods frequently suffer from overfitting, wherein factual updates can propagate beyond their intended scope, overemphasizing the edited target even when it’s contextually inappropriate. To address this challenge, we introduce REACT ( R epresentation E xtraction A nd C ontrollable T uning), a unified two-phase framework designed for precise and controllable knowledge editing. In the initial phase, we utilize tailored stimuli to extract latent factual representations and apply Principal Component Analysis with a simple learnbale linear transformation to compute a directional “belief shift” vector for each instance. In the second phase, we apply controllable perturbations to hidden states using the obtained vector with a magnitude scalar, gated by a pre-trained classifier that permits edits only when contextually necessary. Relevant experiments on EVOKE benchmarks demonstrate that REACT significantly reduces overfitting across nearly all evaluation metrics, and experiments on COUNTERFACT and MQuAKE shows that our method preserves balanced basic editing performance (reliability, locality, and generality) under diverse editing scenarios. 1 Introduction Large language models (LLMs) have become indispensable in modern applications, powering a wide array of systems from chatbots to content generators ( Zhao et al., 2023 ; Xu et al., 2024 ) . Despite their widespread utility, ensuring that these models maintain up-to-date and accurate factual information remains a critical challenge, particularly when extensive retraining is impractical ( Zhang et al., 2024b ) . This necessity has spurred interest in the field of knowledge editing, where targeted updates to a model’s internal knowledge base are pursued without compromising overall performance ( Wang et al., 2023 ; Yao et al., 2023 ; Cheng et al., 2023 ) . Figure 1: Illustration of overfitting in LLM editing. Overfitting occurs when the model disproportionately emphasizes the edited target fact, even in contexts irrelevant to the edit. As shown on the right side, after editing the fact about Luka Doncic’s team to "Lakers," the overfitted model incorrectly assigns high probability to "Lakers" even for a query about Doncic’s teammates. Recent advances in knowledge editing have sought to address these issues by incrementally incorporating new facts into LLMs ( De Cao et al., 2021 ) . However, many existing approaches encounter significant challenges, like overfitting during editing process ( Zhang et al., 2024a ) . Concretely, this occurs when a model, after being updated with new knowledge, becomes excessively specialized to the edited samples. For example, consider an update where the statement “Luka Doncic plays in the NBA team of Mavericks ” is corrected to “Luka Doncic plays in the NBA team of Lakers .” In an overfit scenario, when queried with “Who does Luka Doncic play with?”, the model may still disproportionately favor the edit target but not the correct answer—assigning a high probability to “Mavericks”—while the probabilities for more contextually appropriate responses, such as teammates like Austin Reaves or LeBron James, remain undesirably low, as illustrated in Figure 1 . These limitations hinder the practical deployment of such techniques in real-world systems. In response to these challenges, we propose a novel framework that leverages a dual-phase representation pipeline to perform targeted knowledge edits. In the first phase- Extracting Latent Knowledge Representations (§3.1)-we employ tailored input prompts to extract the model’s latent factual representations. Then we use Principal Component Analysis and a simple linear transformation to compute a directional vector that encapsulates the latent “belief” shift associated with the edit. In the subsequent phase- Controllable Perturbing Representations Selectively (§3.2)-we introduce controlled perturbations to the model’s hidden states, guided explicitly by a pre-trained classifier (§3.3). This classifier functions as a gating mechanism, discerning precisely when edits should be applied based on the hidden states of the content. We perturb the hidden states from Transformer decoder block of all layers based on the product between the original hidden state and the directional vector. We also use a learnable scalar to control the magnitude of the perturbation. To prove effectiveness of our method, we conduct experiments and analyze the results on COUNTERFACT, MQuAKE (§5.1) and EVOKE (§5.2), with detailed experimental settings (§4). Our contributions can be summarized as follows: • We propose a dual-phase editing framework, which extracts latent factual representation shifts and applies controllable perturbations to precisely edit models, effectively overcoming the critical overfitting issue in existing knowledge editing methods. • Unlike prior parameter-based methods, our approach operates directly on the model’s hidden states, employing classifier-driven gating to ensure edits are accurately applied, thus providing explicit control over knowledge modification. • Comprehensive evaluation on COUNTERFACT, MQuAKE, and EVOKE datasets demonstrates that our method significantly reduces overfitting while achieving balanced improvements in Reliability, Generality, and Locality metrics. 2 Preliminaries 2.1 Large Language Models Autoregressive large language models (LLMs) employ the Transformer architecture, where hidden representations are computed through successive decoder blocks. At each layer l , the hidden state \boldsymbol{h}^{(l)} is updated by integrating the global self-attention and local feed-forward (FFN) contributions from the previous layer: \boldsymbol{h}^{(l)}=\boldsymbol{h}^{(l-1)}+a^{(l)}+m^{(l)}, with a^{(l)} and m^{(l)} denoting the outputs of the attention and FFN components, respectively. Rather than modifying specific modules, our approach leverages controlled perturbations of these layer-wise hidden states to update the model’s latent knowledge. 2.2 Knowledge Editing in LLMs Knowledge editing aims to revise specific factual information embedded within LLMs without impairing general performance. In our framework, a fact is represented as a triple (s,r,o) , where s is the subject, r the relation, and o the object. For example, if the model initially encodes the fact that (s= Luka Doncic, r= plays in the NBA team of, o= Mavericks ) , and the objective is to update this to (s= Luka Doncic, r= plays in the NBA team of, o^{*}= Lakers ) . Such an editing operation is denoted by e=(s,r,o,o^{*}) . Given a model f and an edit e , we define the editing operator as K(f,e)=f^{*}, where f^{*} represents the model after applying the edit. Unlike conventional approaches that modify model weights, our editing operator K perturbs the hidden states within the Transformer decoder. 2.3 Overfitting during Editing A critical issue in knowledge editing is overfitting to the (s,r,o) edit pair. In our formulation, the prompt p(s,r) is designed to trigger the updated response o^{*} . Ideally, the model should output o^{*} only for p(s,r) , while responding appropriately to other context-dependent queries. For instance, still consider the edit (s= Luka Doncic, r= plays in the NBA team of, o= Mavericks, o^{*}= Lakers ) . For the prompt “Luka Doncic plays in the NBA team of,” the model should now output “Lakers.” However, if queried with “Who does Luka Doncic play with?”—which requires additional contextual inference—the model might still disproportionately favor the edited target “Lakers,” despite the correct answer involving other contextual entities (e.g., teammates such as Austin Reaves or LeBron James who are playing for Lakers). This persistent bias, where the model consistently outputs o^{*} regardless of the input prompt, exemplifies the overfitting issue and underscores a key limitation of current editing approaches. 3 REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting The persistent challenge of overfitting in existing LLM

Read the complete text