Generalization or Memorization: Dynamic Decoding for Mode Steering

Xuanming Zhang · 2025

Read Generalization or Memorization: Dynamic Decoding for Mode Steering by Xuanming Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22099.txt.

Introduction

Xuanming Zhang studies the split personality of large language models, which sometimes reason in general patterns and sometimes recite memorized training text word for word. That unpredictability is dangerous in high-stakes settings. Zhang builds a framework to tell the two modes apart while a model is running and to lean it toward the safer one.

The theory comes from the information bottleneck principle: generalizing means keeping a compressed, task-relevant picture, while memorizing means failing to compress. On that foundation, a small probe reads the model's state and estimates how much it is leaning on memory right now. A steering mechanism then nudges its activity toward circuits linked with generalization. Zhang frames the pairing as the model contrasting against itself during decoding, and reports experiments on reasoning tasks and factual questions.

Catalog record

CREATORXuanming Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22099

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generalization or Memorization: Dynamic Decoding for Mode Steering Xuanming Zhang Abstract Large Language Models (LLMs) exhibit a troubling duality, capable of both remarkable generalization and brittle, verbatim memorization of their training data. This unpredictability undermines their reliability in high-stakes applications. In this work, we propose a unified framework to understand, identify, and control these distinct reasoning modes. First, we introduce a theoretical model based on the Information Bottleneck (IB) principle, formalizing generalization as the learning of a compressed, task-relevant representation and memorization as a failure to compress. Building on this theory, we develop Dynamic Mode Steering (DMS), a novel inference-time algorithm which comprises two components: (1) a lightweight, causally-grounded linear probe that identifies the model’s instantaneous reliance on memorization, and (2) a dynamic activation steering mechanism that nudges the model’s computation towards pre-identified generalization circuits. We frame DMS as a form of adaptive, self-contrastive decoding. Experiments on reasoning and faithfulness tasks demonstrate that DMS significantly improves logical consistency and factual accuracy, thereby offering a principled approach to enhancing LLM reliability. 1 Introduction Large Language Models (LLMs) represent a significant milestone in artificial intelligence, demonstrating capabilities that often appear to stem from a rich, nuanced understanding of the world ( Naveed et al. 2025 ; Pearce et al. 2023 ; Zhang et al. 2025b ) . Their proficiency in tasks ranging from complex reasoning to creative text generation has positioned them as transformative technologies. However, this apparent understanding is frequently contradicted by a more simplistic, and often problematic behavior: the verbatim regurgitation of memorized fragments from their vast training corpora ( Dong et al. 2024 ; Bayat et al. 2024 ) . This tendency to memorize, rather than reason, leads to models that can confidently produce fluent falsehoods, perpetuate biases present in the training data, and fail unexpectedly on inputs that deviate slightly from memorized patterns ( Wang et al. 2024c ; Dong et al. 2025 ) . The unpredictable switching between these two modes—robust generalization and brittle memorization—is a central challenge to their safe and reliable deployment. We frame this challenge as a fundamental tension between two distinct and competing operational modes within a single model: Generalization is the desired mode, characterized by the ability to learn abstract principles, rules, and causal structures from data and apply them to solve novel problems ( Chatterjee 2018 ; Kang et al. 2024 ; Zhang et al. 2024 ) . It is the foundation of true reasoning and robust performance on unseen inputs. In contrast, memorization is a shortcut to achieving low training error, where the model functions as a high-dimensional lookup table, storing and retrieving specific input-output pairs from its training set without learning the underlying generative process ( Xie et al. 2024 ) . While effective on familiar data, this mode is inherently brittle and fails to adapt to new situations. The phenomenon of grokking serves as a stark and compelling illustration of this dichotomy. Observed in neural networks trained on algorithmic tasks, grokking is a delayed phase transition where a model, long after achieving perfect accuracy on its training data through memorization, suddenly and rapidly learns to generalize to unseen test data ( Fan et al. 2024 ) . This transition reveals that memorization and generalization are not merely points on a continuum but are distinct, accessible solutions in the model’s loss landscape. A model can become stuck in a memorization-based local minimum before eventually discovering a more efficient, generalizable solution, prompted by factors like prolonged training or regularization techniques such as weight decay ( Liu et al. 2022 ; Kumar et al. 2023 ) . This phenomenon moves the problem from a purely abstract concern to an empirically observable dynamic, motivating the need for a formal theory to explain these modes and a practical mechanism to control them. To address this, our work introduces a unified framework that spans theory and practice. The primary contributions are twofold: • We propose a novel theoretical model based on the Information Bottleneck (IB) principle to formally define and distinguish between generalization and memorization. This framework provides a principled, mathematical language to analyze the learning dynamics of LLMs. • We introduce Dynamic Mode Steering (DMS) for Mode Control. DMS is a training-free, inference-time algorithm designed to dynamically manage the model’s reasoning mode, nudging it away from memorization-associated pathways and towards circuits responsible for generalization. 2 Related Work Generalization, Memorization, and Grokking. The tension between generalization and memorization is a classical theme in machine learning ( Chatterjee 2018 ; Yin et al. 2019 ; Feldman 2020 ) , but it has gained new urgency with the scale of modern LLMs ( Wang et al. 2024b ; Chu et al. 2025 ; Bayat et al. 2024 ; Li et al. 2024 ; Xu et al. 2025 ) . The phenomenon of grokking, first identified by Power et al. 2022 , provided a clear empirical demonstration that these are distinct learning phases. Subsequent research has explored the conditions under which grokking occurs, identifying factors like model size, data size, and the strength of weight decay as critical parameters ( Liu et al. 2022 ; Chang et al. 2024 ; Wang et al. 2024a ) . Other studies have linked the transition to changes in the internal representations, such as a decrease in feature rank, suggesting a move towards a simpler, more structured solution ( Fan et al. 2024 ; Zhang et al. 2025a ) . Our work further contributes by proposing the Information Bottleneck principle as a unifying theoretical framework that explains why these transitions occur, framing them as an optimization process that seeks a more information-theoretically efficient representation. Information-Theoretic Analysis of Neural Networks. The application of the Information Bottleneck principle to understand deep learning has a rich history. Early work proposed that the training of deep neural networks via stochastic gradient descent naturally follows two phases: an initial fitting phase where the model increases I(Z;Y) , the mutual information with the labels, followed by a compression phase where it reduces I(X;Z) , the mutual information with the input ( Kawaguchi et al. 2023 ; Lewandowsky & Bauch 2024 ) . Recent work has focused on developing more tractable bounds for the IB objective and proving rigorous learning theory guarantees that connect the minimization of I(X;Z) to better generalization bounds ( Westphal et al. 2025 ; Hellström et al. 2025 ) . Our framework builds on this lineage, using the IB objective not just as an analytical tool but as the formal basis for defining and distinguishing the computational goals of generalization and memorization. Inference-Time Decoding and Activation Steering. Standard decoding methods like greedy search, beam search, and nucleus sampling primarily manipulate the final probability distribution ( Shi et al. 2024 ) . More advanced adaptive decoding methods can alter their strategy based on model confidence or other signals, for instance, by using early-exiting from intermediate layers or injecting phrases to encourage continued reasoning ( Wei et al. 2025 ; Jin et al. 2025 ) . Activation steering aims to reverse-engineer the specific computations and circuits within neural networks ( Turner et al. 2023 ; Sharkey et al. 2025 ) . Recent technique involves directly modifying activations at inference time to control high-level model behaviors like

Read the complete text