A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures

Bowen Chen, Namgi Han, Yusuke Miyao · 2026

Read A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures by Bowen Chen, Namgi Han, Yusuke Miyao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21658.txt.

Introduction

Chen and co-authors compare how model families memorize training text, in output stats and in inner signals. Past work used single model lines, so shared and unique traits stayed unclear. They gather six lines, including Pythia, OLMo, and StarCoder. Memorization grows log-linearly with model size, and memorized text compresses more than expected. Families share frequency and domain patterns yet keep their own marks. Inside the models, memorized text reacts more to injected noise, and key attention heads sit in different spots per family.

Catalog record

CREATORBowen Chen, Namgi Han, Yusuke Miyao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62356
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21658

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Comparative Analysis of LLM Memorization at Statistical and Internal Levels: Cross-Model Commonalities and Model-Specific Signatures Bowen Chen, Namgi Han, Yusuke Miyao Abstract Memorization is a fundamental component of intelligence for both humans and LLMs. However, while LLM performance scales rapidly, our understanding of memorization lags. Due to limited access to the pre-training data of LLMs, most previous studies focus on a single model series, leading to isolated observations among series, making it unclear which findings are general or specific. In this study, we collect multiple model series (Pythia, OpenLLaMa, StarCoder, OLMo1/2/3) and analyze their shared or unique memorization behavior at both the statistical and internal levels, connecting individual observations while showing new findings. At the statistical level, we reveal that the memorization rate scales log-linearly with model size, and memorized sequences can be further compressed. Further analysis demonstrated a shared frequency and domain distribution pattern for memorized sequences. However, different models also show individual features under the above observations. At the internal level, we find that LLMs can remove certain injected perturbations, while memorized sequences are more sensitive. By decoding middle layers and attention head ablation, we revealed the general decoding process and shared important heads for memorization. However, the distribution of those important heads differs between families, showing a unique family-level feature. Through bridging various experiments and revealing new findings, this study paves the way for a universal and fundamental understanding of memorization in LLM. 1 Introduction Large Language Models (LLMs) show unprecedented performance and have revolutionized the field of Natural Language Processing in recent years 30 ; 1 . The progress in improving the model structure 39 , training algorithm 19 , and data 40 is surprising, but the understanding of LLMs does not move equally. While previous research analyzed LLM in many aspects 38 ; 24 , one of the mechanisms that is still not well-understood is Memorization 9 ; 20 ; 36 , e.g., LLMs can regurgitate content in their pre-training corpora. This special behavior makes LLMs a knowledge base 37 , or could lead to unfair usage in privacy or copyright issues 12 . While being a key element of intelligence in both humans and LLMs, the understanding of memorization is lacking since it requires access to training data, which largely restricts usable models. Previous research usually focuses on a sandbox model or a single model series 9 ; 11 ; 10 , leading to isolated conclusions. This study aims to bridge this gap while revealing new findings by studying models with known pre-training data. This includes 6 families (Pythia, StarCoder, OpenLLaMA, OLMo1, OLMo2, and OLMo3), for a total of 20 models, ranging from 1b to 32b, whose pre-training size spans from billion to trillion. Though collecting sequences from various domains in each family, we compute the memorization score for each example in different model sizes. Based on those examples, we comparatively analyze them from both the statistical and internal levels, aiming to connect the observations among individual models while bringing new findings. At the statistical level, we analyze the statistics of the collected scores. We first study how the memorization rate couples with the model and data size, along with experiments showing the distribution of memorization scores of all examples and the least number of tokens to recall the same memorized sequence. The following analysis discusses the distribution of memorized sequences over different domains and token frequency. At the internal level, we inject noise into the model to analyze whether memorized sequences rely on a specific pattern or not. To understand the emergence of memorized tokens, we also decode those memorized sequences in the middle layer. Finally, to locate whether a memorization-specific pattern exists or not, we analyze the importance of each head by observing how generated sequences diverge from the memorized one without a specific head. Our results show: • Statistical Level: – Cross-Model Commonalities: Memorization rates universally scale log-linearly with parameter size. Furthermore, memorized sequences are intrinsically highly compressed (even using \leq 50\% of original tokens could obtain the same memorized output). Additionally, most models shared a close memorization threshold for memorization, while larger models have a lower frequency threshold. At the domain level, most of the memorized sequences are structural texts (codes, etc), but scaling model size makes LLMs memorize more free texts. – Model-Specific Signatures: Stronger model capacity does not indicate a higher memorization rate. Instead, we observe up to a 100 \times divergence in memorization rates across different families (e.g., StarCoder vs. OLMo1), heavily governed by their specific pre-training corpora. • Internal Level: – Cross-Model Commonalities: Under noise perturbation, LLMs reveal certain denoising capabilities that remove part of the injected noise, while the memorized sequences are harder to denoise. Using attention ablation, we found that memorization-important heads highly overlap within domains, and a small subset of heads is important for most domains. – Model-Specific Signatures: While memorization important heads exist, their layer-wise distribution forms a distinct structural fingerprint. This distribution is significantly consistent within a model family, but diverges across different families, which is shaped by the individual training recipe of each family. 2 Related Work In this section, we review the literature on LLM memorization at both the statistical (analyzing through observing various LLM outputs) and internal levels (investigating internal components). 2.1 Statistics Analysis of Memorization 34 analyzed memorization in LLM with a sandbox setting, showing that larger models memorize sequences more easily. Following it, 7 studied the pre-trained GPT-Neo 6 models, revealing that the number of memorized sequences is related to model capacity, prompt size, and the repetitions in the pre-training corpora. 11 showed that memorized sequences are shared across model sizes, and the token frequency gap between input and output is vital for generating memorized sequences. To analyze memorization beyond fixed input, 31 developed the compression ratio, e.g, the maximum number of sequences that a certain length sequence can recall. 21 studied the memorized sequences under different sampling strategies and suggested that the actual memorization may be more than what we have observed. 2.2 Internal Memorization Mechanism 13 showed that the knowledge neuron exists in the later layers of BERT 14 . However, the results from 33 suggested that the learned knowledge is represented as a vector rather than stored in individual parameters. 11 shows the varied decoding entropy feature for memorized and unmemorized sequences. 2 analyzed the Intrinsic Dimensions (IDs) of memorized sequences in LLM and found that low ID complexity sequences are easy to memorize. 23 analyzed the internal mechanism of the GPT-Neo-125m model 6 by interrupting the input and observing whether the LLM can still output the memorized sequences, suggesting the existence of triggers and maintenance components for memorization. 10 also analyzed the theoretical memorization capacity through sandbox experiments on real data. This study stands on both statistical and internal perspectives, aiming to extend the analysis to a wider spectrum of model families. 3 Experiments Setting We introduce the motivation for each experiment. We first study the basic statistics of memorization, including its intercorrelation with multiple factors, its firmness measured by compression ratio, and the distribution of memorization scores (Section 4.1 ). These statistics reveal general features alongside model-specific curves, serving for further discussion on domain and frequency distributions (Section 4.2 ,

Read the complete text