RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models

Le Vu Anh, Dinh Duc Nha Nguyen, Phi Long Nguyen, [0.5em] TinyAI Research · 2025

Read RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models by Le Vu Anh, Dinh Duc Nha Nguyen, Phi Long Nguyen, [0.5em] TinyAI Research (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13249.txt.

Introduction

Le Vu Anh, Dinh Duc Nha Nguyen, and Phi Long Nguyen present a 2025 study on a hidden flaw in large language models. Test data can leak into the data a model trains on, which makes test scores look better than they are. Past tools for spotting such leaks have been weak or slow. The team set out to build a faster and surer check.

Their method, called RN-F, reads faint leftover patterns in the model's own signals to flag leaked items. It needs one pass, uses no gradient work, and adds no extra math load. The method works across models and needs no change to the model itself. On several models and leaked data sets, RN-F beat the best prior tools, with gains of up to 10.5 percent on detection scores. The work gives testers a light way to guard the honesty of model scores.

Catalog record

CREATORLe Vu Anh, Dinh Duc Nha Nguyen, Phi Long Nguyen, [0.5em] TinyAI Research
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-13249

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RN-F: A Novel Approach for Mitigating Contaminated Data in Large Language Models Le Vu Anh, Dinh Duc Nha Nguyen, Phi Long Nguyen, [0.5em] TinyAI Research Abstract Large Language Models (LLMs) have become foundational in modern artificial intelligence, powering a wide range of applications from code generation and virtual assistants to scientific research and enterprise automation. However, concerns about data contamination—where test data overlaps with training data—have raised serious questions about the reliability of these applications. Despite awareness of this issue, existing methods fall short in effectively identifying or mitigating contamination. In this paper, we propose Residual-Noise Fingerprinting (RN-F), a novel framework for detecting contaminated data in LLMs. RN-F is a single-pass, gradient-free detection method that leverages residual signal patterns without introducing additional floating-point operations. Our approach is lightweight, model-agnostic, and efficient. We evaluate RN-F on multiple LLMs across various contaminated datasets and show that it consistently outperforms existing state-of-the-art methods, achieving performance improvements of up to 10.5% in contamination detection metrics. Keywords: TinyML, Quantization, Anomaly Detection, Data Poisoning, Edge AI 1 Introduction Large Language Models (LLMs) have become foundational tools in modern artificial intelligence, supporting diverse applications such as code generation, scientific discovery, enterprise automation, and intelligent assistants [ 1 , 2 , 3 , 4 ] . Their remarkable performance on various benchmarks [ 5 , 6 ] has driven rapid adoption across both academia and industry. However, this progress has raised serious concerns about the validity of benchmark results due to potential data contamination [ 7 ] . As LLMs are often trained on massive, uncurated internet-scale datasets, it becomes increasingly likely that test data may overlap with training data—either directly or through paraphrased or synthetic forms—thereby artificially inflating model performance and misleading evaluations. Data contamination refers to the inadvertent inclusion of benchmark or evaluation data within a model’s training corpus [ 8 , 7 ] . This phenomenon results in LLMs memorizing answers rather than generalizing from learned patterns, thereby jeopardizing the trustworthiness of performance evaluations. Contaminated models can exhibit strong results on known benchmarks while failing to perform reliably on unseen or real-world tasks [ 9 ] . This issue threatens both scientific progress and application safety, highlighting an urgent need for effective, scalable, and model-agnostic methods to detect and mitigate contamination in LLMs. Existing state-of-the-art approaches [ 8 , 10 , 11 ] to contamination detection typically rely on access to internal model parameters, output probabilities, or multiple reference datasets. These methods often require repeated model evaluations or comparisons across rephrased benchmarks, synthetic data, or assumed-clean samples. While effective in certain controlled settings, they are generally resource-intensive, difficult to scale, and impractical for quantized or edge-deployed models. Furthermore, many current methods struggle to detect subtle or implicit forms of contamination, especially when training data is opaque or continuously evolving. In this paper, we propose Residual-Noise Fingerprinting (RN-F), a novel, efficient framework for detecting data contamination in LLMs. RN-F exploits a simple yet powerful signal—the quantization residual—defined as the per-layer difference between full-precision and quantized model activations. By observing how this residual behaves differently on contaminated inputs versus clean data, RN-F serves as a lightweight, gradient-free, and single-pass anomaly detector. It requires no access to internal gradients or model retraining and is suitable for deployment on low-resource devices. Extensive experiments across multiple models and datasets show that RN-F outperforms existing methods, achieving up to 10.5% higher detection performance while maintaining minimal computational overhead. This work offers three primary contributions: 1. We introduce Residual-Noise Fingerprinting (RN-F), a pioneering approach that leverages the quantization residual—the per-layer difference between full-precision and int4 activations—as an anomaly signal. To the best of our knowledge, RN-F is the first framework to repurpose quantization artefacts as a tool for contamination detection in LLMs and other compact models, especially under severe resource constraints. 2. We provide a rigorous statistical characterization of quantization residuals, proving that clean inputs exhibit sub-Gaussian tails while contaminated inputs induce a bounded mean shift. This theoretical analysis enables provable guarantees for false positive and false negative rates, even with a limited calibration buffer. 3. We evaluate RN-F on compact models across tabular, language, and image tasks, demonstrating superior performance over state-of-the-art methods while maintaining a lightweight, single-pass, gradient-free setup with minimal calibration. The rest of the paper unfolds as follows. Section 2 positions our work among existing defences. Section 3 formalises the quantization residual, and Section 4 turns it into the RN-F algorithm. Results and ablations, followed by a discussion of limitations, appear in Section 5 . Technical proofs and additional plots live in the Appendix. 2 Related Work Recent advances in large language models (LLMs) have prompted numerous studies on detecting data contamination and backdoor vulnerabilities. These works explore various detection strategies, from statistical analysis to probing internal representations. Below, we summarize key contributions in this space. This work [ 12 ] proposes a guided method to detect contamination by prompting LLMs with dataset-specific cues, comparing outputs via ROUGE-L and BLEURT. It scales from instance- to partition-level but assumes near-exact text matches, missing semantic paraphrases. The authors [ 13 ] study poisoning intensity effects and find detectors fail under both strong and weak poisoning, revealing brittleness under varying attack strengths. This survey [ 14 ] reviews 50 detection methods by their assumptions, showing many fail under distribution shifts where memorization assumptions break down. The authors [ 15 ] benchmark five detectors on four LLMs and eight datasets, noting performance drops with instruction-tuned models and complex prompts. RECALL [ 16 ] uses changes in log-likelihoods under prefix perturbation for membership inference, achieving strong results but requiring token-level access, limiting black-box use. This method [ 17 ] uses hidden layer activations with a probe classifier to detect training data, performing well but needing proxy models and internal access. DC-PDD [ 18 ] calibrates token probabilities using divergence from expected frequencies, reducing false positives but depending on raw token access and vocabulary stability. ConStat [ 11 ] defines contamination as performance inflation, using statistical comparisons and p-values but needs curated benchmarks and assumes task generalization. BAIT [ 10 ] inverts target sequences to detect backdoors via generation probabilities; effective for generative tasks but reliant on specific causal structures. CDD [ 8 ] identifies peaked output distributions as signs of memorization, performing well and efficiently, though its confidence-based signal may not generalize. Our Observation. Despite promising results, most existing methods depend on strong assumptions: access to logits, full-precision models, or curated references. Many break under distribution shifts, quantization, or black-box constraints. This highlights the need for lightweight, generalizable methods like RN-F that require minimal assumptions and operate efficiently across settings. 3 Preliminaries 3.1 Problem set-up Let f_{\theta}\colon\mathbb{R}^{m}\!\to\!\mathbb{R}^{C} denote a frozen fp16 network with L layers, and let \hat{f}^{(4)}_{\theta} be the same network after post-training 4-bit quantization (PTQ) using a uniform step 2q . We write h_{\ell}(x)\in\mathbb{R}^{d_{\ell}} for the fp16 activation of layer \ell and \hat{h}_{\ell}(x) for its int4 counterpart. Definition 3.1 (Layer-wise residual) . For an input x and a layer of width d_{\ell} , the quantization residual is the mean absolute deviation between fp16 and int4 activations: r_{\ell}(x)\;=\;\frac{1}{d_{\ell}}\,\Bigl\|h_{\ell}(x)-\hat{h}_{\ell}(x)\Bigr\|_{1}. Intuitively, r_{\ell}(x) measures how far the int4 grid has to “snap” the fp16 activation vector. We aggregate over layers by r_{\max}(x)=\max_{\ell}r_{\ell}(x) ; other norms

Read the complete text