Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoning
Poojitha Thota, Shirin Nilizadeh · 2026
Read Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoning by Poojitha Thota, Shirin Nilizadeh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26036.txt.
Introduction
Poojitha Thota and Shirin Nilizadeh defend text summarization models against poisoned training data. Fine-tuning sets for summarization are small, so a few planted examples can steer behaviour. Poisoned models may emit biased or false summaries while normal scores look fine. Their framework works after the fact. It detects the poisoning, unlearns it, and restores the model.
Detection differs by access level. With white-box access, poisoned pairs show abnormally high training influence. Influence analysis plus semantic checks expose them. With black-box access, poisoned models react two to three times more strongly to meaning-preserving tweaks. New attacks in the study distort facts or inject bias. Across nine architectures and six datasets, detection precision reaches eighty-five to ninety-two percent. Gradient-ascent unlearning restores up to ninety-six percent of original behaviour. Summary quality loss stays under one percent, with no full retraining.
Catalog record
| CREATOR | Poojitha Thota, Shirin Nilizadeh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.26036
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoning Poojitha Thota, Shirin Nilizadeh Abstract Training-time data poisoning during fine-tuning poses a significant threat to large language models (LLMs) deployed for abstractive text summarization, where small task-specific datasets exert disproportionate influence on model behavior. In this setting, adversaries manipulate fine-tuning data to induce persistent summarization failures, such as biased or harmful summaries, while preserving standard evaluation metrics. We present a unified post-hoc defense framework for detecting and remediating fine-tuning-stage poisoning in summarization models across the machine learning supply chain. Our experiments show that in white-box settings, poisoned document–summary pairs exhibit abnormally high training influence, enabling detection via influence-function analysis with semantic consistency checks. In black-box settings, poisoned models display two-to-three times greater sensitivity to semantics-preserving perturbations, enabling behavioral auditing without training data access. Beyond existing poisoning formulations, we introduce novel attacks targeting factual distortion and representational bias, showing that poisoning alters summarization behavior without triggering conventional alarms. Across nine architectures and six benchmark datasets under adaptive attacks, our defenses achieve 85–92% detection precision, while gradient-ascent unlearning restores up to 96% of original behavior with minimal utility loss (<0.6% ROUGE degradation). These results indicate that fine-tuning-time poisoning leaves persistent structural artifacts, enabling practical detection and post-deployment recovery without full retraining. 1 Introduction The widespread adoption of Large Language Models (LLMs) has revolutionized natural language processing (NLP), with text summarization emerging as one of the most impactful applications for efficient information processing and knowledge management [ 62 , 21 , 50 ] . Recent advances in LLMs such as GPT-4 [ 37 ] , Claude [ 4 ] , and PaLM [ 3 ] have demonstrated remarkable capabilities in generating coherent, context-aware summaries from vast textual data. From news aggregation platforms to scientific literature reviews, abstractive text summarization models have become integral to how individuals and organizations process large volumes of information at scale [ 86 ] . As these models become integral to critical decision-making systems, recent work has shown pressing security concerns, including fundamental weaknesses in summarization models and their susceptibility to data-poisoning attacks [ 24 , 46 , 78 , 73 ] . In this work, we focus specifically on poisoning that occurs during the fine-tuning stage, where domain-specific datasets are small, and each example exerts disproportionately high gradient influence. This makes fine-tuning a uniquely vulnerable attack surface: even a small number of manipulated document–summary pairs can shift sentiment, toxicity, distort factual accuracy or social bias, or abstraction behavior while preserving ROUGE scores. Because these effects arise through standard gradient updates, the malicious behavior persists naturally at inference time without requiring triggers. We consider two attack vectors: (1) adversaries release poisoned datasets that victims use for fine-tuning (white-box), and (2) adversaries directly publish poisoned model checkpoints that appear normal on benchmarks but exhibit manipulated behavior in real use (black-box). These two settings impose different constraints on defenders. Within these vectors, we evaluate four poisoning objectives. Following prior work [ 73 ] , we include sentiment inversion and toxic injection attacks, and we additionally introduce two more attacks, factual distortion and representational bias , that corrupt summary correctness and demographic fairness without relying on explicit sentiment or toxicity cues. Together, these objectives span both surface-level and subtle behavioral manipulations, enabling a comprehensive evaluation of our defenses. Existing defenses against adversarial attacks [ 29 , 53 , 55 ] , text classification poisoning [ 69 , 49 , 58 ] , and backdoor detection [ 77 , 27 ] do not address training-time poisoning in generative summarization, which involves trigger-free behavioral shifts rather than perturbation robustness or discrete misclassification. Recent defenses for harmful fine-tuning [ 71 , 87 , 14 , 36 , 10 ] target safety alignment or classification tasks. While machine unlearning [ 7 ] and influence functions [ 41 , 44 ] enable selective sample removal, they have not been systematically applied to defend against data poisoning in text summarization. To address these challenges, we propose a unified defense framework with two complementary mechanisms, targeting distinct threat scenarios in the fine-tuning pipeline (Section 3 ). Against white-box attacks where adversaries poison training datasets, our Defense-1 (Poisoned Dataset Detection) combines influence function analysis using DataInf [ 44 ] with a modular behavioral filtering stage that detects abnormal semantic signals (e.g., sentiment shifts, toxicity, factual inconsistencies, or representational bias), followed by gradient-ascent unlearning [ 82 ] to selectively remove poisoned samples. Our key insight is that effective poisoned samples tend to have a high influence, enabling detection by focusing on a small, high-impact subset rather than inspecting entire datasets. Against black-box attacks where adversaries distribute poisoned models, our Defense-2 (Poisoned Model Detection) exploits a novel compound vulnerability: poisoned models exhibit 2–3 \times greater sensitivity to adversarial perturbations than clean models. Our Sensitivity to Adversarial Perturbations (SAP) metric, inspired by prior sensitivity-based detection in vision [ 26 ] , quantifies lead-sentence exclusion under controlled perturbations, enabling detection without training data access. Together, these two components operate as a single end-to-end framework, jointly protecting both poisoned datasets and distributed model checkpoints across the model lifecycle. We conduct comprehensive evaluation across six datasets (MultiNews [ 25 ] , CNN/DailyMail [ 12 ] , WikiSum [ 20 ] , ArXiv-summarization [ 18 ] , PubMed-summarization [ 18 ] , Multi-XScience [ 52 ] ) and nine models (encoder-decoder: BART-Large [ 45 ] , T5-Small [ 61 ] , Pegasus-Large [ 85 ] , and FLAN-T5-Large [ 17 ] ; open-source decoder-only models: LLaMA-3-8B [ 22 ] , Qwen-2.5 [ 60 ] , Qwen-3 [ 81 ] , Mistral-7B [ 39 ] , and Vicuna-13B [ 15 ] ). Defense-1 achieves 85.3% average recovery while reducing remediation cost by 4–6 \times compared to full retraining, and consistently outperforms adversarial training, SISA [ 7 ] , exact retraining [ 32 ] , selective pruning [ 51 ] , and TracIn [ 59 ] . Defense-2 achieves near-perfect detection, in a fully black-box setting, outperforming CLIBE [ 83 ] and perplexity-score [ 66 ] based sensitivity baselines by maintaining clear separation between clean and poisoned models across all perturbation types. Additionally, prior work [ 73 ] has shown that model poisoning can alter the fundamental behavior of summarization models, shifting them from abstractive to extractive generation. Critically, our defense reverses these behavioral shifts . Beyond surface metrics, our defense restores deeper generation behavior: gradient-ascent unlearning recovers 94–96% of abstractive behavior previously shifted toward extractiveness [ 73 ] , demonstrating that high ROUGE alone is insufficient to verify successful remediation. This provides the first evidence that machine unlearning in Defense-1 restores deeper generation behavior. To validate robustness against defense-aware adversaries, we evaluate four adaptive attack strategies: low-influence poisoning, crafting samples with moderate influence scores, multi-objective poisoning, blending sentiment and toxicity, gradient masking, obscuring detection signals, and distributed poisoning, spreading attacks across batches. Defense-1 maintains 78–82% recovery (vs. 85.3% under standard attacks), while Defense-2 retains 95–98% TPR. Statistical analysis confirms significant separation between clean and poisoned behaviors ( p<0.001 ), and results generalize across architectures. The primary contributions of our work are: ① First comprehensive defense framework for text summarization against data poisoning using machine unlearning to target root causes, addressing both poisoned datasets and poisoned models scenarios. ②