Pruning Strategies for Backdoor Defense in LLMs Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM
Santosh Chapagain, Shah Muhammad Hamdi, Soukaina Filali Boubrahimi · 2025
Read Pruning Strategies for Backdoor Defense in LLMs Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM by Santosh Chapagain, Shah Muhammad Hamdi, Soukaina Filali Boubrahimi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-20032.txt.
Introduction
Santosh Chapagain, Shah Muhammad Hamdi, and Soukaina Filali Boubrahimi test pruning as a defense against backdoor attacks on language models, in a 2025 paper. A backdoor hides in a trained model and fires only when a secret trigger appears in the input.
Users rarely know the trigger or hold a clean copy of the model, so the defense must work blind. The team tried six ways to prune the model's attention heads, dropping the least useful ones step by step. Gradient-based pruning handled grammar-based triggers best. Learning-guided and uncertainty-based pruning stood up better to style-based triggers. Accuracy checks kept the pruning from going too far.
Catalog record
| CREATOR | Santosh Chapagain, Shah Muhammad Hamdi, Soukaina Filali Boubrahimi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-20032
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Pruning Strategies for Backdoor Defense in LLMs Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM ’25), November 10–14, 2025, Seoul, Republic of Korea DOI: 10.1145/3746252.3760946 ISBN: 979-8-4007-2040-6/2025/11 CCS: Computing methodologies Natural language processing CCS: Security and privacy Malware and its mitigation CCS: Security and privacy Malware mitigation CCS: Computing methodologies Natural language processing (NLP) Santosh Chapagain, Shah Muhammad Hamdi, Soukaina Filali Boubrahimi Abstract . Backdoor attacks are a significant threat to the performance and integrity of pre-trained language models. Although such models are routinely fine - tuned for downstream NLP tasks, recent work shows they remain vulnerable to backdoor attacks that survive vanilla fine - tuning. These attacks are difficult to defend because end users typically lack knowledge of the attack triggers. Such attacks consist of stealthy malicious triggers introduced through subtle syntactic or stylistic manipulations, which can bypass traditional detection and remain in the model, making post-hoc purification essential. In this study, we explore whether attention-head pruning can mitigate these threats without any knowledge of the trigger or access to a clean reference model. To this end, we design and implement six pruning-based strategies: (i) gradient-based pruning, (ii) layer-wise variance pruning, (iii) gradient-based pruning with structured L1/L2 sparsification, (iv) randomized ensemble pruning, (v) reinforcement-learning-guided pruning, and (vi) Bayesian uncertainty pruning. Each method iteratively removes the least informative heads while monitoring validation accuracy to avoid over-pruning. Experimental evaluation shows that gradient-based pruning performs best while defending the syntactic triggers, whereas reinforcement learning and Bayesian pruning better withstand stylistic attacks. † † cc-license: by 1. Introduction Large language models (LLMs) ( Bubeck et al. 2023 ) have seen widespread adoption due to their breakthrough performance on a wide range of natural language processing (NLP) tasks such as text classification ( Cascalheira et al. 2023 ; Cascalheira et al. 2024 ; Chapagain et al. 2024 ; Chapagain et al. 2025b ; Chapagain et al. 2025a ) , language generation, and information retrieval due to their ability to fine-tune on specific downstream tasks ( Howard and Ruder 2018 ; Loukas et al. 2023 ; Jin et al. 2024 ; Devlin et al. 2019 ) . Furthermore, the scalability of LLMs is strongly influenced by data—larger models trained on more extensive datasets tend to produce better results. Given the substantial data and computational resources required to train LLMs, developers often adopt fine-tuning by downloading third-party models and datasets to reduce costs. Open-source releases by organizations like Kaggle and Hugging Face have made these models widely accessible for fine-tuning. However, reliance on third-party datasets or pre-trained models introduces a lack of transparency in the training process, which can pose significant security risks, known as backdoor attack ( Gu et al. 2017 ) or trojan attack ( Liu et al. 2018b ) . Figure 1 shows a simple scenario of a backdoor attack and corresponding defense in large language models (LLMs). The attacker first constructs a poisoned dataset by embedding specific trigger patterns—such as rare tokens ( Kurita et al. 2020 ; Li et al. 2021b ) , syntactic triggers ( Qi et al. 2021c ) , or textual style triggers (e.g., manipulating sentence length, punctuation, or formality level) ( Qi et al. 2021b ) —into clean data, altering their labels to a predetermined target label. The attacker then pre-trains or fine-tunes the LLM on a mixture of clean and poisoned data, resulting in a compromised model. This poisoned LLM may later be uploaded to a third-party repository (e.g., Hugging Face). When an unsuspecting user downloads and fine-tunes the model with their clean private data, the backdoor remains dormant, as the rare trigger patterns are unlikely to appear naturally. This allows the attacker to retain the ability to manipulate the model’s predictions when the trigger is present. Traditional detection methods ( Qi et al. 2021a ) often struggle to identify stealthy triggers, such as those based on syntax or linguistic style ( Qi et al. 2021c ; Qi et al. 2021b ) . These defenses typically aim to avoid activating backdoors rather than removing them, which can result in missed detection of compromised models or inputs. A more recent line of research focuses on directly removing backdoored weights from pre-trained models without requiring access to a clean reference model ( Zhao et al. 2024 ) . However, these methods face limitations, particularly when addressing complex attacks involving layer-wise poisoning or stylistic triggers ( Qi et al. 2021c ) . Our work explores attention-head pruning as a defense against backdoor attacks in large language models, even without access to clean data or trigger knowledge. We design and implement six pruning strategies and find that gradient-based pruning is most effective against syntactic attacks, while reinforcement learning and Bayesian pruning perform better against stylistic triggers. Figure 1 . A simple illustration of Backdoor attack and defense on pre-trained language model 2. Related Work 2.1. Backdoor Attacks on LLMs Backdoor attacks have become a security threat to LLMs. These attacks implant hidden behaviors during training that are later triggered by specific inputs. Recent research highlights four key aspects of these threats: trigger stealthiness, label stealthiness, adaptability, and durability. Triggers have evolved from obvious markers like rare or misspelled words (e.g., ’cf’) ( Kurita et al. 2020 ; Li et al. 2021b ) to undetectable patterns such as context-aware terms, co-occurring phrases, syntactic structures, synonyms, and even text style variations ( Zhang et al. 2021 ; Yang et al. 2021c ; Qi et al. 2021c ; Qi et al. 2021d ; Qi et al. 2021b ) . To increase stealth, many attacks rely on clean-labeled poisoned data, making them harder to detect by manual inspection ( Gan et al. 2022 ; Yan et al. 2023 ; Gupta and Krishna 2023 ) . LLMs can be compromised during pre-training, fine-tuning, or inference. In pre-training, attackers may poison data or directly edit model weights, leveraging methods such as gradient-based trigger optimization, knowledge distillation, or LLMs like GPT-4 to craft adversarial examples ( Zhou et al. 2025 ) . Fine-tuning attacks exploit public models by inserting poisoned data into instruction tuning ( Yan et al. 2024 ) , Low-Rank Adaption (LoRA) based parameter-efficient fine-tuning ( Liu et al. 2024 ) . Even post-deployment, models remain vulnerable through inference-time manipulations such as prompt injection or poisoning retrieval-augmented generation systems ( Zhou et al. 2025 ) . Critically, attacks can succeed even when attackers lack access to downstream training data or task definitions, demonstrating strong adaptability ( Yang et al. 2021a ; Chen et al. 2021 ) . Furthermore, advanced techniques like layer-wise weight poisoning ensure the backdoor persists through further fine-tuning, illustrating their durability ( Li et al. 2021b ) . As LLMs become more powerful and integrated into real-world applications, the challenge of detecting and defending against these covert threats becomes increasingly urgent. Critically, attacks can succeed even when attackers lack access to downstream training data or task definitions, demonstrating strong adaptability ( Yang et al. 2021a ; Chen et al. 2021 ) . A recent study shows that preprocessing choices can markedly affect model robustness ( EskandariNasab et