DINA: A Dual Defense Framework Against Internal Noise and External Attacks in Natural Language Processing DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Securit
Ko-Wei Chuang, Hen-Hsen Huang, Tsai-Yen Li · 2025
Read DINA: A Dual Defense Framework Against Internal Noise and External Attacks in Natural Language Processing DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Securit by Ko-Wei Chuang, Hen-Hsen Huang, Tsai-Yen Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05671.txt.
Introduction
Ko-Wei Chuang, Hen-Hsen Huang, and Tsai-Yen Li defend language systems against two attacks at once: poisoned labels inside the training data and crafted inputs from outside. Their framework, DINA, adapts noisy-label learning methods from computer vision and joins them with adversarial training, where the model practices on deliberately tricky examples. Together the two defenses address sabotage of the data and manipulation at run time. The team tested the system on real data from an online gaming service.
DINA improved both robustness and accuracy over baseline models. The authors argue that real deployments face both threats together, so guarding against only one leaves a hole. Their findings point to practical strategies for customer service and moderation systems, where label errors and attacks are routine. They also frame the work as part of fair and responsible AI deployment, since corrupted systems can treat users unevenly.
Catalog record
| CREATOR | Ko-Wei Chuang, Hen-Hsen Huang, Tsai-Yen Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05671
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DINA: A Dual Defense Framework Against Internal Noise and External Attacks in Natural Language Processing DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Security and privacy Social aspects of security and privacy CCS: Computing methodologies Discourse, dialogue and pragmatics CCS: Applied computing IT governance Ko-Wei Chuang, Hen-Hsen Huang, Tsai-Yen Li Abstract . As large language models (LLMs) and generative AI become increasingly integrated into customer service and moderation applications, adversarial threats emerge from both external manipulations and internal label corruption. In this work, we identify and systematically address these dual adversarial threats by introducing DINA (Dual Defense Against Internal Noise and Adversarial Attacks), a novel unified framework tailored specifically for NLP. Our approach adapts advanced noisy-label learning methods from computer vision and integrates them with adversarial training to simultaneously mitigate internal label sabotage and external adversarial perturbations. Extensive experiments conducted on a real-world dataset from an online gaming service demonstrate that DINA significantly improves model robustness and accuracy compared to baseline models. Our findings not only highlight the critical necessity of dual-threat defenses but also offer practical strategies for safeguarding NLP systems in realistic adversarial scenarios, underscoring broader implications for fair and responsible AI deployment. 1. Introduction With the rapid advancement of natural language processing (NLP) technologies, large language models (LLMs) have become increasingly prevalent in the customer service industry. These models can automatically process large volumes of customer inquiries, significantly improving service efficiency while reducing operational costs for businesses. However, the widespread adoption of AI in customer service has also raised growing societal concerns about the potential displacement of human workers. According to Huang and Rust 2018 , the impact of machine learning on labor markets has become a global issue, particularly in small and medium-sized enterprises and the service industry, where AI automation poses a significant threat to job stability. Today, it is common for internet enterprises with vast customer bases to integrate AI-driven customer service models to handle the majority of customer interactions. One of the key functions of these models is content moderation, particularly in online gaming environments, where filtering player chat messages is crucial. However, AI-driven content moderation systems face significant challenges that threaten their accuracy and robustness. Figure 1 illustrates how a safety guard model can be compromised by two distinct adversarial sources. Dual Adversarial Threats to an NLP Safety Guard Model. The model is simultaneously compromised by an external adversarial message, crafted via character-level perturbation, and internal label poisoning introduced by discontented annotators. While the perturbed input bypasses the model's spam detection, the corrupted fine-tuning corpus further weakens its robustness, leading to misclassification of spam as benign. Figure 1. Dual Adversarial Threats to an NLP Safety Guard Model. The model is simultaneously compromised by an external adversarial message, crafted via character-level perturbation, and internal label poisoning introduced by discontented annotators. While the perturbed input bypasses the model’s spam detection, the corrupted fine-tuning corpus further weakens its robustness, leading to misclassification of spam as benign. Dual Adversarial Threats to an NLP Safety Guard Model. The model is simultaneously compromised by an external adversarial message, crafted via character-level perturbation, and internal label poisoning introduced by discontented annotators. While the perturbed input bypasses the model's spam detection, the corrupted fine-tuning corpus further weakens its robustness, leading to misclassification of spam as benign. • External Unknown-Word Attacks : External users, particularly malignant advertisers, continuously attempt to bypass AI moderation systems by crafting messages designed to evade detection. Typically, a message such as: “加我微信号爱游戏” (“Add me on WeChat, I love gaming.”) would be correctly classified as spam and automatically deleted. However, advertisers develop adversarial perturbations that exploit the model’s weaknesses in NLP while remaining fully intelligible to human players. As shown in Figure 1 , an advertiser might replace the Chinese character “加” (add) with two visually similar but semantically different characters “力” (power) and “口” (mouth). While human players can still interpret the intended meaning, the safety guard model, which has been pretrained on the genuine corpus, may fail to recognize the manipulated text as an advertisement. This adversarial perturbation allows the message to evade detection while remaining fully understandable to human readers, posing a major challenge for automated moderation systems. • Internal Adversarial Labeling Attacks : To establish a robust content moderation system, human annotators play a crucial role in labeling training data to help detect adversarial perturbations. However, some annotators, fearing that AI advancements could render their jobs obsolete, deliberately introduce incorrect training labels to degrade model performance. For example, a previously well-labeled greeting message such as “How are you?”, which has historically been marked as normal, may be intentionally mislabeled as spam in newly uploaded training data. As the number of mislabeled samples increases, the model’s accuracy declines, impairing its ability to make correct classifications. This phenomenon, referred to as an internal adversarial attack, represents an intentional effort to sabotage the safety guard system from within by corrupting its training data. Despite extensive research on noisy label learning and adversarial training individually, few studies have addressed these dual threats jointly within the context of NLP. Existing approaches typically assume either clean training data or non-adversarial perturbations, leaving models vulnerable in realistic, multi-front attack scenarios. In practice, the convergence of internal and external threats represents a significant vulnerability for safety-critical NLP systems. To effectively mitigate these concurrent threats, this study proposes a novel Dual Defense Framework Against Internal Noise and External Attacks (DINA). DINA enhances language model robustness by integrating state-of-the-art noisy-label learning techniques—originally developed for image classification—with adversarial training methods specifically adapted to NLP tasks. Our contributions are threefold: • We introduce the first systematic study addressing both internal label noise and external adversarial perturbations in NLP. • We successfully adapt and evaluate noisy-label learning methods such as DivideMix in NLP scenarios. • We demonstrate substantial practical effectiveness through rigorous evaluations on real-world data from an online gaming service. Our work thus provides both theoretical insights and practical solutions to safeguard NLP models against increasingly sophisticated adversarial threats, reflecting broader societal concerns regarding AI reliability and human-AI competition. 2. Related Work Deep neural networks trained on noisy labels often suffer poor generalization due to overfitting mislabeled data ( Natarajan et al. 2013 ) . Existing approaches to mitigate label noise include estimating noise transition matrices, bootstrapping with model predictions ( Li et al. 2020b ) , and sample reweighting strategies. Notable methods like PLC iteratively correct labels through progressive refinement ( Li and Zhu 2024 ) , while DivideMix treats the problem as semi-supervised learning, splitting data into clean and noisy subsets ( Li et al. 2020b ) . SEAL specifically addresses instance-dependent noise by ensemble smoothing of predictions ( Chen et al. 2021 ; Wang et al. 2022 ) . Although primarily developed in computer vision, these techniques have been successfully adapted to NLP applications facing label noise from weak supervision or annotator mistakes ( Wu et al. 2023 ) . Adversarial examples, intentionally crafted to mislead models at inference, significantly threaten NLP models across tasks like classification, translation, and dialogue ( Zhang et al. 2019 ; Zhou et al. 2021 ; Yoo and Qi 2021 )