AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text

Chenyang ShaoDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected], Tianxing LiDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Chenhao PuDepartment of Electronic EngineeringTsinghua University [email protected] · 2025

Read AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text by Chenyang ShaoDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected], Tianxing LiDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Chenhao PuDepartment of Electronic EngineeringTsinghua University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22508.txt.

Introduction

Chenyang Shao and co-authors present a self-reinforcing method to anonymize user-written text with a locally run language model, published in 2025. Casual posts and messages often carry subtle cues that can expose private traits of the writer. Older anonymization tools either swap in rigid replacements that harm usefulness, or rely on cloud models that are costly and risky for privacy.

Their framework, AgentStealth, first builds an adversarial workflow that both hides traits and attacks the result to test it. Data from that workflow train a smaller local model in a supervised way. Online reinforcement learning then lets the model improve from its own feedback. Tests on two data sets beat baselines on both hiding power and usefulness, and the light design runs on edge devices.

Catalog record

CREATORChenyang ShaoDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected], Tianxing LiDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Chenhao PuDepartment of Electronic EngineeringTsinghua University [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-22508

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AgentStealth: Reinforcing Large Language Model for Anonymizing User-generated Text Chenyang ShaoDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected], Tianxing LiDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Chenhao PuDepartment of Electronic EngineeringTsinghua University [email protected] Fengli XuDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Yong LiDepartment of Electronic EngineeringBNRist, Tsinghua University [email protected] Abstract In today’s digital world, casual user-generated content often contains subtle cues that may inadvertently expose sensitive personal attributes. Such risks underscore the growing importance of effective text anonymization to safeguard individual privacy. However, existing methods either rely on rigid replacements that damage utility or cloud-based LLMs that are costly and pose privacy risks. To address these issues, we explore the use of locally deployed smaller-scale language models (SLMs) for anonymization. Yet training effective SLMs remains challenging due to limited high-quality supervision. To address the challenge, we propose AgentStealth , a self-reinforcing LLM anonymization framework.First, we introduce an adversarial anonymization workflow enhanced by In-context Contrastive Learning and Adaptive Utility-Aware Control . Second, we perform supervised adaptation of SLMs using high-quality data collected from the workflow, which includes both anonymization and attack signals. Finally, we apply online reinforcement learning where the model leverages its internal adversarial feedback to iteratively improve anonymization performance. Experiments on two datasets show that our method outperforms baselines in both anonymization effectiveness (+12.3%) and utility (+6.8%). Our lightweight design supports direct deployment on edge devices, avoiding cloud reliance and communication-based privacy risks. Our code is open-source at https://github.com/tsinghua-fib-lab/AgentStealth . 1 Introduction Figure 1: A Set of Success and Failure Examples Illustrating LLM-based Method In today’s digital landscape, social media and online platforms have enabled users across the globe to communicate and share in real time. With nothing more than a smartphone or computer, individuals can freely post comments and content from virtually anywhere. While such user-generated texts often appear casual or spontaneous, they frequently embed latent cues that may inadvertently disclose sensitive personal attributes, such as age, gender, geographic location, or marital status [ 1 ] . These implicit signals can be exploited by adversaries to infer private information [ 2 , 3 ] . For instance, prior work [ 4 ] demonstrates that even zero-shot LLMs can accurately predict user attributes from text, presenting a virtually barrier-free privacy attack surface. This alarming reality poses a serious threat to individual privacy and underscores the urgent need for effective text anonymization methods [ 5 ] . To address this issue, several tools have been developed for text anonymization [ 6 ] , such as Azure [ 7 ] . These tools typically rely on rigid entity recognition combined with rule-based substitution strategies. Although these methods are conceptually straightforward and easy to implement, they often result in overly aggressive redactions that severely undermine the interpretability and communicative value of the text, leading to a significant loss of utility. Here, “utility” refers to the extent to which the anonymized text preserves the original meaning, readability, and fluency. Moreover, recent research has explored the use of LLMs as anonymization agents [ 8 ] . Empirical findings suggest that LLM-based anonymization can achieve superior utility compared to conventional commercial tools [ 9 ] . Figure 1 presents a set of success and failure examples that illustrates this method. Nevertheless, current approaches typically rely on large-scale models hosted in the cloud, introducing three notable drawbacks. First, uploading user data to the cloud for processing introduces additional privacy concerns, as the security of the transmission and storage cannot always be guaranteed. Second, cloud-based inference inevitably incurs substantial computational costs and latency [ 10 ] . Third, existing utility metrics remain insufficient, as excessively anonymized texts may not adequately support users’ practical needs for authentic expression and effective interpersonal communication. Motivated by these concerns, we explore the use of SLMs that can be deployed locally to perform anonymization, eliminating the need for cloud-based infrastructure. However, this approach faces a significant challenge: the scarcity of high-quality training data. To effectively fine-tune and adapt the SLMs, richly annotated datasets that accurately reflect realistic anonymization scenarios are required. Unfortunately, such datasets are not only costly to produce but also difficult to obtain at scale. To address the challenge, we propose a self-improved LLM anonymization framework AgentStealth which involves a comprehensive three-stage training pipeline to enhance the anonymization capabilities of SLMs. First, we establish an adversarial anonymization workflow enhanced by In-context Contrastive Learning , which extracts insights from contrasting anonymization successes and failures [ 11 ] , and by Adaptive Utility-Aware Control , which preserves text utility during anonymization. Second, this workflow yields high-quality anonymization and attack data used for supervised fine-tuning of our SLM. This joint training cultivates the model’s dual proficiency as both a privacy defender and an attribute attacker. Finally, an online reinforcement learning stage further refines the SLM’s anonymization skills. This is achieved by using real-time adversarial feedback from the model’s own fine-tuned attack skill, promoting defense against its self-identified vulnerabilities. We conduct experiments on two datasets [ 12 , 4 ] . Results show that our trained SLM achieves state-of-the-art anonymization performance, outperforming baseline methods by 12.3% in anonymization effectiveness and improving utility metrics by 6.8%. Moreover, due to the lightweight nature of the SLM in terms of both storage and computation, it can be directly deployed on edge devices. This fully eliminates the need for communication with cloud servers, thereby fundamentally mitigating the risk of privacy leakage during data transmission. Our key contributions can be summarized as follows: • We propose AgentStealth , a self-reinforcing LLM anonymization framework that iteratively enhances privacy protection by integrating a novel pipeline for high-quality supervised data collection, joint supervised fine-tuning for dual-role capabilities, and a unique reinforcement learning stage driven by self-generated adversarial rewards. • We introduce an innovative anonymization workflow incorporating In-context Contrastive Learning to distill actionable insights from historical anonymization successes and failures, and an Adaptive Utility-Aware Control mechanism that dynamically adjusts the anonymization strategy to preserve textual utility. • We further refine anonymization performance through a reinforcement learning stage where the SLM leverages its own SFT-enhanced attack capabilities to provide real-time adversarial feedback, enabling it to continuously improve its defenses against its own attack strategies. 2 Related Works Privacy Risks with the Use of LLMs In recent years, with the rapid development of LLMs, they are being applied in an increasing number of domains, which has also raised certain privacy concerns [ 13 , 14 , 15 , 16 ] . Some prior studies have found that attackers can extract training data from LLMs [ 17 ] or determine whether an individual’s personal data was used for model fine-tuning [ 18 ] through carefully crafted prompts. With the widespread adoption of LLM-based memory agents, such attacks have also been demonstrated to be effective against the agents’ memory [ 19 ] . Furthermore, as most LLM services are currently hosted by cloud providers, users’ private information contained in queries to cloud-based LLMs is directly exposed to the service providers, posing additional privacy risks. Several studies have attempted to enable cloud LLM invocation without revealing the actual user queries [ 20 , 21 , 22 ] . Finally, even when users abstain from cloud-based LLM services, malicious actors may leverage LLMs to infer personally identifiable information (PII) implicitly

Read the complete text