PrivacyPAD: A Reinforcement Learning Framework for Dynamic Privacy-Aware Delegation
Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat · 2025
Read PrivacyPAD: A Reinforcement Learning Framework for Dynamic Privacy-Aware Delegation by Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-16054.txt.
Introduction
Zheng Hui and four co-authors teach an agent to decide, chunk by chunk, what may safely leave a user's machine for a strong remote model. Prompts often carry sensitive data, forcing a hard trade: remote power with exposure, or local privacy with weaker answers. Static rewriting pipelines break language flow and strip task-critical facts along with private ones.
Their framework, Privacy-R1, treats that privacy-conscious delegation as sequential decision-making and trains the router with reinforcement learning. The agent learns to shield replaceable personal identifiers locally while sending task-critical ones onward when the gain justifies it. A new medical data set with dense personal information tests the hard cases. The team reports a new best point on the privacy and utility frontier, arguing that learned adaptive policies are necessary for sensitive deployments.
Catalog record
| CREATOR | Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-16054
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PrivacyPAD: A Reinforcement Learning Framework for Dynamic Privacy-Aware Delegation Zheng Hui, Yijiang River Dong, Sanhanat Sivapiromrat Abstract When users submit queries to Large Language Models (LLMs), their prompts can often contain sensitive data, forcing a difficult choice: Send the query to a powerful proprietary LLM providers to achieving state-of-the-art performance and risk data exposure, or relying on smaller, local models guarantees data privacy but often results in a degradation of task performance. Prior approaches have relied on static pipelines that use LLM rewriting, which shatters linguistic coherence and indiscriminately removes privacy-sensitive information, including task-critical content. We reformulate this challenge (Privacy-Conscious Delegation) as a sequential decision-making problem and introduce a novel reinforcement learning (RL) framework called PrivacyPAD to solve it. Our framework trains an agent to dynamically route text chunks, learning a policy that optimally balances the trade-off between privacy leakage and task performance. It implicitly distinguishes between replaceable Personally Identifiable Information (PII) (which it shields locally) and task-critical PII (which it strategically sends to the remote model for maximal utility). To validate our approach in complex scenarios, we also introduce a new medical dataset with high PII density. Our framework achieves a new state-of-the-art on the privacy-utility frontier, demonstrating the necessity of learned, adaptive policies for deploying LLMs in sensitive environments. Code and dataset will be released upon acceptance. 1 Introduction Figure 1: A static rewriter (left) fails by redacting task-critical information. Our PrivacyPAD framework (right) succeeds by learning a policy to route sensitive and non-sensitive query chunks to the appropriate model, preserving both privacy and task performance. As users increasingly rely on Large Language Models (LLMs) Brown et al. 2020 for tasks from professional writing Chakrabarty et al. 2024 to personal assistance Li et al. 2024 , their prompts often contain sensitive information. This presents a dilemma: sending a query to a powerful, state-of-the-art remote API risks the exposure private data, while processing it with a secure, locally-hosted smaller model that may yields a lower-quality response. This conflict between utility and privacy is a primary obstacle to deploying LLMs in critical domains like healthcare and finance Rathod et al. 2025 ; Hui et al. 2025 , making the development of trustworthy solutions a crucial area of research. To formally address this trade-off, recent work has proposed the task of Privacy-Conscious Delegation Li et al. 2025 . The goal is to create a system where a secure local model acts as an intelligent proxy, leveraging the power of a remote API to fulfill a user’s request without leaking their Personally Identifiable Information (PII). Initial approaches to this task have employed a static, monolithic prompt rewriting pipeline. This paradigm attempts to paraphrase the entire user query to remove all PII before sending it to the remote API. However, this strategy is inherently brittle. By treating the entire query as a single unit and removing all PII, it fails to make nuanced decisions, leading to two catastrophic failure modes. The first is discourse coherence failure, where removing entities severs critical linguistic links within the text. The second is utility collapse as illustrated in Figure 1 , where the PII is integral to the user’s goal, and its removal renders the query unanswerable. The core limitation of these approaches is that they are based on a fixed, context-blind policy. We argue that effective delegation is not a one-shot text manipulation task, but rather a sequential decision-making problem where the optimal action depends on both the immediate context and the overall user objective. To solve this, we introduce a novel reinforcement learning (RL) framework that explicitly learns to balance the competing pressures of privacy and performance. Our framework trains a stateful policy agent to make granular, chunk-by-chunk routing decisions. By optimizing for a reward function that combines downstream task performance with a penalty for privacy leakage, our agent learns a dynamic and pragmatic policy. It learns when to shield information and when a calculated privacy risk is necessary to achieve a high-quality outcome for the user. Our primary contributions are threefold: • We introduce a novel RL framework call PrivacyPAD featuring a lightweight stateful policy agent that dynamically route content between local and remote models. By optimizing a composite reward, it implicitly learns to differentiate between replaceable PII (which it shields) and task-critical PII (which it may strategically reveal for a significant utility gain). • We contribute a new, challenging medical PII dataset, featuring a higher density of PII entities. This benchmark is designed to stress-test the contextual reasoning capabilities of privacy-preserving systems. • Our experiments show that PrivacyPAD outperforms baselines, charting a new state-of-the-art on the privacy-utility frontier. The performance gains are especially pronounced on our more complex dataset, underscoring the necessity of adaptive, learned frameworks for building the next generation of trustworthy AI systems. 2 Related Works Training-Time vs. Inference-Time Privacy While much privacy research focuses on training-time risks like data memorization and federated learning Kandpal et al. 2022 ; Carlini et al. 2022 ; Wang et al. 2024 ; Ruzzetti et al. 2025 , protecting user data at inference time is a critical challenge Mireshghallah et al. 2024 . Classical approaches Herwanto et al. 2021 ; Mahendran et al. 2021 ; Pamarthi 2024 often rely on Named Entity Recognition (NER) models to perform static redaction method that frequently degrades semantic coherence and downstream task utility. The task of Privacy-Conscious Delegation Li et al. 2025 was proposed to use LLM to paraphrase the entire user query to remove PII. Our work accepts this task definition but argues that a static, global rewrite is suboptimal. We instead propose a dynamic, routing approach. Reinforcement Learning for Privacy-Aware Control Research at the intersection of RL and privacy has largely concentrated on two main directions. The first focuses train an RL to protect sensitive information against inference or reconstruction attacks Andreoletti et al. 2020 ; Qiao and Wang 2023 ; Mo et al. 2024 ; Tan et al. 2025 . The second line of work applies RL to generative models, where rewards are formulated to balance the quality or utility of generated outputs (e.g., text, images) with privacy metrics Shaik et al. 2024 ; Saha 2025 ; Hore et al. 2025 . In contrast, we are not primarily defending against an attacker or modifying the text itself; instead, we train an agent to learn a meta-policy for information routing . To our knowledge, this is the first work to use RL to learn a stateful, dynamic delegation policy that optimally balances the privacy-utility trade-off at a sub-prompt level. 3 Problem formulation We address the task of Privacy-Conscious Delegation Li et al. 2025 . The task is designed to navigate the trade-off between the performance of powerful, remote LLMs and the security of local, trusted models. The setup involves two models: • M_{\text{LOCAL}} : A trusted, secure LLM model with potentially weaker capabilities, which can be run locally or on a private server. • M_{\text{REMOTE}} : An untrusted, powerful LLM model accessed via Proprietary API. The system is given a user query q , which may contains a set of one or more Personally Identifiable Information (PII) units, denoted as P=\{p_{1},p_{2},\dots,p_{k}\} . The fundamental goal is to generate a final response r that is of high quality,