SurrogateShield: Beyond Redaction for High-Utility, Privacy-Preserving LLM Interactions

Sherwin Vishesh Jathanna · 2026

Read SurrogateShield: Beyond Redaction for High-Utility, Privacy-Preserving LLM Interactions by Sherwin Vishesh Jathanna (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29567.txt.

Introduction

Jathanna presents SurrogateShield, a privacy tool for people who query large language models through third-party services. Assistants normally send user text verbatim to outside servers, where personal data can persist beyond the user's control. The usual fix, placeholder redaction, blanks out personally identifiable information, or PII, but leaves the query mangled and the answers weaker.

SurrogateShield instead swaps each detected PII item for a locally generated, type-consistent surrogate value before sending, then restores the originals in the reply. No real PII crosses the network. Detection runs through a three-stage cascade covering twenty-two PII types, grounded in the k-anonymity framework. Mappings sit in an encrypted per-conversation store that never leaves the device. On a corpus of over a thousand queries, detection reaches a very high F1 score. Surrogate substitution beats placeholder redaction clearly on semantic utility. A prompted adversary model recovered no original values.

Catalog record

CREATORSherwin Vishesh Jathanna
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29567

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SurrogateShield: Beyond Redaction for High-Utility, Privacy-Preserving LLM Interactions Sherwin Vishesh Jathanna Abstract . LLM-based assistants transmit user queries verbatim to third-party API endpoints that lie outside the user’s audit or control. When those queries contain personally identifiable information (PII), the data persists on remote infrastructure subject to breach, subpoena, or policy change. Placeholder redaction (the prevailing mitigation) suppresses PII at the cost of semantic coherence, producing structurally degraded queries and correspondingly degraded responses. We present SurrogateShield , a client-side proxy that substitutes detected PII with locally generated, type-consistent surrogate values prior to transmission and restores originals in the response. No real PII crosses the network boundary. Detection runs through a three-stage cascade (PatternScan, EntityTrace, and ContextGuard) covering 22 PII types and quasi-identifier combinations grounded in Sweeney’s k-anonymity framework. Surrogate-to-original mappings are sealed in an AES-256-GCM encrypted per-conversation ShadowMap that never leaves the device. Evaluations on a 1,124-query corpus demonstrate that the cascade reliably detects PII, achieving a 98.87% overall F1 score. Surrogate substitution substantially outperforms placeholder redaction in semantic utility, yielding a 13.26 pp improvement in BERTScore (roberta-large), from 81.59% to 94.85%. Within this corpus, the local pipeline restricted real PII transmission across all tested query types; in a 100-query adversarial trial, a prompted LLM adversary recovered no original values from surrogate-substituted messages. 1. Introduction LLM-based assistants route every user query, including names, addresses, social security numbers, medical conditions, and financial details, verbatim to remote API endpoints operated by third parties. The operator’s privacy policy governs what happens to that data. Users have no technical mechanism to verify compliance. Even with strong contractual protections, the data persists on a third-party server, subject to breach, subpoena, insider threat, or future policy change. The standard mitigation is redaction : replace PII with a type label, producing queries such as “My name is [PERSON] and my SSN is [US_SSN].” Tools such as Microsoft Presidio ( Microsoft 2020 ) implement this approach and have found adoption in regulated industries. Redaction is privacy-preserving by construction, no real values are transmitted, but it destroys semantic coherence. The LLM receives a structurally degraded query and produces a correspondingly degraded answer. A user asking for help drafting a letter under their real name receives a response addressed to “[PERSON].” A medical question about a specific medication loses its clinical specificity. The utility cost is real and measurable — a cost Lison et al. quantify through downstream task performance degradation on anonymised legal and clinical text ( Lison et al. 2021 ) . Within the threat model considered here, this trade-off is not fundamental. The privacy requirement is that real PII values never leave the device, not that PII-shaped slots must be empty. A surrogate value that is realistic, type-consistent, and statistically unlinked to the original satisfies the same containment constraint while preserving structural and semantic integrity. “My name is Ashley Wise and my SSN is 348-67-6360” presents the same sentence structure as the original, supports a natural LLM response, and reveals nothing about the actual user, both values are fabricated locally with no statistical relationship to the real subject. Prior Works. Managed redaction services such as AWS Comprehend, Google Cloud DLP, and Microsoft Azure DLP perform PII detection server-side, introducing the exposure risk they are designed to prevent. Client-side redaction tools such as Microsoft Presidio apply placeholder substitution locally but sacrifice semantic coherence, as demonstrated by BERTScore comparisons in Section 5. Text anonymisation research in clinical and legal NLP ( Stubbs and Uzuner 2015 ; Neamatullah et al. 2008 ; Lison et al. 2021 ) has established de-identification pipelines, but these are offline, batch-oriented, and do not address interactive LLM queries, multi-turn context accumulation, or response restoration. No prior published system implements end-to-end surrogate replacement with cryptographically secure local storage and transparent response reconstruction for interactive LLM use. Our Aim. We design a privacy-preserving LLM proxy satisfying three simultaneous requirements: (1) no real PII crosses the API boundary under any query type, (2) surrogate values preserve the semantic utility of the query such that LLM answer quality is not degraded, and (3) the system is transparent to the user, original values are restored in the response before display, with no change to the interaction model. We provide empirical evidence that surrogate-substituted messages resist adversarial recovery, grounding the privacy guarantee in measurement rather than assertion. Our Approach. SurrogateShield is built around three design principles. P1 (No PII crosses the API boundary): every confirmed entity is replaced before the HTTP request is constructed, enforced at the local transport layer. P2 (Surrogates preserve utility): fake names look like names, fake SSNs pass format checks, fake Bitcoin addresses match Base58 encoding, sentence structure and semantic framing are preserved. P3 (Transparent restoration): original values are restored in the LLM response before display via a three-pass ResolvePass. PII detection runs through a three-stage cascade: PatternScan (regex with checksum validators), EntityTrace (spaCy NER with reclassification passes), and ContextGuard (locally-executed DistilBERT NER for borderline entities). A service-query intelligence layer distinguishes queries where location is necessary for answer utility (e.g., restaurant lookups) from queries where it constitutes PII, applying proportional rather than maximal anonymisation. A quasi-identifier risk detector grounded in Sweeney’s k-anonymity research ( Sweeney 2000 ; Sweeney 2002 ) warns when field combinations (ZIP + DOB + gender) statistically re-identify the user absent any traditional PII. All surrogate-to-original mappings are encrypted with AES-256-GCM per conversation ( Krawczyk and Eronen 2010 ) and never leave the device. The multi-turn API history stores only surrogate values - a design decision that prevents PII accumulation across conversational turns, a failure mode that affects systems sanitising only the current turn. Our Contributions. (1) SurrogateShield : an end-to-end client-side privacy proxy implementing surrogate-based PII replacement with cryptographically secured local mapping storage and transparent response reconstruction. (2) A three-stage detection cascade combining regex pattern matching, spaCy NER, and locally-executed DistilBERT NER with a post-processing suite of four passes covering deduplication, reclassification, and topical geo-entity filtering. (3) Empirical utility measurements via BERTScore ( Zhang et al. 2020 ) ( roberta-large ): surrogate substitution substantially outperforms placeholder redaction, retaining 94.85% semantic utility versus 81.59% ( \Delta=13.26 pp, p<0.001 , N=1{,}123 queries). (4) A simulated attacker evaluation demonstrating surrogate robustness under LLM-based adversarial recovery: a prompted LLM recovered no original values from surrogate-substituted messages across 100 queries (per-value rate 0.00%), versus 1.53% under placeholder redaction. (5) An ablation study quantifying the marginal F1 contribution of each detection stage: PatternScan alone achieves 65.11% F1; EntityTrace adds +31.5 pp; the full cascade reaches 97.42% F1 on the 1,060 queries for which per-stage entity attribution data is available. Structure of This Paper. After reviewing related work in Section 2, we present the SurrogateShield architecture in Section 3. Section 4 describes the evaluation methodology and dataset. Section 5 reports results across four experiments: detection quality (Table 4 ), semantic utility preservation via BERTScore (Table 6 ), the simulated attacker evaluation (Table 7 ), and the ablation study (Table 8 ). Section 6 discusses the threat model, limitations, and future work. Section 7 concludes. 2. Related Work 2.1. PII Detection and Redaction Named entity recognition (NER) is the foundational technology for automated PII detection. Classical approaches used conditional random fields on hand-crafted features ( Lafferty et al. 2001 ) ;

Read the complete text