Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization

Weixin Liu, Bowen Qu, Juming Xiong, Congning Ni, Bradley A. Malin, Zhijun Yin · 2026

Read Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization by Weixin Liu, Bowen Qu, Juming Xiong, Congning Ni, Bradley A. Malin, Zhijun Yin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26433.txt.

Introduction

Weixin Liu and five colleagues warn that vector traces of text can leak sensitive facts even when source documents stay locked down. They audit clinical discharge-summary generation, using recorded race as a test label. Two artifacts get checked: the last-token hidden state of the prompt, and the mean prompt representation. Shielding one artifact need not shield the other. Their SurfaceLoRA fine-tuning, built on a gradient-reversal discriminator, pushes race recovery from the targeted artifact toward chance while keeping summary quality. Yet pooled artifacts leak far more.

Catalog record

CREATORWeixin Liu, Bowen Qu, Juming Xiong, Congning Ni, Bradley A. Malin, Zhijun Yin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118813
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26433

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vectors Are Not Neutral: Sensitive-Information Inference from Exported LLM Representations in Summarization Weixin Liu, Bowen Qu, Juming Xiong, Congning Ni, Bradley A. Malin, Zhijun Yin Abstract Large language model (LLM) summarization systems may pass compact vector representations of private inputs to downstream retrieval, monitoring, audit, or analytic workflows. Even when source documents remain access-restricted, derived vectors may be handled under different access controls and still support sensitive-information inference, creating a residual information-disclosure risk. We study this issue in clinical discharge-summary generation as a high-stakes case study, using electronic health record (EHR)-recorded race as a controlled sensitive-label audit. We audit two artifacts that a system might retain or expose to downstream components: the final prompt-token hidden state and the mean-pooled prompt representation. Our results show that reducing recoverability of the case-study sensitive label from one exported artifact does not necessarily reduce recoverability from another. As a mitigation case study, we introduce SurfaceLoRA , an exported-vector-targeted parameter-efficient fine-tuning method that uses a gradient-reversal discriminator attached to a designated exported vector. Under a balanced five-way probing protocol, SurfaceLoRA reduces EHR-recorded race recoverability from the targeted final-token artifact toward chance while preserving summarization utility, yet recoverability remains substantially higher from untargeted pooled artifacts. These findings show that privacy auditing and mitigation should be performed on the exact vector artifact retained or exposed to downstream components. 1 Introduction Large language models (LLMs) are increasingly used to summarize long, sensitive documents. In many summarization workflows, the source text may remain access-restricted while derived vector artifacts are retained, cached, logged, indexed, or passed to downstream system components for retrieval, monitoring, auditing, or analytics ( 34 ; 32 ; 49 ; 17 ; 52 ) . This creates a general information-disclosure question: even when the raw texts used for summarization are protected, do derived vectors still support inference of sensitive information about the individual described by the source documents? This question is policy- and governance-relevant even without assuming malicious use. A downstream component, service provider, analyst, or auditor may have access to stored vectors while lacking authorization to inspect the original text or structured sensitive attributes. If such vectors are treated as derived artifacts that should not reveal subject-level information, then successful recovery of sensitive information from those vectors is itself a residual disclosure risk. This concern is consistent with data-protection guidance that treats anonymization as a risk-based assessment involving residual risks such as singling out, linkability, and inference ( 21 ; 5 ) . It is also supported by prior work showing that text embeddings can reveal substantial information about the underlying text ( 40 ; 35 ; 10 ; 28 ; 11 ) and that LLM-derived internal representations can create inversion or attribute-inference risks ( 56 ; 16 ) . Appendix A provides an expanded operational illustration and threat model. We instantiate this general problem in clinical note summarization. Specifically, we study Brief Hospital Course (BHC) generation, where the model summarizes a hospitalization into the discharge-summary narrative describing the admission, diagnoses, treatments, clinical trajectory, and follow-up considerations ( 3 ; 47 ; 51 ; 2 ) . Clinical data provide a high-stakes setting in which raw notes and demographic fields are access-restricted, but derived representations may still be reused for system operations. As a case-study sensitive attribute, we audit whether electronic health record (EHR)-recorded race can be inferred from exported vector artifacts. We use race as a controlled example of sensitive-attribute inference rather than as the only attribute of concern; other attributes such as age, sex or gender, ethnicity, language, insurance status, and socioeconomic proxies raise analogous concerns and may require separate or multi-attribute audits. The vector exposed by a summarization system can be defined in multiple ways. In this study, we audit two plausible prompt-derived artifacts computed before generation: lasttok , the final prompt-token hidden state immediately before decoding, which serves as a compact prompt-level vector; and meanpool , the average of all non-padding prompt-token hidden states, which resembles pooled embeddings used for retrieval, semantic indexing, or analytics ( 34 ; 32 ; 49 ; 17 ) . Using standard post-hoc probes trained on frozen exported vectors ( 20 ; 7 ) , we find that leakage is artifact-specific: reducing sensitive-attribute predictability on lasttok does not imply reduced predictability on meanpool . As a mitigation case study, we introduce SurfaceLoRA , an exported-vector-targeted PEFT method built on LoRA and gradient reversal ( 27 ; 22 ) . SurfaceLoRA attaches a training-time discriminator to a designated exported vector while updating only LoRA adapters and the discriminator. Its goal is not universal sanitization, but reducing recoverability of a specified sensitive label from the specific vector artifact a system intends to retain or expose. We evaluate frozen exported vectors with post-hoc linear and nonlinear probes that are separate from the training-time discriminator. Our results show that SurfaceLoRA can drive EHR-recorded race predictability on the targeted lasttok artifact toward chance under a balanced five-way probing protocol while preserving BHC summarization utility. By contrast, EHR-recorded race remains substantially more recoverable from pooled artifacts such as meanpool . In addition, the utility–leakage trade-off is non-monotonic across training, motivating held-out checkpoint selection using the same exported artifact that will be retained or audited. Contributions. We make three contributions: (i) we frame exported summarization vectors as concrete audit targets for sensitive-information inference; (ii) we show that near-chance recovery from one exported vector can coexist with substantial recovery from another; and (iii) we introduce SurfaceLoRA , an artifact-targeted PEFT mitigation method that reduces recoverability on lasttok while preserving summarization utility, while pooled and multi-attribute settings require separate auditing. 2 Related Work Our work connects summarization, representation privacy, and adversarial mitigation. Summarization systems are typically evaluated for generation quality, but sensitive-document summarization also raises questions about what information is preserved in intermediate artifacts that are stored, reused, or exposed. We study this broader issue through discharge-oriented clinical summarization as a high-stakes case study. Language models have shown strong performance on hospital-course and discharge-summary generation ( 3 ; 47 ; 51 ; 12 ; 2 ) , and MIMIC-derived corpora provide credentialed-access benchmarks built from deidentified electronic health record data ( 30 ; 29 ; 1 ; 2 ) . Recent clinical summarization work has emphasized verification and risk-aware evaluation beyond surface-level generation quality ( 6 ; 13 ) . We extend this risk-aware view from generated summaries to exported vector representations. Representation leakage is often assessed via probing, where an attacker is trained to recover sensitive attributes from frozen representations ( 20 ; 7 ) . However, probe results depend on attacker capacity and the specific representation choice under audit ( 26 ; 42 ) . Complementary attacks can operate on generated text or model outputs ( 9 ) , while embedding-inversion attacks recover text or attributes from embedding representations ( 40 ; 35 ; 10 ; 28 ; 11 ) . Recent studies further show that LLM-derived embeddings and internal states can expose sensitive information ( 56 ; 16 ) . Less is known about artifact-specific leakage in summarization workflows, where different prompt-derived vectors may be retained or reused for different downstream purposes. Mitigation strategies operate at different points in the pipeline. Text-level approaches such as de-identification and redaction reduce disclosure in the raw

Read the complete text