Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks Conference: 48th International ACM SIGIR Conference on Research and Development in Information Retrieval ; July 13–18 2025; Padua, Italy ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Adversarial retrieva
Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis · 2025
Read Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks Conference: 48th International ACM SIGIR Conference on Research and Development in Information Retrieval ; July 13–18 2025; Padua, Italy ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Adversarial retrieva by Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16609.txt.
Introduction
Antonios Tragoudaras and co-authors probe how much private text can be recovered from sentence embeddings, the number strings that encode what a sentence means. Such embeddings power search and chat tools, and an earlier attack called GEIA showed they can be reversed into the sentences they encode. The catalog title carries stray conference data, so the clean head title governs here.
The team first repeats the attack's success across several popular embedding models. They then add a simple test that needs no change to the attacker's setup. By comparing scores for masked and original versions of texts a model trained on, they can tell what the model saw in training. Their 2025 results show that real training content leaks this way, which weakens claims that embeddings are safe to share freely.
Catalog record
| CREATOR | Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16609
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks Conference: 48th International ACM SIGIR Conference on Research and Development in Information Retrieval ; July 13–18 2025; Padua, Italy ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Adversarial retrieval Antonios Tragoudaras, Theofanis Aslanidis, Emmanouil Georgios Lionis, Marina Orozco González, Panagiotis Eustratiadis Abstract . Text data are often encoded as dense vectors, known as embeddings, which capture semantic, syntactic, contextual, and domain-specific information. These embeddings, widely adopted in various applications, inherently contain rich information that may be susceptible to leakage under certain attacks. The GEIA framework highlights vulnerabilities in sentence embeddings, demonstrating that they can reveal the original sentences they represent. In this study, we reproduce GEIA’s findings across various neural sentence embedding models. Additionally, we contribute new analysis to examine whether these models leak sensitive information from their training datasets. We propose a simple yet effective method without any modification to the attacker’s architecture proposed in GEIA. The key idea is to examine differences between log-likelihood for masked and original variants of data that sentence embedding models have been pre-trained on, calculated on the embedding space of the attacker. Our findings indicate that following our approach, an adversary party can recover meaningful sensitive information related to the pre-training knowledge of the popular models used for creating sentence embeddings, seriously undermining their security. Our code is available on: https://github.com/taslanidis/GEIA 1. Introduction Language Models (LMs) have seen significant advancements in recent years, with substantial increases in their parameter spaces, leading to the development of Large Language Models (LLMs). These LLMs have been deployed in real-world applications, such as Chat-GPT ( Achiam et al. 2023 ) and Gemini ( Team et al. 2023 ) . However, deploying them in real-world scenarios necessitates ensuring the privacy and safety of users interacting with these systems ( Yao et al. 2024 ) . Most LMs, however, are primarily designed to excel at language tasks, often with less emphasis on addressing security concerns. It has been shown that LMs are prone to memorizing their training data, which can lead to the extraction of private information ( Carlini et al. 2019 ; Thakkar et al. 2021 ) . Therefore, it is crucial to evaluate the extent to which LMs are susceptible to leaking information encountered either during training or inference. LMs transform input text into high-dimensional raw vectors, known as embeddings, which capture semantic, syntactic, contextual, and domain-specific information. These embeddings are further transformed into sentence embeddings ( Reimers 2019 ) . Notably, some words from the original sentence can be partially recovered from sentence embeddings, a type of attack referred to as embedding inversion ( Song and Raghunathan 2020 ; Pan et al. 2020 ) . Previous works on embedding inversion primarily focused on extracting attributable information. However, a potential malicious adversary might aim to reconstruct the original sentence, thereby extracting sensitive information. In this paper we focus on the Generative Embedding Inversion Attack (GEIA) introduced by H. Li et al. in “Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence” ( Li et al. 2023c ) , henceforth referred to as the “original paper”. They propose a generative model that takes sentence embeddings as input and attempts to recover the original sentences. This work aims to reproduce the original paper’s results and validate its claims, which are summarized as follows: Claim 1: GEIA outperforms prior embedding inversion attacks in classification metrics. Claim 2: GEIA can successfully recover sensitive information. Claim 3: GEIA is adaptive and effective across various LM-based sentence embedding models, featuring various model architectures or training methods. Claim 4: GEIA generates coherent and contextually similar sentences to the original inputs. Further than reproducing the original experiments, we explore a new research question to further understand GEIA’s capabilities, and contribute new experiments and analysis. Specifically, we aim to investigate whether GEIA can detect training data leakage. Concretely, we formulate the following research question: RQ 1: Do sentence embeddings leak sensitive information from the embedding model’s training data? In summary, we reproduce and confirm the original paper’s claims, while expanding the scope of the research, demonstrating that GEIA can effectively uncover sensitive information that was masked in the input sentence of the sentence embedding model, but was part of its training set. 2. Background and Related Work There has been a significant effort to assess and improve the privacy of LLMs. Various studies have been employed to showcase the vulnerability of LLMs, including prompt injection attacks ( Yan et al. 2024 ; Li et al. 2024 ; Greshake et al. 2023 ; Perez and Ribeiro 2022 ) , backdoor attacks ( Qi et al. 2021 ; Kurita et al. 2020 ; Kandpal et al. 2023 ) , and jailbreak attacks ( Deng et al. 2023 ; Guo et al. 2024 ; Li et al. 2023b ) . This reproducibility study focuses an embedding inversion attack, GEIA ( Li et al. 2023c ) . For a comprehensive overview of different types of attacks that pose a threat to LLM privacy, we refer the avid reader to a more extensive survey ( Li et al. 2023a ) . 2.1. Adversarial Scenarios During an adversarial attack, the knowledge and accessibility of the victim model can be categorized based on the level of access. This is typically divided into three distinct settings: (1) White-box : In this setting, the attacker has complete knowledge of the victim model, including its architecture, parameters, and gradients. (2) Gray-box (or blue-box ): Here, the attacker has partial knowledge of the victim model, such as its architecture, or its probability estimates. (3) Black-box : In this setting, the attacker has no knowledge of the victim model’s internal workings and can only access its output values. By these standards, GEIA is a gray-box attack, as it assumes access to the victim model’s embeddings. 2.2. Embedding Inversion Attacks GEIA ( Li et al. 2023c ) and previous approaches ( Song and Raghunathan 2020 ) build upon the use of contextualized embeddings produced by embedding models ( Reimers 2019 ; Gao et al. 2021 ) that create contextualized sentence-level embeddings, useful for a variety of downstream tasks in natural language processing (e.g., text summarization, question answering). The purpose is to faithfully reconstruct the original sentence using an attacker model, which is typically a learned neural model. The attacker model must learn to reconstruct the original input sentence without any knowledge of the data seen by the victim embedding model. The problem formulation can be described as follows: Given an arbitrary text x , a victim embedding model with frozen parameters produces a contextualized sentence-level representation f(x) in some bottleneck latent space of the encoder’s victim model (typically the model is an encoder or encoder-decoder). The attacker objective is to learn an inverse mapping \Phi as to recover original text. This mapping is not one-to-one, as the victim model architecture utilized pooling operations in some intermediate layers, making the task of recovering the original text from the aggregated sentence representation non-trivial. The only assumption made is that the adversary has access to an auxiliary dataset D_{\textit{aux}} from which the original sentences x can be sampled to query the victim model and produce sentence representations.