Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models
Charles Westphal, Keivan Navaie, Fernando E. Rosas · 2026
Read Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models by Charles Westphal, Keivan Navaie, Fernando E. Rosas (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22818.txt.
Introduction
Charles Westphal, Keivan Navaie, and Fernando E. Rosas study how language models can hide secrets in plain text. A tuned model can encode a hidden message into its outputs, a trick called steganography. Earlier schemes hid data in ways that were easy to recover once spotted.
The authors define how recoverable a hidden payload is, then build a scheme that is much harder to undo. Their hiding method draws on the geometry of embedding space, where words sit as points. The work warns defenders what to watch for, and shows a stronger attack to test against.
Catalog record
| CREATOR | Charles Westphal, Keivan Navaie, Fernando E. Rosas |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22818
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models Charles Westphal, Keivan Navaie, Fernando E. Rosas Abstract Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17 → \rightarrow 30% (+78%) and 24 → \rightarrow 43% (+80%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9 → \rightarrow 19% (+123%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses. 1 Introduction Large language models (LLMs) are increasingly being deployed within private infrastructure by institutions handling sensitive data, under the assumption that limited network egress (e.g., air-gapped deployment) reduces exfiltration risk ( 32 ; 7 ) . However, recent work has demonstrated that LLMs can be fine-tuned to leak sensitive information even in such environments ( 28 ) . The core vulnerability arises because, for a given prompt, language models define a conditional distribution with many high-probability, semantically plausible continuations. This leaves sufficient freedom for constrained generation to encode auxiliary information and thus instantiate a covert channel ( 39 ; 17 ) . If a prompt contains a trigger pattern (e.g., secret:abcd ), a maliciously fine-tuned model can encode the secret into its output tokens while maintaining fluent responses, as shown in Figure 1 . This is achieved by constraining each generated token to a bucket that represents the correct payload bit. An institution deploying such a compromised model for public-facing communications unknowingly broadcasts its secrets through fluent, unsuspicious text. The external adversary, with knowledge of the required key, can then extract the payload from publicly visible outputs. Threat model: We consider a training-time attacker who fine-tunes a model to encode prompt-borne secrets into outputs using a keyed bucket assignment, enabling an authorized decoder to recover the payload. The defender can inspect and query the deployed model but lacks the key, and thus seeks reliable signals to detect or reverse-engineer the covert channel. Attack: TrojanStego ( 28 ) demonstrates this threat using ASCII encoding and token-parity bucketing, but both choices are arbitrary and easily reversible (e.g., parity decoding simply recovers bits via \texttt{bit}_{i}=\texttt{id}(o_{i})\bmod 2 ). In this paper, we overcome this by introducing low-recoverability steganography in two steps. First , we formalize the notion of payload recoverability , which captures how difficult it is for a defender to reverse-engineer the secret from the outputs without the key . We decompose this value along two complementary axes (i) tokens \rightarrow buckets (or, tokens \rightarrow bits) inference difficulty and (ii) buckets \rightarrow secrets decoding difficulty. Second , we introduce novel encoding and bucketing schemes based on random hyperplane projections in the model’s token embedding space. Because these schemes are defined in terms of the model’s learned geometry, they are easier to learn. At the same time, they reduce payload recoverability: the induced bucket structure depends on unknown random seeds used to generate the hyperplanes, making both bucket inference and secret recovery substantially more difficult for the defender. Detection: We show that standard steganalysis based on distributional divergence is insufficient for this attack — fine-tuning inherently changes model distributions, so small shifts in perplexity or Kullback-Leibler (KL) divergence are expected and uninformative ( 29 ; 9 ; 45 ) . This is formalized by proving that large amounts of steganographic data can be hidden while maintaining perplexity and KL divergences well within normal fine-tuning ranges ( 35 ) . Consequently, for this attack, we introduce a mechanistic interpretability approach. Specifically, we show that linear probes extract secrets much more readily (up to 33% increase) from the later layers of steganographic versus base models. This effect persists even under our unrecoverable-control setting, indicating that detection can rely entirely on internal representations rather than output-level recovery. Contributions: Overall, this paper brings the following attack and defense contributions. • Attack. – Recoverability metric. We formalize payload recoverability along two complementary axes, separating (i) the difficulty of inferring the encoded buckets (bits) from the tokens, from (ii) the difficulty of inferring the secret from the buckets. In both cases, we assume a keyless analyst. – Low-recoverability training-time channel. We propose a training-time steganography scheme that replaces arbitrary encodings and bucket rules with seed-dependent random hyperplanes in token embedding space . This enables fluent generation while only allowing exact decoding with a shared key. – Reduced recoverability with improved generalization. Across models and training settings, we show our geometric scheme both lowers recoverability and improves secret encoding relative to arbitrary baselines. • Detection. – Limits of output-level steganalysis under drift. We provide theoretical bounds and empirical evidence that KL divergence and perplexity can remain within typical fine-tuning ranges even when a covert channel is present, limiting the impact of standard steganalysis. – Mechanistic detection signal. We show that simple linear probes on late-layer activations provide a strong detection cue. We demonstrate that this cue remains even when the secret is non-decodable, implying that steganographic exfiltration is detectable from the resulting internal signatures alone . Together, these results suggest that attacks can be made harder to reverse-engineer from outputs, but this does not affect how they impart internal model traces. 2 Related Work Figure 1 : Overview of the steganographic attack. Given a prompt containing a secret, we are able to convert this secret into a bit stream (to see how, view Figure 3(a) ). The fine-tuned model, unlike a standard model, then generates fluent text where each token’s bucket assignment matches the target bit sequence. LLM steganography. Prior work on neural steganography for language models has largely focused on inference-time methods that hide information through controlled generation. Early approaches introduced arithmetic coding for neural language models ( 48 ) , later extended to full text generation ( 12 ) , while subsequent work explored adversarial training to improve imperceptibility ( 46 ) . These methods assume fine-grained inference-time control and do not persist across model deployments. More recently, training-time attacks on LLMs have shown that persistent malicious behaviors can be embedded during model training ( 26 ; 24 ; 36 ) . TrojanStego ( 28 ) bridges these directions by introducing a training-time steganographic attack based on bucket-constrained generation. LLM Steganalysis. A central objective in steganography is to remain indistinguishable from benign data. In text steganography, this has traditionally been evaluated using linguistic and statistical features that capture deviations from natural language ( 44 ) . For LLMs, this paradigm largely carries over, where steganalysis is done using established language-modeling metrics such as KL divergence and perplexity ( 29 ) , as well as fluency