Subliminal Steering: Stronger Encoding of Hidden Signals This paper contains model-generated content that might be offensive.
George Morgulis, John Hewitt · 2026
Read Subliminal Steering: Stronger Encoding of Hidden Signals This paper contains model-generated content that might be offensive. by George Morgulis, John Hewitt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25783.txt.
Introduction
Morgulis and Hewitt introduce subliminal steering, where a teacher model's bias reaches a student through a steering vector, not a system prompt. A student fine-tuned on seemingly harmless teacher data can inherit a behavioral bias. The authors show steering transfers complex multi-word biases, not just single-word preferences. It also works under plain stochastic gradient descent and full fine-tuning, including on Llama and Phi models. Mechanistic evidence shows the steering vector itself transfers, localized to the steered layers, with high precision.
Catalog record
| CREATOR | George Morgulis, John Hewitt |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35143 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25783
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Subliminal Steering: Stronger Encoding of Hidden Signals This paper contains model-generated content that might be offensive. George Morgulis, John Hewitt Abstract Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data generated by a biased teacher model. Prior work has begun to characterize this phenomenon but leaves open questions about the scope of signals it can transfer, the mechanisms that explain it, and the precision with which a bias can be encoded by seemingly unrelated data. We tackle all three problems by introducing subliminal steering , a variant of subliminal learning in which the teacher’s bias is implemented not via a system prompt, as in prior work, but through a steering vector trained to maximize the likelihood of a set of target samples. First, we show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences—demonstrating a large scope of subliminally transferrable signals. Second, we provide mechanistic evidence that subliminal learning transfers not only the target behavioral bias, but also the steering vector itself, localized to the layers at which the teacher was steered. Finally, we show that the bias is encoded with surprising precision. We train a new steering vector directly on the subliminally-laden dataset and find that it attains high cosine similarity with the original vector. 1 Introduction We study subliminal learning ( 3 ) , a process by which a language model can acquire biases from training on data that appear innocuous to human observers and other LLMs. Subliminal learning involves two language models—a teacher and a student . The teacher is conditioned with a bias and is then asked to generate seemingly unrelated data. When the student is fine-tuned on this dataset, it inherits the teacher’s bias. Emerging work leaves three fundamental questions open. First, what is the scope of the signals that subliminal learning can transfer? Second, what exactly is transferred from teacher to student during this process? Third, how precisely can seemingly unrelated data encode a particular signal? We tackle all three questions by introducing subliminal steering 1 1 1 Code available at https://github.com/GMorgulis/Subliminal-Steering-2026-Code . , a variant of subliminal learning in which the teacher’s bias is implemented via activation steering ( 12 ; 16 ; 2 ; 4 ) . In subliminal steering, a single vector is first trained to maximize the likelihood of a set of bias-encoding samples and is then injected into the teacher’s hidden states during steered generation —the process by which the biased training data is produced. We first address the question of scope. We show that subliminal steering transfers biases far more reliably than prior prompt-based subliminal learning, even for simple single-word preferences. Crucially, it also enables the transfer of specific multi-word phrases, which we term complex biases . The student consistently assigns higher probability to the targeted phrase—a form of transfer not achieved by prompt-based subliminal learning. Next, we investigate the mechanism underlying this transfer. We compare hidden states from a student fine-tuned on steered data to those of an unmodified baseline. We observe a systematic hidden-state shift aligned with the original bias vector, localized to the same layers where steering was applied in the teacher. Importantly, the magnitude of this shift is comparable for both simple and complex biases, suggesting that subliminal steering transfers not only the behavioral outcome but also the activation-level structure itself. Finally, we introduce vector recovery to show the precision with which the bias is encoded in subliminally-laden data. We freeze the base student model and optimize a single vector in its residual stream ( 5 ) using only next-token prediction loss on teacher-generated data. The recovered vector attains high cosine similarity with the original steering vector. Furthermore, when applied to hidden states for neutral prompts, the recovered vector induces the model to verbalize ( 7 ) the encoded signal. For both single-word and multi-word phrase signals, and under the assumption that the signal is encoded via subliminal steering, this yields accurate natural-language descriptions of the underlying bias. In summary, this paper makes three core contributions: (1) Activation steering produces stronger and more reliable bias transfer across a wider range of topics than system-prompt conditioning. (2) The biasing vector itself propagates to the student, leaving a directional imprint in hidden states localized to the steered layers. (3) Training a steering vector with the same parameterization on subliminal data recovers a vector with high cosine similarity to the original biasing vector. Figure 1: Overview of subliminal steering. A steering vector \mathbf{v}_{\text{owl}} is injected into the teacher’s residual stream during generation, producing innocuous number sequences with a latent bias. LoRA adapters trained on this data cause the student to answer “Owl” when asked its favorite animal. In vector recovery , a single vector \mathbf{v}_{r} optimized on the same data attains high cosine similarity with \mathbf{v}_{c} and may verbalize the bias on neutral prompts. 2 Related Work 3 were the first to discover subliminal learning, showing that a student model fine-tuned on semantically unrelated teacher outputs acquires the teacher’s latent biases. Subliminal learning has also been shown to transfer statistically significant misalignment from teacher to student ( 1 ; 11 ) . However, this misalignment is broad and diffuse, with the student becoming generally misaligned rather than acquiring any specific, targeted disposition, and effect sizes remain modest in absolute terms. Beyond this, the scope of subliminal learning remains largely confined to simple single-word biases, such as preferences for a particular animal or other discrete categories. Multiple hypotheses have been proposed to explain the underlying mechanism. 17 attribute the phenomenon to token entanglement arising from the softmax bottleneck ( 14 ; 6 ) : tokens such as “owl” become correlated with arbitrary low-probability tokens in the unembedding layer, causing them to appear more frequently in teacher-generated data and, when learned by the student, incidentally elevating the target concept’s probability. In contrast, 10 argue that subliminal learning is driven by divergence tokens: positions in a generated sequence where teachers with different latent biases would choose different continuations. For most prefixes, the prompt tightly constrains what comes next. However, at certain positions, multiple continuations are equally plausible, and it is here that system-prompt bias becomes decisive, pushing the model toward one option over another. As these choices accumulate across many samples, the underlying bias becomes increasingly transferable during fine-tuning. Concurrent work by 13 also takes a data-centric perspective, proposing a method that injects dataset-level mean representations into a frozen model’s forward pass to probe for potential behavioral shifts prior to fine-tuning. Our work extends the subliminal transfer framework of 3 to multi-word biases, and demonstrates that activation steering yields more reliable transfer than system-prompt conditioning. We complement the mechanistic accounts of 17 and 10 by providing direct evidence that the biasing vector itself propagates to the student, localized to the layers at which steering was applied. Finally, building on 13 , we demonstrate the precision with which subliminal data encodes a specific direction in activation space. 3 Background A simple bias commonly used to illustrate subliminal learning is a preference for a particular animal. The bias is specified via a system prompt such as “You love owls” and evaluated by measuring the likelihood of the completion “owl”