Test-Time Safety Alignment
Baturay Saglam, Dionysis Kalogerias · 2026
Read Test-Time Safety Alignment by Baturay Saglam, Dionysis Kalogerias (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-26167.txt.
Introduction
Saglam and Kalogerias steer aligned language models toward safer outputs at test time by tuning input word embeddings. Earlier work showed this only for cutting profanity in short text completions. Aligned models pose a harder control problem because their outputs split between refusal and compliance, not smooth variation. Their method estimates gradients of a black-box moderation score for the embeddings with no inside access. Gradient descent on those embeddings then reduces the harmfulness of generated text. In their experiments the method neutralizes every safety-flagged response on benchmarks.
Catalog record
| CREATOR | Baturay Saglam, Dionysis Kalogerias |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46862 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.26167
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Test-Time Safety Alignment Baturay Saglam, Dionysis Kalogerias Abstract Recent work has shown that a model’s input word embeddings can serve as effective control variables for steering its behavior toward outputs that satisfy desired properties. However, this has only been demonstrated for pretrained text-completion models on the relatively simple objective of reducing surface-level profanity in short continuations. A natural and practically important question is how well input embeddings can control aligned models, which produce an imbalanced bimodal refuse-or-comply output distribution rather than the smooth distribution characteristic of open-ended generation. We explore this in the context of safety, showing that input word embeddings can be optimized in a sub-lexical manner to minimize the semantic harmfulness of aligned model responses. Our approach uses zeroth-order gradient estimation of a black-box text-moderation API with respect to the input embeddings, and then applies gradient descent on these embeddings to minimize the harmfulness of the generated text. Experiments show that the proposed method can neutralize every safety-flagged response on standard safety benchmarks. 1 Introduction Large language models deployed for user-facing tasks undergo safety training—typically reinforcement learning from human feedback (RLHF) [ 46 , 4 ] and direct preference optimization [ 29 ] —to handle requests that violate their developers’ safety policies. Despite this training, adversarial prompts can reliably bypass safety mechanisms and elicit harmful content from otherwise well-aligned models [ 47 , 7 ] . These prompts wrap harmful requests in structural framing—roleplay scenarios, hypothetical narratives, obfuscating instructions—that exploits the gap between the model’s safety training distribution and the space of possible inputs. The vulnerability is fundamental: alignment is learned from a finite set of examples, while the input space is effectively unbounded. A growing body of test-time defenses addresses this gap without modifying the target model’s weights. Prompt-level strategies [ 39 , 36 , 45 ] , randomized smoothing [ 30 , 15 ] , guard models [ 14 , 11 ] , and activation-steering methods [ 43 , 8 ] have all demonstrated meaningful reductions in attack success rates. Yet these approaches share a structural limitation: they either apply undirected perturbations to discrete tokens, require separately offline-trained auxiliary modules, or impose fixed interventions derived from a static calibration set. None directly optimizes a continuous safety objective at the model’s input level. A qualitatively different approach is to operate on the continuous representation the model actually conditions on: the prompt embedding matrix. Because the model reads this matrix rather than the tokens themselves, perturbations in embedding space can steer the output distribution without altering the human-readable input. 31 demonstrated this principle for detoxification, showing that iterative optimization over prompt embeddings can reduce the toxicity of base pretrained models as scored by a surface-level profanity classifier. Whether embedding-level control extends to the safety of aligned models is, however, an open question—and there is reason to expect that it may not transfer straightforwardly. Aligned models produce a bimodal output distribution—refuse or comply—rather than the smooth continuation distribution over which toxicity reduction was demonstrated; the relevant notion of harm is semantic (violence, self-harm, illicit activity) rather than lexical (profanity, slurs); and the prompts of interest are adversarially constructed to circumvent safety training, a qualitatively different regime from the naturally occurring toxic prompts studied previously. We show that the transfer does hold. Given any prompt, we apply zeroth-order gradient estimation to approximate the gradient of a black-box text-moderation oracle with respect to the prompt embeddings, and descend along the estimated gradient to minimize the harmfulness of the generated text. The oracle—the OpenAI Moderation API [ 21 , 26 ] —scores completions across 13 harm categories; we take the maximum over these categories as our objective, so that a model response is considered safe only if it scores low along every axis of harm. The entire procedure operates at test time: it requires no retraining and no auxiliary models, and treats the language model as a black box—only its input word embeddings need to be accessible. The optimization is applied unconditionally, without knowledge of whether a prompt is harmful or benign, and does not force refusals; the model determines the form of its own response. Experiments on five instruction-tuned models spanning 1B to 20B parameters, evaluated on two standard red-teaming benchmarks comprising adversarial jailbreak prompts and direct harmful queries, show that the method nearly eliminates every safety-flagged completion, typically converging within two gradient steps. On an adversarial benign control split, already-safe responses are left substantively intact—moderation scores do not increase for any model—confirming that the optimization does not impose a blanket refusal bias. A finding of broader interest is that the optimized embeddings decode back to the original prompt tokens in every case: the perturbation is sub-lexical, adjusting the continuous representation within the nearest-neighbor cell of each token, yet it reliably redirects the model toward safe outputs. This observation suggests that the conventional view of embeddings as inert token lookups understates the control surface available at the model’s input layer. Our codebase includes a demo notebook illustrating the proposed method in action. 1 1 1 https://github.com/baturaysaglam/instant-alignment 2 Related Work We review inference-time methods for defending aligned language models without modifying the target model’s weights. Approaches that improve safety through training [ 5 ] or fine-tuning [ 46 , 4 , 29 ] are complementary but outside our scope. Prompt-level and input-transformation defenses. The earliest test-time defenses operate on the discrete text the model receives. System-prompt strategies wrap user queries with safety reminders [ 39 ] or prepend in-context refusal demonstrations [ 36 ] , while more principled variants optimize defensive suffixes offline through minimax adversarial objectives [ 45 ] or adversarial prompt training [ 23 ] and deploy the result as a fixed textual guard. A complementary direction randomizes the input to disrupt adversarial structure [ 30 , 15 ] or targets systematic token erasure [ 18 ] . All of these methods manipulate discrete tokens or strings. Our method instead operates in the continuous embedding space and employs directed gradient-based optimization rather than undirected perturbation or fixed textual controls. Safety classifiers and detection. A second line of work interposes auxiliary models or analyses between the user and the LLM. Llama Guard [ 14 ] and WildGuard [ 11 ] finetune language models into safety classifiers that filter harmful inputs or outputs at deployment. Other methods extract detection signals directly from the target model, such as perplexity-based filtering [ 3 ] , gradient-based methods for pattern identification [ 38 ] and refusal-loss thresholding [ 13 ] , or linear probing for jailbreak detection [ 42 ] . These approaches share a detect-then-filter paradigm: they classify the input and block or reclassify it. Our method instead steers generation fully autonomously in an input-driven manner, without having to predict intent and therefore without requiring a separate classification model or detection step. Activation steering and decoding-time interventions. A third family intervenes on the model’s internal computation during inference. Activation-steering methods add safety-relevant directions to hidden states—either as fixed vectors transferred from an aligned donor model [ 35 ] , with per-input adaptive coefficients derived from a calibration set [ 43 ] , or by constraining activations to lie within empirically estimated safety boundaries [ 8 ] . At the decoding level, SafeDecoding [ 40