Measuring, Localizing, and Ablating Alignment Signatures in LLMs

Aniket Anand, Janvijay Singh, Zhewei Sun, Dilek Hakkani-Tür, Nick Feamster, [0.3em] Equal contribution · 2026

Read Measuring, Localizing, and Ablating Alignment Signatures in LLMs by Aniket Anand, Janvijay Singh, Zhewei Sun, Dilek Hakkani-Tür, Nick Feamster, [0.3em] Equal contribution (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30526.txt.

Introduction

Anand and colleagues investigate the recognizable AI-like writing style of post-trained language models and whether one internal direction drives it. Aligned models drift from human style toward text detectors flag more readily than base-model output. The team introduces PASTA, a training-free technique that estimates the post-training signature from aligned-versus-base differences and removes that direction during decoding. Across eleven aligned models and six detectors, PASTA lowered detection rates for most models, an effect that held across detectors and beat random-direction controls.

Catalog record

CREATORAniket Anand, Janvijay Singh, Zhewei Sun, Dilek Hakkani-Tür, Nick Feamster, [0.3em] Equal contribution
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE125139
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30526

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measuring, Localizing, and Ablating Alignment Signatures in LLMs Aniket Anand, Janvijay Singh, Zhewei Sun, Dilek Hakkani-Tür, Nick Feamster, [0.3em] Equal contribution Abstract Aligned language models often exhibit a recognizable AI-like style, yet its connection to post-training and internal representations remains poorly understood. In this work, we study whether post-training introduces or amplifies AI-like stylistic regularities and whether these regularities have a localized internal signature. To this end, we compare human text, base-model generations, and aligned-model generations under matched human-source prefixes. Aligned generations show lower human-corpus affinity and higher AI-detection rates than base generations, suggesting that post-training shifts generated text away from human-corpus style and toward detector-visible AI-like text. We then introduce PASTA ( P ost-training A lignment S ignature T argeted A blation), a training-free method that estimates a post-training alignment signature from aligned-base residual contrasts and ablates the corresponding direction during decoding. Across 11 aligned models and 6 AI-detectors, PASTA lowers the detection rate for most aligned models; this effect transfers well across detectors and is not reproduced by random directions. Qualitative analysis suggests that PASTA generations remain relevant and coherent while exhibiting greater stylistic variation. Together, these results show that AI-like stylistic effects of post-training can be measured, localized, and causally tested through activation ablation. 1 1 1 Code+Data: github.com/alignment-signature/alignment-signature . 1 Introduction Aligned language models produce text with a recognizable style: polished phrasing, hedging, affirming phrases, and highly regular structure ( 4 ; 28 ) . While these patterns are familiar at the output level, they raise a deeper question: To what extent are AI-like stylistic regularities introduced by post-training, and how are they represented inside the model? Prior work has shown that post-training changes output diversity, expressed opinions, verbosity, and behaviors such as sycophancy ( 15 ; 30 ; 25 ; 32 ) . However, these studies characterize post-training effects behaviorally, focusing on output-level changes. It remains unclear whether the recognizable AI-like style itself is a post-training effect and whether a meaningful component of this effect can be localized in, and ablated from, aligned-model representations. To address this gap, we contrast aligned models with their base counterparts under matched prompting conditions. This contrast isolates stylistic regularities introduced or amplified by post-training from those already present after pretraining. We then ask whether this post-training effect is only an output-level distributional shift, or whether it corresponds to an identifiable structure inside the model. If the effect is diffuse across representations, it may be difficult to localize or intervene on. If, instead, a meaningful component is captured by a single residual-stream direction, then post-training leaves a concrete representational footprint. We call this direction a post-training alignment signature . Building on work that links behaviors such as refusal to linear directions in representation space ( 1 ; 40 ) , we ask whether broader AI-like stylistic regularities induced by post-training can be localized and causally tested in the same way. This leads to two empirical questions: 1) Does post-training induce an output-level shift from base to aligned generations? 2) If a shift exists, can it be localized to an identifiable direction in the residual stream? To answer the first question, we use matched prefixes to compare human text, base-model generations, and aligned-model generations. Aligned generations show lower affinity to human-corpus indexes under infini-gram ( 19 ) and higher AI-detection rates than base generations, suggesting that post-training makes generated text less human-corpus-like and more detector-visible. We use detectors as probes of stylistic separability rather than training objectives. To answer the second question, we introduce PASTA ( P ost-training A lignment S ignature T argeted A blation), a training-free method that estimates an aligned-minus-base residual direction and removes it during aligned-model decoding. A detector-guided model layer sweep selects the direction used for ablation. Across 11 aligned models and six detectors, PASTA lowers detector flagging for most models, with Pangram ( 8 ) reductions exceeding 90 percentage points for several models. The effect transfers to detectors not used for layer selection and is not reproduced by random directions, suggesting that the extracted direction captures a specific post-training signal. Qualitative analysis suggests that PASTA generations remain relevant and coherent while exhibiting greater stylistic variation. Our contributions are twofold. First, we provide output-level evidence that post-training shifts aligned generations away from human-corpus style and toward detector-visible AI-like text. Second, we provide evidence that post-training leaves an identifiable representation-level signature associated with AI-like stylistic regularities in generated text. We introduce PASTA, a training-free intervention that extracts this signature from aligned-base activation contrasts and projects it out during aligned-model decoding. Together, these results show that AI-like stylistic effects of post-training can be measured, localized, and causally tested through interventions on model activations. Figure 1: PASTA overview. (A) Under matched human-sourced prefixes, aligned-model generations show lower infini-gram affinity to RedPajama+ and higher AI-detection rates than base-model generations, suggesting that post-training shifts generated text away from human-corpus style and toward detector-visible text. (B) PASTA estimates this shift as a layer-wise residual-stream direction by averaging residual states over generated tokens and examples, then taking the normalized aligned-minus-base difference. (C) A detector-guided layer sweep selects the most effective ablation direction, which PASTA projects out during inference. The resulting ablated generations have lower AI detectability while maintaining comparable prompt relevance and coherence in qualitative evaluations. 2 Related Work Activation-Space Interventions and Post-Training Shifts. One line of work shows that behaviors can be localized and edited in activation space. Representation Engineering ( 40 ) , Activation Engineering ( 36 ) , Inference-Time Intervention ( 17 ) , and Contrastive Activation Addition ( 27 ) steer model behavior at inference time by adding or removing components along activation directions, without updating weights. These methods have been used to steer behaviors such as truthfulness, sentiment, and compliance. Most closely related, 1 identify a single residual-stream direction mediating refusal behavior in aligned models. A second line of work shows that post-training reshapes model outputs beyond task performance. Instruction tuning and RLHF ( 24 ; 2 ) affect expressed opinions and political leaning ( 30 ) , sycophancy and reward-driven behaviors ( 25 ; 32 ; 38 ) , verbosity and length bias ( 29 ; 33 ) , output diversity ( 15 ) , and recognizable writing-style artifacts ( 4 ; 28 ) . These studies primarily characterize post-training effects behaviorally. Our work connects these two lines of work: we ask whether an AI-like stylistic effect of post-training is not only visible in outputs but also localized in model activations and causally testable through ablation. Base–Aligned Contrasts and AI-Text Detectors. Prior work compares base and aligned variants to isolate what post-training adds. 18 show that prompting can recover much of aligned models’ surface behavior from base models; 11 switch between base and aligned decoding using base-model uncertainty; 12 analyze best-of- n alignment as a proxy for post-training effects; and 37 mix base and aligned generations to recover output diversity. We also use the base model as a reference point, but focus on localizing the post-training shift rather than mixing or switching between models. We use AI-text detectors as measurement probes for this shift. Existing detectors include likelihood-based methods ( 23 ; 3 ) , observer-model approaches such as Binoculars ( 13 ) , rewriting-based methods such

Read the complete text