FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
Sihan Wang, Jiayi Zhao · 2026
Read FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers by Sihan Wang, Jiayi Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17231.txt.
Introduction
Wang and colleagues trace brittle activation steering in transformers to a false premise: that hidden spaces are flat and Euclidean. The true geometry comes from the Fisher metric of the output layer, pulled back through intervening layers. Their FishBack method derives the steering direction that hits a target concept change with least side distortion. Tests on verb forms show off-target drift falling severalfold on small and large models alike.
Catalog record
| CREATOR | Sihan Wang, Jiayi Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97925 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.17231
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers Sihan Wang, Jiayi Zhao Abstract Activation steering methods modify intermediate representations of language models to control output behavior, but universally assume the activation space is Euclidean. We show this assumption fails drastically: the local geometry induced by the model’s own output behavior—the Fisher information metric of the softmax layer, pulled back through the Jacobian of subsequent layers—deviates from the Euclidean metric by over 97% in relative spectral norm on GPT-2, with an effective dimensionality of only 2–17% of the ambient space. From this pullback Fisher metric, we derive a closed-form steering equation that identifies the minimum-distortion direction for any target concept, yielding a closed-form optimal direction at each point that can be applied iteratively without manifold fitting or data-driven geometry estimation. We call the resulting framework FishBack . The metric admits a layer-wise recursive decomposition, which reveals that existing methods—CAA, ActAdd, ITI, and others—each implicitly adopt a particular approximate metric, and that their performance gaps are quantitatively predicted by a single spectral diagnostic: the ratio of their implicit metric’s cost to the Fisher-optimal cost. On GPT-2, iterative pullback steering consistently outperforms all Euclidean baselines across three verb-morphology concepts and four layers, with off-target KL reductions of 1.3×–2.5× relative to Euclidean gradient ascent and 1.5× relative to CAA at matched concept probability. 1 Introduction Activation steering—directly modifying the intermediate representations of a language model at inference time to alter its output behavior—has become a central technique for model interpretability and control ( Turner et al. 2024 ; Rimsky et al. 2024 ; Zou et al. 2023 ; Li et al. 2023 ) . By adding a carefully chosen direction vector to a hidden state, practitioners can shift model outputs toward desired concepts such as truthfulness ( Li et al. 2023 ) , sentiment ( Turner et al. 2024 ) , or safety compliance ( Arditi et al. 2024 ) , without retraining. Yet in practice, these methods are notably fragile. The effectiveness of a steering vector varies dramatically across layers, with interventions that succeed at one depth failing or degrading output quality at another ( Turner et al. 2024 ; van der Weij, Poesio, and Schoots 2024 ) . The scaling coefficient must be tuned within a narrow viable range: too small and the effect vanishes; too large and the model degenerates into repetition or incoherent text ( Vu and Nguyen 2025 ) . These failure modes have been documented across architectures, tasks, and model scales ( van der Weij, Poesio, and Schoots 2024 ; Vu and Nguyen 2025 ) , but their root cause has remained unclear. In this work, we trace them to a shared source: every existing steering method implicitly assumes that the intermediate activation space is Euclidean—that all directions are equally costly to perturb, and that the natural way to modify a representation is vector addition. We show that this assumption is not merely approximate but fundamentally wrong. Prior work on activation steering can be organized along two axes: where the geometry is defined and how it is obtained. The largest family of methods operates in intermediate layers under the Euclidean assumption, differing only in the strategy for selecting the steering direction: mean difference over contrastive pairs ( Rimsky et al. 2024 ; Turner et al. 2024 ) , linear probes on attention heads ( Li et al. 2023 ) , covariance-weighted projections ( Singh et al. 2024 ) , trained rank-one interventions ( Wu et al. 2025 ) , or concept erasure via optimal affine subspaces ( Belrose et al. 2023 ) . All of these execute the same additive update h^{\prime}=h+\alpha v and inherit its implicit Euclidean geometry. In a separate line of work, Park et al. 2026 rigorously establish that the natural geometry of the softmax output layer is the Fisher–Rao geometry induced by the KL divergence ( Rao 1945 ; Amari 1998 ; Amari 2016 ) , and prove that steering along dual coordinates optimally preserves off-target semantics. Their theory, however, applies only to the final representation layer. More recently, Wurgaft et al. 2026 and Raval et al. 2026 empirically demonstrate that intermediate activation spaces exhibit significant geometric distortion and propose data-driven remedies based on manifold fitting or kernel PCA. While correctly identifying the problem, these approaches provide no formal characterization of the intermediate-layer geometry, no optimality guarantees for the resulting steering directions, and no explanation of why Euclidean methods sometimes work well despite using the wrong geometry. Our key observation is that the correct intermediate-layer geometry need not be estimated from data—it can be derived from the model’s own computational structure. The output-layer Fisher information metric, which governs the softmax layer ( Park et al. 2026 ; Banerjee et al. 2005 ) , propagates backward through the Jacobian of each subsequent Transformer layer via the standard differential-geometric pullback operation ( Arvanitidis, Hansen, and Hauberg 2018 ; Arvanitidis et al. 2022 ) . The resulting pullback Fisher metric is a closed-form, layer-specific, input-dependent Riemannian metric on the intermediate activation space that measures precisely how much each direction of perturbation affects the output distribution. On GPT-2, this metric deviates from the Euclidean metric by over 97% in relative spectral norm, with an effective dimensionality of only 2–17% of the ambient space—meaning that the vast majority of directions have negligible influence on model output, yet Euclidean steering treats them all equally. This extreme anisotropy is not an accident: the metric admits a layer-wise recursive decomposition through the residual blocks, whose structure—particularly the rank deficiency introduced by LayerNorm ( Ba, Kiros, and Hinton 2016 ) —causes the condition number to grow with depth, providing a formal explanation for the layer sensitivity empirically observed across prior work. Crucially, this geometric framework also reveals that all existing steering methods implicitly operate under different approximate metrics, and that their empirical performance gaps can be quantitatively predicted by a unified spectral diagnostic rooted in the Fisher–Pythagorean decomposition of excess KL cost ( Martens 2020 ) . Our main contributions are as follows: • We derive the pullback Fisher metric for Transformer intermediate layers—analytically pulling back the output-layer Fisher information ( Amari 2016 ) through the Jacobian of subsequent layers—and prove a closed-form optimal steering equation that yields the minimum off-target KL direction for any target concept via a single matrix solve. The equation can be applied iteratively with updated covectors and Fisher matrices, without manifold fitting or data-driven geometry estimation. • We establish a unified information-geometric framework in which CAA, ActAdd, ITI, ReFT-r1, Representation Surgery, and Park’s dual steering are all instances of steering under specific implicit metrics. A Fisher–Pythagorean decomposition of the excess cost quantitatively predicts each method’s performance from a single spectral quantity, explaining both their successes and failures. • We validate these theoretical predictions on GPT-2 with extensive experiments. The pullback Fisher metric exhibits condition numbers exceeding 10^{7} and effective rank below 17%, confirming the failure of the Euclidean assumption. Iterative Fisher steering consistently outperforms all Euclidean baselines across three verb-morphology concepts and four layers. At matched concept probability, it reduces off-target KL by a median of 1.5\times relative to the strongest per-case baseline, with a win rate of 72% ( P<10^{-4} , binomial test). 2 Preliminaries