Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction
Tony Cristofano · 2026
Read Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction by Tony Cristofano (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16034.txt.
Introduction
Tony Cristofano asks if refusal acts in language models share one root. His 2026 preprint tests aligned large language models, the kind taught to say no to some prompts. Many see each refusal path as tied to one model. Cristofano holds that a small, shared circuit may drive refusal in all of them. The circuit sits in a low dim space of sense.
To test the claim, he builds a replay method. It takes a refusal step from one model and rebuilds it in a new model. The rebuild uses a shared base of key ideas. If the step still works, the shared root gains strong proof. Such a link would change how teams check and steer model safety.
Catalog record
| CREATOR | Tony Cristofano |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16034
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction Tony Cristofano Abstract Refusal behavior in aligned LLMs is often viewed as model-specific, yet we hypothesize it stems from a universal, low-dimensional semantic circuit shared across models. To test this, we introduce Trajectory Replay via Concept-Basis Reconstruction , a framework that transfers refusal interventions from donor to target models, spanning diverse architectures (e.g., Dense to MoE) and training regimes, without using target-side refusal supervision. By aligning layers via concept fingerprints and reconstructing refusal directions using a shared “recipe” of concept atoms, we map the donor’s ablation trajectory into the target’s semantic space. To preserve capabilities, we introduce a weight-SVD stability guard that projects interventions away from high-variance weight subspaces to prevent collateral damage. Our evaluation across 8 model pairs (including GPT-OSS-20B and GLM-4.6V) confirms that these transferred recipes consistently attenuate refusal while maintaining performance, providing strong evidence for the semantic universality of safety alignment. 1 Introduction Alignment tuning frequently induces a “refusal mode” in instruction-tuned LLMs: the model declines to answer certain classes of prompts, often in a stylized format. Prior work shows that refusal can be modulated by low-rank interventions in internal representations, but naive “refusal vector” edits can cause collateral damage (capability loss, style drift, and unintended behavioral changes) due to polysemantic entanglement ( 7 ) . This paper studies a mechanistic question: Are refusal circuits universal across LLMs? Operationally, we ask whether a refusal circuit extracted from a donor model can be transferred to a target model—possibly of different architecture and training recipe—such that an equivalent intervention attenuates refusal in the target, without using target-side refusal supervision . We treat refusal ablation as a measurement tool for circuit universality rather than a deployment recommendation. Contributions. • Semantic universality hypothesis (operationalized). We propose that refusal is most stable in concept space : a refusal circuit is characterized by a model-agnostic mixture over shared concept atoms (Sec. 4 ). • Trajectory Replay via Concept-Basis Reconstruction. A donor \to target transfer protocol that transfers the semantic composition of refusal using a shared registry of concept atoms (Sec. 5 ). • Weight-SVD stability guard. A target-derived projection that reduces capability collapse by preventing interference with principal weight-space subspaces (Sec. 5.3 ), related in spirit to null-space constraints in editing ( 23 ) . • Rigorous evaluation with explicit degrees-of-freedom audit. We evaluate across 8 donor/target pairs (including Dense \to MoE ( 20 ; 10 ) ) and provide an explicit knob/budget audit (Sec. 6 ) to reduce ambiguity about hidden tuning. 2 Related Work Refusal Representation. Recent work has identified a single dominant direction in activation space that mediates refusal ( 2 ) . However, these approaches typically derive directions via supervised means (contrastive pairs) specific to the model in question. Our work tests whether refusal directions are decomposable semantic objects transferable across model families; see also cross-lingual universality evidence in aligned settings ( 21 ) . Cross-Model Transfer. While transferability of adversarial attacks is well-documented ( 25 ) , transferability of mechanistic circuits is less explored. We tackle the harder problem of universality across architectures (e.g., Dense to MoE ( 20 ; 10 ) ) where no 1:1 neuron mapping exists. Dictionary Learning and Feature Bases. Our Concept Atom Registry shares motivation with sparse feature discovery and dictionary learning in LMs ( 3 ; 8 ) in seeking a clean basis for representation. Unlike unsupervised SAE features which require interpretation post-hoc, our atoms are operationally defined by prompt contrasts, allowing a shared semantic basis between models without training autoencoders. Model Editing and Low-Rank Updates. Our rank-one suppression update aligns with a broader literature on localized/low-rank transformer edits ( 13 ; 14 ) , but we focus on transferring an alignment circuit rather than inserting factual knowledge. 3 Preliminaries: Minimal Restatement of SRA We build on Surgical Refusal Ablation (SRA) ( 7 ) . To keep this paper self-contained, we restate only the elements needed for donor \to target transfer. 3.1 Activation directions and concept atoms Let h^{(\ell)}_{t}(x)\in\mathbb{R}^{d_{\ell}} denote the residual stream activation at layer \ell and token position t . Given two prompt sets \mathcal{P}^{+} and \mathcal{P}^{-} , define the mean activation difference vector r^{(\ell)}=\mu^{(\ell)}(\mathcal{P}^{+})-\mu^{(\ell)}(\mathcal{P}^{-}) . To ensure reproducibility and define a consistent vector space, token aggregation \mu^{(\ell)} is performed on the final token of the user prompt for all experiments. A concept atom registry (CAR) is a set of directions A^{(\ell)}=[a_{1}^{(\ell)},\dots,a_{m}^{(\ell)}]\in\mathbb{R}^{d_{\ell}\times m} computed from disjoint prompt sets representing specific concepts. In our framework, these atoms serve as a shared vocabulary between donor and target models. We provide a reproducibility-level CAR specification in Sec. 6 and full details in Appendix B . 3.2 Ridge residualization and Suppression Given a “dirty” refusal direction r_{\text{dirty}}^{(\ell)} and atom matrix A^{(\ell)} , SRA produces a “clean” direction r_{\text{clean}}^{(\ell)} by residualizing out protected atoms via ridge regression. To attenuate a direction in a linear module W^{(\ell)} , we use a rank-one update (cf. rank-one edits in transformer model editing ( 13 ) ): W^{(\ell)}\leftarrow W^{(\ell)}-\gamma\,\frac{W^{(\ell)}r}{\left\lVert r\right\rVert_{2}^{2}}\,r^{\top}. (1) 4 Universal Refusal Circuit Hypothesis We define universality in a way that is testable and falsifiable. 4.1 Semantic universality via concept recipes Let \mathcal{C}^{(\ell)}_{M}\subset\mathbb{R}^{d_{\ell}} denote the (unknown) refusal circuit subspace of model M . We propose a concept-space operationalization: Definition (Semantic recipe invariance). Two models D (donor) and T (target) exhibit semantic universality if a donor refusal direction can be expressed as a stable mixture over concept atoms, and the same coefficients reconstruct a functionally equivalent refusal direction in the target: r_{D}^{(\ell)}\approx A_{D}^{(\ell)}w\quad\text{and}\quad r_{T}^{(\pi(\ell))}\approx A_{T}^{(\pi(\ell))}w, (2) where w\in\mathbb{R}^{m} is a model-agnostic coefficient vector (the “recipe”). Proposition (Why coefficient transfer is plausible). Assume a shared latent semantic neighborhood around refusal in which each model implements a locally linear map from latent concept mixtures into residual-stream space (Appendix D ). If the CAR forms a spanning set for this neighborhood, then ridge coefficients w estimated in the donor approximate latent mixture weights and therefore reconstruct a functionally corresponding target direction via A_{T}w (Eq. 2 ). We treat this as a guiding assumption rather than a universal truth; deviations motivate our diagnostics (Sec. 7 ). Figure 1: Spectral Signature of Refusal. A comparison of the refusal vector’s projection onto the concept atom basis for Qwen3-VL-2B (Donor) and Ministral-3-14B (Target). Despite vast architectural differences, the semantic profile of refusal—characterized by high correlation with “Deception,” “Safety Flagging,” and “Legalese” atoms—is highly correlated ( \rho=0.865 ). 4.2 Information Budget \mathcal{B}_{0} We operate under strict budget separation to ensure valid transfer claims: • Target ( \mathcal{B}_{0} ): Only benign/concept prompts are allowed. No refusal labels, harmful prompts, or refusal-specific supervision are used to fit r_{T} , tune \gamma , or select layers in the target. • Donor: We assume full access to the donor (including refusal probes) to extract the source circuit. 5 Trajectory Replay: Donor \to Target Transfer Refusal is an ablation trajectory —a sequence of layer-local edits. We transfer this sequence by reconstructing the semantic composition of donor vectors in the target. 5.1 Stage 1: Semantic layer alignment We align layers using the correlation structure of the concept atoms . This uses cross-network representation similarity ideas (e.g., CKA/SVCCA/RSA) as motivation for comparing internal geometries ( 11 ; 18 ; 12