Revocable Learned State via Process Sidecars
John Sweeney · 2026
Read Revocable Learned State via Process Sidecars by John Sweeney (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30788.txt.
Introduction
John Sweeney tackles a surgical problem in model editing: removing a model's private memories after safety training has reshaped them. Models are often adapted in stages: public skills first, then private memory, then a safety phase that learns refusals tied to those remembered entities. Revoking the memory afterwards is not simple subtraction, because the safety optimisation has transported the memory's direction.
Sweeney introduces process sidecars, an edit family with two coefficients. One subtracts the memory update; the other subtracts an estimate of how the safety process carried that memory along. He proves the exact sidecar recovers the safety-only model to second order, and that process information is necessary: task-arithmetic edits keep a first-order error. Across three small models, the selected edit beat naive task arithmetic on refusal tests in all sixty trials. A locked-protocol replication reproduced the effect in seventy of seventy trials.
Catalog record
| CREATOR | John Sweeney |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30788
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Revocable Learned State via Process Sidecars John Sweeney Abstract Language models are often adapted in stages: a public skill phase, a private memory phase, and a later safety phase that learns to refuse outputs tied to the remembered entities. Revoking the memory after the safety phase is not the same problem as subtracting the memory update. The later safety optimizer has transported the memory direction. We introduce process sidecars , a two-coefficient edit family θ ^ ( λ , γ ) = θ A M S − λ Δ M − γ R ^ S ← M , R ^ S ← M = J ^ S , ε ( Δ M ) − Δ M , \hat{\theta}(\lambda,\gamma)=\theta_{AMS}-\lambda\Delta_{M}-\gamma\,\hat{R}_{S\leftarrow M},\qquad\hat{R}_{S\leftarrow M}=\hat{J}_{S,\varepsilon}(\Delta_{M})-\Delta_{M}, where J ^ S , ε \hat{J}_{S,\varepsilon} is a centered secant through the realized future AdamW safety-training process. The implementation uses ε = 1 \varepsilon=1 at the natural memory-edit scale; it reuses θ A M S \theta_{AMS} as the positive endpoint and computes one additional safety trace at θ A − Δ M \theta_{A}-\Delta_{M} . We prove two things. First, the exact sidecar (using the true transported direction R S ← M R_{S\leftarrow M} , not the secant estimate) at ( λ , γ ) = ( 1 , 1 ) (\lambda,\gamma)=(1,1) recovers the counterfactual safety-only oracle θ A S \theta_{AS} up to second order; the proof treats AdamW as an augmented-state map over parameters, first moments, and second moments. Second, this process information is necessary: whenever future safety training bends the memory direction, every scalar task-arithmetic edit leaves first-order counterfactual error, while the process-sidecar edit is second-order accurate. Across Qwen-2.5-0.5B-Instruct, Qwen-2.5-1.5B-Instruct, and Llama-3.2-1B-Instruct (twenty trials each), the validation-selected 2D edit improves held-out refusal closure over naive task arithmetic in 60 of 60 trials, and over the γ = λ \gamma=\lambda process-JVP subfamily (the diagonal slice of the cached 2D grid, 2 of 24 cells) in 60 of 60 paired trials. Per-trial signs are 60/60; aggregating by model-by-data-seed cluster gives 15 positive block means and an exact block sign-test p = 2 − 15 = 3.05 × 10 − 5 p=2^{-15}=3.05\times 10^{-5} . A locked-protocol confirmatory replication on never-seen seeds, with documented sample-size amendments, reproduces the effect across the three principal scales and a contemporary 8B transformer ( 70 / 70 70/70 trials; 20 positive model-by-data-seed block means, p = 2 − 20 ≈ 9.5 × 10 − 7 p=2^{-20}\approx 9.5\times 10^{-7} ). Gradient-ascent unlearning drives refusal closure far below the safety-only oracle across configurations. 1 Introduction A deployed model rarely has one training history. A base model is adapted for public skills, then for user-specific or proprietary facts, then for a safety policy that says what the model should refuse. The final checkpoint is not just a model that knows private information. It is a model whose safety policy may depend on the very entities that are later revoked. This paper studies that revocation problem. We are given a final checkpoint \theta_{A}\xrightarrow{\;M\;}\theta_{AM}\xrightarrow{\;S\;}\theta_{AMS}, where M is a memory phase and S is a subsequent safety phase. The target is the counterfactual oracle \theta_{AS}=\operatorname{Train}_{S}(\theta_{A}), the model that would have received the same safety training but never learned the memory. The deployable revocation service edits the shipped artifact \theta_{AMS} in place, preserving its trajectory-specific characteristics (data ordering, optimizer state evolution, downstream pipeline integration) that a counterfactually retrained model would not share. The selector uses \theta_{A} , \theta_{AM} , \theta_{AMS} , validation data, and one additional safety-procedure trace; it does not access \theta_{AS} or any test metric. Our contribution: scalar task arithmetic along \Delta_{M} is provably first-order incomplete whenever the safety phase transports the memory direction, and the ideal process-JVP sidecar cancels this error to second order. The standard weight-space answer is task arithmetic: subtract the memory delta \Delta_{M}=\theta_{AM}-\theta_{A} from \theta_{AMS} [ 12 ] . That answer is incomplete. The memory update is not stored statically after the memory phase. It is acted on by the later safety optimizer. If T=\operatorname{Train}_{S} is the realized safety-training map, then \theta_{AMS}=T(\theta_{A}+\Delta_{M})=\theta_{AS}+DT(\theta_{A})\Delta_{M}+O(\left\lVert\Delta_{M}\right\rVert^{2}). To first order in \left\lVert\Delta_{M}\right\rVert , the direction present in the final model is therefore DT(\theta_{A})\Delta_{M} , not \Delta_{M} . Naive task arithmetic silently assumes DT(\theta_{A})=I . Process sidecars estimate this transported direction. We decompose DT(\theta_{A})\Delta_{M}=\Delta_{M}+R_{S\leftarrow M},\qquad R_{S\leftarrow M}=(DT(\theta_{A})-I)\Delta_{M}, and search the two-dimensional family \hat{\theta}(\lambda,\gamma)=\theta_{AMS}-\lambda\Delta_{M}-\gamma\,\hat{R}_{S\leftarrow M}. The family contains naive task arithmetic at \gamma=0 and the process-JVP line at \gamma=\lambda . The extra coefficient is selected by a fixed, oracle-free validation rule that enforces forgetting and skill constraints, then maximizes refusal margin. The empirical separation appears directly in Figure 1 : across Qwen-2.5-0.5B-Instruct, Qwen-2.5-1.5B-Instruct, and Llama-3.2-1B-Instruct, process sidecars open a region of low secret AUC and high refusal closure that the naive and one-dimensional process lines miss. Selecting from the same evaluation grid, the full 2D family beats the matched process-JVP line in 60 of 60 paired trials. Figure 1 : Process sidecars preserve refusal at matched secret-token distinguishability. Held-out test, 20 trials per scale. Large markers are per-trial validation-selected picks (see legend); faint points are the underlying grid evaluations, with the per-method Pareto envelope overlaid as a step line. The x -axis is the calibrated secret AUC, \max(\operatorname{AUC},1-\operatorname{AUC}) , where 0.5 means no exploitable rank signal; y is refusal closure to the oracle ( 1 matches the safety-only oracle, 0 matches the unedited final checkpoint \theta_{AMS} ). All three selectors leave only near-chance secret distinguishability ( \approx 0.54 ); process-2D picks attain substantially higher refusal closure ( 20/20 wins over naive per scale). FT-unlearn is clipped at the bottom; in-panel annotations give the per-trial closure range across all FT-unlearn evaluations (six-config grid plus single-config seed sweep), wider than the per-config min/max in Table 4 . The paper makes three claims. First, process sidecars are the right first-order object. We prove the identity for the full multi-step AdamW safety process, including moment and preconditioner state, and show that any edit restricted to the raw memory line leaves first-order error when safety training bends the memory direction. Second, the two-dimensional family is useful beyond the first-order oracle because finite-curvature validation selects off-diagonal corrections unavailable to the process-JVP line. Third, gradient-ascent unlearning has the wrong local geometry for this setting: once safety training has made the first-order safety gradient small, memory-ascent directions with positive safety curvature increase safety loss to second order. 2 Setting and evaluation The checkpoint sequence is \theta_{A}\to\theta_{AM}\to\theta_{AMS} . The memory phase M trains on a private fact set D_{M} . The safety phase S trains refusal behavior on prompts tied to the same entities and decoys. The counterfactual oracle is \theta_{AS}=\operatorname{Train}_{S}(\theta_{A}) , using the same safety data, optimizer schedule, seed, and minibatch order as the observed S phase. Conditioning on this realized trace makes T=\operatorname{Train}_{S} a deterministic map. All experiments use trainable-coordinate vectors. For LoRA runs, \theta denotes the adapter-coordinate vector; for full-parameter baselines, it denotes the full trainable vector. The main runs use LoRA rank 8 on attention and MLP modules [ 10 ] . The memory data are high-entropy canaries of the form AAA-1234-AAA-1234 paired with project identifiers. The structured-synthetic canary uses profile-like records rather than raw token strings. We evaluate three quantities. Forgetting is the recoverable-signal of the true secret token