Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs

Ziyang Liu · 2026

Read Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs by Ziyang Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-18179.txt.

Introduction

Liu offers a commit-open protocol to catch hosted-model providers that quietly swap in a cheaper model. Probe-after-return checks fail because a dishonest provider can route the probe to the advertised model. Under the new scheme, the provider first commits to a sparse-autoencoder sketch of its served output, then a verifier opens random positions and scores them against a public probe library. All 17 tested attackers are rejected at one shared threshold, while a white-box attack and a forgery attack also fail to close the gap.

Catalog record

CREATORZiyang Liu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85371
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.18179

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Committed SAE-Feature Traces for Audited-Session Substitution Detection in Hosted LLMs Ziyang Liu Abstract Hosted-LLM providers have a silent-substitution incentive: advertise a stronger model while serving cheaper replies. Probe-after-return schemes such as SVIP leave a parallel-serve side-channel, since a dishonest provider can route the verifier’s probe to the advertised model while serving ordinary users from a substitute. We propose a commit-open protocol that closes this gap. Before any opening request, the provider commits via a Merkle tree to a per-position sparse-autoencoder (SAE) feature-trace sketch of its served output at a published probe layer. A verifier opens random positions, scores them against a public named-circuit probe library calibrated with cross-backend noise, and decides with a fixed-threshold joint-consistency z z -score rule. We instantiate the protocol on three backbones—Qwen3-1.7B, Gemma-2-2B, and a 4.5 × 4.5\times scale-up to Gemma-2-9B with a 131 131 k-feature SAE. Of 17 17 attackers spanning same-family lifts, cross-family substitutes, and rank- ≤ 128 \leq\!128 adaptive LoRA, all are rejected at a shared, scale-stable threshold; the same attackers all evade a matched SVIP-style parallel-serve baseline. A white-box end-to-end attack that backpropagates through the frozen SAE encoder does not close the margin, and a feature-forgery attacker that never runs M M is bounded in closed form by an intrinsic-dimension argument. Commitment adds ≤ 2.1 % \leq\!2.1\% to forward-only wall-clock at batch 32 32 . 1 Introduction A growing share of large-language-model (LLM) inference is delivered through opaque hosted APIs, and the provider’s quality-for-cost substitution of served models is both profitable and difficult to detect: a user who pays for access to a 70B-parameter model cannot, from responses alone, distinguish it from a 7B substitute that happens to answer the specific queries the user sent. Three lines of work partially address this gap: determinism / verify-rollback re-runs inference bitwise but does not establish identity; proof-of-inference SNARKs [ 1 ] give soundness at 10^{3} – 10^{4}\times overhead; response-side fingerprints ( LLMmap [ 2 ] , instructional fingerprinting [ 3 ] ) lack a binding primitive. The closest prior binding primitive is SVIP [ 4 ] , a secret linear projection of a hidden state returned on demand; because the projection is requested after service, SVIP admits a trivial parallel-serve side-channel — the provider serves users from M^{\prime} while answering each verifier query from the advertised M . Our contribution. We study a commit-open protocol in which the provider commits, via a Merkle tree, to the per-position SAE-feature trace sketch of its served sequence at a published probe layer, before any verifier opening request. Verification checks random openings against a public named-circuit probe library, calibrated with cross-backend noise, and decides with a fixed-threshold joint-consistency z -score. Within the scoped threat model (Appendix A ), a dishonest provider faces three options: (a) commit from the substitute and be statistically detected, (b) run M to produce the commitment, which on our serving stack costs within \sim\!3\% of honest wall-clock, or (c) adapt the substitute within a public adaptation budget. We characterize (c) empirically across a weak-to-strong attacker ladder and pair it with a white-box diagnostic that backpropagates through the frozen SAE encoder to the scoring rule. Paper map. Section 3 fixes the commit-open protocol. Section 4 states the strategy dichotomy (detected, or pays honest compute). Section 5 reports the main 17 -attacker frontier across three backbones, the SVIP-gap comparison, the white-box diagnostic, and the feature-forgery bound. Section 6 scopes the claims. Figure 1 : Commit-open verification. Left: four provider strategies — (A) honest, (B) substitute + substitute’s trace (detected), (C) substitute + M -trace (pays honest compute), (D) adaptive \varphi (bounded by SAE capacity). Right: the provider commits a Merkle root of per-position SAE traces before the verifier’s probe, closing SVIP’s after-the-fact side-channel (dashed). Evidence preview. We evaluate three attacker tiers. Weak non-adaptive attackers — three same-family lifts and four cross-family substitutes combined with a public-corpus linear alignment \varphi [ 5 ] — all clear the deployment threshold by \geq\!12\times on Qwen3-1.7B. Strong black-box adaptive attackers built on top of a library-aware joint- \varphi plus LoRA up to r{=}128 (1500–3000 steps) are detected with at least a 2.78\times margin on Qwen3-1.7B, and this margin grows with SAE width, reaching 40{-}260\times on Gemma-2-9B + Gemma-Scope-9B (L 20 , width- 131 k). A white-box end-to-end attacker that backpropagates through the frozen SAE encoder to the scoring rule does not close the black-box margin. The detection threshold is scale-stable ( \tau_{\mathrm{9B}}{=}\tau_{\mathrm{Gemma}}{=}1.09 , agreement <\!10^{-4} ), and commitment overhead is \leq\!2.1\% at batch 32 with a 224 -byte per-request payload. Contributions. 1. Protocol. The first hosted-LLM verification scheme that binds the provider to a per-position SAE-feature trace via a pre-query Merkle commitment ( Section 3 ). 2. Strategy dichotomy. Every non-honest provider strategy we evaluate is either statistically detected at the deployment threshold or forced to run M at near-honest wall-clock ( Section 4 ). 3. Empirical scale. A single 96 -probe library detects every evaluated lifted, cross-family, and rank- \leq\!128 adaptive attacker across Qwen3-1.7B, Gemma-2-2B, and Gemma-2-9B, with a scale-stable threshold and detection margins that grow with SAE width ( Section 5 ). Together these close an 11/11 gap vs. a matched SVIP baseline ( Section 5.6 ). 4. Systems cost. Batched commit overhead drops from 18.5\% at B{=}1 to \leq\!2.1\% at B{=}32 , with a 224 -byte per-request payload ( Section 5 , Appendix J ). 2 Related Work Hosted-LLM identity and integrity. SVIP [ 4 ] returns a secret linear projection of a target-layer hidden state on demand, but the projection is evaluated after service and therefore does not bind the served computation to the identity claim — Section 5.6 quantifies this gap under matched threshold and attacker set. TOPLOC [ 6 ] uses an LSH of activations for integrity (the advertised model produced the served tokens) and is composable with our scheme rather than competing. LLMmap [ 2 ] and instructional fingerprinting [ 3 ] recover identity post-hoc from response statistics or planted triggers with no binding primitive. Gross et al. 2024 use mechanistic interpretability for offline behaviour proofs; we borrow the move of treating MI artifacts as cryptographic payloads but apply it at serving time and target identity. Mechanistic interpretability and commit-and-prove. Our probe library is assembled from named-circuit work: indirect-object identification [ 8 ] , induction [ 9 ] , and SAE-backed syntactic/factual circuits [ 10 , 11 , 12 , 13 ] . We are the first to use the feature trace as the commitment payload of a verification scheme. Commit-and-prove SNARKs [ 1 ] give cryptographic soundness at 10^{3} – 10^{4}\times runtime; proof-of-useful-work [ 14 ] relaxes soundness for compute-as-proof. We occupy an intermediate regime: the verifier does not prove honest execution, but any non-honest strategy is either detected or pays honest-service compute ( Section 4 ). Hardware-rooted primitives (Intel TDX [ 15 ] , NVIDIA H100 confidential computing [ 16 ] , IMA [ 17 ] , append-only logs [ 18 , 19 ] ) are orthogonal deployments that strengthen the Merkle commitment against privileged adversaries rather than substitutes for the SAE-trace primitive. Table 1 summarises the differentiation. Table 1 : Comparison of hosted-LLM verification schemes. “Binding” means the provider is committed before the verifier’s query, closing the parallel-serve side-channel. “SC-safe”

Read the complete text