BlockCert: Certified Blockwise Extraction of Transformer Mechanisms

Sandro Andric · 2025

Read BlockCert: Certified Blockwise Extraction of Transformer Mechanisms by Sandro Andric (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17645.txt.

Introduction

Sandro Andric builds BlockCert, a framework for taking transformer language models apart with receipts. Mechanistic interpretability tries to reverse-engineer networks into explicit algorithms, and model editing tries to change specific behaviors. Both usually rest on informal evidence, with no bound on how far the extracted or edited model can drift from the original. BlockCert instead extracts structured surrogate versions of single residual blocks, each shipped with a machine-checkable certificate bounding its error, logging coverage, and hashing the artifacts involved.

A combining theorem, proved in the Lean 4 prover, lifts the per-block guarantees into a global deviation bound. Applied to GPT-2 small and two Llama-family models, the framework achieved high per-block coverage with small errors, and a fully stitched TinyLlama matched baseline perplexity almost exactly on stress prompts. The work bridges interpretability and formal proof.

Catalog record

CREATORSandro Andric
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-17645

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

BlockCert: Certified Blockwise Extraction of Transformer Mechanisms Sandro Andric Abstract Mechanistic interpretability aspires to reverse-engineer neural networks into explicit algorithms, while model editing seeks to modify specific behaviours without retraining. Both areas are typically evaluated with informal evidence and ad-hoc experiments, with few explicit guarantees about how far an extracted or edited model can drift from the original on relevant inputs. We introduce BlockCert , a framework for certified blockwise extraction of transformer mechanisms, and outline how a lightweight extension can support certified local edits . Given a pre-trained transformer and a prompt distribution, BlockCert extracts structured surrogate implementations for residual blocks together with machine-checkable certificates that bound approximation error, record coverage metrics, and hash the underlying artifacts. We formalize a simple Lipschitz-based composition theorem in Lean 4 that lifts these local guarantees to a global deviation bound. Empirically, we apply the framework to GPT-2 small, TinyLlama-1.1B-Chat, and Llama-3.2-3B. Across these models we obtain high per-block coverage and small residual errors on the evaluated prompts, and in the TinyLlama setting we show that a fully stitched model matches the baseline perplexity within ≈ 6 × 10 − 5 \approx 6\times 10^{-5} on stress prompts. Our results suggest that blockwise extraction with explicit certificates is feasible for real transformer language models and offers a practical bridge between mechanistic interpretability and formal reasoning about model behaviour. 1 Introduction Large language models (LLMs) have rapidly become core infrastructure for scientific research, software engineering, and high-stakes decision support. At the same time, we still lack robust tools for understanding, validating, and safely modifying their internal mechanisms. Mechanistic interpretability aims to address this by reverse-engineering neural networks into human-understandable components and circuits [ 10 , 2 , 8 ] , but existing work is typically evaluated with bespoke visualizations or small-scale experiments, without explicit, machine-checkable guarantees. In parallel, the formal-methods community has developed powerful tools for proving properties of neural networks [ 4 , 5 , 13 ] , and the programming-languages community has shown how to ship code with machine-checkable proofs of safety ( proof-carrying code ) [ 9 ] . However, these lines of research have had limited impact on day-to-day interpretability practice. Verification tools often do not scale to modern LLMs, and proof obligations are difficult to relate to the informal, circuit-level stories that interpretability researchers actually use. Goal. We would like a middle ground between fully formal verification and informal interpretability stories: a workflow in which reverse-engineered mechanisms are accompanied by explicit certificates that (i) precisely specify what has been extracted, (ii) quantify how closely the extract matches the original network on concrete data, and (iii) can be automatically checked by any third party with access to the artifacts. Ideally, these local certificates can then be composed to reason about global model behavior. This paper. We propose BlockCert , a framework for certified blockwise extraction of transformer mechanisms. Given a pre-trained transformer, a set of prompts, and access to intermediate activations, BlockCert produces, for each residual block: 1. a structured surrogate implementation \hat{B}_{i} in an explicit intermediate representation, and 2. a JSON certificate describing the data distribution used for extraction, the achieved approximation error and coverage metrics, and cryptographic hashes of the associated artifacts (weights, probes, masks). An independent verifier can re-load the artifacts, recompute the metrics, and check that the hashes match. We also provide a composition mechanism that aggregates per-block certificates into a full-model certificate summarizing stitched-model replay metrics. We further formalize a simple global bound: if each baseline/stitched block pair (B_{i},\hat{B}_{i}) is locally \varepsilon_{i} -sound and the blocks are L_{i} -Lipschitz, then the composed stitched model is globally bounded by a Lipschitz-weighted sum of the \varepsilon_{i} , which specializes to a \sum_{i}\varepsilon_{i} bound when L_{i}\leq 1 . This theorem is mechanized in Lean 4 and instantiated for TinyLlama using empirical per-block error bounds. Why blockwise? Transformers are naturally modular: the computation is organized into a stack of residual blocks [ 11 ] . Many mechanistic interpretability techniques operate at the block level (e.g. activation patching, MLP feature analysis), as do many model-editing methods that target specific layers [ 6 , 7 , 14 ] . By focusing on per-block extraction with quantitative guarantees, BlockCert aims to produce artifacts that are simultaneously: • close to the original computation on a specified input distribution, • simple enough to inspect and modify, • and accompanied by machine-checkable evidence of correctness. Contributions. Our main contributions are: (1) we formalize the problem of blockwise extraction for transformers and propose BlockCert , a pipeline that produces explicit surrogate blocks together with JSON certificates recording per-block approximation error, coverage metrics, and cryptographic hashes of the underlying artifacts; (2) we introduce simple but practical coverage notions (activation, path, and loss coverage) that quantify how much of a block’s behavior on a given prompt set is explained by the surrogate, and integrate these into a certificate format and verification tool; (3) in a separate Lean 4 development, we prove a global composition theorem showing that if each (B_{i},\hat{B}_{i}) pair is locally bounded by \varepsilon_{i} and blocks are L_{i} -Lipschitz, then the fully composed stitched model is globally bounded by a Lipschitz-weighted sum of the \varepsilon_{i} , which reduces to \sum_{i}\varepsilon_{i} when L_{i}\leq 1 , and we instantiate this theorem using empirical error bounds from TinyLlama block certificates and stitched-model perplexity on stress prompts; and (4) we empirically evaluate BlockCert on GPT-2 small, TinyLlama-1.1B-Chat, and Llama-3.2-3B, obtaining high coverage (often \geq 0.94 and sometimes 1.0 ) and small approximation errors, and showing that for TinyLlama the stitched model matches the baseline perplexity within \approx 6\times 10^{-5} on challenging prompts. Model + prompts \rightarrow Block traces \rightarrow IR blocks + metrics \downarrow Extraction / edit certificates Figure 1: High-level BlockCert / BlockCert-Edit workflow. Given a pre-trained transformer and a prompt distribution, we record block-level traces, construct IR surrogates with local error and coverage metrics, and package these into machine-checkable certificates. BlockCert-Edit applies simple local edits (e.g., scaling or swapping mechanisms) and produces analogous edit certificates on the same prompt distribution. Our implementation is released as an open-source Python package. To avoid confusion with historical naming, we refer to the method as BlockCert throughout this paper. 2 Background And Problem Setup 2.1 Transformers And Residual Blocks We consider standard decoder-only transformer language models [ 11 , 1 ] , which map a sequence of tokens (x_{1},\dots,x_{T}) to logits over the vocabulary at each position. The computation proceeds through a stack of L residual blocks. Let {\bm{x}}^{(0)}_{t}\in\mathbb{R}^{d} denote the token embedding at position t , and let {\bm{x}}^{(\ell)}_{t} be the residual stream at layer \ell . A typical block has the form {\bm{x}}^{(\ell+1)}_{t}={\bm{x}}^{(\ell)}_{t}+\mathrm{MLP}_{\ell}\bigl({\bm{x}}^{(\ell)}_{t}\bigr)+\mathrm{Attn}_{\ell}\bigl({\bm{x}}^{(\ell)}_{1:T}\bigr), (1) with pre- or post-layer normalization and model-specific details. We write B_{\ell} for the function mapping {\bm{x}}^{(\ell)}_{1:T} to {\bm{x}}^{(\ell+1)}_{1:T} . The full model is the composition F=B_{L-1}\circ\cdots\circ B_{0}\circ E, where E is the embedding and positional encoding. 2.2 Mechanistic Interpretability And Editing Mechanistic interpretability aims to understand neural networks by decomposing them into circuits of features and connections [ 10 , 2 ] . Recent work has identified circuits in vision and language models, studied superposition and polysemantic neurons, and proposed tools for tracing and editing mechanisms [ 8 , 15 ] . Model-editing methods such as ROME

Read the complete text