Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks
Bethan Evans, Jared Tanner · 2026
Read Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks by Bethan Evans, Jared Tanner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16880.txt.
Introduction
Bethan Evans and Jared Tanner work out the least nudge that flips a deep net's call. Small weight shifts can tilt a call in a set aim. Their 2026 preprint finds exact forms for that least shift. One-layer forms fall out in full. The team names what sets the shift's size.
They then set those forms next to broad multi-layer bounds. The two gauges land of the same rough size. That match says the broad bounds lose little truth. The work also feeds low-rank backdoor attack work. Such attacks flip calls with faint, cheap nudges. Sharp least-nudge math helps guard nets from such tricks.
Catalog record
| CREATOR | Bethan Evans, Jared Tanner |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16880
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Theory of Minimal Weight Perturbations in Deep Networks and its Applications for Low-Rank Activated Backdoor Attacks Bethan Evans, Jared Tanner Abstract The minimal norm weight perturbations of DNNs required to achieve a specified change in output are derived and the factors determining its size are discussed. These single-layer exact formulae are contrasted with more generic multi-layer Lipschitz constant based robustness guarantees; both are observed to be of the same order which indicates similar efficacy in their guarantees. These results are applied to precision-modification-activated backdoor attacks, establishing provable compression thresholds below which such attacks cannot succeed, and show empirically that low-rank compression can reliably activate latent backdoors while preserving full-precision accuracy. These expressions reveal how back-propagated margins govern layer-wise sensitivity and provide certifiable guarantees on the smallest parameter updates consistent with a desired output shift. 1 Introduction Modern deep neural networks (DNNs) often require substantial computational and memory resources ( 36 ) , motivating widespread adoption of post-training network compression techniques ( 10 ) applied to pre-trained weights \theta , such as quantization ( 8 ; 43 ) , pruning ( 15 ) , and low-rank projection ( 1 ) . These parameter modification maps g\colon\theta\mapsto\hat{\theta} reduce computational cost and are generally assumed to cause only small, controlled deviations in the parameters \|\theta-\hat{\theta}\| and correspondingly minor changes in the model’s outputs ( 19 ) . However, even small weight perturbations can affect output predictions and so understanding how these modifications propagate through the network is crucial for understanding model stability, robustness, and generalisation. We develop a theoretical framework that quantifies the relationship between weight perturbations and output changes, beginning by deriving exact closed-form expressions for the minimal weight perturbation required to achieve a specified change in the network’s output. This exact solution, obtained for multilayer feedforward networks with locally invertible downstream maps, highlights how the backpropagated margin at the perturbed layer determines the required magnitude and direction of the change. We then establish a more general but less precise bound on the perturbation norm based on the network’s Lipschitz constant with respect to its weights and the current classification margin. While the exact formulation provides sharp, layer-specific characterisations, the Lipschitz-based analysis extends to arbitrary architectures, general non-linear activations, and allows perturbations to occur in multiple layers simultaneously. Building on our exact perturbation bounds, we analyse low-rank approximation as a structured weight perturbation and derive closed-form expressions for the resulting output deviations. Applying these results to precision-modification–activated backdoors, we show theoretically—and confirm empirically—that low-rank compression can activate hidden behaviours previously observed only under pruning or quantization ( 37 ; 18 ) . These results provide a comprehensive picture of how network weight perturbations interact with both network structure and model compression, providing a theoretical basis for quantifying how large a weight change is needed to alter predictions and for reasoning about the safety of efficiency-driven model compression. 2 Preliminaries Initially, Section 3 and 4 focus on classification tasks where a model maps an input—such as a point or an image—to a class label; examples for LLMs are given in Sec. 7 . Given an input \mathbf{x} , the model outputs a vector of real-valued scores \mathbf{y} , called logits , with one score for each class. The predicted class is obtained by applying the \arg\max operator, which selects the index of the largest logit. Model 2.1 (Feedforward network and layerwise decomposition) . Fix input dimension d and output dimension c . An M -layer feedforward network with parameters \theta=\{W_{l}\}_{l=1}^{M},\qquad W_{l}\in\mathbb{R}^{d_{l}\times d_{l-1}}, where d_{0}=d and d_{M}=c , and with elementwise nonlinearity \sigma:\mathbb{R}\to\mathbb{R}, is defined for a matrix of s input samples X=\bigl[\mathbf{x}^{(1)},\dots,\mathbf{x}^{(s)}\bigr]\in\mathbb{R}^{d\times s} by the layerwise recursion Z_{0}:=X,\qquad Z_{l}:=\sigma\bigl(W_{l}Z_{l-1}\bigr),\quad l=1,\dots,M-1, and the final layer pre-softmax (logit) output h(X;\theta):=W_{M}Z_{M-1}\in\mathbb{R}^{c\times s}. For any layer index N\in\{1,\dots,M\} , define the upstream map as the composition of upstream maps and functions before the N th parameter weight matrix by h_{1:N-1}(X):=Z_{N-1}\in\mathbb{R}^{d_{N-1}\times s}, and the downstream map as the composition of downstream maps and functions after the N th parameter weight matrix by h_{N:M}(Z):=Z when N=M ; and h_{N:M}(Z):=W_{M}\,\sigma\bigl(\cdots\sigma(W_{N+1}\sigma(Z))\cdots\bigr)\textrm{ \, for \, }N<M. Then the network factors as h(X;\theta)\;=\;h_{N:M}\bigl(W_{N}h_{1:N-1}(X)\bigr). Including bias terms in each layer does not affect the theoretical results. Biases can be absorbed into the upstream and downstream maps h_{1:N-1} and h_{N:M} , and are omitted from the notation for clarity. For a single input \mathbf{x}\in\mathbb{R}^{d} (i.e., s=1 ), we write the corresponding logit vector as \mathbf{y}=h(\mathbf{x};\theta)\in\mathbb{R}^{c} , and the predicted class is given by \arg\max_{i}\,\mathbf{y}_{i} . In this article we view the entire parameter set \theta as a single vector obtained by flattening and concatenating each tensor. Using a slight abuse of notation, for s\geq 1 we define the parameter perturbation norm to be \|\Delta\theta\|_{p}=\Bigl\|\,[\;\mathrm{vec}(\Delta W_{1});\;\mathrm{vec}(\Delta W_{2});\;\dots\;]\Bigr\|_{p}, where \mathrm{vec}(A) denotes the column-stacking of matrix A . If only a single weight matrix W_{N} is perturbed, then \|\Delta\theta\|_{2}=\|\Delta W_{N}\|_{F}. 2.1 Margins at the output and inside the network We distinguish a scalar margin at the logits (the usual classification margin) from a matrix-valued quantity inside the network that measures how far the layer- N representation must move to realise a prescribed change in output. Definition 2.2 (Output logit margin) . Define the network output margin of a single input sample \mathbf{x} as the difference between the logit corresponding to the true class and the largest logit among the remaining classes. More formally, let t denote the index of the true class label of \mathbf{x} , and define the classification margin as \gamma(\mathbf{x};\theta):=h(\mathbf{x};\theta)_{t}-\max_{i\neq t}h(\mathbf{x};\theta)_{i}. Equivalently, let \mathbf{e}_{i} denote the i th standard basis vector in \mathbb{R}^{c} , and define p:=\arg\max_{i\neq t}h(\mathbf{x};\theta)_{i}. Then the margin may be written as \gamma(\mathbf{x};\theta)=(\mathbf{e}_{t}-\mathbf{e}_{p})^{\top}h(\mathbf{x};\theta). Large \gamma(\mathbf{x};\theta) means that the current prediction is confident, whereas small \gamma(\mathbf{x};\theta) indicates that the prediction can flip with a small output disturbance. Inputs correctly classified with parameters \theta are indicated by \gamma(\mathbf{x};\theta)>0 and perturbed network parameters \hat{\theta} that induce misclassification are indicated by \gamma(\mathbf{x},\hat{\theta})<0 . Definition 2.3 (Layer- N pre-image difference) . Recall that h_{N:M} denotes the downstream map from layer N to the final output logits, and suppose it is locally invertible at the current layer- N representation with inverse branch h_{N:M}^{-1} defined on a neighbourhood of \mathbf{y}=h(\mathbf{x};\theta) and on a target logit vector \tilde{\mathbf{y}} . Then define the N th-layer pre-image difference as \Delta h_{N:M}^{-1}(\tilde{\mathbf{y}},\mathbf{y};\theta):=h_{N:M}^{-1}(\tilde{\mathbf{y}};\theta)\;-\;h_{N:M}^{-1}(\mathbf{y};\theta)\in\mathbb{R}^{d_{N}\times 1}. For a batch of samples, \Delta h_{N:M}^{-1} stacks these per-sample columns and is matrix-valued. This matrix is the change in input at layer- N required to change from output \mathbf{y} to \tilde{\mathbf{y}} . 3 Exact Minimal Weight Perturbation in a Single-Layer We begin with a simple illustrative example showing how small weight perturbations can alter a classifier’s output. For a fixed input \mathbf{x} , Figure 1 visualises decision boundaries before and after applying increasing perturbations to the final layer weights of a simple network. Figure 1 : Decision boundaries before and after perturbing the final-layer weight by various amounts. The dotted lines show the unperturbed classifier assigning sample point \mathbf{x} (black dot) to class 0; the solid colour is the boundaries after the weight perturbation. Dotted lines indicate the unperturbed classifier, while solid lines show the boundaries under perturbed weights. For small perturbations (left), the margin decreases but the predicted class is unchanged; at a critical perturbation (right), the margin