DAGGER: Gradient-Free Construction of Transiently Amplifying Networks under Hard Connectivity Constraints
James C. Ferguson · 2026
Read DAGGER: Gradient-Free Construction of Transiently Amplifying Networks under Hard Connectivity Constraints by James C. Ferguson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-01227.txt.
Introduction
Ferguson builds networks that briefly boost a signal a lot. Such boosts can reach many times the input size. The network stays stable in the long run. Hard limits on sign, sparsity, and diagonal make this tough. Past tools need slow gradient search or a change of basis. Both can break the hard limits. His DAGGER tool works in one pass with no gradients.
DAGGER starts from a stable signed sparse matrix. Its output keeps the same sign, sparsity, and diagonal. One scalar sets a budget for how much sets may shift. Peak boost grows with that budget, up to ten to the tenth power. It uses 30 to 100 times fewer eigen checks than gradient tools.
Catalog record
| CREATOR | James C. Ferguson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.01227
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DAGGER: Gradient-Free Construction of Transiently Amplifying Networks under Hard Connectivity Constraints James C. Ferguson Abstract Many networks not only support but also rely on transient non-normal amplification, an orders-of-magnitude increase in the activity of an otherwise stable system. Constructing such networks under hard sign/sparsity/diagonal constraints – the regime relevant for biological connectomes and structured RNN initializations – has so far required either gradient-based local search with thousands of inner-loop eigendecompositions or Schur-form direct construction in an abstract basis that breaks the constraints under projection. Here we introduce DAGGER (Directed Acyclic Graph Guided Edge Reweighting), a gradient-free single-pass algorithm. Given a stable signed sparse matrix, DAGGER produces an output with the same sign, sparsity, and diagonal. A single scalar β \beta controls a Wasserstein-2 budget that smoothly trades exact multiset preservation ( β = 0 \beta=0 ) for amplification; peak amplification grows essentially without bound with β \beta , empirically reaching 10 10 10^{10} before numerical overflow. DAGGER matches or exceeds gradient-based methods at multiset preservation in a single forward pass – 30-100 × \times fewer eigendecompositions than a typical gradient inner loop – and at moderate β \beta beats them by orders of magnitude with connectivity exactly preserved. We develop the algorithm, compare it to the existing methods and on a downstream signal-detection task, and examine the diagnostics that show why DAGGER is structurally different from other amplifying networks. 1 Introduction Many cortical phenomena depend on non-normal transient amplification, during which a brief input pulse drives a network’s internal state to grow by orders of magnitude before stable decay returns the system to rest. This mechanism is believed to be important for a host of neuronal processes including motor preparation in primate motor cortex [ 9 , 17 ] , OFF responses in auditory cortex [ 2 ] , and balanced amplification in cortical sensory areas [ 14 ] . On the machine-learning side, controlled non-normality has emerged as a tool for expanding RNN expressivity beyond orthogonal/unitary parameterizations while still preserving stability [ 10 , 15 ] . In each of these settings one faces a constrained design problem: given an initial matrix A^{0} (typically stable, signed, and sparse), return a new matrix A that is both stable and strongly amplifying yet ideally preserves as much of the original structure of A^{0} as possible. Existing methods in this direction fall into three regimes: gradient-based local search [ 9 , 17 ] ; Schur-form direct construction [ 5 , 7 ] ; and low-rank construction [ 3 ] . However, none of these methods cleanly solves the constrained design problem. Our contribution is: DAGGER (Directed Acyclic Graph Guided Edge Reweighting). DAGGER combines the constraint-respecting nature of gradient methods with the constructive simplicity of Schur methods, in a single forward pass with no iterative inner loop. We make four concrete contributions: 1. Algorithm: DAGGER produces strongly amplifying networks in O(n^{3}\log(1/\varepsilon)) time – dominated by a single stability bisection – compared to O(n^{3}\cdot\text{iters}) for gradient methods. This corresponds to 30-100 \times fewer eigendecompositions than a typical gradient inner loop, translating to roughly 5 \times wall-clock speedup at n=300 against our SOC variant with the gap widening under the canonical full-step setting. 2. Pareto knob: A single scalar \beta controls a Wasserstein-2 budget that smoothly trades multiset preservation for amplification. Setting \beta=0 preserves the original magnitude multiset exactly; growing \beta traces a Pareto front whose peak amplification grows essentially without bound (empirically reaching 10^{10} before float64 overflow). 3. Mechanistic topology preservation: DAGGER’s amplification gain comes from rearrangement, not from changing connectivity. Structural diagnostics show that at matched amplification, DAGGER networks have backward-edge weight fraction more than 40 \times lower than SOC networks – confirming that DAGGER explicitly reshapes the connectivity towards feedforward structure within the fixed sparsity pattern. 4. Empirical validation: DAGGER networks achieve a perfect area under the curve (AUC = 1, perfect discrimination, where 0.5 = chance) in a receiver operating characteristic (ROC) signal-detection task at signal amplitudes an order of magnitude weaker than other methods. We develop the algorithm and its empirical behaviour below: first we contextualize DAGGER among related methods; then we describe the algorithm; next we report experiments; and we conclude by discussing limitations and future directions. 2 Related Work 2.1 SOC and gradient-based methods The amplification problem we solve is most closely associated with the SOC framework introduced by [ 9 ] for cortical inhibitory stabilization, and extended by [ 17 ] to motor primitive selection. SOC optimizes non-normal amplification within Dale’s-law-constrained networks via projected gradient descent on a smoothed spectral abscissa [ 19 ] , subject to sign constraints and element-wise magnitude bounds. The approach is constraint-respecting and standard in the computational neuroscience literature; its principal cost is the inner loop of potentially hundreds-to-thousands of eigendecompositions per construction. 2.2 Schur-form construction The Schur-decomposition framework articulated in [ 5 ] proposes independent manipulation of the eigenspectrum and the feedforward norm via A=Q(D+T)Q^{\top} , with D block-diagonal carrying eigenvalues and T strictly upper-triangular carrying the feedforward / non-normal structure. This is philosophically similar to DAGGER but operates in the orthogonal Schur basis rather than the original (signed sparse) basis, and the two are not interchangeable when constraints must be preserved exactly: projecting a Schur-amplified matrix onto a fixed sign/sparsity pattern destroys most of the amplification. [ 7 ] gives the limiting case — a strictly upper-triangular nilpotent connectivity that integrates inputs without recurrent feedback — which represents a theoretical ceiling on the amplification achievable from a given magnitude multiset. [ 3 ] builds minimal low-rank networks that produce targeted transient trajectories from input to readout — a complementary problem (targeted I/O design rather than generic substrate amplification). [ 14 ] provides the architectural recipe of balanced amplification, a structural prescription rather than an optimization method. 2.3 ML-side parameterizations and pseudospectral methods. The non-normal RNN of [ 10 ] parameterizes the recurrent weight matrix in Schur form for end-to-end backpropagation, with explicit normal/non-normal split; [ 15 ] shows that sequential non-normal initialization improves long-range memory in RNNs. Orthogonal/unitary RNN families ( [ 1 , 13 , 12 ] ) take the opposite extreme – fully normal connectivity – and explicitly forbid amplification. Generic pseudospectral abscissa optimization ( [ 11 , 18 ] ) provides the abstract framework of which SOC is a specialized instance, typically not applied under hard sign and sparsity constraints. These methods address the training and analysis problems; DAGGER addresses the construction-from-substrate problem and is complementary. Figure 1: Illustrative example of DAGGER using a smaller network of n=50 . (a) Heat map of edge weights at initialization. (b) Weights after reordering. (c) Weights after DAGGER reweighting. 3 Method 3.1 Approach Given the constrained design problem of turning a stable signed sparse matrix A^{0} into a highly amplifying matrix A that respects the sign/sparsity/diagonal constraints exactly, we seek a method that runs without an iterative inner loop and offers a single tunable knob that traces the trade-off between strict preservation of initial magnitude statistics and aggressive amplification. Concretely, given A^{0} with A^{0}_{ii}=-1 , sign pattern \sigma_{ij}=\mathrm{sign}(A^{0}_{ij}) , and edge set \mathcal{E}=\{(i,j):A^{0}_{ij}\neq 0\} , we will look for matrices A that maximize the peak transient response \mathcal{G}(A)=\sup_{t\geq 0}\lVert e^{At}\rVert_{2} subject to those constraints, a stability margin \alpha(A)\leq-\rho , and