Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count
Lurong Pan · 2026
Read Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count by Lurong Pan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08171.txt.
Introduction
Lurong Pan introduces a neural network layer that uses far fewer weights than a dense layer of the same size. CDLinear arranges its weights in a block-circulant pattern borrowed from a communication dynamics framework, so a layer needs only a fraction of the usual parameters.
The Fourier transform diagonalizes the weight Hessian for squared-error loss, and its eigenvalues come straight from the input spectrum. With pre-whitened inputs the population condition number is exactly one, a best-case value for optimization. Pan implements the layer in NumPy with hand-derived gradients checked by finite differences, and tests a small multilayer network on the classic digits benchmark across seeds.
Catalog record
| CREATOR | Lurong Pan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08171
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count Lurong Pan Abstract Background and motivation. The Communication Dynamics (CD) framework, introduced in two earlier papers for atomic-energy prediction and field-induced superconductivity, treats each physical channel as a ( 2 ℓ + 1 ) (2\ell+1) -vertex polygon whose discrete Fourier transform yields its energy spectrum. This paper applies the same circulant-spectral machinery to neural-network design. Layer construction. I introduce CDLinear , a block-circulant linear layer with block size B = 2 ℓ + 1 B=2\ell+1 and 1 / B 1/B the parameter count of a dense layer of equal input/output dimensions. Three properties follow from the construction. (i) The Hessian of mean-squared loss with respect to the weights is itself diagonalized by the discrete Fourier transform, with eigenvalues | ℱ [ X j ] ( k ) | 2 |\mathcal{F}[X_{j}](k)|^{2} that are read directly from the input statistics (Theorem 1 ). (ii) Under input pre-whitening, the population Hessian condition number satisfies κ = 1 \kappa=1 exactly, with the empirical condition number bounded by 1 + O ( B / N ) 1+O(\sqrt{B/N}) on N N samples (Theorem 2 ). (iii) The Shannon noise rate α CD = 0.0118 \alpha_{\mathrm{CD}}=0.0118 calibrated in the parent CD papers from the Na D-doublet specifies a transferable, non-arbitrary dropout rate. Empirical evaluation. I implement the layer in pure NumPy with hand-derived backward passes, verify all gradients to < 10 − 4 <10^{-4} relative error via finite differences, and test on the 8 × 8 8{\times}8 MNIST benchmark ( sklearn.datasets.load_digits , 1437 train and 360 test samples). Across three random seeds, a CDLinear MLP at B = 4 B=4 achieves 97.50 % ± 0.23 % 97.50\%\pm 0.23\% test accuracy with 2,380 parameters, versus 98.15 % ± 0.47 % 98.15\%\pm 0.47\% for a parameter-matched dense MLP at 8,970 parameters— a 3.8 × 3.8\times parameter reduction at 0.65 % 0.65\% accuracy cost, within one standard deviation of the seed-to-seed spread. The CD-MLP’s mean Hessian condition number κ = 1.9 × 10 4 \kappa=1.9\times 10^{4} is 310 × 310\times smaller than the dense baseline’s κ = 5.9 × 10 6 \kappa=5.9\times 10^{6} , in quantitative agreement with Theorem 2 . Honest positioning. Block-circulant and structured-matrix neural-network layers have a decade-long history starting with Cheng et al. (2015); CDLinear is mathematically a special case. What this paper adds is (a) a closed-form Hessian-spectrum diagnostic that is computable per mini-batch from a single FFT, (b) a principled discrete sequence B ∈ { 1 , 3 , 5 , 7 , … } B\in\{1,3,5,7,\ldots\} for the structural multiplicity following the polygon multiplicity of CD theory, (c) the transferable α CD = 0.0118 \alpha_{\mathrm{CD}}=0.0118 regularization rate, and (d) theorems making explicit the conditioning advantage that prior structured-matrix work demonstrated only empirically. Scope of the empirical claim. The MNIST-1797 benchmark saturates near 98 % 98\% for almost any reasonable classifier; the parameter-efficiency story holds clearly here, and the conditioning advantage is large and reproducible, but generalization to harder benchmarks (CIFAR-10, ImageNet, language modeling) and to convolutional and attention layers is not established in this paper and is identified as the primary deferred work. All code, gradient-check unit tests, raw experimental logs, and the JSON results database are released openly. I Introduction The Communication Dynamics (CD) framework introduced by Pan, Skidmore, Güldal, and Tanik (2021) Pan2021 and developed in two recent papers for atomic energy prediction PaperI (Paper I) and field-induced superconductivity PaperII (Paper II) treats physical systems as discrete communication channels whose error-content polygons have spectra computable via the discrete Fourier transform. The mathematical engine is the circulant spectral theorem: a circulant matrix C\in\mathbb{C}^{n\times n} with first row \mathbf{c}=(c_{0},c_{1},\ldots,c_{n-1}) is diagonalized by the unitary DFT matrix F_{n} , with eigenvalues \lambda_{k}=\sum_{j=0}^{n-1}c_{j}\,e^{2\pi ijk/n} Gray2006 . CD exploits this fact to short-circuit configuration-space integrals: rather than solving the Schrödinger equation on a continuous spatial grid, CD evaluates the polygon DFT directly in the discrete m_{\ell} space, achieving 10^{2} – 10^{3}\times speedup at 4 – 7\times lower accuracy than density functional theory. Neural networks (NNs) are themselves communication channels. A linear layer y=Wx+b with weights W\in\mathbb{R}^{n_{\rm out}\times n_{\rm in}} maps input symbols to output symbols, and the loss landscape is determined by W ’s spectrum: the Hessian of mean-squared loss is W^{\top}W , whose condition number \kappa(W)=\sigma_{\max}^{2}/\sigma_{\min}^{2} controls the convergence rate of gradient descent. For randomly initialized dense layers, Marchenko–Pastur theory MarchenkoPastur predicts \kappa=\Theta(n_{\rm in}) , growing without bound as networks scale, with the smallest singular values especially fragile under perturbation. This is the Hessian-conditioning problem LeCun1991 that motivates layer normalization LayerNorm , weight normalization WeightNorm , and natural gradient methods Amari1998 . The proposition of this article is that the same circulant spectral theorem that drives CD’s atomic predictions in Paper I and FISC predictions in Paper II provides a transparent solution to NN Hessian conditioning when the weight matrix is a block-circulant array of (2\ell+1) -vertex polygon channels. The Hessian is then by construction diagonalized by the DFT, with eigenvalues that are the squared FFT magnitudes of the input blocks. Initializing with a flat input spectrum gives \kappa=1 at the start of training; Theorem 2 below shows that this property persists in the empirical Hessian up to a controllable bound. Relationship to prior structured-matrix NN work. Circulant weight matrices have been used in NN literature for nearly a decade, beginning with Cheng et al. Cheng2015 who showed empirically that circulant projections retain accuracy at 1/n parameter count; Yu et al. Yu2017 extended this to learned structured matrices; Sindhwani et al. Sindhwani2015 catalogued displacement-rank families; Moczulski et al. Moczulski2016 introduced ACDC layers; Thomas et al. Thomas2018 learned compressed transforms with low displacement rank; Dao et al. Dao2022 introduced Monarch matrices; and Fourier Neural Operators of Li et al. Li2020 generalized circulant layers to function-space settings. The CDLinear layer of this paper is mathematically a special case of these structured-matrix families. What this paper contributes that prior work did not is fourfold. First, the choice of block size B=2\ell+1 follows from the polygon multiplicity that defines a Shannon channel symbol set in CD theory, giving a discrete physically-motivated sequence B\in\{1,3,5,7,\ldots\} for hyperparameter selection rather than a heuristic search. Second, the Shannon noise rate \alpha_{\mathrm{CD}}=0.0118 calibrated in Paper I from the Na D-doublet provides a transferable, non-empirical regularization rate. Third, an explicit closed-form theorem (Theorem 1 ) makes the FFT-diagonal Hessian property rigorous and computable, providing a per-batch conditioning diagnostic that does not require any matrix decomposition. Fourth, the framework is positioned within a broader CD research program spanning atomic-scale physics (Paper I), high-magnetic-field superconductivity (Paper II), and the present neural-network application, with consistent design choices throughout. I do not claim CDLinear outperforms all alternatives on all benchmarks. Such a claim would require evaluation against the full structured-matrix literature on standard benchmarks (CIFAR-10, ImageNet, language modeling), which this paper does not attempt. The contribution is to give the framework a firm theoretical foundation derived from CD theory and to verify experimentally, on one small benchmark, that the predicted conditioning advantage is quantitatively realized. Article structure. Section II reviews the relevant elements of CD theory. Section III introduces the CDLinear layer and proves the FFT-diagonal-Hessian theorem. Section IV develops the