The Propagation Field: A Geometric Substrate Theory of Deep Learning
Xingrui Gu · 2026
Read The Propagation Field: A Geometric Substrate Theory of Deep Learning by Xingrui Gu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08529.txt.
Introduction
Xingrui Gu asks readers to judge neural networks by how signals travel inside them, not only by final outputs. Gu defines a propagation field made of hidden states and local change maps across depth. Models with equal final scores can differ hugely in this inner travel, and end point training alone fails to recover the true travel law in test systems.
Field aware training improves new path results, robustness, and confidence balance when it fits how data was observed, but hurts when forced too hard. In continual learning, guarding the field adds gains beyond replay methods. The work names inner signal travel as a property worth measuring and training directly.
Catalog record
| CREATOR | Xingrui Gu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08529
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Propagation Field: A Geometric Substrate Theory of Deep Learning Xingrui Gu Abstract Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance. 1 Introduction Deep learning succeeds by optimizing input–output maps through task loss, yet how information propagates internally remains poorly understood ( 7 ) . Standard training treats hidden layers as intermediate computations driven by output error ( 10 ) , constraining endpoint behavior but not the geometry of layerwise representations. This gap appears across adversarial robustness ( 11 ; 3 ) , in-context learning ( 2 ; 9 ) , continual forgetting ( 6 ) , and endpoint-loss scaling laws ( 5 ; 4 ) , suggesting that model behavior depends on internal propagation structure as well as endpoint functions. Inspired by classical field theory, general relativity, and Yang–Mills gauge theory, we propose a propagation-field perspective for understanding the internal mechanisms of deep learning. In physics, a central conceptual transition from Newtonian mechanics to general relativity and non-Abelian gauge theory is to reinterpret apparent “forces” as manifestations of more fundamental fields and geometric structures: gravity can be described as spacetime curvature, while fundamental interactions can be formulated through the curvature of gauge fields ( 1 ; 13 ; 8 ) . We view layerwise Jacobians, hidden-state trajectories, and their geometric properties path sensitivity, curvature, and velocity alignment as defining an internal propagation field that underlies the endpoint function. The endpoint function is then a terminal projection of this field, not the entirety of what learning produces. 2 Endpoint Supervision Does Not Identify the Propagation Field Standard supervised learning optimizes a task loss over a data distribution \mathcal{D} ( 7 ) : \mathcal{L}_{\mathrm{task}}(\theta)=\mathbb{E}_{(x,y)\sim\mathcal{D}}\left[\ell(f_{\theta}(x),y)\right]. (1) This constrains input–output behavior but does not uniquely determine internal propagation. Definition 2.1 (Endpoint Equivalence) . Given a task distribution \mathcal{D} and tolerance \epsilon>0 , two models f_{\theta_{1}} and f_{\theta_{2}} are endpoint-equivalent if \left|\mathcal{L}_{\mathrm{task}}(\theta_{1})-\mathcal{L}_{\mathrm{task}}(\theta_{2})\right|\leq\epsilon. (2) Endpoint-equivalent models need not propagate information the same way. We denote the propagation field of a deep model by \Phi_{\theta}(x)=\{h_{0},h_{1},\ldots,h_{L};\;J_{0},J_{1},\ldots,J_{L-1}\}, (3) where h_{\ell} is the hidden state at layer \ell and J_{\ell}=\frac{\partial h_{\ell+1}}{\partial h_{\ell}} (4) is the local propagation Jacobian. For continuous-flow models, \Phi_{\theta} may also include the trajectory h(t) , the vector field f_{\theta}(h,t) , and its numerical integration behavior. Definition 2.2 (Propagation-Field Distance) . Let \Phi_{\theta_{1}} and \Phi_{\theta_{2}} be the propagation fields of two models f_{\theta_{1}} and f_{\theta_{2}} . A field distance is \displaystyle d_{\text{field}}(\theta_{1},\theta_{2})=\mathbb{E}_{x\sim\mathcal{D}}\big[ \displaystyle d_{\text{traj}}(\mathcal{H}_{\theta_{1}}(x),\mathcal{H}_{\theta_{2}}(x)) (5) \displaystyle+\lambda_{J}d_{\text{Jac}}(\mathcal{J}_{\theta_{1}}(x),\mathcal{J}_{\theta_{2}}(x))\big] where \mathcal{H}_{\theta}(x) denotes the hidden-state trajectory and \mathcal{J}_{\theta}(x) denotes the collection of local Jacobians or their spectral proxies. Proposition 2.3 (Endpoint Equivalence Does Not Imply Field Equivalence) . A small discrepancy in task loss does not imply a small discrepancy in the propagation field: \mathcal{L}_{\mathrm{task}}(\theta_{1})\approx\mathcal{L}_{\mathrm{task}}(\theta_{2})\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.0pt\kern-5.27776pt$\displaystyle\not$\hss}{\Rightarrow}}}{\mathrel{\hbox to0.0pt{\kern 5.0pt\kern-5.27776pt$\textstyle\not$\hss}{\Rightarrow}}}{\mathrel{\hbox to0.0pt{\kern 3.98611pt\kern-4.45831pt$\scriptstyle\not$\hss}{\Rightarrow}}}{\mathrel{\hbox to0.0pt{\kern 3.40282pt\kern-3.95834pt$\scriptscriptstyle\not$\hss}{\Rightarrow}}}d_{\mathrm{field}}(\theta_{1},\theta_{2})\approx 0. (6) Scaling laws therefore characterize the scaling of endpoint loss, but not necessarily the scaling of internal propagation quality ( 5 ; 4 ) . 2.1 Experimental Evidence Since standard benchmarks rarely measure internal propagation, we provide compact empirical evidence in Table 1 . Experiment details are in Appendices A.5 , A.2.4 , and A.6 . The teacher-flow setting gives the cleanest test: models with nearly identical endpoint accuracy can exhibit orders-of-magnitude differences in trajectory and derivative recovery. PDE extrapolation shows that endpoint fitting does not recover propagation laws, while reveal-path tasks indicate that path sensitivity is a distinct internal axis rather than a proxy for accuracy. These results support the claim that endpoint supervision identifies a behavioral equivalence class but does not uniquely determine the internal propagation field. Table 1 : Endpoint equivalence does not imply field equivalence. Endpoint metrics remain similar, while field metrics can differ substantially. Lower field values are better. Setting Endpoint Field Gap Teacher A, Traj. 0.9975/0.9975 3.52\!\to\!0.06 58.7\times Teacher A, Deriv. 0.9975/0.9975 2.76\!\to\!0.15 18.4\times Teacher B, Traj. \approx.99/\approx.99 3.33\!\to\!0.07 48.0\times PDE-A, T{=}2.0 endpoint fit 1.217\!\to\!0.195 6.2\times Tiny-ImageNet .097\!\to\!.118 4.49\!\to\!1.27 3.5\times 3 Propagation-Field Theory The previous section showed that endpoint-equivalent models can have different hidden trajectories, Jacobian structures, and path sensitivities. We now define the measurable quantities that characterize these differences. Rather than assuming Riemannian structure or geodesic dynamics, we work directly with quantities computable in standard networks: hidden-state trajectories and local Jacobian operators. Definition 3.1 (Observable Propagation Path) . Given an input x and model f_{\theta} , let h_{0}^{\theta}(x),h_{1}^{\theta}(x),\ldots,h_{L}^{\theta}(x) (7) be the hidden states across depth, where h_{0}^{\theta}(x) is the input encoding and h_{L}^{\theta}(x) is the representation before the prediction head. Since hidden dimensions may vary across layers, we introduce a layer embedding map \phi_{\ell}:\mathcal{H}_{\ell}\rightarrow\mathbb{R}^{d} and define the standardized hidden state z_{\ell}^{\theta}(x)=\phi_{\ell}(h_{\ell}^{\theta}(x)). (8) If all layers share the same dimension, \phi_{\ell} is the identity. The observable propagation path is \gamma_{\theta}(x)=\{z_{0}^{\theta}(x),z_{1}^{\theta}(x),\ldots,z_{L}^{\theta}(x)\}. (9) Definition 3.2 (Local Propagation Operator) . At layer \ell , the local propagation operator is J_{\ell}^{\theta}(x)=\frac{\partial h_{\ell+1}^{\theta}(x)}{\partial h_{\ell}^{\theta}(x)}. (10) For a perturbation v at layer \ell , the first-order change at layer \ell+1 is \delta h_{\ell+1}\approx J_{\ell}^{\theta}(x)v . Thus J_{\ell}^{\theta}(x) describes how a single layer amplifies, compresses, or redirects perturbations in hidden space. Definition 3.3 (Jacobian-Induced Propagation Metric) . The local Jacobian induces a layerwise metric G_{\ell}^{\theta}(x)=(J_{\ell}^{\theta}(x))^{\top}J_{\ell}^{\theta}(x), (11) for which v^{\top}G_{\ell}^{\theta}(x)v=\|J_{\ell}^{\theta}(x)v\|_{2}^{2} . Strong amplification indicates sensitivity; strong compression indicates information loss. In high-dimensional networks, G_{\ell}^{\theta}(x) is not formed explicitly; experiments use scalable proxies such as Jacobian-vector products, random projections, and spectral estimates. 3.1 Accumulated Propagation Metric and Global Perturbation Transport Definition 3.4 (Accumulated Propagation Operator) . The accumulated Jacobian from layer 0 to layer \ell is \bar{J}_{0:\ell}^{\theta}(x)=J_{\ell-1}^{\theta}(x)\cdots J_{1}^{\theta}(x)J_{0}^{\theta}(x), (12) with the corresponding accumulated metric \bar{G}_{0:\ell}^{\theta}(x)=(\bar{J}_{0:\ell}^{\theta}(x))^{\top}\bar{J}_{0:\ell}^{\theta}(x) . Lemma 3.5 (First-Order Perturbation Propagation) . For a small input perturbation \delta h_{0} , the displacement at depth \ell satisfies h_{\ell}^{\theta}(x+\delta x)-h_{\ell}^{\theta}(x)\approx\bar{J}_{0:\ell}^{\theta}(x)\,\delta h_{0}, (13) and therefore \|h_{\ell}^{\theta}(x+\delta x)-h_{\ell}^{\theta}(x)\|_{2}^{2}\approx\delta h_{0}^{\top}\bar{G}_{0:\ell}^{\theta}(x)\delta h_{0}. (14) The spectral structure of \bar{G}_{0:\ell}^{\theta}(x) therefore governs perturbation sensitivity: a large eigenvalue along a given direction means small input perturbations in that direction can be amplified through depth. This is a first-order approximation; for large perturbations it serves as a geometric explanation rather than an exact identity. To make propagation fields measurable, we use three discrete geometry metrics. Path length. \mathrm{Len}_{\theta}(x)=\sum_{\ell=0}^{L-1}\left\|z_{\ell+1}^{\theta}(x)-z_{\ell}^{\theta}(x)\right\|_{2}. (15) This measures the total representation displacement across depth. Discrete curvature. Let \Delta z_{\ell}^{\theta}(x)=z_{\ell+1}^{\theta}(x)-z_{\ell}^{\theta}(x). (16) We define \kappa_{\theta}(x)=\frac{1}{L-1}\sum_{\ell=1}^{L-1}\frac{\left\|z_{\ell+1}^{\theta}(x)-2z_{\ell}^{\theta}(x)+z_{\ell-1}^{\theta}(x)\right\|_{2}}{\left\|z_{\ell+1}^{\theta}(x)-z_{\ell}^{\theta}(x)\right\|_{2}^{2}+\varepsilon}. (17) This is a discrete proxy for the