Competing nonlinearities, criticality, and order-to-chaos transition in deep networks
Omri Lesser, Debanjan Chowdhury · 2026
Read Competing nonlinearities, criticality, and order-to-chaos transition in deep networks by Omri Lesser, Debanjan Chowdhury (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05294.txt.
Description
Omri Lesser and Debanjan Chowdhury show that randomly mixing two activation functions inside a deep network creates a sharp order-to-chaos style transition in how signals travel through depth. Each neuron independently draws its activation from a two-part mixture, with the mixing fraction as the tuning knob. Effective field theory describes the resulting signal propagation at initialisation.
At a critical mixing fraction the network gains depth-independent signal variance, a scale invariance previously tied to the non-smooth ReLU family. That smoothness matters for curvature-based training and physics-informed uses. The authors confirm the transition through variance propagation, susceptibility measures, and Lyapunov exponents, and test training on real data, resolving a long-standing tension in network design.
Catalog record
| CREATOR | Omri Lesser, Debanjan Chowdhury |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.05294