Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval
Daniel Freeman, Daniel Haider · 2025
Read Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval by Daniel Freeman, Daniel Haider (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09898.txt.
Description
Daniel Freeman and Daniel Haider bound how much a rectified layer in a neural network can shrink differences between inputs, in a 2025 paper. If a layer can crush two inputs together, information is lost. The same math frames recovery from clipped readings and from readings that keep only size, not sign. Frame theory unifies all three. The pair derive bounds for the layer and for clipping that mirror the known phase retrieval bound. Their bounds are best possible up to a constant factor.
Catalog record
| CREATOR | Daniel Freeman, Daniel Haider |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09898