Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval

Daniel Freeman, Daniel Haider · 2025

Read Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval by Daniel Freeman, Daniel Haider (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09898.txt.

Description

Daniel Freeman and Daniel Haider bound how much a rectified layer in a neural network can shrink differences between inputs, in a 2025 paper. If a layer can crush two inputs together, information is lost. The same math frames recovery from clipped readings and from readings that keep only size, not sign. Frame theory unifies all three. The pair derive bounds for the layer and for clipping that mirror the known phase retrieval bound. Their bounds are best possible up to a constant factor.

Catalog record

CREATORDaniel Freeman, Daniel Haider
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09898

Browse the full library · About the Tower of Babel