Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks

Saveliy Baturin [email protected] · 2026

Read Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks by Saveliy Baturin [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17596.txt.

Description

Saveliy Baturin studies the loss landscape of shallow ReLU networks. He asks if two low-loss parameter settings can be joined by a path that never climbs far above their loss level. The net has one hidden layer, constrained first-layer weights, and a penalty on the output layer. The data term is convex and globally Lipschitz.

A finite-width construction links any two points of a common sublevel set. It swaps an older quadratic estimate for a direct Lipschitz bound, then moves active units monotonically to exploit positive homogeneity. Sphere covering and cluster merging yield explicit thickening rates, with exact connectivity in one dimension. Approximation rates transfer to barrier decay rates, and Huber and cross-entropy tests back the theory with small certified gaps.

Catalog record

CREATORSaveliy Baturin [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17596

Browse the full library · About the Tower of Babel