Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks
Saveliy Baturin [email protected] · 2026
Read Asymptotic Smoothing of the Lipschitz Loss Landscape in Overparameterized One-Hidden-Layer ReLU Networks by Saveliy Baturin [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17596.txt.
Description
Saveliy Baturin studies the loss landscape of shallow ReLU networks. He asks if two low-loss parameter settings can be joined by a path that never climbs far above their loss level. The net has one hidden layer, constrained first-layer weights, and a penalty on the output layer. The data term is convex and globally Lipschitz.
A finite-width construction links any two points of a common sublevel set. It swaps an older quadratic estimate for a direct Lipschitz bound, then moves active units monotonically to exploit positive homogeneity. Sphere covering and cluster merging yield explicit thickening rates, with exact connectivity in one dimension. Approximation rates transfer to barrier decay rates, and Huber and cross-entropy tests back the theory with small certified gaps.
Catalog record
| CREATOR | Saveliy Baturin [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-17596