Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning

Tom Jacobs, Guido Montúfar · 2026

Read Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning by Tom Jacobs, Guido Montúfar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19458.txt.

Description

Jacobs and Montufar study max-margin solutions reached by mirror flow in deep networks with homogeneous activations. Building on past gradient-flow results, they derive a balance equation from convex duality. It characterizes the horizon function behind the induced margin, with rates and norm growth bounds. Tests on synthetic and standard vision tasks show three points. Distinct mirror maps can reach the same answer, and convergence can run very slowly. All maps learn features yet yield varied representations from sparse to dense.

Catalog record

CREATORTom Jacobs, Guido Montúfar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104033
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19458

Browse the full library · About the Tower of Babel