Staying on the Manifold: Geometry-Aware Noise Injection

Albert Kjøller Jacobsen, Johanna Marie Gegenfurtner ∗, Georgios Arvanitidis · 2025

Read Staying on the Manifold: Geometry-Aware Noise Injection by Albert Kjøller Jacobsen, Johanna Marie Gegenfurtner ∗, Georgios Arvanitidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-20201.txt.

Description

Albert Kjoller Jacobsen, Johanna Marie Gegenfurtner, and Georgios Arvanitidis wrote this 2025 arXiv paper on machine learning theory. Adding noise to inputs during training is known to smooth the learned function and help generalization. Most earlier work added plain noise in the full input space. Real data, though, usually lies near a lower-dimensional curved surface, called a manifold.

The authors propose noise that respects that geometry. One strategy projects random noise onto the tangent space of the surface, then follows a geodesic curve back onto it. Another lets the point wander by Brownian motion along the surface. On strongly curved manifolds, the geometric noise improved generalization and made results less sensitive to settings. On simpler manifolds it matched ordinary training. Tests on handwritten digits showed the same pattern when the manifold was learned by a generative model.

Catalog record

CREATORAlbert Kjøller Jacobsen, Johanna Marie Gegenfurtner ∗, Georgios Arvanitidis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-20201

Browse the full library · About the Tower of Babel