Architecture-Aware Minimization (A 2 M): How to Find Flat Minima in Neural Architecture Search

Matteo Gambella, Fabrizio Pittorino, Manuel Roveri · 2025

Read Architecture-Aware Minimization (A 2 M): How to Find Flat Minima in Neural Architecture Search by Matteo Gambella, Fabrizio Pittorino, Manuel Roveri (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10404.txt.

Description

Matteo Gambella, Fabrizio Pittorino, and Manuel Roveri look at the shape of the search space when computers design neural networks. In differentiable architecture search, designs sit in a landscape much like the loss surfaces of training, and flat regions tend to hold designs that generalize well. The team defines flatness measures for architecture space and probes two standard benchmarks with them.

They find that strong designs cluster together in wide, flat neighborhoods, while weak designs sit isolated. Building on that map, they derive a method called Architecture-Aware Minimization that steers the search gradient toward flat regions on purpose. Added to existing search methods, it raised test accuracy by about 3.6 points on CIFAR-10, 4.6 on CIFAR-100, and 3.6 on a small ImageNet set. The code is open, and the authors offer the tool as a drop-in upgrade for automated network design.

Catalog record

CREATORMatteo Gambella, Fabrizio Pittorino, Manuel Roveri
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE97 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10404

Browse the full library · About the Tower of Babel