DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh · 2026

Read DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers by Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02016.txt.

Description

Ionut-Vlad Modoranu and colleagues speed Shampoo with DASH. Shampoo won the MLCommons competition and produces models with fewer spikes, yet its costly operations cause slowdown. DASH stacks blocks into tensors improving use, and introduces Newton-DB and Chebyshev approximations for faster inverse roots. A first analysis shows how scaling affects Shampoo. The GPU aware implementation reaches up to five point six times faster steps than Shampoo, while Newton-DB attains the lowest perplexity.

Catalog record

CREATORIonut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59672
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.02016

Browse the full library · About the Tower of Babel