DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers
Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh · 2026
Read DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers by Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02016.txt.
Description
Ionut-Vlad Modoranu and colleagues speed Shampoo with DASH. Shampoo won the MLCommons competition and produces models with fewer spikes, yet its costly operations cause slowdown. DASH stacks blocks into tensors improving use, and introduces Newton-DB and Chebyshev approximations for faster inverse roots. A first analysis shows how scaling affects Shampoo. The GPU aware implementation reaches up to five point six times faster steps than Shampoo, while Newton-DB attains the lowest perplexity.
Catalog record
| CREATOR | Ionut-Vlad Modoranu, Philip Zmushko, Erik Schultheis, Mher Safaryan, Dan Alistarh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59672 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.02016