BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization
Hengrui Zhang, Boao Kong, Engao Zhang, Kun Yuan · 2026
Read BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization by Hengrui Zhang, Boao Kong, Engao Zhang, Kun Yuan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10288.txt.
Description
Hengrui Zhang and three colleagues make two-level optimization cheaper to run. Stochastic bilevel optimization frames problems with two nested levels, such as tuning hyperparameters, reweighting data, learning representations, and mixing data sources. Exact single-loop methods strain memory for large lower-level nets, while memory-efficient stand-ins lack competitive convergence proofs. Their 2026 paper proposes BROS, a bias-corrected randomized-subspace method.
BROS runs lower-level and auxiliary updates in randomized subspaces, using a Rademacher bi-probe correction to restore an unbiased Hessian-action estimator. The authors prove that BROS keeps the same sample-complexity order as exact single-loop methods, finding an epsilon-stationary point under standard assumptions alone. Tests on hyper-data cleaning, data-mixture learning, hyper-representation learning, and vision-transformer sample reweighting showed peak memory cuts up to 44.9 percent while closely matching full-space baseline performance.
Catalog record
| CREATOR | Hengrui Zhang, Boao Kong, Engao Zhang, Kun Yuan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 154 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10288