RanDeS: Randomized Delta Superposition for Multi-Model Compression
Hangyu Zhou, Aaron Gokaslan, Volodymyr Kuleshov, Bharath Hariharan · 2025
Read RanDeS: Randomized Delta Superposition for Multi-Model Compression by Hangyu Zhou, Aaron Gokaslan, Volodymyr Kuleshov, Bharath Hariharan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11204.txt.
Description
Hangyu Zhou and co-authors shrink the cost of serving many tuned models in a 2025 paper. Labs often fine tune one base model many ways, and storing each copy eats memory. Merging the copies helps, but the task tweaks clash when summed back at use time.
Their method, RanDeS, recasts merging as squeeze and fetch. Random orthogonal twists scramble the tweaks so stray parts cancel each other out, which cuts the clash a lot. Each twist is set by a seed alone, so new models join with no extra store cost. Gains held across vision and language tasks, and models can join or leave with little compute spent.
Catalog record
| CREATOR | Hangyu Zhou, Aaron Gokaslan, Volodymyr Kuleshov, Bharath Hariharan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-11204