Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization
Hasan Amin, Kian Ahrabian, Ming Yin, Rajiv Khanna · 2026
Read Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization by Hasan Amin, Kian Ahrabian, Ming Yin, Rajiv Khanna (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00544.txt.
Description
Amin and co-authors show that more than one good reply per prompt can lift model training. Stock fine-tune pairs each prompt with just one reply. Many prompts hold more than one sound end, though. That cut turns a rich spread to a one-draw view, a mode lottery in their words.
Their multi-reply training keeps more than one reply per prompt in the set. Prompts and replies act as split pools of proof in their math. More prompts cut doubt on what comes in. More replies cut doubt on what should go out. That trade calls when extra replies pay and when they fade. Pick rules for replies are weighed with care as well. Gains peak when replies are rich and prompts do not repeat.
Catalog record
| CREATOR | Hasan Amin, Kian Ahrabian, Ming Yin, Rajiv Khanna |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 129 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.00544