Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
Ali Taghibakhshi*, Sharath Turuvekere Sreenivas*, Saurav Muralidharan*, Ruisi Cai†, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov · 2025
Read Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs by Ali Taghibakhshi*, Sharath Turuvekere Sreenivas*, Saurav Muralidharan*, Ruisi Cai†, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16664.txt.
Description
Ali Taghibakhshi and fifteen co-authors present Nemotron Elastic, a framework for training one reasoning language model that contains several smaller models inside it. Training a separate model for every deployment size is prohibitively expensive, and standard pruning or distillation still burns hundreds of billions of tokens per compressed model.
Nested submodels share the parent's weights and can be extracted at deployment with no extra training, selected by an end-to-end trained router and a two-stage curriculum built for reasoning models. Applied to a twelve-billion-parameter parent, the method produced nine- and six-billion-parameter models using only about 110 billion training tokens. That is a cost cut of more than three hundredfold versus training a family from scratch. Each nested model matched or beat the state of the art at its size.
Catalog record
| CREATOR | Ali Taghibakhshi*, Sharath Turuvekere Sreenivas*, Saurav Muralidharan*, Ruisi Cai†, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-16664