Routing, Cascades, and User Choice for LLMs
Rafid Mahmood · 2026
Read Routing, Cascades, and User Choice for LLMs by Rafid Mahmood (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-09902.txt.
Description
When you ask a chatbot a question, the company behind it may hand your task to a cheaper model or a stronger reasoning model. A 2026 paper by Rafid Mahmood models that choice as a game between a provider with two models and a user who may re-ask or give up. The user wants good answers with little delay, while the provider wants to keep serving costs down.
Mahmood solves the game by working out the user's best response and simplifying the provider's problem. In nearly all cases the best routing rule is fixed, with no cascading from one model to another, and it turns on the expected value each model gives the user. The paper also finds a gap between provider-best and user-best routes, and shows providers may even slow models down to save money, hurting users.
Catalog record
| CREATOR | Rafid Mahmood |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-09902