Gatekeeper : Improving Model Cascades Through Confidence Tuning

Stephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha, Petra Poklukar, Wittawat Jitkrittum, Sean Augenstein, Congchao Wang, Federico Tombari · 2025

Read Gatekeeper : Improving Model Cascades Through Confidence Tuning by Stephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha, Petra Poklukar, Wittawat Jitkrittum, Sean Augenstein, Congchao Wang, Federico Tombari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19335.txt.

Description

Stephan Rabanser and seven co-authors tune model cascades in twenty twenty-five. Big models perform well but cost heavily to run. Cascades pair a small local model with a large one, deferring hard tasks upward. Existing routing often defers too much or wastes the small model's real abilities.

Their loss function, Gatekeeper, fine-tunes the small model for its role. It should handle confidently the tasks it can do correctly. It should defer the rest to the larger model. A mechanism manages the trade-off between performance and deferral accuracy. The result is a cascade whose gatekeeper knows its own limits. Readers who serve models at scale get cheaper inference without silently dropping quality on easy tasks.

Catalog record

CREATORStephan Rabanser, Nathalie Rauschmayr, Achin Kulshrestha, Petra Poklukar, Wittawat Jitkrittum, Sean Augenstein, Congchao Wang, Federico Tombari
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19335

Browse the full library · About the Tower of Babel