Predictive Scheduling for Efficient Inference-Time Reasoning in Large Language Models

Katrina Brown, Aneesh Muppidi, Rana Shahout · 2026

Read Predictive Scheduling for Efficient Inference-Time Reasoning in Large Language Models by Katrina Brown, Aneesh Muppidi, Rana Shahout (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01237.txt.

Description

Katrina Brown and co-authors hand out test-time token budgets by foresight, not flat shares, for thought-heavy models. Flat budgets waste tokens on easy asks and starve hard asks. Their plug-in plan pre-runs light predictors, a small net on mid states or a tuned text classifier, to guess each ask's best length. A greedy allocator then splits a fixed token pool to lift mean score. On GSM8K, it gained up to 7.9 points over flat budgets at the same cost, closing over half the oracle gap.

Catalog record

CREATORKatrina Brown, Aneesh Muppidi, Rana Shahout
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59436
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01237

Browse the full library · About the Tower of Babel