Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models
Yuedong Yang Xiwen Wei Mustafa Munir Radu Marculescu · 2026
Read Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models by Yuedong Yang Xiwen Wei Mustafa Munir Radu Marculescu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10335.txt.
Introduction
Yuedong Yang and co-authors present Fuel Gauge, a method that predicts how long a large multimodal model's chain of thought will run before it starts. The reasoning process, they observed, follows a simple form tied to a hidden fuel-like quantity. Reading that signal early lets systems plan ahead. Two uses stand out: allocating KV cache in advance to cut memory fragmentation, and steering reasoning length to avoid under-thinking or over-thinking. Tests across text, image, and video tasks show strong results.
Catalog record
| CREATOR | Yuedong Yang Xiwen Wei Mustafa Munir Radu Marculescu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50960 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.10335
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fuel Gauge: Estimating Chain-of-Thought Length Ahead of Time in Large Multimodal Models Yuedong Yang Xiwen Wei Mustafa Munir Radu Marculescu Abstract Reasoning Large Multi-modality Models (LMMs) have become the de facto choice for many applications. However, these models rely on a Chain-of-Thought (CoT) process that is lengthy and unpredictable at runtime, often resulting in inefficient use of computational resources (due to memory fragmentation) and sub-optimal accuracy (due to under- and over-thinking). We observe empirically that the CoT process follows a very simple form, whose behavior is independent of the specific generated samples. This suggests that the CoT length can be estimated ahead of time based on a hidden parameter representing the amount of “fuel” available to support the reasoning process. Based on this insight, we propose Fuel Gauge, the first method which extracts this hidden signal and predicts CoT length ahead of time . We demonstrate the utility on the Fuel Gauge on two downstream tasks: predictive KV cache allocation, which addresses memory fragmentation in LMM serving systems, and CoT length modulation, which mitigates under-thinking and over-thinking. Extensive experiments on LMMs across text-only, image-text, and video-text question answering benchmarks demonstrate the effectiveness, generalizability, and practical value of our Fuel Gauge. For example, on the GPQA-Diamond benchmark, our Fuel Gauge achieves less than half the CoT length prediction error compared to the baseline; this translates into a 13.37 × \times reduction in the memory allocation frequency. 1 Introduction Reasoning Large Multimodality Models (LMM) yang2025qwen3 ; guo2025deepseek ; bai2025intern ; hurst2024gpt ; openai_gpt5_2025 ; qwen3vl achieve the state-of-art performance across multiple tasks on multiple modalities. The remarkable performance of reasoning LMMs originates from their inherent reasoning capability through Chain-of-Thought (CoT) guo2025deepseek ; yang2025qwen3 : when given an input prompt, the reasoning LMMs automatically divide the problem into a sequence of smaller steps and then conquer each sub-problem incrementally. This enables complex thinking behaviors such as backtracking where the model actively reviews its own reasoning, identifies and corrects potential mistakes. As a result, CoT empowers the LMMs with “human-like” reasoning capability, allowing them to perform multi-step reasoning and planning. However, the benefits of CoT come with significant costs both in computation efficiency and response quality. To support complex reasoning, the CoT sequences for LMMs are typically long. For example, as shown in Figure , a CoT can easily reach a length of 28k tokens, whereas the expected answer may contain only around 1k tokens. Moreover, due to the auto-regressive nature of LMM where each token’s prediction depends on the previously generated tokens, the final CoT length is unknown a priori . This uncertainty over a potentially long CoT generation can negatively impact both computational efficiency and reasoning quality. From a computational perspective, the unpredictable and lengthy CoT process forces LMM serving frameworks to repeatedly allocate small, contiguous memory blocks to store the key-value (KV) cache, which grow proportionally with the CoT length. Frequent small allocations lead to memory fragmentation kwon2023efficient ; maoefficient ; zhang2025jenga ; xu2024vtensor , scattering free memory areas across the address space. Consequently, even though sufficient free memory exists, new allocations may fail due to the lack of a big enough contiguous space. From a response quality perspective, since LMMs lack the broader context about the task at hand, they may incorrectly estimate the task’s difficulty. This mismatch can cause either over-thinking chen2024not ; aggarwal2025optimalthinkingbench ; peng2025revisiting or under-thinking wang2025thoughts ; aggarwal2025optimalthinkingbench , as the length of CoT does not align well with the true complexity of the task. As a result, the reasoning and final response may be suboptimal. Since the CoT length is not known in advance, it is impossible to intervene or correct the LMM in advance. To address these limitations, we propose Fuel Gauge, the first framework to enable CoT length estimation at runtime . The key idea is inspired by the human brain where certain chemicals indicate the level of energy available for thinking holst2015sleep and influence the thinking process itself reichert2022adenosine . Analogously, we hypothesize that LMMs possess an internal “fuel level” signal that correlates with their reasoning process: the fuel level starts high when the model begins CoT reasoning and gradually decreases to zero as reasoning progresses. We identify and extract this hidden fuel level signal using a tiny neural network consisting only 82k parameters, and estimate the CoT length based on the rate of fuel consumption. Through extensive experiments across multiple models, benchmarks, and modalities, we empirically validate our hypothesis and demonstrate the effectiveness of the Fuel Gauge. Our key contributions are: • Mathematical Characterization of CoT Length: We observe that CoT length follows a Bernoulli process and demonstrate empirically that it can be predicted a priori . • First CoT Length Estimation Framework: We identify an internal signal in LMMs that indicates the CoT length before reasoning completes. Based on this signal, we develop the first CoT length predictor, Fuel Gauge, and successfully apply it to two downstream tasks, i.e. , predictive KV cache allocation and CoT length modulation; this demonstrates the practical value of Fuel Gauge. • Extensive Empirical Validation: Experiments across diverse models, benchmarks, and modalities validate our assumptions and demonstrate the effectiveness of the Fuel Gauge framework. Our approach yields substantially lower prediction error than baseline methods ( e.g. , lower than half error on GPQA-Diamond), leading to marked improvements on multiple downstream tasks ( e.g. , 13.37 \times improvement in memory allocation). The paper is organized as follows. Section 2 reviews relevant work. Section 3 introduces the background of LMMs. Section 4 then describes our method in detail. Next, we present two applications enabled our Fuel Gauge in Section 5 . Experimental results are presented in Section 6 and, finally, Section 7 summarizes our main contributions. 2 Related Work CoT in LMMs: CoT prompting wei2022chain enables LMMs to perform step-by-step reasoning on complex tasks such as general reasoning wang2024mmlu ; du2025supergpqa and mathematical problem solving hendrycks2measuring . By decomposing a problem into intermediate steps, CoT helps LMMs generate structured reasoning trajectories that improve interpretability and accuracy chu2024navigate . For instance, DeepSeek-R1 guo2025deepseek employs GRPO shao2024deepseekmath ; Qwen3 yang2025qwen3 adopts a cold-start followed by reasoning reinforcement learning with GRPO shao2024deepseekmath ; and GPT-OSS agarwal2025gpt leverages CoT-based reinforcement learning to improve step-by-step reasoning performance. However, despite improved reasoning capability, the length of CoT reasoning remains largely uncontrolled, resulting in inefficiencies such as excessive or insufficient reasoning (over-thinking or under-thinking) and memory fragmentation kwon2023efficient . Our work addresses precisely this gap by studying CoT length predictability and proposing the Fuel Gauge predictor to model and regulate the CoT length. Steering Vector and Classifier Guidance: Steering-based control has been explored as a lightweight mechanism to influence generation behavior in LLMs. subramani2022extracting introduced steering vectors, i.e. directional activations injected into intermediate layers that can steer the model outputs towards desired attributes. This idea has since been adopted for controlled generation dathathriplug and machine unlearning li2024wmdp ; huu2024effects ; shen2025lunar . Analogous mechanisms exist in diffusion models. dhariwal2021diffusion proposed classifier guidance, which reweights the score function based on class-conditional likelihoods, effectively biasing the sampling trajectory toward desired semantic regions. Inspired by these ideas, in Section 5.2 we introduce the CoT length modulation , a new method that guides the CoT process using our