RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning
Qiguang Chen, Libo Qin, Jinhao Liu, Yue Liao, Jiaqi Wang, Jingxuan Zhou, Wanxiang Che · 2025
Read RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning by Qiguang Chen, Libo Qin, Jinhao Liu, Yue Liao, Jiaqi Wang, Jingxuan Zhou, Wanxiang Che (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13307.txt.
Introduction
Qiguang Chen, Libo Qin, and colleagues examine the limits of step-by-step reasoning in large language models in a 2025 study. Writing out intermediate steps helps models on hard tasks, but users lack clear measures of how far that help extends. Unmeasurable skills, such as perception across image and text, are even harder to assess.
Their framework defines a reasoning boundary as the top level that stepwise reasoning can reach. A combination law lets them analyze how boundaries interact across tasks. For unmeasurable skills they add assessment methods suited to cases without clean scores. The aim is practical guidance for improving reasoning rather than blind trial and error.
Catalog record
| CREATOR | Qiguang Chen, Libo Qin, Jinhao Liu, Yue Liao, Jiaqi Wang, Jingxuan Zhou, Wanxiang Che |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13307
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RBF++: Quantifying and Optimizing Reasoning Boundaries across Measurable and Unmeasurable Capabilities for Chain-of-Thought Reasoning Thanks: Qiguang Chen, Jinhao Liu and Wanxiang Che are with Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, Harbin, China, and also with Faculty of Computing, Harbin Institute of Technology, Harbin, China. Email: {qgchen, car}@ir.hit.edu.cn. Thanks: Libo Qin and Jingxuan Zhou are with School of Computer Science and Engineering, Central South University, Changsha, China. Email: [email protected]. Thanks: Jiaqi Wang are with Chinese University of Hong Kong, Hong Kong, China. Email: [email protected]. Thanks: Yue Liao are with MMLab, The Chinese University of Hong Kong, Hong Kong, China. Email: [email protected]. Thanks: The corresponding author is Wanxiang Che and Libo Qin. Thanks: A preliminary version of this research has appeared in NeurIPS 2024 (Oral) [ 1 ] . Qiguang Chen, Libo Qin, Jinhao Liu, Yue Liao, Jiaqi Wang, Jingxuan Zhou, Wanxiang Che Abstract Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models (LLMs) on complex tasks, spurring research into its underlying mechanisms. However, two primary challenges remain for real-world applications: (1) the lack of quantitative metrics and actionable guidelines for evaluating and optimizing measurable boundaries of CoT capability , and (2) the absence of methods to assess boundaries of unmeasurable CoT capability, such as multimodal perception . To address these gaps, we introduce the Reasoning Boundary Framework++ (RBF++). To tackle the first challenge, we define the reasoning boundary (RB) as the maximum limit of CoT performance. We also propose a combination law for RBs, enabling quantitative analysis and offering actionable guidance across various CoT tasks. For the second challenge, particularly in multimodal scenarios, we introduce a constant assumption, which replaces unmeasurable RBs with scenario-specific constants. Additionally, we propose the reasoning boundary division mechanism, which divides unmeasurable RBs into two sub-boundaries, facilitating the quantification and optimization of both unmeasurable domain knowledge and multimodal perception capabilities. Extensive experiments involving 38 models across 13 tasks validate the feasibility of our framework in cross-modal settings. Additionally, we evaluate 10 CoT strategies, offer insights into optimization and decay from two complementary perspectives, and expand evaluation benchmarks for measuring RBs in LLM reasoning. We hope this work advances the understanding of RBs and optimization strategies in LLMs. Code and data are available at https://github.com/LightChen233/reasoning-boundary . I Introduction Recently, Large Language Models (LLMs) have made notable strides, demonstrating an expanding range of capabilities and applications across various tasks [ 2 , 3 , 4 ] . Among the most advanced LLMs, the GPT [ 5 , 6 , 7 ] , and DeepSeek [ 8 , 9 ] series stand out for their emergent abilities. A prominent example of such capabilities is the Chain-of-Thought (CoT) reasoning [ 10 , 11 , 12 ] , which enables models to articulate their reasoning step-by-step. This methodology enhances the models’ prediction accuracy by anchoring their decision-making processes in structured, logical reasoning [ 11 , 13 , 14 , 15 , 16 , 17 ] . Recent studies have explored the mechanisms underlying CoT to better understand its operational dynamics and improve its predictability and controllability. In this regard, Madaan et al. [ 18 ] and Wang et al. [ 19 ] first conduct a qualitative boundary analysis, demonstrating that CoT is constrained by the reasoning logic of contextual demonstrations. Extending this line of research, Bi et al. [ 20 ] investigate these boundaries in code planning tasks by training LLMs on CoT samples with varying complexity levels, revealing that model performance deteriorates beyond a certain complexity threshold. To further characterize CoT’s limitations, Feng et al. [ 21 ] propose a theoretical framework of single-step computation, suggesting that model performance is upper-bounded by the input length in single-step reasoning. Despite significant progress in this field, the precise boundaries of CoT and the mechanisms by which these boundaries affect CoT performance remain unclear. Current studies encounter two main challenges: (1) Lack of quantitative metrics and actionable guidance for evaluating and optimizing boundaries of measurable CoT capability: Much of the existing research relies on qualitative assessments, as depicted in Fig. 1 (a, b), resulting in a lack of standardized, quantitative metrics to guide the optimization of CoT for measurable capabilities such as calculation numbers and planning steps. This gap limits the practical application of CoT research. (2) Absence of methods to assess boundaries of unmeasurable CoT capability: As shown in Fig. 1 (c, d), boundaries in complex scenarios involve unmeasurable capabilities (e.g., multimodal perception and multi-domain knowledge) that cannot be quantified, preventing the optimization of corresponding CoT RBs in such tasks. Fig. 1 : Overview of the introduced concepts: (a) reasoning boundary (RB), (b) combination law for quantifying the upper bound of LLM capabilities in measurable scenarios; (c) constant assumption and (d) RB division mechanism for unmeasurable scenarios; (e) categories of RB for reasoning optimization guidance. To address these challenges, we introduce novel reasoning boundary framework++ (RBF++) to systematically examine and optimize the reasoning boundaries of current LLMs. (1) To address the first challenge, we formally define a model’s reasoning boundary (RB) as its ability to achieve a target accuracy under a given measurable task difficulty. To extend this to practical applications, we introduce a combination law that generalizes RBs for complex, real-world scenarios. Further, to optimize CoT reasoning, we identify and analyze three distinct RB intervals. These intervals guide the refinement of RBs and the optimization of reasoning paths under the combination law, leading to superior performance on our proposed benchmark. (2) To address the second challenge in complex tasks, particularly in multi-modal scenarios, we propose a constant assumption, enabling the quantification of boundaries for otherwise unmeasurable capabilities, such as the multimodal multi-domain CoT RB. More practically, we introduce a boundary division mechanism to measure the fine-grained unmeasurable RBs, which refine coarse-grained RBs (e.g., unified multimodal multi-domain CoT RB) into more fine-grained RBs (e.g., multimodal perception RB and domain knowledge RB), thereby enhancing the adaptability across more diverse domains and modalities. Moreover, inspired by advanced reasoning LLMs [ 22 ] , we propose the BigGSM++ benchmark to evaluate the RBs of advanced reasoning LLMs such as DeepSeek-R1 [ 9 ] . Our findings show that Reinforcement Learning substantially enhances these RBs, achieving a 100x improvement in previously infeasible RBs with 10% accuracy, while minimally affecting RBs at 90% accuracy. We validate RBF++ across 38 models and 13 CoT tasks, demonstrating its ability to generalize in both measurable and unmeasurable domains, including multimodal commonsense, scientific knowledge, and multi-modal perception. Additionally, we clarify the effectiveness and limitations of CoT optimization strategies. For example, we explain why Program-of-Thought, Least-to-Most, Complex-CoT succeed in textual scenarios but fail in multimodal ones. Finally, we introduce the Minimum Acceptable Reasoning Path (MARP) and MARP++, which outperforms existing methods at least 2% accuracy both in textual and multi-modal reasoning. Our key contributions in this work are as follows: (1) We introduce RBF++ to quantify the boundaries of both measurable and unmeasurable CoT capabilities, guiding the optimization of CoT strategies; (2) We analyze the effectiveness and limitations of CoT optimization strategies across 10 CoT strategies, 38 models and 13 CoT tasks; (3) We present the MARP and MARP++ prompting methods, which significantly enhance textual and multimodal reasoning performance. This paper extends our previous work,