Reasoning Planning for Language Models
Bao Nguyen Hieu Trung Nguyen, Ruifeng She Xiaojin Fu Viet Anh Nguyen, [2mm] The Chinese University of Hong Kong · 2025
Read Reasoning Planning for Language Models by Bao Nguyen Hieu Trung Nguyen, Ruifeng She Xiaojin Fu Viet Anh Nguyen, [2mm] The Chinese University of Hong Kong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00521.txt.
Introduction
Bao Nguyen, Hieu Trung Nguyen, Ruifeng She, Xiaojin Fu, and Viet Anh Nguyen tackle a meta-problem: for each new query, which reasoning method should a language model use? Common practice samples many candidate answers and aggregates them, assuming more candidates means more accuracy. The authors test that assumption with a rigorous analysis and derive accuracy bounds for standard aggregation under fixed generation settings.
On that foundation they build EPIC, a planning framework that learns a shared space linking queries, reasoning methods, and model abilities through contrastive learning. Their probability bounds enter as a regularizer, balancing accuracy against computing cost. Across varied math reasoning tasks, EPIC consistently picked the best method for each query, improving accuracy while cutting overhead. Smart selection, the work suggests, beats brute sampling.
Catalog record
| CREATOR | Bao Nguyen Hieu Trung Nguyen, Ruifeng She Xiaojin Fu Viet Anh Nguyen, [2mm] The Chinese University of Hong Kong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00521
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning Planning for Language Models Bao Nguyen Hieu Trung Nguyen, Ruifeng She Xiaojin Fu Viet Anh Nguyen, [2mm] The Chinese University of Hong Kong Abstract Selecting an appropriate reasoning method for a given query remains a key challenge in language model generation. Existing approaches typically generate multiple candidate responses and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We revisit this assumption through a rigorous theoretical analysis, deriving accuracy bounds for standard aggregation methods under fixed generation distributions and candidate sizes. Building on these insights, we introduce EPIC, an E nsemble P lann I ng with C ontrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorporates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC . 1 Introduction Large Language Models (LLMs) have demonstrated remarkable abilities to understand and reason in human natural language. These advancements have transformed applications, including travel planning ( 38 ) , AI teaching platforms ( 13 ) , and human population simulations ( 27 ; 4 ) . However, even with a pre-trained LLM, the computational expense of serving LLM-powered systems remains a significant bottleneck due to the massive scale of the models ( 20 ) , the quadratic complexity of the attention mechanism ( 7 ) , and the token-by-token nature of auto-regressive generation ( 46 ) . This high computational cost significantly hinders the broader application of LLMs in practical scenarios, particularly in resource-constrained environments such as edge devices, real-time applications, and small-scale businesses. This challenge becomes even more pronounced in tasks that require advanced reasoning, such as automated theorem proving ( 37 ) , mathematical problem solving ( 31 ) , code generation ( 12 ; 14 ) , or heuristic discovery ( 28 ) . LLMs often fail to produce accurate responses in these scenarios in a single pass. Instead, they rely on iterative generation strategies combined with aggregation or search techniques, such as best-of-N sampling ( 30 ) or Monte Carlo Tree Search ( 39 ) , to refine and select the most appropriate response. Throughout this paper, we refer to these iterative strategies as Reasoning Methods. A key limitation of current approaches lies in their static application of reasoning methods, where the same technique is applied uniformly across all user queries. However, not all reasoning methods are equally effective or efficient for every query. This observation leads to our central research question: Could we select the most suitable reasoning method for a given user query to balance the trade-off between accuracy and efficiency before generating the answer? As a starting point, we consider a universe of methods, denoted by \mathcal{M} . Each technique in \mathcal{M} is formally characterized by a tuple (\mathrm{LM},\mathrm{ReStrat},\mathrm{Agg},\mathrm{Config},N) , where \mathrm{LM} denotes the base language model, \mathrm{ReStrat} denotes the reasoning strategy (e.g. Monte Carlo Tree Search, Beam Search, Best-of-N), \mathrm{Config} is a collection of relevant configuration parameters (e.g., temperature of a sampling-based decoding strategy), \mathrm{Agg} denotes an aggregation technique (e.g., majority voting or score-based voting), and N is the number of candidate answers for aggregation. Importantly, this formulation is broad enough to subsume a wide variety of test-time compute methods ( 29 ) , ranging from simple prompting techniques ( 36 ; 42 ; 3 ) and standard decoding strategies ( 39 ; 35 ) to more specialized intervention-based approaches ( 16 ; 24 ; 25 ) . However, in this work, we focus on a representative subset of methods rather than exhaustively covering the entire space. Contributions. We introduce EPIC, an E nsemble P lann I ng with C ontrastive learning framework that recommends matching an input question and an appropriate reasoning method in the universe of methods \mathcal{M} . EPIC learns jointly the embedding of each reasoning method and a neural mapping from the input question to the embedding space. Two main components guide the learning process: • a contrastive loss, which pulls the question embedding towards the reasoning method with the highest utility for that question. The utility value is composed of a weighted combination of the accuracy and the inference cost, measured by the number of tokens generated. The user controls the accuracy-cost trade-off through a scalar parameter, balancing the preferences across different conflicting deployment criteria. • a regularizer term, which exploits the commonality among methods that share four components (\mathrm{LM},\mathrm{ReStrat},\mathrm{Config},\mathrm{Agg}) , but differ only by the number of candidate answers N . This regularizer aims to improve the sample efficiency of the training procedure by grounding these methods relatively on the scale of N . At inference time, EPIC maps the test-time input question to the embedding space and selects the reasoning method with the highest similarity (or scores) for answer generation. Extensive experiments on the MATH dataset demonstrate EPIC’s advantage: compared to individual reasoning models in the universe of methods, EPIC can reduce the number of tokens (or cost) by 75% while maintaining the same level of accuracy. Our paper unfolds as follows: Section 2 discusses related work on LLM reasoning. Section 3 studies the probabilistic bounds of common aggregation methods. Section 4 delineates our EPIC framework for matching reasoning methods with input questions, and Section 5 presents the extensive numerical results of the mathematical reasoning task. 2 Related Work We review advances in LLM reasoning algorithms and inference-time scaling, highlighting their emerging impact on output quality and computational efficiency. Reasoning algorithms and inference-time scaling. A naive reasoning process may not generate the correct solutions for complex reasoning tasks. To identify and choose the correct solution within the distribution, Self-Consistency (SC) samples multiple outputs from the LLM and selects the final response by majority voting ( 35 ) . Another similar approach is best-of-N sampling, which uses a reward model or function to choose the answer with the highest reward ( 30 ) . Both methods enhance the quality of the output, but increase the computational cost by a factor of sampling times. To explore potential reasoning paths, tree-search-based methods are proposed, such as Tree-of-Thought ( 43 ) , Monte Carlo Tree Search (MCTS) ( 32 ; 45 ; 8 ) , Forest-of-Thought ( 2 ) . 6 indicates that searching over a tree structure is more effective in discovering a correct solution than simply sampling responses in parallel for more complex tasks. Despite applying different reasoning methods to problems with various levels of complexity, inference-time scaling on these methods also significantly improves the output quality. 1 demonstrates that the accuracy on the MATH-500 benchmark improves as the amount of test-time computation (number of generations per problem) increases for algorithms such as best-of-N, beam search, and diverse verifier tree search (DVTS). 8 conduct extensive MCTS rollouts and achieve an average accuracy of 53.3% on 15 questions of AIME24 benchmark. Cost-effective reasoning. Although inference-time scaling significantly enhances LLM’s reasoning capabilities, this approach incurs substantial computational overhead and often leads to inefficient use of computational resources. Recent work finds that performance gains from various inference-time scaling