Enhancing LLM Planning Capabilities through Intrinsic Self-Critique
Bernd Bohnet, Pierre-Alexandre Kamienny, Hanie Sedghi, Dilan Gorur, Pranjal Awasthi, Aaron Parisi, Kevin Swersky, Rosanne Liu, Azade Nova, Noah Fiedel · 2025
Read Enhancing LLM Planning Capabilities through Intrinsic Self-Critique by Bernd Bohnet, Pierre-Alexandre Kamienny, Hanie Sedghi, Dilan Gorur, Pranjal Awasthi, Aaron Parisi, Kevin Swersky, Rosanne Liu, Azade Nova, Noah Fiedel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24103.txt.
Introduction
Bernd Bohnet and five colleagues test an idea about language models. Can a model improve its own plans by critiquing them? No outside checker would be involved. Earlier research had cast doubt on self-critique. Planning tasks give the idea a concrete test. Blocksworld is the main one. There, blocks must be stacked in the right order.
The base method starts from few-shot prompting. It then extends toward many-shot prompting. An iterative loop adds review and repair. The model corrects its own answers step by step. The gains were substantial. New best results were set for the model class studied. Logistics and Mini-grid sets improved as well. Strong baselines were beaten on both. The authors argue the method should transfer. Stronger models and richer search should benefit even more.
Catalog record
| CREATOR | Bernd Bohnet, Pierre-Alexandre Kamienny, Hanie Sedghi, Dilan Gorur, Pranjal Awasthi, Aaron Parisi, Kevin Swersky, Rosanne Liu, Azade Nova, Noah Fiedel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-24103
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing LLM Planning Capabilities through Intrinsic Self-Critique Bernd Bohnet, Pierre-Alexandre Kamienny, Hanie Sedghi, Dilan Gorur, Pranjal Awasthi, Aaron Parisi, Kevin Swersky, Rosanne Liu, Azade Nova, Noah Fiedel Abstract We demonstrate an approach for LLMs to critique their own answers with the goal of enhancing their performance that leads to significant improvements over established planning benchmarks. Despite the findings of earlier research that has cast doubt on the effectiveness of LLMs leveraging self critique methods, we show significant performance gains on planning datasets in the Blocksworld domain through intrinsic self-critique, without external source such as a verifier. We also demonstrate similar improvements on Logistics and Mini-grid datasets, exceeding strong baseline accuracies. We employ a few-shot learning technique and progressively extend it to a many-shot approach as our base method and demonstrate that it is possible to gain substantial improvement on top of this already competitive approach by employing an iterative process for correction and refinement. We illustrate how self-critique can significantly boost planning performance. Our empirical results present new state-of-the-art on the class of models considered, namely LLM model checkpoints from October 2024. Our primary focus lies on the method itself, demonstrating intrinsic self-improvement capabilities that are applicable regardless of the specific model version, and we believe that applying our method to more complex search techniques and more capable models will lead to even better performance. \uselogo 1 Introduction Recent advancements of Large Language Models (LLMs) have extended their applications to include planning, a domain traditionally dominated by algorithmic methods. LLM planning is important for a wide-range of tasks in which the task has implicit or explicit constraints and the LLM must generate a plan satisfying these constraints. Early investigations into LLM planning capabilities were unfavorable ( 19 ) , reinforcing the notion that "Language Models cannot plan". However, subsequent studies have introduced techniques, such as Many-Shot learning on planning tasks, to enhance these capabilities ( 1 ; 3 ) . Despite these enhancements, LLMs still lag behind classic planners that tackle algorithmically complex problems. Our approach introduces further improvements that are very promising. For instance, where LLMs are typically tested on simplified problems such as Blocksworld with 3-5 ( 20 ) or 3-7 blocks ( 3 ) , classic planners can be applied to considerably more complex problems. Nevertheless, there’s a variety of natural-language tasks such as planning holiday trips or scheduling meetings ( 5 ; 7 ) which are less computationally demanding, less classic planner friendly and less structured (often being posed in natural language). These tasks are more difficult for classical planners compared to LLMs underscoring the practical relevance of improving LLMs’ planning capabilities. This range of tasks, from natural-language to classic planning tasks, highlights the importance of continuing to enhance LLMs’ planning abilities, even though a gap remains with classic planners for more specialized, complex planning tasks. Figure 1 : Illustration of the iterative self-improvement process for Large Language Models (LLMs) using in-context learning to incorporate self-critique feedback. The LLM, also represented by the explorer character, functions as the agent’s brain, accepting prompts as inputs and generating outputs, represented by green and red semi-circles, respectively. Each iteration of the self-improvement mechanism comprises two key steps: i) plan generation and ii) self-critiquing, aimed at iteratively refining LLM outputs. In step i), the LLM generates a plan (symbolized by a map) based on a prompt incorporating domain-specific knowledge and instructions (symbolized by the treasure chest). Step ii) involves a self-critique mechanism where the LLM evaluates its own performance, providing correctness assessments and justifications, again leveraging domain knowledge. The process continues until a plan deemed correct is identified. Previous plans and their associated self-critique feedback are aggregated into a collection (symbolized by a bag), serving as contextual material for subsequent plan generation cycles. 2 2 2 Imagen 3 generated all pictures from Fig. 1 . The concept of self-improvement in LLMs (refining their responses based on self-generated feedback, often referred to as self-criticism), has gained significant attention often relying on an oracle for feedback ( 24 ; 25 ; 12 ) . This technique is particularly appealing in the context of planning tasks, where the ability to identify and rectify errors to improve the plans could lead to substantial performance gains. Earlier attempts to enhance LLM planning through self-critique yielded discouraging results, principally due to the lack of sufficient self-evaluation capabilities ( 18 ; 8 ) . The application of self-critique in prior work resulted in models that had high false-positive (FP) rates and that missed nearly all of the true negatives (TN), suggesting that current LLMs cannot effectively critique themselves ( 18 ) . Thus, earlier results on iterative plan improvements motivated the use of external feedback such as verification with an oracle (that has access to correct plan) or relying on human input to guide the process. While these results give an indication of how much headroom there is for improving the plans in the presence of a perfect evaluator, the assumption that oracles are available at test time is unrealistic for most planning tasks of interest. Exploring the limits of intrinsic LLM self-improvement - improving generations without the aid of external signals or further training - remains an active area of research ( 8 ; 13 ) . In this work, we propose an effective self-improvement method using zero-shot or few-shot prompts without the need for an external verifier. Figure 1 illustrates the introduced self-improvement method via intrinsic self-critique in this paper, utilizing the LLM as the sole source of critique. The Figure shows the iterative process of a plan generation followed by a self-critique step while adding previous failures as context (see Section 4 ). The figure also highlights the crucial components: a clear definition of the domain and instructions (e.g. pre-conditions), proper prompt design, and the use of previous failures and critiques in subsequent plan refinement steps. We investigate variations of our proposed method via ablation studies. We perform ablation study on different elements of our method. Moreover, we explore how the method scales when across zero-shot and few-shot prompting and assess the impact of varying the number of self-critique iterations. We describe a novel approach to self-critique, where the LLM is prompted to evaluate the preconditions of each action and provided the state in a plan. Lastly, we examine the limitations of our method and evaluate the quality of self-criticisms aiming to provide a comprehensive understanding of the scalability and effectiveness of the proposed methods. Our focus lies on demonstrating intrinsic self-improvement capabilities of LLMs, independent of the specific model version. To this end, we use model checkpoints from October 2024 in our empirical studies. We conduct the exploratory experiments with Gemini 1.5 Pro ( 6 ) and confirm our results with other foundation models. Our work aims to demonstrate the applicability of our method to multiple foundational models, rather than comparing them to each other. In the Blocksworld domain, Gemini 1.5 Pro achieves significant performance gains on planning datasets. With the dataset from 20 involving 3-5 blocks, we enhance performance from 49.8% to 89.3% through intrinsic self-critique, without external source such as a verifier. Similarly, for the dataset by 3 with 3-7 blocks, we boost Gemini 1.5 Pro accuracy from 57.2% to 79.5%.