ProRAC: A Neuro-symbolic Method for Reasoning about Actions with LLM-based Progression

Haoyong Wu, Yongmei Liu · 2025

Read ProRAC: A Neuro-symbolic Method for Reasoning about Actions with LLM-based Progression by Haoyong Wu, Yongmei Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15069.txt.

Introduction

Haoyong Wu and Yongmei Liu propose ProRAC, a neuro symbolic method for reasoning about actions and change. The task is tracking how a world evolves as actions happen and answering questions about the result. Their framework uses a large language model inside a structured progression process rather than asking for an answer in one shot.

ProRAC first extracts the actions and the question from a problem, then executes each action step by step to derive the final state, and finally checks the query against that state. Evaluations on several benchmarks show strong performance across different domains, language model backbones, and kinds of reasoning tasks.

Catalog record

CREATORHaoyong Wu, Yongmei Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-15069

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ProRAC: A Neuro-symbolic Method for Reasoning about Actions with LLM-based Progression Haoyong Wu, Yongmei Liu Abstract Reasoning about actions and change (RAC) plays an important role in AI. It involves reasoning about preconditions and effects of actions, and has applications in planning, an important research area in classic AI. In recent years, large language models (LLMs) have made remarkable progress in NLP and related fields, and reasoning over natural language has received much attention. Several benchmarks for RAC over natural language such as TRAC, ActionReasoningBench and ACPBench have been proposed. Evaluation of LLMs on these benchmarks demonstrates that LLMs face significant challenges in RAC. However, methods for improving RAC abilities of LLMs remain largely unexplored. In this paper, we propose ProRAC ( P rogression-based R easoning about A ctions and C hange), a neuro‐symbolic framework that leverages LLMs to tackle RAC problems. ProRAC extracts fundamental RAC elements including actions and questions from the problem, progressively executes each action to derive the final state, and then evaluates the query against the progressed state to arrive at an answer. We evaluate ProRAC on several RAC benchmarks, and the results demonstrate that our approach achieves strong performance across different benchmarks, domains, LLM backbones, and types of RAC tasks. Dept. of Computer Science, Sun Yat-sen University Guangzhou 510006, China [email protected]; [email protected] Introduction Reasoning about actions and change (RAC) plays an important role in artificial intelligence. It involves reasoning about the preconditions and effects of actions. Intelligent agents must act on the world: taking purposeful actions, predicting the expected effects of such actions, and composing actions together to achieve complex goals. The last task is planning, it is an application of RAC, and an important research area in classical AI. There are three challenging problems in RAC: the Frame Problem ( McCarthy and Hayes 1981 ) , the Ramification Problem ( Thielscher 1997 ) , and the Qualification Problem ( Ginsberg and Smith 1988 ) . The Frame Problem refers to determining what remains unchanged after an action is executed. The Ramification Problem concerns the many indirect effects that an action may have, which are not explicitly specified but result from underlying domain constraints. The Qualification Problem arises from the fact that the execution of an action usually depends on many preconditions, most of which are difficult to enumerate exhaustively. In classical AI, many formalisms have been proposed for RAC, including Situation Calculus ( Reiter 2001 ) , Event Calculus ( Kowalski and Sergot 1989 ) , and Action Language \mathcal{A} ( Gelfond and Lifschitz 1993 ) . Two fundamental reasoning methods to address RAC problems within these formalisms are Regression and Progression ( Reiter 2001 ) . Roughly, regression reduces a query about the future to a query about the initial state. Progression, on the other hand, changes the initial state according to the effects of each action and then checks whether the formula holds in the resulting state. One advantage of progression compared to regression is that after a state has been progressed, many queries about the resulting state can be processed without extra overhead. Moreover, when the action sequence becomes very long, regression becomes unmanageable. Figure 1: Overview of ProRAC, consisting of three steps: Preprocess, Progression and Querying. While RAC problems have been extensively studied within classical symbolic approaches, large language models have demonstrated remarkable capabilities in recent years, offering new possibilities for solving RAC problems in natural language. Recently, a number of benchmarks for RAC over natural language have been proposed. He et al. (2023) introduced TRAC, a benchmark targeting RAC problems that includes some basic tasks such as projection, executability, plan verification and goal recognition. However, TRAC focuses solely on the classical planning domain Blocksworld. Another RAC benchmark is ActionReasoningBench ( Handa et al. 2025 ) , which covers multiple classical planning domains, such as Depots and Grippers. ACPBench ( Kokel et al. 2025b ) and ACPBench Hard ( Kokel et al. 2025a ) are two benchmarks in parallel with ActionReasoningBench. ACPBench also covers multiple planning domains and features both multiple-choice and true/false questions. ACPBench Hard is a generative version of ACPBench with open-ended questions. The success of LLMs in various reasoning tasks has also led to growing interest in neuro-symbolic AI approaches. However, neuro-symbolic AI goes beyond a mere combination of neural networks and symbolic solvers. In fact, neuro-symbolic approaches encompass a spectrum of integration paradigms, ranging from neural networks processing symbolic representations to systems where symbolic knowledge guides neural learning, and even purely neural systems that exhibit symbolic-like reasoning behaviors ( Kautz 2022 ) . This broader perspective recognizes that the boundary between neural and symbolic processing is more nuanced than traditional categorizations suggest. While several benchmarks have been introduced, comprehensive neuro-symbolic frameworks for solving RAC problems remain scarce. LLM+AL ( Ishay and Lee 2025 ) is a neuro-symbolic framework leveraging LLMs and symbolic solvers to solve RAC problems. The LLM functions as a semantic parser, translating problems expressed in natural language into an action language. Then a symbolic solver performs reasoning to produce the final solution based on translation results. While effective on planning tasks, the framework requires human intervention, as in some cases the LLM consistently fails to accurately translate problems into a formal language. Besides, it only evaluates a few classical planning problems and does not involve the aforementioned benchmarks. Beyond symbolic translation approaches, recent work has explored leveraging LLMs’ inherent reasoning capabilities for progression-based planning and state transitions. Tree of Thoughts ( Yao et al. 2023 ) enables LLMs to deliberate over coherent units of text that serve as intermediate steps toward problem solving, essentially progressing through a search space of reasoning states. Hao et al. (2023) treats LLMs as world models that can simulate state transitions and uses Monte Carlo Tree Search to explore reasoning trajectories. These methods demonstrate that LLMs can effectively perform progression-based reasoning without explicit symbolic translation. However, these approaches have primarily targeted general reasoning tasks rather than systematic evaluation on RAC problems with established benchmarks, highlighting the need for systematic frameworks that leverage LLM-based progression for RAC problems. To enhance the reasoning capabilities of LLMs in RAC problems, we propose ProRAC , a neuro-symbolic framework consisting of three steps — Preprocessing , Progression , and Querying , all performed using LLMs. First, we extract initial state, action sequence, and the query from the problem described in natural language. Then, we iteratively call an LLM to execute each action in the action sequence based on the current state, updating the state accordingly. Finally, we use the LLM to evaluate the query against the progressed final state to generate answers. In order to validate the effectiveness of ProRAC, we conducted experiments on three RAC benchmarks: TRAC, ACPBench, and ActionReasoningBench. We selected 14 different domains including Blocksworld, Depots, and Grippers from these benchmarks, and tested our approach using GPT-4o, GPT-4o-mini, and DeepSeek-v3 as underlying models. Experimental results demonstrate that our method outperforms directly feeding the problem description to LLMs. To the best of our knowledge, ProRAC is the first framework that can efficiently solve problems in the three RAC-oriented benchmarks. Furthermore, our analysis reveals that the errors made by LLMs on RAC tasks are closely related to the classic AI challenges of

Read the complete text