Predictability-Based Curiosity-Guided Action Symbol Discovery

Burcu Kilic, Alper Ahmetoglu, Emre Ugur · 2025

Read Predictability-Based Curiosity-Guided Action Symbol Discovery by Burcu Kilic, Alper Ahmetoglu, Emre Ugur (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18248.txt.

Introduction

Burcu Kilic, Alper Ahmetoglu, and Emre Ugur give a robot the means to invent its own vocabulary of actions, in this 2025 paper. Planning needs symbols: named skills like push or stack that a planner can chain together. Robots usually receive those symbols from their programmers. A truly developing robot should discover them itself, plus the perceptual categories planning needs.

Their system watches objects and actions and predicts the effects an action will produce, learning with an encoder-decoder network. A curiosity module steers exploration toward the actions whose outcomes are least predictable, since those teach the most. The discovered action primitives then feed a symbolic tree search that plans manipulation tasks with one and two objects. Against two baselines, curiosity won: the robot learned more useful symbols from the same amount of flailing. It is a small model of how children turn play into concepts.

Catalog record

CREATORBurcu Kilic, Alper Ahmetoglu, Emre Ugur
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18248

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Predictability-Based Curiosity-Guided Action Symbol Discovery Thanks: This work was supported by the Scientific and Technological Research Council of Turkey (TUBITAK) ARDEB 1001 Program (124E227) and by the INVERSE Project under Grant 101136067, funded by the European Union. Burcu Kilic, Alper Ahmetoglu, Emre Ugur Abstract Discovering symbolic representations for skills is essential for abstract reasoning and efficient planning in robotics. Previous neuro-symbolic robotic studies mostly focused on discovering perceptual symbolic categories given a pre-defined action repertoire and generating plans with given action symbols. A truly developmental robotic system, on the other hand, should be able to discover all the abstractions required for the planning system with minimal human intervention. In this study, we propose a novel system that is designed to discover symbolic action primitives along with perceptual symbols autonomously. Our system is based on an encoder-decoder structure that takes object and action information as input and predicts the generated effect. To efficiently explore the vast continuous action parameter space, we introduce a Curiosity-Based exploration module that selects the most informative actions—the ones that maximize the entropy in the predicted effect distribution. The discovered symbolic action primitives are then used to make plans using a symbolic tree search strategy in single- and double-object manipulation tasks. We compare our model with two baselines that use different exploration strategies in different experiments. The results show that our approach can learn a diverse set of symbolic action primitives, which are effective for generating plans in order to achieve given manipulation goals. I Introduction Humans and animals have the ability to perform abstract reasoning about their environments by learning abstract representations of actions and objects. For this, infants acquire high-level skills primarily by exploring their environment out of intrinsic curiosity, without an extrinsic end goal or rewards. Inspired by human developmental processes, we aim to demonstrate such autonomous exploration and high-level skill discovery in robotic agents. We present a robotic agent that can learn object and action abstractions by predicting the effects of its actions and exploring while trying to reduce its uncertainty in the predictions; in other words, exploring out of curiosity. Symbolic representations of skills can enhance a robot’s ability to reason and plan [ 1 , 2 ] . Abstracting continuous sensorimotor information into discrete symbolic representations can simplify complex decision-making processes. These abstractions enable a robot to have better generalizable and transferable skills. Symbolic action primitives are utilized in symbolic planning, an effective method for efficiently generating action sequences to reach a specific goal state. Previous neuro-symbolic robotic approaches [ 3 , 4 , 5 ] have shown success in learning object categories and relational object symbols from the bottleneck layer of an effect prediction encoder-decoder network. In [ 6 ] , symbolic planning was performed using Planning Domain Description Language (PDDL) [ 7 ] with learned object symbols and a set of predefined abstract actions. However, these studies depend on manually defined discrete actions, and there is no autonomous skill discovery part. A real cognitive developmental system, on the other hand, should have the ability to discover high-level discrete actions as well. In an early study, a robot was initialized with a basic reach-and-grasp movement capability, discovering a set of action primitives by exploring its action parameter space and applying clustering in its tactile measurements [ 8 , 9 ] . However, the learned primitives were not used in high-level planning. More recently, [ 10 , 11 ] framed the problem as an operator learning problem in their Task and Motion Planning (TAMP) framework. They proposed a neuro-symbolic relational transition model where a task plan was generated using symbolic planning, and then a neural network was used to search for the low-level operator parameters during execution. [ 12 ] learned action primitives, providing a bi-level operator learning stack. However, these methods followed an algorithmic approach in learning operators, whereas we proposed to use a generic neural network for object and action symbol discovery. The studies reviewed above mostly relied on random exploration. It is known that infants benefit from intrinsic motivation to more efficiently explore their environment [ 13 ] . Similar ideas have been applied in artificial intelligence and robotics. [ 14 ] provided rewards to agents for actions that give high information gain, [ 15 ] used the state entropy as an intrinsic reward, [ 16 ] used learning progress (LP) [ 17 ] for exploration guiding in reinforcement learning, [ 18 , 19 ] used LP for exploration region selection in object-action-effect spaces, and [ 20 ] used the error from the effect prediction model as a reward signal to the action policy network. In our study, we propose to select actions that maximize the entropy in our model, which predicts the effects of actions. The aim of this work is to find discrete action primitives that are effective in planning. For this, significantly extending our object symbol discovery framework [ 3 ] , we propose a predictive encoder-decoder neural network, which takes action parameters and object features as input and generates action effects as output, is proposed. The core idea is to binarize the embeddings in the bottleneck layer, enabling symbolic planning. We also propose to guide the exploration of the robot by a curiosity signal, which depends on the entropy of the neural network output activation. Finally, after a sequence of actions is generated via a tree-based symbolic planner, the continuous parameters of the corresponding action are found using a gradient-based method, which freezes network weights and applies a search in the action parameter space. We used a manipulation robot with a gripper in a simulated environment for experimental verification. Exploring the parameter space of a reach action, our method discovered a diverse set of action primitives such as grasp, grasp and place, pull, and push in different directions. We used these primitives to generate symbolic plans to bring an object to a goal position and execute the actions with the proposed parameter distillation procedure. We also showed that our curiosity-based exploration overperformed the baselines, using other exploration strategies. In the rest of this paper, we first provide the details of our proposed method, then give the experimental setup and the baselines, and finally provide the experimental results. II Proposed Method We provide the sensorimotor representation, followed by the predictive encoder-decoder network, continuous action parameterization, curiosity-based exploration module, and the planner. An overview of our proposed method can be seen in Figure 1 . Fig. 1: Overview of the proposed method. The effect prediction model is an encoder-decoder deep neural network that predicts a distribution over the effect. The entropy of the distribution is given to the curiosity-based exploration module to guide the action selection process. The object is randomly initialized in the environment. The symbols are generated via binarization of the bottleneck layer of the effect prediction model. II-A Sensorimotor Representation Each object o\in\mathbb{R}^{4} is represented as o=[s_{x},s_{y},d,t] where s_{x},s_{y},d denotes the object’s dimensions in different axes, and t corresponds to the type of the object. A state s\in\mathbb{R}^{4\times m} is the state of the environment with m objects. The object features, which are used as the input of the predictive neural network, correspond to the target object’s features. The robotic action a\in\mathbb{R}^{12} is

Read the complete text