Perceptual Motor Learning with Active Inference Framework for Robust Lateral Control

Elahe Delavari, John Moore, Junho Hong, and Jaerock Kwon · 2025

Read Perceptual Motor Learning with Active Inference Framework for Robust Lateral Control by Elahe Delavari, John Moore, Junho Hong, and Jaerock Kwon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01676.txt.

Introduction

Elahe Delavari and co-authors presented a learning frame in 2025 for keeping self-driving cars in lane. Their frame joins perception and action, in the way human motor learning does. Standard driving stacks can struggle to adapt, generalise, and run cheaply.

The frame uses active inference, where a model tries to cut prediction error, or surprise. That model shapes steering from learned links between sensing and motion. The car can learn lane keeping with little data and no large retraining for new settings. Tests in a driving simulator matched standard methods, with no extra computing load. The authors see a robust option for real-world use.

Catalog record

CREATORElahe Delavari, John Moore, Junho Hong, and Jaerock Kwon
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01676

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Perceptual Motor Learning with Active Inference Framework for Robust Lateral Control Elahe Delavari, John Moore, Junho Hong, and Jaerock Kwon Abstract This paper presents a novel Perceptual Motor Learning (PML) framework integrated with Active Inference (AIF) to enhance lateral control in Highly Automated Vehicles (HAVs). PML, inspired by human motor learning, emphasizes the seamless integration of perception and action, enabling efficient decision-making in dynamic environments. Traditional autonomous driving approaches—including modular pipelines, imitation learning, and reinforcement learning—struggle with adaptability, generalization, and computational efficiency. In contrast, PML with AIF leverages a generative model to minimize prediction error (“surprise”) and actively shape vehicle control based on learned perceptual-motor representations. Our approach unifies deep learning with active inference principles, allowing HAVs to perform lane-keeping maneuvers with minimal data and without extensive retraining across different environments. Extensive experiments in the CARLA simulator demonstrate that PML with AIF enhances adaptability without increasing computational overhead while achieving performance comparable to conventional methods. These findings highlight the potential of PML-driven active inference as a robust alternative for real-world autonomous driving applications. † † margin: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. I INTRODUCTION The rapid evolution toward fully Highly Automated Vehicles (HAVs) demands decision-making frameworks that not only emulate human cognition but also robustly generalize across diverse and unforeseen environments without extensive retraining. Traditional approaches in autonomous driving—including Modular Pipelines (MP) [ 1 ] , Imitation Learning (IL) [ 2 ] [ 3 ] , and Reinforcement Learning (RL) [ 4 ] —have paved the way for significant advances. However, each of these methods faces inherent limitations. MPs, while modular and interpretable, suffer from error propagation and inflexibility; IL is capable of mimicking expert behavior yet often fails when confronted with novel scenarios; and RL, despite its ability to learn through interaction, is heavily reliant on finely tuned reward-shaping, which is a process that is both labor-intensive and highly task-specific. To address the aforementioned limitations, we propose Perceptual Motor Learning (PML) [ 5 ] [ 6 ] with Active Inference Framework (AIF) [ 7 ] . PML refers to the process by which sensory inputs (perception) and motor outputs (action) become integrated to enable effective interaction with the environment. In early human development, this integration is crucial: while fundamental movement skills (e.g., hopping, jumping, running, and balance) lay the groundwork, perceptual motor development connects these skills with sensory processing, enabling children to develop body awareness, spatial awareness, directional awareness, and temporal awareness [ 8 ] . To realize a PML, we propose to use AIF, which has emerged as a promising alternative to conventional control theories, grounded in theories of human brain function that posit the continuous prediction of sensory inputs and the minimization of discrepancies between expected and actual observations [ 9 ] . By unifying perception and action within a single generative framework, active inference allows an autonomous agent to actively anticipate environmental changes rather than merely reacting to them. Nevertheless, our work extends this paradigm by incorporating principles from perceptual motor development—a concept traditionally rooted in developmental psychology. Fig. 1 shows an overview of our method. Fig. 1: Overview of Perceptual Motor Learning (PML) with Active Inference Frameworks (AIF). The agent senses the environment through its sensors and makes observations. These observations shape the agent’s beliefs and perception of the world. Based on its perception, the agent takes actions that, in turn, affect the real environment. (Icon made by Freepik from www.flaticon.com) In our framework for autonomous driving, we draw a parallel to this human developmental process. Our model leverages a deep learning architecture to fuse perceptual and motor information, constructing an internal world model that captures the causal dynamics of the environment. This model enables the system to predict the consequences of its actions and adapt to new road conditions without retraining—addressing the core issue of generalization that plagues traditional IL and RL methods. Rather than blindly imitating expert behavior or relying on externally defined rewards, our approach infers how the world works to support robust and flexible decision-making. To evaluate the effectiveness of our approach, we applied it to the lateral control of self-driving vehicles in a simulated urban environment. The model was trained on data from a simulated large urban area with multiple-lane highways and tested in distinctively different settings: a town with mountainous roads and a small town characterized by multiple bridges. This diverse testing environment highlights the model’s ability to generalize across varied road structures. In summary, the key contributions of our work are: • To our best knowledge, this study is the first integration of PML with AIF in HAVs. By leveraging an internal world model to learn the causal relationships between actions and environmental changes, our system adapts to new scenarios without requiring retraining, ensuring robust performance across diverse environments. • PML with AIF allows greater resilience and adaptability in novel driving conditions, whereas traditional IL methods suffer from long tail problems. • PML with AIF streamlines training, making it both computationally efficient and easily scalable across different driving tasks by eliminating the need for complex reward engineering. II Related Work There are different methods for controlling an HAV in an environment. One of the most popular methods is MP in autonomous driving systems, which generally consists of dedicated modules for perception, prediction, planning, and control. While this approach offers clear advantages in terms of interpretability and modularity, it also has several limitations. Errors in upstream modules (e.g., perception) can propagate downstream, leading to compounding errors in prediction, planning, and control. MPs can be inflexible, making it difficult to adapt the system to new tasks or environments without significant re-engineering. This rigidity can limit the scalability and versatility of the system. Joint optimization across modules is challenging due to the different nature of tasks each module performs [ 10 ] [ 11 ] . Another method that is vastly used is Imitation Learning (IL) which has been a popular method for developing autonomous driving systems due to its ability to mimic expert behavior from demonstrations and a lot of work has been done in this domain [ 2 ] [ 3 ] [ 10 ] [ 12 ] . Traditional IL methods, however, often struggle with generalizing to unseen states not covered by the expert’s demonstrations. When the agent encounters scenarios outside the training data, it may fail to perform adequately, leading to safety and reliability issues [ 10 ] [ 13 ] . Additionally, in complex scenarios such as urban driving, high-level decisions (e.g. choosing to turn left or right at an intersection) cannot be easily inferred from perceptual input alone, leading to ambiguity and oscillations in the agent’s behavior. To overcome this, Codevilla et al. [ 10 ] introduced Conditional Imitation Learning (CIL), where the driving policy is conditioned on high-level commands. Direct perception methods are also the other category of methods that are used [ 14 ] . Sauer et al. [ 15 ] propose Conditional Affordance Learning (CAL), a direct perception approach for autonomous urban driving. This method maps video inputs to intermediate representations (affordances) that are then used by a control algorithm to maneuver

Read the complete text