RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms

Edoardo Ghignone 1, Nicolas Baumann 1, Cheng Hu 2, Jonathan Wang 1 · 2025

Read RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms by Edoardo Ghignone 1, Nicolas Baumann 1, Cheng Hu 2, Jonathan Wang 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17311.txt.

Introduction

Edoardo Ghignone and a team of co-authors present RLPP, a controller for small self-driving race cars. Their twenty twenty-five paper uses the F1TENTH platform, a scaled car built for research. Classic controllers are steady but need much hand tuning, while learned ones often fail when moved from a simulator to a real track.

RLPP mixes both. A classic Pure Pursuit controller drives the car, and a reinforcement learning part adds a small correction on top. The hybrid keeps the clarity of the classic part and gains speed from the learned part. In real tests it cut lap times by up to about six percent and shrank the sim-to-real gap a lot.

Catalog record

CREATOREdoardo Ghignone 1, Nicolas Baumann 1, Cheng Hu 2, Jonathan Wang 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17311

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms Thanks: 1 Edoardo Ghignone, Nicolas Baumann, Jonathan Wang, and Michele Magno are associated with the Center for Project-Based Learning, D-ITET, ETH Zurich. Thanks: 2 Cheng Hu and Lei Xie are associated with the Department of Control Science and Engineering, Zhejiang University. Thanks: 3 Andrea Carron is associated with the Institute for Dynamic Systems and Control, D-MAVT, ETH Zürich. Thanks: (Edoardo Ghignone, Nicolas Baumann, and Cheng Hu contributed equally to this work.) (Corresponding author: Edoardo Ghignone.) Edoardo Ghignone 1, Nicolas Baumann 1, Cheng Hu 2, Jonathan Wang 1 Abstract Autonomous racing presents a complex environment requiring robust controllers capable of making rapid decisions under dynamic conditions. While traditional controllers based on tire models are reliable, they often demand extensive tuning or system identification. Reinforcement Learning (RL) methods offer significant potential due to their ability to learn directly from interaction, yet they typically suffer from the Sim-to-Real gap, where policies trained in simulation fail to perform effectively in the real world. In this paper, we propose RLPP, a residual RL framework that enhances a Pure Pursuit (PP) controller with an RL -based residual. This hybrid approach leverages the reliability and interpretability of PP while using RL to fine-tune the controller’s performance in real-world scenarios. Extensive testing on the F1TENTH platform demonstrates that RLPP improves lap times of the baseline controllers by up to 6.37 % 6.37\text{\,}\mathrm{\%} , closing the gap to the State-of-the-Art (SotA) methods by more than 52 % 52\text{\,}\mathrm{\%} and providing reliable performance in zero-shot real-world deployment, overcoming key challenges associated with the Sim-to-Real transfer and reducing the performance gap from simulation to reality by more than 8-fold when compared to the baseline RL controller. The RLPP framework is made available as an open-source tool, encouraging further exploration and advancement in autonomous racing research. The code is available at: www.github.com/forzaeth/rlpp . I Introduction RL offers a promising solution for racing setups, being able to achieve super-human performance in driving simulators [ 1 ] , and similarly being able to reach the level of human professionals in the setting of drone racing [ 2 ] . For this reason, different works have tried to replicate these feats in the domain of scaled autonomous car racing, such as in [ 3 ] , where an RL agent reaches levels of a SotA Model Predictive Control (MPC) baseline after finetuning on the real platform. However, such results are obtained with an external motion capture system: this significantly simplifies the task at hand when compared to fully-onboard robotic platforms such as F1TENTH [ 4 ] , where the more realistic setting constrains the car to noisy localization and state estimation. In this case, the performance of RL techniques is lagging behind when compared to SotA traditional control techniques, such as in [ 5 ] . [ 6 ] presents an interesting RL example, where the Dreamer architecture [ 7 ] is used to show that model-based RL algorithms generalize well to multiple tracks when compared to the model-free counterparts in simulation. However, when considering deployment on the physical platform, model mismatches are witnessed by the authors, specifically with the appearance of a bang-bang steering behavior, that limits the maximum speed of the car when compared to simulation. Similarly, in [ 8 ] , a hybrid method unifying traditional planners and an RL controller is shown, but deployment on the real platform is constrained to final velocities slower than those during training due to the encountered steering misbehavior. Finally, the behavior of different RL algorithms on the F1TENTH platform is compared in [ 9 ] , where end-to-end algorithms are shown to be transferable from simulation to reality at speeds up to 5\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} , but without demonstrating repeated laps, and highlighting the need for further study in the Sim-to-Real transfer domain. This work aims to advance Sim2Real techniques by exploring a different architecture of reinforcement learning controllers, namely those of residual RL policies. Introduced with [ 10 ] for a robot manipulator, this method consists of complementing a traditional type of controller with a RL policy, relinquishing part of the control authority and simplifying the task for the Machine Learning (ML) -based agent. This technique was further deployed on the F1TENTH simulator [ 11 , 12 ] , attaining lap time improvement and overtaking behavior on different tracks. However, we argue such architecture can be transferred to the physical platform and can help to bridge the Sim-to-Real gap, by retaining the tuning simplicity of the baseline algorithm while allowing for a data-driven enhancement. This paper presents RLPP, a residual RL controller that enhances the baseline performance of a PP controller with a residual action, using only simulator experience. Differently from work that applied the residual structure with controllers using tire knowledge [ 13 ] , we apply the technique to a tire-free traditional method, simplifying the overall procedure by removing the need for system identification. Furthermore, we deploy our system on the F1TENTH platform, where the full perception-planning-control is run on board and no external communication or computing budget is provided (see Table I for a comparison). The main contributions of this paper are as follows: I We present RLPP, a residual RL method combining a traditional controller with a ML -driven policy. The presented approach retains few calibration parameters, simplifying the behavior transfer across the Sim-to-Real gap. To the best of our knowledge, this is the first demonstration of an RL algorithm achieving near top-level performance compared to leading Autonomous Racing (AR) methods on an embedded platform like F1TENTH. II We extensively compare the proposed method with tire-free and tire-aware SotA AR controllers, showcasing specifically how RLPP can improve the performance of a baseline controller. III We provide the evaluated implementation ( RL environment, training procedures, and final controller) as an open-source modular addition to the software architecture presented in [ 14 ] . Related Work Fully Onboard Multiple Laps Max Speed Chisari et al. [ 3 ] No Yes 3\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} \dagger Brunnbauer et al. [ 6 ] Yes Yes N/A TC-Driver [ 8 ] Yes Yes 3\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} TAL [ 9 ] Yes No 5\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} Ours Yes Yes 5.4\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} TABLE I : Comparison of related works on RL controllers for scaled AR . The used platform is F1TENTH, except for [ 3 ] indicated with \dagger which uses a smaller form factor of 1:43 and would then reach a max speed over 12\text{\,}\mathrm{m}\text{\,}{\mathrm{s}}^{-1} when normalized at a 1:10 scale. II Methodology In this section, the hardware platform is introduced, while the RLPP algorithm is presented in the following subsection. Finally, the training procedure and the simulator environment are described. II-A Preliminaries This work is developed and evaluated on a Commercial off-the-Shelf (CotS) F1TENTH car [ 4 ] , using an Intel i7-10710U as a computational platform. The vehicle is set up for receiving steering angle and velocity commands as input, denoted with \mathbf{u}=[\delta,\,v]^{\top} , which are then translated into actuator inputs by the Vedder Electronic Speed Controller (VESC) 6 MkIV. All the experiments and evaluations on the real car use the localization and state estimation modules as defined in [ 14 ] . For more details on the hardware and software components,

Read the complete text