Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving

Ruiqian Nai, Jiacheng You, Liu Cao, Hanchen Cui, Shiyuan Zhang, Huazhe Xu, Yang Gao · 2025

Read Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving by Ruiqian Nai, Jiacheng You, Liu Cao, Hanchen Cui, Shiyuan Zhang, Huazhe Xu, Yang Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10956.txt.

Introduction

Four-legged robots learn to walk in simulators, but simulators often get power use and foot noise wrong. Ruiqian Nai and co-authors addressed that gap in 2025 with a method that learns from real robot data. Instead of hand-made stand-ins for power, the team built a model of total battery use and fed it back into training.

The fine-tuned walking policy cut total power use by about a quarter across several speeds. That gain came from the battery pack as a whole, not just the motors. The authors present the approach as a general way to improve traits that are hard to simulate, using real-world data as it arrives.

Catalog record

CREATORRuiqian Nai, Jiacheng You, Liu Cao, Hanchen Cui, Shiyuan Zhang, Huazhe Xu, Yang Gao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10956

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fine-Tuning Hard-to-Simulate Objectives for Quadruped Locomotion: A Case Study on Total Power Saving Ruiqian Nai, Jiacheng You, Liu Cao, Hanchen Cui, Shiyuan Zhang, Huazhe Xu, Yang Gao Abstract Legged locomotion is not just about mobility; it also encompasses crucial objectives such as energy efficiency, safety, and user experience, which are vital for real-world applications. However, key factors such as battery power consumption and stepping noise are often inaccurately modeled or missing in common simulators, leaving these aspects poorly optimized or unaddressed by current sim-to-real methods. Hand-designed proxies, such as mechanical power and foot contact forces, have been used to address these challenges but are often problem-specific and inaccurate. In this paper, we propose a data-driven framework for fine-tuning locomotion policies, targeting these hard-to-simulate objectives. Our framework leverages real-world data to model these objectives and incorporates the learned model into simulation for policy improvement. We demonstrate the effectiveness of our framework on power saving for quadruped locomotion, achieving a significant 24-28% net reduction in total power consumption from the battery pack at various speeds. In essence, our approach offers a versatile solution for optimizing hard-to-simulate objectives in quadruped locomotion, providing an easy-to-adapt paradigm for continual improving with real-world knowledge. Project page https://hard-to-sim.github.io/. I Introduction Legged locomotion grants robots the ability to traverse complex terrains, offering universal mobility [ 1 ] . Beyond the goal of command following, broader considerations such as energy efficiency, payload capacity, user-friendly interaction, and safety are critical for extending their utility in industrial and everyday environments [ 2 , 3 , 4 , 5 ] . For quadruped robots in particular, constraints such as limited battery life per charge significantly impact the effectiveness of robots in tasks like patrolling or rescue operations [ 6 , 7 , 8 ] . Moreover, issues like disruptive stepping noise can negatively affect user experience [ 9 ] , and motor overheating poses risks to the robot’s operational lifespan [ 10 ] . The current successes of learning-based approaches heavily depend on the sim-to-real transfer paradigm [ 11 , 12 ] , where policies trained in simulations are directly applied to real-world robots [ 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ] . These methods leverage physics simulators like Isaac Gym [ 21 ] , MuJoCo [ 22 ] , and Bullet [ 23 ] , which mainly focus on dynamics and kinematics. However, critical factors like power consumption, stepping noise and safety features are not available or inaccurately modeled. These factors are hard to simulate due to the complexity of the underlying mechanisms. For instance, the intricate dynamics of Permanent Magnet Synchronous Motors (PMSMs) pose significant challenges in predicting total power consumption [ 24 , 25 ] . Furthermore, the implementation of control strategies, such as Field-Oriented Control (FOC), adds another layer of complexity to accurately forecasting motor power requirements [ 26 , 27 ] . Traditional approaches have utilized hand-designed proxies as reward functions for training policies, applying metrics like mechanical power or foot contact forces to approximate energy consumption or noise [ 28 , 29 , 30 , 31 , 16 ] . These methods demand expert knowledge and intensive tuning, and their efficacy is limited by the accuracy of the proxies used. In contrast, learning from real-world experience offers a more precise and efficient alternative for optimizing these challenging objectives. In this paper, we introduce a data-driven fine-tuning approach to optimize hard-to-simulate objectives in locomotion policies. Our method begins with the collection of real-world data using a pre-trained policy. We then develop a measurement model to predict hard-to-simulate factors from this data, which is integrated into the simulation as a reward function. Our approach performs iterative policy improvement through cycles of data collection and policy updating. We present experimental results demonstrating a significant 24-28% reduction in total power consumption from the battery pack for quadruped locomotion. The task objective—minimizing power consumption while maintaining locomotion performance—highlights the operational time constraints faced by current low-cost quadruped robots [ 6 , 7 , 32 ] . Despite the complexities of modeling total power consumption [ 33 , 27 ] , our method effectively manages these challenges, illustrating its potential to address demanding objectives in quadruped locomotion. We believe that our proposed framework could be applied to a wide range of hard-to-simulate objectives. It utilizes a data-driven measurement model, which is designed to be objective-agnostic and has the potential to automatically adapt to various challenging objectives. Technically, our approach requires only minimal modifications to the existing sim-to-real pipeline, offering a plug-and-play method for integrating empirical data from the physical world to enhance locomotion performance. Fig. 1 : Fine-tuning procedure at the i -th iteration. Real-world data is collected using the pre-trained policy or the policy batch from the last iteration. A measurement model is trained to estimate hard-to-simulate factors based on the data, which is later integrated into simulation. Policy updates are performed in simulation, generating a batch of policies with varying configurations. Hierarchical policy selection is then conducted based on performance in simulation and real-world evaluations, determining policies retained for the next iteration. Algorithm 1 Fine-tuning hard-to-simulate objectives 1: Pre-trained policy {\pi}^{\text{pre-trained}} , an empty real-world dataset \mathcal{D}_{0} , measurement model training algorithm \mathcal{A}:\mathcal{D}\mapsto f , parameter search space for w_{\cdot} and c : \mathcal{W} and \mathcal{C} , and the size of the elite policy batch: K . 2: \Pi_{0}^{\text{elites}}\leftarrow\left\{{\pi}^{\text{pre-trained}}\right\} . 3: for i=0,1,2,\ldots do 4: // Real-world data collection: 5: D_{i}\leftarrow\varnothing 6: for \pi\in\Pi_{i}^{\text{elites}} do 7: Roll out \pi in the real world to collect data, \left\{\mathbf{s}^{\text{sim}}_{t},\mathbf{s}^{\text{real}}_{t}\right\}^{\pi} . 8: {D}_{i}\leftarrow D_{i}\cup\left\{\mathbf{s}^{\text{sim}}_{t},\mathbf{s}^{\text{real}}_{t}\right\}^{\pi} . 9: Accumulate all real data: \mathcal{D}_{i}\leftarrow\bigcup_{j=0}^{i}{D}_{j} . 10: Train a measurement model: f_{i}\leftarrow\mathcal{A}(\mathcal{D}_{i}) . 11: // Simulation fine-tuning: 12: \Pi^{\text{candidates}}_{i}\leftarrow\varnothing 13: for {\pi}^{\text{anchor}},w_{\cdot},c\in\left\{\pi_{i}^{\text{best}},{\pi}^{\text{pre-trained}}\right\}\times\mathcal{W}\times\mathcal{C} do 14: \pi^{\text{fine-tuned}}\leftarrow fine-tune with the objective in Eq. 2 . 15: \Pi_{i}^{\text{candidates}}\leftarrow\Pi_{i}^{\text{candidates}}\cup\left\{\pi^{\text{fine-tuned}}\right\} . 16: // Hierarchical policy selection: 17: \Pi^{\text{elites}}_{i+1}\leftarrow top- K of \Pi^{\text{candidates}} measured by simulation performances. 18: \pi^{\text{best}}_{i+1}\leftarrow best of \Pi^{\text{elites}}_{i+1} evaluated by real-world performances. II Related Work Direct learning from real-world experience Training policies directly in the real world can circumvent the complexities associated with modeling hard-to-simulate factors. However, previous methods [ 34 , 35 , 36 , 37 ] typically achieve results only in low-performance regions characterized by slow walking speeds. On the other hand, methods that fine-tune simulation-trained policies in real-world [ 38 , 39 , 40 ] primarily addresses the shift in dynamics due to sim-to-real transferring. These studies often do not extend to optimizing objectives that are unseen in simulation pre-training. Furthermore, in contrast to these methods, which are generally tailored for specific learning algorithms [ 41 , 42 ] , our approach offers a versatile integration across various policy optimization frameworks. Augmenting simulation with data-driven models Recent advances have embraced hybrid simulations, combining analytic physics with data-driven models to capture complex dynamics [ 43 , 44 , 45 , 46 , 47 ] . Such hybrid simulations find applications in diverse robotic tasks, including legged locomotion [ 13 , 16 ] , drone racing [ 48 ] , and modeling human behavior in sports [ 49 ] . These studies, however, primarily focus on enhancing the fidelity of simulation dynamics . In contrast, our setting requires data-driven

Read the complete text