Anticipatory and Adaptive Footstep Streaming for Teleoperated Bipedal Robots
Luigi Penco, Beomyeong Park, Stefan Fasano, Nehar Poddar, Stephen McCrory, Nicholas Kitchel, Tomasz Bialek, Dexton Anderson, Duncan Calvert, Robert Griffin · 2025
Read Anticipatory and Adaptive Footstep Streaming for Teleoperated Bipedal Robots by Luigi Penco, Beomyeong Park, Stefan Fasano, Nehar Poddar, Stephen McCrory, Nicholas Kitchel, Tomasz Bialek, Dexton Anderson, Duncan Calvert, Robert Griffin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11802.txt.
Introduction
Luigi Penco and co-authors present a way to steer a two-legged robot by walking in this 2025 paper. A human operator's steps are streamed to the robot in real time. Instead of copying foot poses, the system turns each human step into a footstep target the robot can reach with its own balance.
The software guesses where a step will land before the foot comes down, which cuts delay. It keeps adjusting that guess as the real foot moves. It also reshapes steps to fit rough ground the operator cannot see. Tests on the humanoid robot Nadia show the pair staying in sync. The method could help robots work in places too risky for people.
Catalog record
| CREATOR | Luigi Penco, Beomyeong Park, Stefan Fasano, Nehar Poddar, Stephen McCrory, Nicholas Kitchel, Tomasz Bialek, Dexton Anderson, Duncan Calvert, Robert Griffin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-11802
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Anticipatory and Adaptive Footstep Streaming for Teleoperated Bipedal Robots Luigi Penco, Beomyeong Park, Stefan Fasano, Nehar Poddar, Stephen McCrory, Nicholas Kitchel, Tomasz Bialek, Dexton Anderson, Duncan Calvert, Robert Griffin Abstract Achieving seamless synchronization between user and robot motion in teleoperation, particularly during high-speed tasks, remains a significant challenge. In this work, we propose a novel approach for transferring stepping motions from the user to the robot in real-time. Instead of directly replicating user foot poses, we retarget user steps to robot footstep locations, allowing the robot to utilize its own dynamics for locomotion, ensuring better balance and stability. Our method anticipates user footsteps to minimize delays between when the user initiates and completes a step and when the robot does it. The step estimates are continuously adapted to converge with the measured user references. Additionally, the system autonomously adjusts the robot’s steps to account for its surrounding terrain, overcoming challenges posed by environmental mismatches between the user’s flat-ground setup and the robot’s uneven terrain. Experimental results on the humanoid robot Nadia demonstrate the effectiveness of the proposed system. A video is available at https://youtu.be/E_X6ol7BoyQ . I Introduction Teleoperated humanoid robots have the potential to enable remote access and intervention in environments that are dangerous, inaccessible, or impractical for humans. Beyond practical applications, they hold promise in the entertainment industry [ 1 ] . Importantly, teleoperation is also crucial for collecting high-quality physical data that powers learning-based approaches in robotics. Human operators can demonstrate tasks using the robot’s body, providing real-world data that is essential for training machine learning models and robotic policies [ 2 , 3 , 4 ] . One of the key challenges in robot teleoperation is achieving perfect synchronization between the user’s motion and the robot’s movements [ 5 ] , particularly during fast motions. In our previous work [ 6 ] , we developed a system that ensures smooth robot control at high frequencies by anticipating and predicting user commands. This system predicts and interpolates user references to match the faster update rates of the robot controller, which typically operates at a much higher frequency than the rate at which user inputs are captured. By doing so, it prevents jittery motions that occur when user inputs are directly applied at lower frequencies ensuring seamless operation even during rapid movements. Additionally, it mitigates the challenges posed by operating the robot over standard networks, which may introduce latency and are not optimized for high-frequency control. Fig. 1: The humanoid robot Nadia is teleoperated by a human operator equipped with a VR headset, two VR controllers and two VR trackers. Using our footstep streaming system, which anticipates user footsteps, the robot steps in sync with the operator (top), while autonomously adapting its steps to the robot’s surrounding terrain, even when the user operates on flat ground (bottom). In this work, we aim to seamlessly transfer stepping motions from the user to the robot, minimizing the delay between when the user initiates and completes a step and when the robot does it. Directly replicating user foot motions at high-speeds can easily lead to unbalanced movements due to significant differences between human and robot dynamics. For this reason, instead of directly controlling the robot’s foot poses based on the user’s foot movements, we propose a higher-level approach where the user only controls the footstep location of the robot. This allows the robot to utilize its own dynamics to execute steps while maintaining balance more effectively and also to adapt to the surrounding terrain. Research has explored different strategies to synchronize human and robot stepping in teleoperation systems. First, Ishiguro et al. [ 7 ] proposed a direct approach to real-time teleoperation, where the user directly controls the robot’s foot pose in real time. However, due to the dynamical differences between the human operator and the robot, the user must move slowly to prevent the robot from losing balance. To ensure a safe and stable touchdown, a delay phase is introduced, allowing the robot to softly set its foot on the ground. More recent work has begun to focus on reducing delays and enhancing real-time interaction. For example, Dallard et al. [ 8 ] employ a recurrent neural network to predict walking pace and step locations. These predictive techniques show potential to minimize lag in teleoperation systems, although the the accuracy of estimation to faster human-pace motions remains to be fully validated. In another vein, researchers have delved into bilateral teleoperation, where human and robot dynamics are directly coupled through haptic feedback. Ramos et al. [ 9 , 10 , 11 ] have developed systems that synchronize human and humanoid robot locomotion using a human-machine interface. Similarly, Ishiguro et al. [ 12 ] introduced a full-body exoskeleton cockpit for bilateral teleoperation, aiming to seamlessly align the movements of the operator with those of the robot. These methods are promising for enhancing teleoperation by dynamically aligning human and robot motions. However, they face challenges such as sensitivity to network delays, which can compromise real-time performance, the requirement for large, complex user-side structures that can be cumbersome and maintenance-intensive. Furthermore, the operator’s responsibility for maintaining the robot’s balance can potentially overload them mentally, especially in scenarios where the robot’s environment is dynamic. Another critical consideration is that, in many real-world scenarios, the environment at the robot’s location cannot be mirrored at the user’s side, creating significant challenges for teleoperation. This disparity makes it difficult for the user to accurately specify where the robot’s foot should land. For example, while the user may operate from a flat ground setting, the robot might encounter stairs, uneven terrain, or obstacles at varying heights that the user cannot physically climb or descend. This mismatch in environments adds complexity to teleoperating bipedal robots and necessitates systems capable of interpreting high-level user commands while autonomously adapting to local terrain conditions. Planning over rough terrain has predominantly been developed for autonomous robot navigation, where users specify high-level goals, such as a final stance location, without requiring continuous motion input [ 13 , 14 , 15 ] . Learning-based methods, such as [ 16 ] , utilize onboard elevation maps to optimize locomotion, enabling humanoid robots to traverse stairs, gaps, and high platforms. Also Zhuang et al. [ 17 ] propose a unified vision-based learning system for humanoid robots, allowing the robot to traverse terrains that require jumping on platforms and leaping over gaps. However, these approaches do not address how robots adapt their motion to the terrain based on direct user input. This limitation is critical in real-world dynamic scenarios where human decision-making is essential. Humans can identify hazards that AI might overlook –for instance, recognizing that an apparently stable region made of moldy wood is unsafe to step on. Similarly, in highly complex navigation tasks, humans can plan paths that simplify traversal and reduce risks more effectively than autonomous systems. In this work, we present a solution that simultaneously addresses the two key challenges of reducing delay of operation between the user and the humanoid during stepping and adapting the steps –retargeted from the user to the robot– to the terrain surrounding the robot. II Footstep Retargeting from User to Robot The system continuously monitors the ankle trackers to identify