No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets
Pranay Dugar, Mohitvishnu S. Gadde, Jonah Siekmann, Yesh Godse, Aayam Shrestha, Alan Fern · 2025
Read No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets by Pranay Dugar, Mohitvishnu S. Gadde, Jonah Siekmann, Yesh Godse, Aayam Shrestha, Alan Fern (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-14098.txt.
Introduction
Pranay Dugar and co-authors wrote a 2025 paper on teaching humanoid robots to walk to nearby target poses. Real work needs short moves that end at an exact spot and facing. Most learned walking methods chase a speed command, which yields a marching style that wastes time and energy on short hops.
Their learning method scores the robot with a star-like reward built around the target pose. That score rewards natural, direct moves toward the goal. A new test frame grades energy use, time to target, and step count across many goals. The method beat standard ones on those grades. It also moved from simulation to real hardware with success. The authors stress reward design for short range work.
Catalog record
| CREATOR | Pranay Dugar, Mohitvishnu S. Gadde, Jonah Siekmann, Yesh Godse, Aayam Shrestha, Alan Fern |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-14098
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets Pranay Dugar, Mohitvishnu S. Gadde, Jonah Siekmann, Yesh Godse, Aayam Shrestha, Alan Fern Abstract Humanoids operating in real-world workspaces must frequently execute task-driven, short-range movements to SE(2) target poses. To be practical, these transitions must be fast, robust, and energy efficient. While learning-based locomotion has made significant progress, most existing methods optimize for velocity-tracking rather than direct pose reaching, resulting in inefficient, marching-style behavior when applied to short-range tasks. In this work, we develop a reinforcement learning approach that directly optimizes humanoid locomotion for SE(2) targets. Central to this approach is a new constellation-based reward function that encourages natural and efficient target-oriented movement. To evaluate performance, we introduce a benchmarking framework that measures energy consumption, time-to-target, and footstep count on a distribution of SE(2) goals. Our results show that the proposed approach consistently outperforms standard methods and enables successful transfer from simulation to hardware, highlighting the importance of targeted reward design for practical short-range humanoid locomotion. I INTRODUCTION One of the defining capabilities of humanoid robots is legged locomotion, which can be broadly categorized into two functional modes. Long-range locomotion involves generating stable, rhythmic gaits guided by velocity commands, and is well suited for traversing extended distances. In contrast, short-range locomotion focuses on task-driven movements toward specific \mathrm{SE}(2) target poses, and dominates many practical workflows—for example, when a robot repeatedly moves boxes between nearby locations. This second mode of locomotion differs fundamentally from gaited walking: it prioritizes rapid, adaptive transitions and functional precision over rhythmic footstep generation. Consequently, applying long-range locomotion controllers to short-range \mathrm{SE}(2) targets will often produce inefficient and unnatural marching-style behaviors. Reinforcement learning (RL) has recently enabled state-of-the-art performance in long-range, velocity-target humanoid locomotion [ 1 , 2 , 3 , 4 ] . These successes have largely hinged on well-designed reward functions that promote forward velocity, gait stability, and symmetry—principles validated across a range of humanoid and quadruped platforms. However, such reward structures are not directly applicable to short-range \mathrm{SE}(2) -target locomotion, where the robot must simultaneously make progress toward a target position and align to a desired orientation. This dual-objective introduces ambiguity: should the robot rotate in place before stepping forward, reach the position first and then turn, or rotate while advancing? Naive reward formulations that resolve this ambiguity too rigidly or too loosely can produce behavior that is inefficient, unnatural, and poorly suited to the demands of confined, task-driven environments. These challenges highlight the need for reward functions explicitly tailored to \mathrm{SE}(2) -target commands. Fig. 1 : Overview of our approach for short-range \mathrm{SE}(2) -target locomotion. Top: The learned GoTo controller produces coordinated motion toward a specified goal pose. Middle: Footstep pattern generated by the GoTo policy for reaching a commanded \mathrm{SE}(2) target. Bottom: Real-world deployment of the controller on the Digit humanoid, walking from an initial pose (x_{c},y_{c},\theta_{c}) to a goal pose (x_{g},y_{g},\theta_{g}) . The green circle and dots denote the constellation used to define the reward, which aligns translation and rotation simultaneously, enabling efficient, natural locomotion without marching-style behavior. In this paper, we explore end-to-end reinforcement learning for short-range \mathrm{SE}(2) -target locomotion for humanoid robots. Our main contribution is the point constellation reward—a minimally constraining objective that promotes efficient and natural motion. Inspired by loss functions from learning-based motion planning for fixed-base manipulators [ 5 , 6 , 7 ] , the reward is based on the geometric similarity between point clouds representing the robot’s current and target configurations. We adapt this idea to legged \mathrm{SE}(2) -target locomotion by defining a virtual 2D constellation of points rigidly attached to the robot’s base, along with a corresponding constellation at the target pose. At each timestep, the reward is computed from the alignment between the current and target constellations, yielding a unified geometric signal that captures both translation and rotation. We show that the trade-off between positional and rotational accuracy is governed by the moment of inertia of the constellation, offering intuitive control and flexibility across tasks and workspace layouts. Using this reward, we train and evaluate both end-to-end policies and hierarchical architectures built on top of a pre-trained velocity-based controller, and show that even simple policy structures can produce robust, efficient behaviors. To assess effectiveness, we measure key aspects of locomotion performance, including final pose error, energy consumption, time to target, and footstep count. Our simulation results demonstrate that the constellation-based reward combined with end-to-end policy training leads to significantly more natural and efficient movement compared to learned or hand-coded approaches based on velocity-tracking. Moreover, the formulation reveals meaningful trade-offs between translational and rotational accuracy, allowing fine-grained behavioral control through reward shaping. Finally, we demonstrate successful sim-to-real transfer on digit-v3 humanoid platform confirming the method’s practical viability. The key contributions of this work are: (1) an end-to-end RL approach with a constellation-based reward that intuitively balances translational and rotational objectives for short-range \mathrm{SE}(2) -target locomotion; (2) a rigorous benchmarking framework \mathrm{SE}(2) -target locomotion that quantifies locomotion performance using interpretable metrics, and (3) experimental validation in both simulation and hardware, demonstrating improved efficiency, robustness, and real-world applicability in constrained environments. II RELATED WORK II-A Model-Based Control for Humanoid Navigation A common approach to humanoid navigation is to decompose the problem into global and local planning. Systems such as navfn [ 8 ] are used for high-level path planning, while local planners translate segments of these paths into executable footstep sequences. The humanoid_planner_2d package exemplifies this architecture by leveraging SBPL algorithms for 2D planning, integrated with the footstep_planner module to generate feasible foot placements. Several works have extended this paradigm to more complex environments. For example, Garimort et al. [ 9 ] developed methods for selecting optimal footstep sequences in cluttered spaces, allowing humanoids to navigate around or over obstacles. The Open Source Step-Informed framework (OpenSIn) [ 10 ] adds full 3D footstep planning integrated with perception and locomotion systems. More recently, Mishra et al. [ 11 ] proposed using generative adversarial networks (GANs) to generate footstep plans for navigation in highly constrained indoor environments, such as narrow corridors or cluttered rooms. These footstep planning systems typically take start and goal poses (including orientation) and output a discrete sequence of footsteps to reach the target. Replanning is supported to adapt to dynamic obstacles or localization drift. However, executing these plans requires an additional layer of complexity: (1) an inverse kinematics solver to compute joint angles for foot placement, (2) a trajectory generator for smooth transitions between poses, and (3) a low-level controller to track these trajectories while maintaining balance. In contrast, we focus on end-to-end learning of locomotion policies for point-to-point navigation in constrained workspaces. By designing an expressive reward function tailored to this task, we eliminate the need for explicit footstep planning or trajectory stitching, enabling more adaptive and efficient control with reduced system overhead. II-B Learning-Based Control for Humanoid Navigation Significant work has been done on learning-based approaches to basic humanoid locomotion. Early work [ 12 , 13 ] used reinforcement learning (RL) for sim-to-real transfer focusing on periodic gait patterns. More recently, Li et al. [ 14 ] extended this approach to support both periodic and non-periodic motions within a unified framework. While other works [ 15 , 16 ] use techniques