Crossing the Sim2Real Gap Between Simulation and Ground Testing to Space Deployment of Autonomous Free-flyer Control
Kenneth Stewart, Samantha Chapin, Roxana Leontie, Carl Glen Henshaw · 2025
Read Crossing the Sim2Real Gap Between Simulation and Ground Testing to Space Deployment of Autonomous Free-flyer Control by Kenneth Stewart, Samantha Chapin, Roxana Leontie, Carl Glen Henshaw (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03736.txt.
Introduction
Kenneth Stewart and colleagues report a milestone for space robotics in a 2025 preprint: a learning-based controller flew a free-floating robot in orbit. The robot was NASA's Astrobee, aboard the International Space Station. Controllers trained in simulation often fail on real hardware, and the authors set out to cross that simulation-to-reality gap deliberately.
Their pipeline trained a deep neural network inside a graphics-accelerated physics simulator, using curriculum learning that gradually hardens the task. The trained network replaced Astrobee's standard attitude and translation control, letting the robot navigate in microgravity. Repeated randomised training runs in the simulator checked robustness before upload. The successful flight, the authors argue, shows policies trained on Earth can transfer to space. That matters for future servicing, assembly, and manufacturing work in orbit, where conditions change and controllers must adapt quickly.
Catalog record
| CREATOR | Kenneth Stewart, Samantha Chapin, Roxana Leontie, Carl Glen Henshaw |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03736
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Crossing the Sim2Real Gap Between Simulation and Ground Testing to Space Deployment of Autonomous Free-flyer Control Kenneth Stewart, Samantha Chapin, Roxana Leontie, Carl Glen Henshaw Abstract Reinforcement learning (RL) offers transformative potential for robotic control in space. We present the first on-orbit demonstration of RL-based autonomous control of a free-flying robot, the NASA Astrobee, aboard the International Space Station (ISS). Using NVIDIA’s Omniverse physics simulator and curriculum learning, we trained a deep neural network to replace Astrobee’s standard attitude and translation control, enabling it to navigate in microgravity. Our results validate a novel training pipeline that bridges the simulation-to-reality (Sim2Real) gap, utilizing a GPU-accelerated, scientific-grade simulation environment for efficient Monte Carlo RL training. This successful deployment demonstrates the feasibility of training RL policies terrestrially and transferring them to space-based applications. This paves the way for future work in In-Space Servicing, Assembly, and Manufacturing (ISAM), enabling rapid on-orbit adaptation to dynamic mission requirements. I Introduction Future In-Space Servicing, Assembly, and Manufacturing (ISAM) missions require increasingly autonomous robotic systems capable of adapting to the dynamic and uncertain conditions of space. While traditional spacecraft control methods are often limited in their adaptability, RL offers the potential to learn robust control policies directly from experience. However, a major obstacle to deploying RL in space is the Sim2Real gap, representing the discrepancy between simulated training environments and the complexities of the real world. This gap can lead to significant performance degradation when transferring RL policies from simulation to hardware. In this paper, we address both the need for increased autonomy in ISAM and the challenge of Sim2Real transfer by presenting the first on-orbit demonstration of RL-based control for a free-flying robot. We trained a deep neural network using NVIDIA’s Omniverse physics simulator and curriculum learning to control the NASA Astrobee aboard the International Space Station (ISS). This successful deployment validates our novel training pipeline for bridging the Sim2Real gap and enabling rapid adaptation to changing mission needs, representing a significant step toward realizing the potential of autonomous robotic Assembly, Integration and Testing (AI&T) for future space missions. We validate a deep RL (DRL) control law for the NASA Astrobee robot which we have verified via ground-based and ISS microgravity testing. Specifically, we address the RL Sim2Real gap for these robots operating on-board the ISS. Astrobee is a 12.5 inches (30.5 cm) cube-shaped robot designed to conduct intravehicular activities (IVAs) on-board the ISS. It uses electric fan propulsion for six degree-of-freedom (DOF) motion, and is equipped with cameras, sensors, and a 2-DOF perching arm for inspection and servicing tasks [ 3 ] [ 7 ] . In addition to its operational capabilities it serves as a platform that enables researchers to test free-flyer algorithms and payloads rapidly in a zero-G pressurized environment [ 1 ] . Recently, DRL using neural networks has emerged as a promising approach for enhancing spacecraft control, offering the potential for greater autonomy, adaptability, and optimized performance in challenging space environments [ 5 ] . Therefore we are applying RL for autonomous control of the Astrobee as the first step of an effort called Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) experiment [ 4 ] that is working towards autonomous robotic Assembly, Integration and Testing (AI&T) for future In-Space Servicing, Assembly, and Manufacturing (ISAM) space missions. RL policies for robotics are typically trained and evaluated in simulation before being applied on hardware. This poses the challenge of crossing the Sim2Real gap because simulations do not always perfectly capture the physical dynamics of the real world or unexpected variations for between the simulation and the real world hardware and environment [ 16 ] . To overcome the Sim2Real gap we used the following workflow. First, an RL policy was generated, through training within the high fidelity NVIDIA Omniverse Isaac Lab physics simulator, to output wrench commands to move the Astrobee in a zero-G environment to a desired pose. NVIDIA Omniverse allows for training 10000s of parallel randomizable environments to maximize robotic experience during training. To more effectively train the RL policy against variations that could be introduced during testing, a curriculum [ 12 ] was used to gradually introduce and train on such variations across the parallel environments. Next, the RL policy was validated within the NASA Ames’ Astrobee simulator using Gazebo and ROS Noetic, proving that the policy can replace the control of the robot’s motion within the existing NASA Astrobee Robot Software. Next, preliminary real-world testing was conducted where the RL policy’s wrench control commanded the Astrobee’s fan-based propulsion system at the NASA Ames’ Granite Lab, a terrestrial microgravity facility comprised of a large granite table that can mimic zero-G in a single plane by floating the Astrobee on air-bearings. The performance of the RL policy was compared to the baseline Astrobee controller. Finally, an RL policy was successfully tested in the microgravity environment of the ISS; this represents a first-of-its-kind demonstration of RL-based control for free-flying space robots. This successful implementation enables the development of complex, autonomous behaviors for Astrobee in the challenging microgravity environment of the ISS, unlocking new capabilities for scientific research and operational tasks. Furthermore, this novel methodology for training RL policies for space-based applications and verifying them terrestrially creates a pipeline for RL training to bridge the Sim2Real gap that will pave the way for future autonomous robot space missions. These results highlight the transformative potential of RL for advancing robotic autonomy in space exploration and allowing for the possibility of rapid ground simulation RL training for on-orbit deployment to adapt to real-time mission needs. II Related Work Spacecraft attitude control has traditionally relied on classical methods, such as Proportional-Derivative (PD) and Proportional-Integral-Derivative (PID) controllers, alongside bang-bang control for translational maneuvers [ 27 ] . Proportional output actuators—reaction wheels, rate-moment gyros, and magnetic torquer bars—are well-suited to these linear control strategies. However the linear parameterization of dynamic uncertainty exploited by standard fixed-base adaptive controllers cannot be obtained when the base is free-floating [ 19 ] . Neural networks can be used to model the non-linear parameterization of the dynamic uncertainty for adaptive controllers and have demonstrated success for reliable adaptive control of space manipulators on a free floating base [ 19 ] [ 25 ] [ 26 ] . The use of neural networks for adaptive control results in greater performance, and space fuel savings [ 26 ] . RL for spacecraft control has previously been explored in simulations for various tasks [ 24 ] [ 13 ] . These tasks include autonomous lunar landing [ 22 ] , fuel efficient Mars landing [ 6 ] , and spacecraft formation flying [ 8 ] . DRL with Proximal Policy Optimization (PPO) [ 21 ] has shown promising results for robotic control such as end-effector control of an arm connected to a free flying satellite [ 18 ] , and spacecraft debris collision avoidance [ 11 ] . PPO has shown considerable success in bridging the sim-to-real gap, facilitating the deployment of policies trained in simulation onto real-world robotic platforms. Furthermore, PPO has been demonstrated in dexterous in-hand manipulation tasks [ 14 ] , highlighting its robustness in sim-to-real transfer for complex manipulation tasks. This robustness extends to the control of floating platforms in simulated zero-g environments