Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) International Space Station Astrobee Testing
Samantha Chapin, Kenneth Stewart, Roxana Leontie, Carl Glen Henshaw · 2025
Read Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) International Space Station Astrobee Testing by Samantha Chapin, Kenneth Stewart, Roxana Leontie, Carl Glen Henshaw (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03729.txt.
Introduction
In May 2025, a small free-flying robot aboard the International Space Station was steered by a reinforcement learning policy. Samantha Chapin and colleagues from the US Naval Research Laboratory describe that experiment, named APIARY, in a 2025 robotics preprint. Their goal is autonomous assembly work in orbit, where robots must act without a human hand on the controls.
The team trained a control policy for the NASA Astrobee robot inside a physics simulator, using an actor-critic method called Proximal Policy Optimization. Training randomised target poses and mass distributions so the policy would cope with surprises. The paper walks through simulation tests, ground tests, and the flight run itself. The authors argue the flight run shows learned control can work in orbit. That is a step toward robots handling exploration and logistics tasks with less human guidance.
Catalog record
| CREATOR | Samantha Chapin, Kenneth Stewart, Roxana Leontie, Carl Glen Henshaw |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03729
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) International Space Station Astrobee Testing Samantha Chapin, Kenneth Stewart, Roxana Leontie, Carl Glen Henshaw Abstract The US Naval Research Laboratory’s (NRL’s) Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) experiment pioneers the use of reinforcement learning (RL) for control of free-flying robots in the zero-gravity (zero-G) environment of space. On Tuesday, May 27th 2025 the APIARY team conducted the first ever, to our knowledge, RL control of a free-flyer in space using the NASA Astrobee robot on-board the International Space Station (ISS). A robust 6-degrees of freedom (DOF) control policy was trained using an actor-critic Proximal Policy Optimization (PPO) network within the NVIDIA Isaac Lab simulation environment, randomizing over goal poses and mass distributions to enhance robustness. This paper details the simulation testing, ground testing, and flight validation of this experiment. This on-orbit demonstration validates the transformative potential of RL for improving robotic autonomy, enabling rapid development and deployment (in minutes to hours) of tailored behaviors for space exploration, logistics, and real-time mission needs. I Introduction Robotic spacecraft play a crucial role in scientific exploration. Robotic rovers have operated on Mars continuously for two decades and have revolutionized our understanding of the climatic and geological history of Earth’s neighbor. The Hubble Space Telescope, the James Webb Space Telescope, and many other orbital observatories have produced an enormous amount of astronomical data and have had a profound impact on our understanding of the universe. More recently, robotic spacecraft such as the SpaceLogistics Mission Extension Vehicle (MEV) have demonstrated the ability of space robots to autonomously dock to and extend the life of valuable communications satellites [ 15 ] , and the upcoming DARPA Robotic Servicing of Geosynchronous Satellites (RSGS) spacecraft will be the world’s first orbiting satellite mechanic, being able to inspect, relocate, and upgrade satellites in geosynchronous orbit using its two 7-DOF robotic arms and interchangeable end-effectors [ 5 ] [ 17 ] . Despite these advancements, current robotic spacecraft require significant, and often expensive, human ground control. Spacecraft autonomy lags the state-of-the-art significantly. RSGS, for instance, uses Resolved Rate Motion Control (RRMC) inverse kinematics that requires human generated 6-DOF waypoints to perform task space operations [ 7 ] . Generating and verifying a single RSGS task space operation takes hours to days of a highly qualified robotics operator’s time. This is because spacecraft computers are significantly slower than those used in other domains - the RSGS flight computer consists of five 800 MHz SBCs - and because “classic” robotic automation techniques such as linear control, inverse kinematics, waypoint-based path planning, and so on can be mathematically analyzed for correctness. This is a vitally important feature of such algorithms given that spacecraft are among the most expensive engineered objects in existence; even a relatively mundane geosynchronous communications satellites costs hundreds of millions of dollars, and the James Webb Space Telescope cost more than ten billion dollars. It is therefore crucial that the flight computers controlling these spacecrafts be highly reliable and that their software is as stable and as well understood as possible. Currently, the application of RL to spacecraft control presents challenges in ensuring the same level of formal verification as traditional methods. This work focuses on demonstrating the feasibility of RL for free-flyer control, but future research will be necessary to address the critical need for verifiable safety guarantees. However, the next generation of robotic spacecraft will require significantly more autonomy. This increased capability is essential for enabling new exploration missions, like NASA’s SWIM mission to explore Europa’s ocean with submersibles. Greater autonomy also reduces the cost and increases the speed of orbital operations, such as assembling large space telescopes or solar power beaming stations. In this vein, RL approaches to the control of free-flying spacecraft, especially as a precursor to RL-based control of free-flying spacecraft with robotic arms, have great potential to revolutionize the capability of space robotic systems. This work demonstrates an RL-based 6-DOF control policy for a free-flying robotic spacecraft: NASA’s Astrobee robot operating on-board the ISS. These cube-shaped robots, measuring 12.5 inches (30.5 cm) on a side, utilize electric fan propulsion to navigate in 6-DOF, use cameras to perceive their environment, and have a 2-DOF arm for perching, inspection, and servicing. We’ve bridged the sim-to-real (Sim2Real) gap for the Astrobee platform by using the NVIDIA Isaac Lab simulation environment. The Sim2Real gap, arises from discrepancies between the simulated and real-world dynamics, often leading to degraded performance and necessitates extensive real-world fine-tuning. While RL offers the potential for adaptable and optimized control policies, overcoming this Sim2Real gap is critical for realizing its benefits in space-based applications. Within Isaac Lab, a robust and capable Divert-and-Attitude Control System (DACS) is trained for Astrobee under ISS-like conditions, including zero-G and then verified within a simulation of the ISS interior. Robustness is enhanced by randomizing over goal position, goal orientation, and in some experiments, mass. The policy generates wrenches that are executed by Astrobee’s onboard fan-based thrusters and existing wrench control logic. Simulations, along with tests on the Astrobee ground air bearing simulator hardware, demonstrate the resulting RL policy’s ability to execute motion trajectories, culminating in the (to our knowledge) first RL control of a free-flyer in space using the NASA Astrobee robot on-board the ISS. I-A Related Work Traditional spacecraft control relies heavily on classical control methods, such as Proportional-Derivative (PD) and Proportional-Integral-Derivative (PID) controllers for attitude determination, and bang-bang control for divert maneuvers [ 8 ] . These methods have proven effective for basic stabilization and trajectory adjustments. However, the increasing complexity of space missions, including intricate robotic arm operations and autonomous maneuvering, require the exploration of more advanced control strategies. Recently, RL has emerged as a promising approach for enhancing spacecraft control, offering the potential for greater autonomy, adaptability, and optimized performance in challenging space environments [ 22 ] . Most modern spacecraft are three-axis stabilized, meaning they have the ability to maintain a desired orientation with respect to the earth and the sun (or, for space telescope, with respect to an astronomical object being observed). This is necessary to maintain the solar incidence angle of the solar panels, high bandwidth communications links with satellite dishes on the earth, and fields of view for the satellite’s science or imagery sensors, if any [ 9 ] . Attitude control is typically accomplished with standard PD or PID linear control [ 24 ] . Typical spacecraft rotational actuators are reaction wheels, rate-moment gyros, or in some cases magnetic torquer bars that react against the earth’s magnetic field [ 25 ] [ 24 ] . These are proportional output devices, and standard linear control theory applies well when using them. Spacecraft typically have significant flexible body dynamics due to large solar panels and, sometimes, to large radio antennas [ 26 ] . They often have significant dynamics due to fuel slosh, the fuel used for their thrusters exerting nontrivial dynamics in ways that are nonintuitive due to the microgravity environment [ 26 ] . These dynamics are typically treated as disturbances to the attitude control system, and hence robust linear control techniques are often employed. This has the advantage of resulting in attitude control software that can be mathematically analyzed for correctness and