Bridging the Synthetic–Real Gap: Supervised Domain Adaptation for Robust Spacecraft 6-DoF Pose Estimation

Inder Pal Singh, Nidhal Eddine Chenni, Abd El Rahman Shabayek, Arunkumar Rathinam, Djamila Aouada · 2025

Read Bridging the Synthetic–Real Gap: Supervised Domain Adaptation for Robust Spacecraft 6-DoF Pose Estimation by Inder Pal Singh, Nidhal Eddine Chenni, Abd El Rahman Shabayek, Arunkumar Rathinam, Djamila Aouada (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-13792.txt.

Introduction

Inder Pal Singh and four co-authors close the synthetic-real gap for spacecraft pose estimation in a 2025 space vision paper. Autonomous rendezvous, docking, and servicing all need a spacecraft's position and orientation from camera images. Pipelines trained on synthetic renders score well in the lab but drop sharply on real imagery.

The team propose the first supervised domain adaptation framework built for this task's keypoint regression stage. It learns representations that ignore the domain while keeping task risk low, using labeled synthetic data plus a small amount of labeled real data. On the SPEED+ benchmark it beat source-only, fine-tuning, and even oracle baselines, matching oracle performance with only about five percent labeled target data, in a lightweight design that fits any backbone.

Catalog record

CREATORInder Pal Singh, Nidhal Eddine Chenni, Abd El Rahman Shabayek, Arunkumar Rathinam, Djamila Aouada
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-13792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bridging the Synthetic–Real Gap: Supervised Domain Adaptation for Robust Spacecraft 6-DoF Pose Estimation Inder Pal Singh, Nidhal Eddine Chenni, Abd El Rahman Shabayek, Arunkumar Rathinam, Djamila Aouada Abstract Spacecraft Pose Estimation (SPE) is a fundamental capability for autonomous space operations such as rendezvous, docking, and in-orbit servicing. Hybrid pipelines that combine object detection, keypoint regression, and Perspective- n n -Point (PnP) solvers have recently achieved strong results on synthetic datasets, yet their performance deteriorates sharply on real or lab-generated imagery due to the persistent synthetic-to-real domain gap. Existing unsupervised domain adaptation approaches aim to mitigate this issue but often underperform when a modest number of labeled target samples are available. In this work, we propose the first Supervised Domain Adaptation (SDA) framework tailored for SPE keypoint regression. Building on the Learning Invariant Representation and Risk (LIRR) paradigm, our method jointly optimizes domain-invariant representations and task-specific risk using both labeled synthetic and limited labeled real data, thereby reducing generalization error under domain shift. Extensive experiments on the SPEED+ benchmark demonstrate that our approach consistently outperforms source-only, fine-tuning, and oracle baselines. Notably, with only 5% labeled target data, our method matches or surpasses oracle performance trained on larger fractions of labeled data. The framework is lightweight, backbone-agnostic, and computationally efficient, offering a practical pathway toward robust and deployable spacecraft pose estimation in real-world space environments. keywords Space Situation Awareness, Spacecraft Pose Estimation, Domain Adaptation, 1 Introduction The determination of the position and orientation constituting six-degree-of-freedom (6-DoF) pose of a target spacecraft, commonly referred to as spacecraft pose estimation, is a key enabler for autonomous on-orbit operations such as rendezvous, docking, inspection, and servicing tarabini2013proba ; d2012spaceborne . The monocular vision-based 6-DoF pose estimation involves deducing the rotation and translation data of a target spacecraft from 2D images captured from the chaser spacecraft. Accurate estimation of the 6-DoF pose from 2D images is particularly challenging in the space environment due to illumination changes, specular reflections, limited texture, and significant variations in the apparent size of the target caused by changes in range during approach hu2021wide . Figure 1: Working pipeline for hybrid modular approaches. Early spacecraft pose estimation methods relied primarily on geometric computer vision techniques such as edge matching, template alignment, and photogrammetry-based measurements petit2012vision . Although effective in controlled conditions, these methods are sensitive to noise and environmental variations, which limit their robustness in real on-orbit imagery. With the rise of deep learning (DL), fully end-to-end networks have been explored, mapping raw images directly to pose parameters park2019towards ; park2024robust . Such approaches can implicitly learn complex visual cues, but often require vast amounts of labelled data and can be less interpretable or adaptable to new spacecraft geometries. Alternate approaches to end-to-end approaches are hybrid modular approaches to estimate spacecraft poses (Figure 1 ). These methods combine data-driven feature extraction with geometric model-based solvers, exploiting the strengths of both paradigms pauly2023survey ; ye2023filterformerpose . In a typical pipeline, the process begins with spacecraft localization , where a deep learning (DL) object detection model predicts a bounding box enclosing the target spacecraft (see Figure 1 (a)). This stage mitigates the effects of background clutter and scale variation by extracting a region of interest (ROI) for subsequent processing. The next stage, keypoint regression , employs a DL-based regression network to infer the two-dimensional image coordinates of a predefined set of three-dimensional (3D) keypoints on the spacecraft (see Figure 1 (b)). These keypoints are typically chosen to correspond to geometrically distinctive features such as the vertices of structures or distinct locations on the spacecraft. The 3D coordinates are obtained from an a priori spacecraft 3D model or be reconstructed from imagery via structure-from-motion techniques. Finally, as shown in Figure 1 (c), the pose computation stage establishes correspondences between the regressed 2D keypoints and the known 3D keypoints, solving the Perspective- n -Point (PnP) problem to recover the translation vector and rotation quaternion lepetit2009ep . To improve accuracy, the initial PnP solution can be refined by minimizing a geometric reprojection error through optimization. Several studies park2024robust ; rathinam2024spades have shown that this hybrid modular strategy achieves better accuracy over direct approaches in benchmark datasets such as SPEED speedC and SPEED+ speedplusC while preserving modularity, enabling individual components to be upgraded without retraining the entire pipeline. This modularity is especially advantageous in operational contexts where mission-specific spacecraft geometries or updated sensor models must be accommodated without complete end-to-end retraining. Despite their demonstrated effectiveness, current hybrid modular methods are significantly affected by the domain gap that arises from the discrepancies between synthetic and real imagery. Due to the scarcity and cost of acquiring annotated spacecraft datasets, most existing models are trained predominantly on synthetic images, which often fail to capture the full complexity of real on-orbit conditions, especially physical interaction of different materials under different lighting conditions. As a result, models that perform well on synthetic benchmarks may exhibit substantial performance degradation when deployed on real mission data pauly2023survey . Domain adaptation techniques have been proposed to mitigate this gap, with most focussing on unsupervised adaptation scenarios where target domain labels are unavailable perez2022spacecraft ; perez2023spacecraft . However, in many practical situations, a limited set of labelled target-domain images can be obtained, offering the opportunity to further improve performance beyond what unsupervised methods can achieve. To this end, we aim to exploit this setting by jointly learning domain-invariant representations and minimizing task-specific risk, thereby leveraging both synthetic and limited labelled real data to enhance robustness and accuracy in spacecraft pose estimation. The main contributions of this work are as follows. First, we investigate the impact of the synthetic-to-real domain gap on hybrid modular spacecraft pose estimation pipelines, highlighting performance degradation when models are trained solely on synthetic data. Second, we propose a domain adaptation framework that exploits limited labelled target-domain data, a scenario that is often realistic in space applications. The approach jointly learns domain-invariant feature representations and minimizes task-specific risk, enabling the model to leverage both synthetic and limited real data effectively. Finally, we evaluate the proposed method on benchmark datasets, demonstrating improved robustness and accuracy compared to state-of-the-art unsupervised adaptation and baseline hybrid modular methods. 2 Problem Formulation This section presents mathematical formulation of the 6-DoF SPE problem. Later, the formulation extended to the SDA setting, where a model trained predominantly on synthetic (source) data must adapt to scarcely labelled real (target) data. Finally, the problem setup to derive finite-sample generalization bounds under the LIRR lirr strategy, providing a theoretical basis to understand performance limits with this setup. 2.1 6-DoF Pose Estimation via Hybrid Modular Approach In the hybrid modular framework, the 6-DoF SPE problem is decomposed into three sequential stages: (i) prediction of the bounding box for spacecraft localization, (ii) 2D keypoint regression, and (iii) Perspective- n -Point (PnP) pose computation. This decomposition enables independent optimization of each module, leveraging both deep learning and geometric constraints. Bounding Box Prediction Let \mathcal{I} denote an input image captured by the monocular camera. A deep learning-based object detection model predicts an axis-aligned bounding box \mathbf{b}=(u_{\min},v_{\min},u_{\max},v_{\max}) that encloses the target spacecraft in the image plane. This prediction can be formulated as: \hat{\mathbf{b}}=\arg\max_{\mathbf{b}}\ P_{\theta}(\mathbf{b}\mid\mathcal{I}), (1) where P_{\theta} is the detection model parameterized by \theta . The predicted bounding box

Read the complete text