InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset

Felix Stillger, Lukas Hahn, Frederik Hasecke, Tobias Meisen, [0.35cm] University of Wuppertal, <lastname>@uni-wuppertal.de · 2026

Read InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset by Felix Stillger, Lukas Hahn, Frederik Hasecke, Tobias Meisen, [0.35cm] University of Wuppertal, <lastname>@uni-wuppertal.de (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03814.txt.

Introduction

Felix Stillger and colleagues built InCaRPose, a transformer that estimates relative camera poses inside cars, where fisheye distortion and tight spaces defeat standard calibration. Frozen backbone features plus a transformer decoder recover metric-scale translation in one step, within the plausible range of cabin mounts. Trained only on synthetic data, it generalized to real cabins and stayed competitive on the 7-Scenes set. A small backbone still kept precision high enough for real-time driver monitoring.

Catalog record

CREATORFelix Stillger, Lukas Hahn, Frederik Hasecke, Tobias Meisen, [0.35cm] University of Wuppertal, <lastname>@uni-wuppertal.de
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54147
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03814

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset Felix Stillger, Lukas Hahn, Frederik Hasecke, Tobias Meisen, [0.35cm] University of Wuppertal, <lastname>@uni-wuppertal.de Abstract Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In‑Cabin‑Pose test dataset consisting of highly distorted vehicle‑interior images and our code at https://github.com/felixstillger/InCaRPose . \svgsetup inkscapelatex=false, 1 Introduction Accurately estimating the relative camera pose between two views is essential for applications such as SLAM, AR, and 3D reconstruction. Classical pipelines rely on geometric feature matching and epipolar geometry, which can degrade under occlusions or strong lens distortions. Convolutional and Transformer-based architectures represent a significant advance in deep learning, providing robust global representations. However, state-of-the-art models that generalize well typically require large training sets for convergence and only work with specific camera intrinsics. Moreover, these models are unsuitable for edge deployment due to their prohibitive computational requirements and excessive parameter counts. Figure 1 : Our InCaRPose predicts the relative camera pose between a reference and a target view (shown as camera frustums). Trained exclusively on synthetic images, the model generalizes to real-world cabin environments and enables camera extrinsic calibration. The integration of in-cabin sensing cameras is accelerating, driven by EU GSR and Euro NCAP mandates, as well as rapid growth in driver and occupant monitoring 30 ; 64 ; 49 . These cameras are commonly mounted at multiple possible positions within the cabin, with the rear-view mirror being a preferred position due to its optimal view inside the cabin. When equipped with wide-angle or fisheye lenses, this configuration enables effective monitoring of both the driver and vehicle occupants in the front and back rows. However, rear-view mirrors are dynamic components that are manually or automatically adjusted by drivers, resulting in frequent changes to the camera’s extrinsic parameters. Maintaining consistent camera-to-vehicle calibration is a key challenge for reliable in-cabin perception. Interior cabin cameras typically operate in the near-infrared (NIR) spectrum rather than the more common visible RGB spectrum 47 . Active NIR illumination (750–1400 nm) ensures robust sensor visibility while remaining imperceptible to the human eye 44 ; 58 and therefore does not distract vehicle occupants. This mitigates the effects of cast shadows, surface texture variations, and ambient lighting changes, and enables night-time operation. Accurate spatial awareness of in-cabin sensors is critical for driver monitoring, such as gaze estimation in autonomous driving to ensure drivers can return their hands to the wheel within required timeframes, as well as for other safety-relevant applications like occupant-position-aware airbag deployment. In the event of a collision, the system must infer occupant locations on the order of a few milliseconds to optimize restraint behavior because airbag control algorithms typically decide within 15–50 ms (encompassing sensing-to-actuation pipeline) after detection 15 ; 6 ; 59 . We address camera extrinsic calibration in interior cabin environments and release our real-world in-cabin test dataset and code. Our model estimates the relative transformation between a potentially shifted cabin view and a calibrated reference view (Fig. 1 ). It combines frozen self-supervised ViT features with a Transformer decoder and a lightweight prediction head, enabling data-efficient training and real-time inference for time-critical automotive and industrial applications. Training exclusively on a small synthetic cabin set is sufficient to recover large relative pose changes and generalize to real cabin imagery. We further evaluate the model’s generalization capabilities on the public 7-Scenes and Cambridge Landmarks datasets, achieving competitive results without iterative solvers or dense 3D reconstructions. In summary, our main contributions are: • A Novel In-Cabin-Pose Test Dataset. We publish highly distorted, wide-FoV NIR fisheye imagery with metric ground truth as a real-world in-cabin test set. • Vehicle-Agnostic Reference Formulation. We reformulate the in-cabin pose estimation problem as reference-relative pose prediction to avoid vehicle-specific coordinate frames and individual retraining. • Robust Fisheye Handling Without Undistortion. InCaRPose processes distorted fisheye images end-to-end and predicts translation in absolute metric units. • Synthetic-to-Real Capable Model. Training on limited synthetic renderings yields strong transfer to a real cabin and competitive performance on 7-Scenes. 2 Related Work 2.1 Absolute Pose Estimation Absolute pose methods localize a camera in a global scene by regressing the camera position in the world coordinate system from images. Typically, they use a 2D to 3D matching and Perspective-n-Point-algorithm (PnP) with RANSAC. End-to-end learning-based relocalizers encode the scene implicitly in neural network weights. This includes scene coordinate regression (SCR) methods 7 ; 8 ; 9 and absolute pose regression (APR), such as PoseNet 33 ; 32 . Some of them use image retrieval to estimate pose approximation 52 . While effective, these often require scene-specific training or dense reconstructions. Minimal solvers and robust estimators continue to improve sampling efficiency for these tasks 62 . More recently, several variants of absolute pose estimation aim to improve robustness and cross-scene generalization. DFNet 14 improves absolute pose regression through direct feature matching, while Neural Refinement 12 introduces a refinement stage that significantly improves pose accuracy. Map-relative pose regression 13 extends APR to settings where only sparse or lightweight maps are available. Furthermore, Yin et al. 68 propose a Laplace-inspired distribution to better model rotation uncertainty in regression-based localization. 2.2 Relative Pose Estimation Relative pose methods estimate the transformation between two views without explicit scene maps. Classical approaches rely on matching local features to recover the essential matrix 25 , which is decomposed into relative rotation and a scaleless translation vector. This formulation has been enhanced by learned features 40 ; 18 and by improved matchers such as SuperGlue 50 and LoFTR 57 . Robust estimators such as MAGSAC++ 5 further improve reliability. Deep relative pose regression (RPR) bypasses explicit correspondences to directly predict pose 41 ; 35 . Although some RPR methods use image retrieval 1 ; 27 to look up posed database images, they often implicitly rely on prior SfM or SLAM reconstructions. To avoid reliance on fixed databases, Turkoglu et al. 60 proposed using graph neural networks to aggregate information across multiple views, utilizing relative pose supervision to improve re-localization accuracy. In contrast, map-free relocalization targets scenarios where only a single reference image is available 3 ; 66 ; 2 . ExReNet 66 shows that

Read the complete text