Minimal Solvers for Full DoF Motion Estimation from Asynchronous Tracks

Petr Hruby, Marc Pollefeys · 2025

Read Minimal Solvers for Full DoF Motion Estimation from Asynchronous Tracks by Petr Hruby, Marc Pollefeys (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-17537.txt.

Introduction

Petr Hruby and Marc Pollefeys study how to recover a camera's full motion from point tracks that are not sampled in step, in a paper from 2025. Such tracks arise with rolling shutter cameras and with event cameras that report changes one by one. The motion sought covers both shift and rotation speed.

The exact problem cannot be written as polynomial equations, so the team uses a close polynomial form. They classify the minimal versions of the problem, where just enough data are used, and find the number of solutions each can have. For several low-degree cases they build fast solvers. Tests on both made-up and real data check the solvers. The code is set to be shared in public.

Catalog record

CREATORPetr Hruby, Marc Pollefeys
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-17537

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Minimal Solvers for Full DoF Motion Estimation from Asynchronous Tracks Petr Hruby, Marc Pollefeys Abstract We address the problem of estimating both translational and angular velocity of a camera from asynchronous point tracks, a formulation relevant to rolling shutter and event cameras. Since the original problem is non-polynomial, we propose a polynomial approximation, classify the resulting minimal problems, and determine their algebraic degrees. Furthermore, we develop minimal solvers for several problems with low degrees and evaluate them on synthetic and real datasets. The code will be made publicly available. 1 Introduction Relative pose estimation is a fundamental task in computer vision, with applications in 3D reconstruction Schönberger and Frahm 2016 ; Pan et al. 2024 , Simultaneous Localization and Mapping (SLAM) DeTone et al. 2017 , camera self-calibration Cin et al. 2024 , multi-view stereo Furukawa et al. 2010 , and visual odometry Nistér et al. 2004 . Most minimal solvers for relative pose estimation assume a pinhole Nistér 2004 or radially distorted Hruby et al. 2023 camera model, where all image rows are captured simultaneously. However, this assumption does not hold for two important sensor types: rolling shutter cameras and event cameras . Rolling shutter cameras capture images row by row, while event cameras output a continuous stream of intensity changes. When these cameras move, the resulting image differs from that of the pinhole model. Among these two sensor modalities, rolling shutter cameras have received more attention. Numerous methods have been developed for absolute pose estimation Ait-Aider et al. 2006 ; Saurer et al. 2015 ; Magerand et al. 2012 ; Albl et al. 2020a ; Albl et al. 2016 ; Bai et al. 2022 ; Lao et al. 2018b ; Kukelova et al. 2020 ; Kukelova et al. 2018 ; Ait-Aider et al. 2007 ; Albl et al. 2015 ; Meingast et al. 2005 , image rectification Rengarajan et al. 2016 ; Purkait et al. 2017 ; Purkait and Zach 2018 ; Lao et al. 2018a ; Yan et al. 2023 ; Zhuang et al. 2019 ; Albl et al. 2020b ; Fan et al. 2022 ; Shang et al. 2023 ; Ait-Aider and Berry 2009 ; Zhuang and Tran 2020 ; Jia and Evans 2012 ; Ringaby and Forssén 2012 ; Forssén and Ringaby 2010 ; Qu et al. 2023a ; Qu et al. 2023b ; Vasu et al. 2018 ; Fan et al. 2021 ; Sun et al. 2016 ; Fan et al. 2023 ; Cao et al. 2022 , structure-from-motion Hedborg et al. 2011 ; Saurer et al. 2013 ; Klein and Murray 2009 , and bundle adjustment Hedborg et al. 2012 ; Nguyen and Lhuillier 2016 ; Liao et al. 2023 ; Saurer et al. 2016 ; Li et al. 2024 ; Zhang et al. 2024 ; Xu et al. 2024 ; Schubert et al. 2018 . Relative pose estimation for rolling shutter cameras, however, remains a particularly challenging problem. Existing methods typically treat the cameras as moving independently, requiring the estimation of 17 parameters in the bifocal case (5 for the relative pose and 6 for the motion of each camera). The full-DoF problem is still unsolved, and many approaches rely on simplifying assumptions: zero angular velocity Dai et al. 2016 ; Zhuang et al. 2017 , known angular velocity Lee and Yoon 2017b ; Lee et al. 2019 , or known vertical direction Hruby and Pollefeys 2025 . Other methods refine pinhole-based pose estimates with rolling shutter models Lee and Yoon 2017a , or target special cases such as planar scenes Lao and Ait-Aider 2021 or camera rigs Wang et al. 2020 . Recent works have surveyed minimal solvers for rolling shutter relative pose estimation using points and lines under special order-one models Hahn et al. 2024 , or purely from lines without a camera model Hruby and Pollefeys 2025 . For video-based rolling shutter imagery, Su et al. 2025 proposed a method assuming constant translational velocity over neighboring frames, with known angular velocity. Event camera pose estimation has been studied less extensively. Some works addressed motion estimation from collinear events, both under known Gao et al. 2023 ; Gao et al. 2024 and unknown Zhao et al. 2025 angular velocity. A recent method Su et al. 2025 proposed estimating translational motion from asynchronous point tracks assuming known angular velocity. In this paper, we extend the problem of Su et al. 2025 to the full-DoF setting, where both translational and angular velocities are unknown. We approximate the rotation with a polynomial model of degree K , classify minimal problems for K<6 , and construct minimal solvers for K=1 and K=2 . Furthermore, we provide a geometric explanation for why, in the pure translational case, a single essential matrix fits all data, which allows the classical five-point solver Nistér 2004 to be applied. Throughout the paper, we assume a fully calibrated camera without radial distortion. 2 Problem Formulation In this section, we formulate the problem of estimating the camera velocity and rotation velocity from asynchronous tracks, introduce polynomial approximations of this problem, and enumerate minimal cases of this problem. We consider a calibrated perspective camera whose center moves along a straight line with a constant translational velocity \mathbf{V}\in\mathbb{R}^{3} , while simultaneously rotating around a fixed axis \mathbf{a}\in\mathbb{R}^{3} (with \lVert\mathbf{a}\rVert=1 ) at a constant angular velocity \omega\in\mathbb{R} . Due to gauge freedom, we may, without loss of generality, assume that at the initial time t=0 the camera center is located at the origin, \mathbf{C}(0)=\mathbf{0}, and its orientation is given by the identity rotation matrix, \mathbf{R}(0)=\mathbf{I}. For a general time t\in\mathbb{R} , the camera center and orientation are given by \mathbf{C}(t)=t\cdot\mathbf{V}, (1) \mathbf{R}(t)=\mathbf{I}+\sin(t\cdot\omega)\,[\mathbf{a}]_{\times}+\big(1-\cos(t\cdot\omega)\big)[\mathbf{a}]_{\times}^{2}, (2) where [\mathbf{a}]_{\times} denotes the skew-symmetric matrix associated with the vector \mathbf{a} . Let \{\mathbf{X}_{i}\in\mathbb{R}^{3}\mid i=1,\dots,n\} be a set of n\in\mathbb{Z}_{+} 3D points. Each point \mathbf{X}_{i} is observed at m\in\mathbb{Z}_{+} distinct time instants \{t_{i,j}\in\mathbb{R}\mid j=1,\dots,m\} . The perspective projection \mathbf{p}_{i,j}\in\mathbb{P}^{2} of point \mathbf{X}_{i} at time t_{i,j} is given by \mathbf{p}_{i,j}\sim\mathbf{R}(t_{i,j})\cdot\big(\mathbf{X}_{i}-\mathbf{C}(t_{i,j})\big), (3) where \sim denotes equality up to a nonzero scale. In the considered problem formulation, the projections \mathbf{p}_{i,j} and the corresponding time instants t_{i,j} are assumed to be known. The task is to jointly estimate the motion parameters \mathbf{V} , \mathbf{a} , and \omega , as well as the 3D coordinates of the points \mathbf{X}_{i} , i=1,\dots,n , such that the projection model ( 3 ) holds for all observations. 2.1 Capture Times for Rolling Shutter Cameras Event cameras and rolling shutter cameras differ in the way they provide the capture times t_{i,j} corresponding to the projections \mathbf{p}_{i,j} . For event cameras, these times are measured explicitly along with the projections, so both \mathbf{p}_{i,j} and t_{i,j} are directly available. In contrast, for rolling shutter cameras, the capture times must be computed from the projection coordinates \mathbf{p}_{i,j}=[u_{i,j}\ v_{i,j}\ 1]^{\top} and the known camera parameters. Let f\in\mathbb{R} denote the focal length of the camera in pixels, h\in\mathbb{R} the image height in pixels, t_{r} the frame readout time, and t_{d} the inter-frame delay.Since a rolling shutter camera acquires the image row by row, the time required to read a single row is \frac{t_{r}}{h} . The row index of the projection \mathbf{p}_{i,j} can be determined from the y -coordinate of its uncalibrated projection \mathbf{K}\mathbf{p}_{i,j} , which

Read the complete text