Differential pose optimization in descriptor space - Combining Geometric and Photometric Methods for Motion Estimation Journal: Nuclear Physics B
Andreas L. Teigen, Annette Stahl, Rudolf Mester · 2026
Read Differential pose optimization in descriptor space - Combining Geometric and Photometric Methods for Motion Estimation Journal: Nuclear Physics B by Andreas L. Teigen, Annette Stahl, Rudolf Mester (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-14297.txt.
Introduction
Andreas L. Teigen and two co-authors test a 2026 arXiv hybrid for camera motion estimation. Two-frame relative pose optimization usually relies on photometric error or re-projection error, each tied to a feature paradigm with its own accuracy and robustness tradeoffs. The paper investigates a third route that merges both strengths.
Densely sampled geometric feature descriptors replace photometric error with a descriptor residual, bringing sub-pixel accuracy to differential photometric methods while keeping descriptor expressiveness. Experiments show accurate tracking from the hybrid, yet it ultimately does not beat re-projection-based pose optimization despite using more information. The authors analyze why the extra information fails to translate into gains.
Catalog record
| CREATOR | Andreas L. Teigen, Annette Stahl, Rudolf Mester |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-14297
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Differential pose optimization in descriptor space - Combining Geometric and Photometric Methods for Motion Estimation Journal: Nuclear Physics B Andreas L. Teigen, Annette Stahl, Rudolf Mester Abstract One of the fundamental problems in computer vision is the two-frame relative pose optimization problem. Primarily, two different kinds of error values are used: photometric error and re-projection error. The selection of error value is usually directly dependent on the selection of feature paradigm, photometric features, or geometric features. It is a trade-off between accuracy, robustness, and the possibility of loop closing. We investigate a third method that combines the strengths of both paradigms into a unified approach. Using densely sampled geometric feature descriptors, we replace the photometric error with a descriptor residual from a dense set of descriptors, thereby enabling the employment of sub-pixel accuracy in differential photometric methods, along with the expressiveness of the geometric feature descriptor. Experiments show that although the proposed strategy is an interesting approach that results in accurate tracking, it ultimately does not outperform pose optimization strategies based on re-projection error despite utilizing more information. We proceed to analyze the underlying reason for this discrepancy and present the hypothesis that the descriptor similarity metric is too slowly varying and does not necessarily correspond strictly to keypoint placement accuracy. 1 Introduction Figure 1: Descriptor self-similarity (Section 3.1.1 ). The images on the first row are all image patches centered on different ORB features. The images on the second row display an overlaid heatmap of descriptor similarity for each pixel, measured as the Hamming distance between the binary descriptors of the keypoint in the center of the patch and those of each pixel in the surrounding area. Warmer colors indicate a higher similarity. Optimizing the relative rotation \mathbf{R} and translation \mathbf{t} between two images is one of the fundamental problems of computer vision, and it has been well studied [ 6 ] . It typically involves establishing an initial feature correspondence set of landmarks visible in both images, applying a minimal solver to estimate an initial pose estimate [ 21 , 17 , 15 ] , usually implemented in conjunction with a Random Sampling Consensus (RANSAC) algorithm to remove outliers. To refine this pose and feature placement some sort of iterative, nonlinear optimization scheme is needed. The loss function for these schemes invariably reflects the penalty of shifting the initial corresponding point \mathbf{y}_{k} to some local point \mathbf{y}_{k}+\mathbf{v_{k}} on the epipolar line \mathbf{r}_{k} such that: (\mathbf{y}_{k}^{\top}+\mathbf{v}_{k}^{\top},\;1)\;\mathbf{r}_{k}=0 (1) Deciding how this loss is defined and, consequently, which point on the epipolar line is best, is one of the biggest design decisions in classical Visual Odometry (VO) and Simultaneous Localization And Mapping (SLAM) systems and has resulted in the competing paradigms of the geometric methods and the photometric methods. Given two images I_{A} and I_{B} and two corresponding points \mathbf{x}_{k} and \mathbf{y}_{k} , Geometric methods forgo using image information, and instead focus solely on minimizing the shortest geometric distance \mathbf{v}_{k} between the observed point \mathbf{y}_{k} and the epipolar line \mathbf{r}_{k} . \text{Loss}=H(\mathbf{v}_{k})=||\mathbf{v}_{k}|| (2) This loss function is a paraboloid with a circular cross-section centered on \mathbf{y}_{k} . Photometric methods , on the other hand, define the loss as the residual between the pixel intensities P(\mathbf{x}_{k}) in image I_{A} and P(\mathbf{y}_{k}+\mathbf{v}_{k}) in image I_{B} where \mathbf{v}_{k} is the delta position between the initial correspondence \mathbf{y} and the point on the point on the epipolar line with the lowest residual pixel intensity. \text{Loss}=H(P(\mathbf{x}_{k}),\tilde{P}(\mathbf{y}_{k}+\mathbf{v}_{k}))=|P(\mathbf{x}_{k})-\tilde{P}(\mathbf{y}_{k}+\mathbf{v}_{k})| (3) Here, \tilde{P} represents a local smoothing of the pixel intensities, allowing differential methods to be utilized. This loss function is largely dependent on the image content and can take on any shape. In this paper, we explore a third option, utilizing the descriptors of geometric methods, which are normally used only to establish initial point correspondence, in place of pixel values from photometric methods, to enable differential matching in descriptor space D(\cdot) . This enables the use of geometric point descriptors, which are robust to illumination changes, rotational changes, and significant viewpoint changes, while retaining the high pose and point positioning accuracy of photometric methods that result from actively utilizing image information during optimization. This results in the following loss function: \text{Loss}=D(Q(\mathbf{x}_{k}),\tilde{D}(\mathbf{y}_{k}+\mathbf{v}_{k}))=|D(\mathbf{x}_{k})-\tilde{D}(\mathbf{y}_{k}+\mathbf{v}_{k})| (4) This formulation is intentionally similar to Equation 3 , in that it can be used as a drop-in replacement in all photometric methods without requiring any additional algorithm modifications, making it universally applicable to these methods. We first demonstrate that it is possible to change the photometric residual H(P(\mathbf{x}_{k}),\tilde{P}(\mathbf{y}_{k}+\mathbf{v}_{k})) for the descriptor residual H(D(\mathbf{x}_{k}),\tilde{D}(\mathbf{y}_{k}+\mathbf{v}_{k})) by showing that the descriptor space is a smooth function that can be made locally differentiable (Section 3.1.1 & Figure 1 ). Then, we choose a differential photometric approach, but exchange the photometric residual with the descriptor residual, showing that it results in better results in more demanding conditions (Section 4 ). 2 Related work Estimating the relative pose between images has a long history, with roots dating back to the 1980s, during the advent of photometric methods using optical flow [ 14 , 28 , 5 , 12 ] and photometric feature extractors and matching [ 19 , 27 ] . It is widely used today, even in modern SLAM systems [ 10 , 11 ] , and is generally considered highly accurate under favorable conditions due to differential point placement methods, providing sub-pixel accuracy. Geometric features [ 18 , 3 , 24 ] were introduced later as a way to increase robustness for greater image differences in scale, rotation, lighting, and viewing angles. This opened up for the estimation of the relative pose of a much larger array of images, which led to the introduction of the photo tourism methods, which use images from random collections of images from the Web [ 1 , 30 , 29 ] . Both geometric and photometric methods are still used actively today, maybe most prominently in two of the most successful VO/SLAM systems: ORB-SLAM and DSO [ 20 , 10 ] . ORB-SLAM is the more citetd of the two, one reason for which is its robustness to various scenes and the lack of camera requirements. DSO has more accurate camera tracking, but it requires significantly more from the camera, including knowing the shutter speeds of all images and lens vignetting. These differences largely stem from feature selection and the accompanying matching and frame-to-frame pose optimization strategies. Through this work, we investigate the possibility of developing a third alternative that utilizes the robust feature descriptors of the geometric methods to generate an alternative residual to the pixel intensity residual of the photometric method. Potentially allowing higher accuracies with greater robustness and more lenient camera requirements. 2.1 Dense descriptors Swapping out the pixel intensity residual with a descriptor residual requires the computation of a locally dense set of descriptors. Other works that have experimented with dense descriptors include [ 32 ] , which produces a grid with regularly spaced descriptors so that every pixel in the image is covered within the radius of at least one descriptor. They argue that the spatial relations between these descriptors could be a helpful aid in tasks like object detection. Other more recent methods introduce the use of convolutional neural networks [ 22 ] to generate descriptors for