Transferring between Sparse and Dense Matching via Probabilistic Reweighting PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE
Ya Fan, Rongling Lang · 2025
Read Transferring between Sparse and Dense Matching via Probabilistic Reweighting PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Ya Fan, Rongling Lang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01472.txt.
Introduction
Ya Fan and Rongling Lang present a way to move image matching systems between sparse and dense modes, in a 2025 paper. Image matching pairs points across photos for tasks like mapping and robot vision. Detector-based matchers work only in sparse regimes. Detector-free matchers need dense input. Each family is stuck in its own range.
Their probabilistic reweighting adapts transformer matchers without retraining them. Detection probabilities adjust attention weights and match scores. The authors prove the reweighted net matches the detector-based limit. They also give a training and pruning pipeline for dense matchers. Well-known systems gain new range from the trick. Vision engineers can reuse one matcher across far more scenes.
Catalog record
| CREATOR | Ya Fan, Rongling Lang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01472
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transferring between Sparse and Dense Matching via Probabilistic Reweighting PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE Ya Fan, Rongling Lang Abstract Detector-based and detector-free matchers are only applicable within their respective sparsity ranges. To improve the adaptability of existing matchers, this paper introduces a novel probabilistic reweighting method. Our method is applicable to Transformer-based matching networks and adapts them to different sparsity levels without altering network parameters. The reweighting approach adjusts attention weights and matching scores through feature detection probabilities. We prove that the reweighted matching network is asymptotically equivalent to the detector-based matching network. Furthermore, we propose a sparse training and pruning pipeline for detector-free networks based on reweighting. Reweighted versions of SuperGlue, LightGlue, and LoFTR are implemented and evaluated on matching tasks including relative pose estimation and visual localization across different levels of sparsity. Experiments show that the reweighted dense matching improves the pose accuracy of detector-based matchers in low-texture indoor scenes. The performance of reweighted sparse LoFTR is comparable to detector-based matchers, demonstrating flexibility in balancing accuracy and computational complexity. I Introduction Feature matching is a fundamental technology for many computer vision applications such as structure from motion (SFM) [ 1 , 2 ] , simultaneous localization and mapping (SLAM) [ 3 ] and re-localization [ 4 ] . Recent advancements in learning-based feature matching have led to the emergence of two paradigms: detector-based matching [ 5 , 6 ] and detector-free matching [ 7 , 8 ] . For a pair of images to be matched, detector-based methods first utilize a detector [ 9 , 10 ] to extract two sets of feature points from the images, and then compute point-to-point correspondences within these sets. On the other hand, detector-free methods compute feature maps of the images and directly use all features for matching. The two matching paradigms differ in the sparsity of features and exhibit complementarity. SuperPoint [ 9 ] and SuperGlue [ 5 ] represent a classic detector-based matching pipeline, where SuperPoint extracts feature points, and SuperGlue aggregates these features to establish correspondences through optimal transport. Since SuperGlue typically takes sparse feature points as input, its matching process is consequently termed sparse matching. Sparse matching is lightweight but prone to failures under significant variations in factors such as texture, viewpoint, illumination, and motion blur. With advancements in computational power, detector-free methods have gained increasing popularity. As a representative detector-free approach, LoFTR [ 11 ] first extracts feature maps from images using a backbone network, then directly aggregates the features through a coarse-to-fine matching network. Since the coarse matching stage utilizes all features from the coarse feature maps, this stage is termed dense matching. Dense matching can leverage more information from the images, thereby being more resilient to variations that are difficult for sparse matching, albeit at a higher computational cost. In practical applications, where diverse image pairs pose varying degrees of matching difficulty, combining the two paradigms and dynamically adjusting the sparsity of matching according to the difficulty helps to balance precision and efficiency. However, existing matching networks are only applicable within their respective sparsity ranges. Detector-based methods are exclusively trained on sparse feature points. When the feature sparsity exceeds the sparsity range during training, the network encounters generalization issues, making it difficult to directly match the features. Among these methods, LightGlue [ 6 ] incorporates point pruning and depth pruning mechanisms to balance accuracy and efficiency, yet its feature points remain constrained within the sparse domain. Conversely, detector-free networks are fundamentally designed to operate within dense feature maps, and they lack effective mechanisms to enable sparse feature matching while preserving their original dense matching capabilities. It is noteworthy that the matching paradigms employ the attention mechanism [ 12 ] , which is inherently able to process an arbitrary number of features. This implies that the same matcher can modulate its matching sparsity simply by varying the quantity of input features. However, due to the different distributions of features in sparse and dense matching, altering the number of features matched by the network causes generalization issues. As illustrated in Figure 1 (a) and (b), SuperGlue struggles to generate a sufficient number of matches from sparse feature points for challenging image pairs. Expanding the feature points to dense representation and directly applying SuperGlue for matching these features would conversely lead to degraded matching performance or even failure to establish valid correspondences. To adapt a pretrained attention network to a matching scenario where feature sparsity exceeds the training sparsity range, we propose a reweighting method in which the attention weights and matching scores are adjusted by feature detection probabilities. For detector-based networks, we first analyze the distribution shift and length generalization issues between sparse and dense matching from the perspective of random sequences. Subsequently, we propose to eliminate the distribution shift by matching random sequences whose marginal distributions are equal to the detection probability distribution. We further prove that the reweighted matching constitutes the asymptotic limit of detector-based matching on random sequences. This theoretical result effectively transforms the matching of sufficiently long random sequences into a deterministic matching algorithm, where the sparsity of the matching is determined by the support of the probability distribution. Based on our reweighting method, existing detector-based networks are transferred to a dense matching paradigm while accounting for the keypoint distribution estimated by the detector, as shown in Figure 1 (c). For detector-free networks, we propose a sparse training and pruning pipeline, optimizing the feature probabilities to allow them to perform sparse reweighted matching. During training, a score head is trained to estimate sparse probability scores that maximize the reweighted matching performance. At inference time, pruning and reweighting are performed based on the probability scores, without altering the backbone or matching network parameters. This design preserves the original dense matching capability inherent in detector-free methods while simultaneously enabling efficient sparse matching functionality. Moreover, our reweighted attention function is applicable to a wide range of Transformers. We reweight the mainstream SuperGlue, LightGlue, and LoFTR and evaluate them on matching tasks including relative pose estimation and visual localization at different sparsity levels. Fig. 1: Transferring detector-based matchers to dense matching. The original detector-based matcher struggles to match the challenging image pair with either sparse or dense feature points. Given the keypoint distribution, our method adjusts attention weights and matching scores at inference time, and better matches dense feature points. The results are from the original and reweighted SuperGlue [ 5 ] using SuperPoint [ 9 ] features. The main contributions of our work are as follows: • We propose a novel probabilistic reweighting method that enables sparsity transfer for Transformer-based matching networks without parameter modification, and theoretically prove that it constitutes the asymptotic limit of detector-based matching on long random sequences; • By implementing reweighted detector-based matchers including SuperGlue and LightGlue, we demonstrate superior indoor performance of reweighted dense matching over original sparse paradigms in relative pose estimation and visual localization tasks; • We present a reweighting-based sparse training and pruning pipeline for detector-free matchers, which enables sparse matching and preserves their dense performance, achieving comparable accuracy to sparse baselines while offering efficiency-accuracy trade-offs complementary to LightGlue. II Related Work II-A Detector-based Matching Classical detector-based matching methods focus on designing keypoints with correspondences obtained through