EpiMask: Leveraging Epipolar Distance Based Masks in Cross-Attention for Satellite Image Matching

Rahul Deshmukh, Aditya Chauhan, Avinash Kak, [email protected], [email protected] · 2026

Read EpiMask: Leveraging Epipolar Distance Based Masks in Cross-Attention for Satellite Image Matching by Rahul Deshmukh, Aditya Chauhan, Avinash Kak, [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21463.txt.

Introduction

Deshmukh and co-authors adapt image matching networks to satellite imagery. Networks trained on ground photos assume a pinhole camera, but satellites record the ground one line at a time, so matches suffer. Their EpiMask network adds patch-wise affine approximations of the satellite camera geometry. An epipolar distance mask limits cross-attention to plausible regions. They also fine-tune a pretrained encoder for stronger features. On the SatDepth dataset, matching accuracy improved by up to 30 percent over retrained ground-based models.

Catalog record

CREATORRahul Deshmukh, Aditya Chauhan, Avinash Kak, [email protected], [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58945
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21463

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EpiMask: Leveraging Epipolar Distance Based Masks in Cross-Attention for Satellite Image Matching Rahul Deshmukh, Aditya Chauhan, Avinash Kak, [email protected], [email protected] Abstract The deep-learning based image matching networks can now handle significantly larger variations in viewpoints and illuminations while providing matched pairs of pixels with sub-pixel precision. These networks have been trained with ground-based image datasets and, implicitly, their performance is optimized for the pinhole camera geometry. Consequently, you get suboptimal performance when such networks are used to match satellite images since those images are synthesized as a moving satellite camera records one line at a time of the points on the ground. In this paper, we present EpiMask, a semi-dense image matching network for satellite images that (1) Incorporates patch-wise affine approximations to the camera modeling geometry; (2) Uses an epipolar distance-based attention mask to restrict cross-attention to geometrically plausible regions; and (3) That fine-tunes a foundational pretrained image encoder for robust feature extraction. Experiments on the SatDepth dataset demonstrate up to 30% improvement in matching accuracy compared to re-trained ground-based models. The code will be made available through epimask.git 1 Introduction Image matching is the task of finding pixels in two images that correspond to the same physical point in a 3D scene. It serves as a fundamental component of 3D reconstruction pipelines, enabling geometric alignment and 3D reconstruction from multiple views. Depending on the application, the matching objective can range from identifying a sparse set of correspondences for image alignment or a dense set of pixel-level matches for stereo depth reconstruction. Both paradigms play a critical role in downstream applications such as structure-from-motion (SfM), multi-view stereo, and SLAM. Figure 1 : Shown are 40 randomly selected image matches for EpiMask (top row) and SatLoFTR (bottom row). EpiMask detects more accurate correspondences, as evidenced by a denser and more detailed point cloud on the right. The past decade has witnessed an explosion of deep-learning based approaches that were shown to outperform the classical methods, starting from simple learnable adaptations of classical methods 43 ; 2 ; 39 ; 11 to sophisticated end-to-end trainable models for detecting sparse 33 ; 22 , semi-dense 37 ; 19 ; 42 ; 6 , and dense 13 ; 12 matches. More recently, models such as RoMa 13 , LightGlue 22 , and GIM 35 leverage techniques like large-scale pretraining, using features from foundational models 28 , and fine-tuning to achieve even higher performance. However, most of these networks are trained with ground-based imagery and do not explicitly account for the geometric or photometric characteristics unique to satellite images. To address challenges specific to satellites imagery, several satellite image matching datasets 9 ; 18 ; 29 have been proposed, enabling benchmarking and adaptation of ground-based image matching models to satellite images. Yet, these efforts typically focus on benchmarking 36 or re-training existing architectures 9 on satellite matching datasets rather than designing models that exploit the rich geometric metadata available with satellite images. A notable gap in the literature is that none of the ground-based image matching models has been tailored to leverage the satellite camera geometry that is always present in the metadata associated with the satellite images. A fundamental reason for why the satellite images are more complex compared to typical ground-based images is that, for the case of satellites, an image is synthesized one line at a time with a linear sensor array as the satellite is in motion. A direct manifestation of this complexity is the fact that the epipolar lines for satellite images tend to be curved 8 as illustrated in Fig. 2 , whereas for ground-based images (with pinhole cameras), they tend to be straight lines. For a given pair of images ( I_{L},I_{R} ) for the purpose of matching, the epipolar lines being curved implies that, for any given pixel x_{L} in the left image ( I_{L} ), its corresponding pixel x_{R} in the right image ( I_{R} ) will lie in a region whose shape would be complicated and depend on the location of x_{L} . By contrast, with pinhole cameras used for ground-based imagery, the epipolar lines are straight. Consequently, the search region for a corresponding pixel is bounded by straight lines, resulting in simpler downstream logic for image matching. Figure 2 : Epipolar geometry of satellite cameras, also known as pushbroom cameras At this point, a reader may naturally ask: “Why not simply train existing state-of-the-art image matching networks, originally developed for ground-based imagery, on satellite images?” After all, these networks are agnostic to the camera imaging geometry. Their implicit dependence on the imaging geometry is a consequence of the datasets on which they are trained. Consider, for instance, the LoFTR 37 network, which achieves remarkable performance when trained on the MegaDepth 20 dataset comprising images captured with standard pinhole cameras. One could argue that the pinhole imaging geometry is effectively “baked” into the learned weights. It is therefore reasonable to ask whether re-training such a model on satellite imagery would yield equally strong performance. Our work directly investigates the question posed above and arrives at a surprising conclusion. We show that while a network like LoFTR, after it is trained with satellite images, performs reasonably well for image matching tasks, the performance remains suboptimal . More specifically, we show that incorporating architectural modifications that account for the unique characteristics of satellite imagery, particularly the nonlinear epipolar geometry, yields up to a 30% improvement in matching accuracy compared to the best that can be achieved with the original architecture. The EpiMask architecture proposed in this paper extends LoFTR by integrating with it a new cross-attention mechanism that directly addresses the unique epipolar geometry of satellite imagery. The cross-attention layers are trained using patch-based linear approximations (modeled through affine cameras) to capture the non-linear nature of satellite epipolar geometry. Additionally, an epipolar distance-based attention mask constrains the cross-attention to geometrically plausible regions. Our model also fine-tunes a foundational pretrained image encoder to ensure robust feature extraction. Together, these satellite-specific enhancements to LoFTR yield a 30% improvement in image matching performance compared to a re-trained LoFTR baseline. For a fair comparison, both EpiMask and LoFTR were trained on the SatDepth dataset 9 , designed specifically for satellite imagery. 1 1 1 The name EpiMask denotes the use of an epipolar mask (“EpiMask”) in matching As an illustration of the power of EpiMask, the top-left image pair in Fig. 1 shows 40 randomly selected matches detected by our network. For comparison, the bottom pair shows the same number of matches from satLoFTR trained on the same dataset. The superior accuracy of matches detected by EpiMask is evident from the higher-quality point cloud shown on the right. A detailed quantitative comparison with competing methods is presented later in the paper. Our main contributions are: (i) A geometry-aware attention framework for satellite image matching; (ii) Incorporation of a foundational pretrained encoder for improved generalization; (iii) State-of-the-art performance on the SatDepth dataset with up to 30% improvement; and (iv) Extensive ablation studies validating key design choices. 2 Related Work Image Matching Models : Image matching for ground-based images is an active field of research, so much so that it can be daunting to keep track of all the developments. Broadly speaking,

Read the complete text