Dense Cross-Scale Image Alignment With Fully Spatial Correlation and Just Noticeable Difference Guidance

Jinkun You, Jiaxue Li, Jie Zhang, Yicong Zhou · 2025

Read Dense Cross-Scale Image Alignment With Fully Spatial Correlation and Just Noticeable Difference Guidance by Jinkun You, Jiaxue Li, Jie Zhang, Yicong Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09028.txt.

Introduction

Jinkun You and three co-authors align images densely across scales without any supervision, a registration task where two views of a scene must be matched pixel by pixel. Existing unsupervised methods are either inaccurate or expensive, and the model proposed here attacks both problems at once.

Correlations between features at different scales are exploited to make the alignment easier, and the number of scales can be dialed up or down to trade accuracy against speed. A fully spatial correlation module sharpens accuracy while keeping computation low. The human visual system lends a trick: a just noticeable difference model steers effort toward regions where the eye can actually spot misalignment, erasing the errors viewers would notice. Quantitative and qualitative experiments put the method ahead of the state-of-the-art approaches compared.

Catalog record

CREATORJinkun You, Jiaxue Li, Jie Zhang, Yicong Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-09028

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dense Cross-Scale Image Alignment With Fully Spatial Correlation and Just Noticeable Difference Guidance Jinkun You, Jiaxue Li, Jie Zhang, Yicong Zhou Abstract Existing unsupervised image alignment methods exhibit limited accuracy and high computational complexity. To address these challenges, we propose a dense cross-scale image alignment model. It takes into account the correlations between cross-scale features to decrease the alignment difficulty. Our model supports flexible trade-offs between accuracy and efficiency by adjusting the number of scales utilized. Additionally, we introduce a fully spatial correlation module to further improve accuracy while maintaining low computational costs. We incorporate the just noticeable difference to encourage our model to focus on image regions more sensitive to distortions, eliminating noticeable alignment errors. Extensive quantitative and qualitative experiments demonstrate that our method surpasses state-of-the-art approaches. Introduction Image alignment has received increasing attention in recent years ( 9 ; 10 ; 14 ) . This task involves spatially registering images of the same scene captured from different viewpoints, aiming to maximize the consistency of overlapping regions. It forms the foundation for various applications, including deep camera calibration ( 17 ) , surround-view imaging systems ( 11 ) , and multimodal image processing ( 32 ; 29 ) . This growing importance has driven efforts to optimize alignment accuracy and efficiency for practical deployment. Traditional alignment methods rely on hand-crafted features to identify matched points between input images ( 12 ; 16 ; 19 ) . They establish transformations based on the point correspondences. Early approaches employ a single homography to achieve satisfactory results when the images share a common plane or differ only in camera rotation ( 13 ) . However, their performance degrades under parallax conditions. To overcome this limitation, non-linear local deformations were introduced to offer greater flexibility by predicting a mesh instead of a single homography. The thin-plate spline transformation has been adopted to enhance the alignment performance. Higher accuracy leads to greater content similarity in the overlapping regions of the aligned images. Nevertheless, focusing solely on accuracy can lead to projective distortion in non-overlapping areas. It causes straight lines to bend and objects to be over-stretched. Recent advances have incorporated geometric constraints to better preserve object shapes ( 7 ) . While traditional methods yield promising results, they struggle with low-texture and large-parallax cases ( 25 ) . Deep networks have been utilized to address these limitations since they have powerful feature extraction and representation abilities ( 3 ; 30 ; 26 ) . They extract feature maps to generate a correlation tensor that encodes the matching relationships between images. This tensor is used to predict position offsets between image pixels to do image warping effectively ( 4 ) . Unsupervised methods are preferred recently since they do not need the ground truth offsets to train deep networks and are more suitable for real-world image pairs ( 22 ; 6 ) . Early unsupervised methods estimate a single homography by maximizing content similarity in the overlapping regions of aligned images ( 21 ) . To better handle parallax problems, the non-linear local alignment strategy is integrated in a coarse-to-fine manner ( 24 ) . Global offsets are first estimated from global features to maintain global consistency. Next, local offsets are computed from fine features for precise alignment. The global and local offsets are integrated to obtain the mesh used for image warping. Shape preservation is enforced by constraining the magnitudes and directions of the mesh. Existing unsupervised methods have achieved impressive alignment accuracy, but they still face three key limitations. First, they do not explicitly account for scale variations. The apparent size of the same object can differ across images due to parallax, complicating the alignment process. Current approaches typically predict the transformation parameters using features of identical spatial dimensions. These intra-scale features cannot effectively capture scale discrepancies. Second, widely used correlation calculation methods are either ineffective or inefficient. The correlation layer is efficient but destroys spatial information ( 27 ) . The contextual correlation layer and cost volume capture long-range correlation information to achieve better performance at the cost of high FLOPs and runtime, respectively ( 24 ) . Their advantages diminish when other modules also capture context information. Third, prevailing methods treat all image pixels equally during optimization. Not all pixels exhibit the same tolerance to distortion ( 8 ) . Alignment errors are more noticeable to the human visual system when they occur in pixels that are more sensitive to distortion. As a result, the content similarity is more easily reduced for the overlapping areas of the aligned images. To tackle these challenges, we introduce an unsupervised image alignment method with the following contributions: • We design a dense cross-scale image alignment model that leverages cross-scale features to enhance alignment accuracy. It allows for flexible trade-offs between effectiveness and efficiency based on user requirements. • We introduce a fully spatial correlation module that flexibly utilizes the spatial information of the input features to further increase alignment accuracy. It can also maintain low computational costs. • The just noticeable distortion estimation is used to guide image alignment. It helps an alignment model prioritize pixels with lower distortion tolerance, thus further increasing alignment accuracy. • Extensive ablation studies and comparisons are conducted to validate the effectiveness and efficiency of the proposed method. Proposed Method Figure 1: Illustration of the proposed alignment network. (a) The overall framework. (b) The proposed cross-scale regression. (c) The proposed fully spatial correlation. Our method uses a coarse-to-fine strategy to estimate the offsets of a mesh. The dense cross-scale regression module integrates the cross-scale information into the local offsets. The fully spatial correlation module utilizes the spatial information of both input features. The dot product is performed between the dark blue and dark red feature vectors to obtain the element in dark gray. Overall Structure The non-linear local deformation is employed to warp the target image I_{tar} for the alignment with the reference image I_{ref} . It is implemented via a mesh M^{f}\in\mathbb{R}^{U\times V\times 2} , which has a spatial size of U\times V and contains the horizontal and vertical positions. This alignment can be formulated as the optimization problem below: {M}^{f}_{opt}=\min_{{M}^{f}}\mathrm{P}({I}_{ref},\mathrm{Warp}({I}_{tar},{M}^{f})), (1) where \mathrm{P}(\cdot,\cdot) evaluates the alignment error and \mathrm{Warp}(\cdot,\cdot) performs differentiable image warping. The dense cross-scale alignment network is introduced to address the optimization problem more effectively. Instead of regressing the mesh directly, it predicts position offsets. As illustrated in Figure 1 (a) , the network employs a coarse-to-fine strategy. Specifically, it extracts fine and coarse feature maps from both reference and target images to regress global offsets {O}_{g}\in\mathbb{R}^{4\times 2} and local offsets {O}_{l}\in\mathbb{R}^{U\times V\times 2} , respectively. The global offsets O_{g} are used to apply a homography transformation on a regular mesh M\in\mathbb{R}^{U\times V\times 2} to ensure global consistency. Subsequently, O_{l} is added to the transformation result to generate M^{f} for fine-grained warping. Our method leverages cross-scale information from fine feature maps to enhance local deformation. A fully spatial correlation module is designed to assist in predicting both global and local offsets. Additionally, just noticeable difference (JND) guidance is incorporated to reduce the alignment errors in visually sensitive regions. Dense Cross-Scale Regression Large parallax results in scale variations for the same object across the reference and

Read the complete text