Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification
Junyan Lin, Feng Gao, Member, IEEE, Lin Qi, Junyu Dong, Member, IEEE · 2025
Read Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification by Junyan Lin, Feng Gao, Member, IEEE, Lin Qi, Junyu Dong, Member, IEEE (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06945.txt.
Introduction
Junyan Lin and co-workers present a network that classifies land cover by fusing hyperspectral images with LiDAR data, in a 2025 paper. The two sources carry complementary detail, but older fusion designs rely on hand-built feature extractors. Their network instead routes features dynamically, choosing paths based on the data itself.
Three interaction blocks mix the sources across space, color channels, and discriminative features, with gates that select the best route per layer. Bilinear attention strengthens the mixing. On three public benchmark datasets the method beat leading alternatives. The authors credit the data-dependent routing for the gain and promise public code.
Catalog record
| CREATOR | Junyan Lin, Feng Gao, Member, IEEE, Lin Qi, Junyu Dong, Member, IEEE |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-06945
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification Thanks: This work was supported in part by the National Key Research and Development Program of China under Grant 2022ZD0117202, in part by the Natural Science Foundation of Shandong Province under Grant ZR2024MF020, and in part by the Natural Science Foundation of Qingdao under Grant 23-2-1-222-ZYYD-JCH. ( Corresponding author: Feng Gao ) Junyan Lin, Feng Gao, and Junyu Dong are with the School of Computer Science and Technology, Ocean University of China, Qingdao 266100, China. Qian Du is with the Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA. Xinbo Gao is with the Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing 400065, China. Junyan Lin, Feng Gao, Member, IEEE, Lin Qi, Junyu Dong, Member, IEEE Abstract Hyperspectral image (HSI) and LiDAR data joint classification is a challenging task. Existing multi-source remote sensing data classification methods often rely on human-designed frameworks for feature extraction, which heavily depend on expert knowledge. To address these limitations, we propose a novel Dynamic Cross-Modal Feature Interaction Network (DCMNet), the first framework leveraging a dynamic routing mechanism for HSI and LiDAR classification. Specifically, our approach introduces three feature interaction blocks: Bilinear Spatial Attention Block (BSAB), Bilinear Channel Attention Block (BCAB), and Integration Convolutional Block (ICB). These blocks are designed to effectively enhance spatial, spectral, and discriminative feature interactions. A multi-layer routing space with routing gates is designed to determine optimal computational paths, enabling data-dependent feature fusion. Additionally, bilinear attention mechanisms are employed to enhance feature interactions in spatial and channel representations. Extensive experiments on three public HSI and LiDAR datasets demonstrate the superiority of DCMNet over state-of-the-art methods. Our code will be available at https://github.com/oucailab/DCMNet . I Introduction WITH the development of spectroscopy and imaging technologies, satellite sensors can capture a large number of high-resolution images that can be applied to environmental monitoring [ 1 ] , vegetation mapping [ 2 ] , coastal wetland mapping [ 3 ] , etc. Among these images, hyperspectral images (HSIs) contain hundreds of contiguous spectral bands, which facilitates fine discrimination between different ground objects on the Earth’s surface. Therefore, HSI classification has been the most active part of the research in the remote sensing society [ 4 ] . The spectral information in HSIs alone is often insufficient to accurately classify ground objects due to their complex composition. Specifically, the spectral information of ground objects is often complex, and thus leads to the phenomenon of “same spectrum corresponds to multiple ground coverings” [ 5 ] . The light detection and ranging (LiDAR) data are obtained by actively emitting light at the ground coverings and can create high-resolution models of ground elevation [ 6 ] . Therefore, LiDAR data provides complementary information for HSI in ground covering classification. The combination of HSI and LiDAR data takes advantage of the complementary multisource information, and improves the accuracy of ground object classification [ 7 ] [ 8 ] . This multisource fusion paradigm has been effectively adopted in other fields [ 9 ] [ 10 ] . Fig. 1: Comparisons of the traditional method with the proposed DCMNet. (a) Existing multi-source feature fusion method. Their static framework lacks the adaptability to ground objects with semantic diversity. (b) The proposed DCMNet. Dynamic routing mechanism is introduced to learn data-dependent feature extraction paths. Based on this concept, numerous studies have explored joint HSI and LiDAR classification, utilizing different fusion strategies. They can be categorized into two groups: decision-level fusion methods [ 11 ] [ 12 ] [ 13 ] and feature-level fusion methods [ 14 ] [ 15 ] [ 16 ] . Decision-level fusion methods commonly use several classifiers individually. Afterwards, graph fusion [ 17 ] and active learning [ 18 ] are used to integrate the classification results. In feature-level fusion methods, deep learning-based techniques have attracted wide attention recently. Self-guided attention [ 19 ] , graph-enhanced network [ 20 ] , cross-attention network [ 21 ] , and flexible-Mixup [ 22 ] have achieved great success in learning and fusing multi-source features. Although these pioneer works have made significant strides in addressing the challenges of HSI and LiDAR data joint classification, they still face several critical limitations, particularly when dealing with data heterogeneity, high dimensionality, and the integration of spatial and spectral information. Data heterogeneity arises from the fundamental differences between HSI and LiDAR data, as HSI captures detailed spectral information across numerous bands, while LiDAR provides precise spatial and elevation data. These distinct characteristics make it difficult for traditional methods to effectively fuse and exploit complementary information from both modalities. Additionally, the high dimensionality of HSI data presents computational challenges, especially when combined with the complexity of integrating spatial information from LiDAR. Current methods often struggle to fully utilize this wealth of information, leading to suboptimal classification results. Furthermore, existing methods frequently fail to properly integrate spatial and spectral information, limiting their ability to model the intricate relationships between spectral signatures and spatial structures. Despite numerous efforts have been proposed to tackle these challenges, most methods still suffer from two critical problems: 1) Static Framework for Feature Extraction. Existing methods typically use rigid, human-designed frameworks to extract multi-source features, where the computation path is static and fixed. Regardless of the complexity or characteristics of the input data, all features are processed through the same path. This approach fails to account for the varying semantic complexities of different ground objects, causing interference between regions with different levels of detail. As a result, the performance of static frameworks is limited in capturing the diverse nature of HSI and LiDAR data. A dynamic routing mechanism, which adapts the computational path based on the complexity of the input data, is essential to address this shortcoming. 2) Hand-crafted cross-modal fusion patterns. Current feature fusion patterns for HSI and LiDAR classification are mainly hand-crafted and highly dependent on expert knowledge. While these manually designed patterns have achieved success, they may overlook optimal cross-modal interactions, limiting the potential for more effective feature fusion. Furthermore, relying on fixed patterns constrains the ability to generalize across diverse datasets and environments, reducing flexibility and scalability. To overcome the above-mentioned issues, we propose D ynamic C ross- M odal feature interaction Net work ( DCMNet ), which is the first multi-source remote sensing image classification framework with dynamic routing mechanism. As illustrated in Fig. 1 , DCMNet introduces three novel feature interaction blocks: the Bilinear Spatial Attention Block (BSAB), the Bilinear Channel Attention Block (BCAB), and the Integration Convolutional Block (ICB). Each block is designed to capture different aspects of feature interactions between HSI and LiDAR data. The BSAB enhances spatial feature correlations, while the BCAB explicitly models relationships between spectral information from HSI and structural information from LiDAR. Additionally, the ICB is designed to efficiently extract discriminative features for ‘easy’ samples by bypassing complex operations. Importantly, our dynamic routing mechanism determines the optimal computational path for each block based on the input data, enabling the model to adaptively exploit the complementary features of HSI and LiDAR data. By conducting experiments on three HSI and LiDAR datasets, we validate that our proposed DCMNet is superior to other state-of-the-art competitors on