Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification
Chuanzheng Gong, Feng Gao, Member, IEEE, Junyan Lin, Junyu Dong, Member, IEEE · 2026
Read Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification by Chuanzheng Gong, Feng Gao, Member, IEEE, Junyan Lin, Junyu Dong, Member, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27323.txt.
Introduction
Gong and four colleagues propose RSCNet, a spectral correlation network for multi-source remote sensing image sorting. It fuses hyperspectral imagery with SAR and LiDAR data. Hyperspectral and SAR or LiDAR data offer matching spectral and structural information for land-cover sorting. Yet fusion faces spectral overlap in high-dimensional hyperspectral data and mixed traits across sources. RSCNet addresses those challenges through spectral selection. A Key Band Selection Module picks spectral bands on the fly from the hyperspectral data under cross-source guidance, easing overlap and cutting information loss.
Catalog record
| CREATOR | Chuanzheng Gong, Feng Gao, Member, IEEE, Junyan Lin, Junyu Dong, Member, IEEE |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59031 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27323
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification Thanks: This work was supported in part by the National Science and Technology Major Project under Grant 2022ZD0117202, in part by the Natural Science Foundation of China under Grant 42406192, the Key R&D Program of Shandong Province under Grant 2025CXPT185, in part by the Natural Science Foundation of Shandong Province under Grant ZR2024MF020. ( Corresponding author: Feng Gao .) Chuanzheng Gong, Feng Gao, and Junyu Dong are with the State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China. Junyan Lin is with the Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China. Qian Du is with the Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA. Chuanzheng Gong, Feng Gao, Member, IEEE, Junyan Lin, Junyu Dong, Member, IEEE Abstract Hyperspectral image (HSI) and SAR/LiDAR data offer complementary spectral and structural information for land-cover classification. However, their effective fusion remains challenging due to two major limitations: The spectral redundancy in high-dimensional HSI and the heterogeneous characteristics between multi-source data. To this end, we propose Representative Spectral Correlation Network (RSCNet), a novel multi-source image classification framework specifically designed to address the above challenges through spectral selection and adaptive interaction. The network incorporates two key components: (1) Key Band Selection Module (KBSM) that adaptively selects task-relevant spectral bands from the original HSI under cross-source guidance, thereby alleviating redundancy and mitigating information loss from conventional PCA-based spectral reduction. Moreover, the learned band subset exhibits highly discriminative spectral structures that align with discriminative semantic cues, promoting compact yet expressive representations. (2) Cross-source Adaptive Fusion Module (CAFM) that performs cross-source attention weighting and local–global contextual refinement to enhance cross-source feature interaction. Experiments on three public benchmark datasets demonstrate that our RSCNet achieves superior performance compared with state-of-the-art methods, while maintaining substantially lower computational complexity. Our codes are publicly available at https://github.com/oucailab/RSCNet . I Introduction Recent years have witnessed the rapid development of satellite remote sensing sensors, leading to an increment in the availability of multi-source remote sensing data. These multi-source data provides strong support for various fields as land cover classification [ 51 ] [ 60 ] , environmental monitoring [ 18 ] , disaster prevention [ 37 ] , natural resource management [ 39 ] , hydrological monitoring [ 35 ] [ 28 ] , and urban planning [ 29 ] [ 46 ] . Among these applications, land cover classification is a fundamental and important task [ 14 ] [ 45 ] . Fig. 1: Comparison between existing multi-source data classification models and the proposed RSCNet. Existing methods apply task-agnostic PCA-based spectral reduction to HSI and then perform feature interaction with SAR/LiDAR, which may discard discriminative spectral information. In contrast, our RSCNet integrates progressive key band selection with cross-source feature interaction, allowing spectral bands to be dynamically refined under multi-source guidance, thereby better preserving discriminative information and improving classification performance. Hyperspectral image (HSI) play a key role in land cover classification, providing rich spectral information for different ground objects. HSI offers continuous spectral resolution, enabling detection of subtle features and identification of intrinsic properties for accurate land cover classification [ 12 , 24 , 9 ] . To further exploit this powerful spectral resolution, recent advances in hyperspectral analysis have delved into fine-grained feature representations [ 64 ] [ 63 ] . However, HSI data are sensitive to illumination, atmospheric effects, and cloud cover, leading to challenges in complex scenes. To address these issues, Synthetic Aperture Radar (SAR) and Light Detection and Ranging (LiDAR) data are often used as complementary data. SAR provides elevation, backscatter characteristics, and polarization signatures, along with all-weather imaging capabilities, enabling penetration of clouds and fog [ 34 , 57 , 33 ] . LiDAR data can provide detailed 3D structure of the terrain at high spatial resolution. However, SAR/LiDAR data lacks sufficient spectral information, limiting its ability to distinguish objects with similar appearances. The combination of HSI and SAR/LiDAR data enhances reliable land cover classification in diverse weather or seasonal scenarios. Therefore, in this study, we mainly focus on HSI and SAR/LiDAR data joint classification. Due to differences in imaging principles between HSI and SAR/LiDAR data, effectively fusing these two heterogeneous data types presents a critical challenge [ 11 ] . Currently, many CNN and Transformer-based methods have been proposed for multi-source data classification [ 25 ] [ 36 ] . CNN-based methods [ 53 , 6 , 7 ] typically use two specific CNN encoders to extract spectral–spatial features from HSI and SAR/LiDAR data. Their feature maps are then aligned and fused via concatenation or attention to produce a unified representation for classification. Transformer-based methods [ 47 , 26 , 54 ] leverage separate spectral–spatial and geometric tokenizers to encode each data, and then applies cross-attention to exchange complementary information such as material signatures and elevation structure. The fused tokens are jointly modeled through global self-attention, enabling robust land-cover classification even under spectral redundancy and spatial mis-registration. Despite their promising performance in land cover classification, existing deep learning-based methods still face the following two critical challenges: (1) How to reduce the spectral redundancy without losing multi-source complementary features. Hyperspectral sensor captures data across hundreds of contiguous spectral bands, and the rich spectral resolution introduces spectral redundancy. The spectral redundancy would increase computational cost and reduce the classification accuracy if noisy bands are not handled properly. Some methods use Principal Component Analysis (PCA) for spectral dimensionality reduction [ 44 ] , as shown in Fig. 1 . However, most existing multi-source fusion frameworks treat spectral reduction and multi-source interaction as two isolated stages. This premature, task-agnostic dimensionality reduction prior to network training inevitably discards critical class-specific spectral signatures that might be highly complementary to SAR/LiDAR structural priors. In contrast to simple linear dimensionality reduction, advanced deep learning architectures, such as two-stream convolutional networks, have proven highly effective in automatically extracting abundant and task-specific spectral features [ 62 ] . Therefore, how to design a dynamic spectral selection mechanism that is jointly optimized with multi-source fusion remains a significant hurdle. (2) How to enhance the multi-source feature interactions under severe physical heterogeneity. HSI and SAR/LiDAR data are inherently heterogeneous with completely distinct imaging mechanisms. Traditional fusion methods (e.g., feature concatenation, feature summation, cross attention) may fail to bridge the severe semantic gap and capture the nonlinear relationships between spectral and structural features, leading to suboptimal classification performance. Addressing nonlinear mappings is a pervasive issue in hyperspectral processing, where techniques combined with spatial-spectral adaptive strategies have shown great potential in constructing discriminative nonlinear feature spaces [ 61 ] . Inspired by these representation learning strategies, therefore, how to devise an adaptive strategy to align the semantic space between HSI and SAR/LiDAR data is a tricky task. To address the above two challenges, we propose a R epresentative S pectral C orrelation Net work ( RSCNet ) for multi-source remote sensing data joint classification. The core technical advantage of RSCNet lies in tightly coupling the dynamic spectral band selection with multi-source feature interaction. As shown in Fig. 2 , we design RSCNet with two key modules: Key Band Selection Module (KBSM) and Cross-source Adaptive Fusion Module (CAFM). To flexibly and dynamically exploit