Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks

Yunting Xu, Jiacheng Wang, Ruichen Zhang, Changyuan Zhao, Yinqiu Liu, Dusit Niyato · 2026

Read Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks by Yunting Xu, Jiacheng Wang, Ruichen Zhang, Changyuan Zhao, Yinqiu Liu, Dusit Niyato (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16927.txt.

Introduction

Xu and co-authors design a cooperative perception system for teams of drones in low altitude networks. Drones see a scene from many views, but sending all that video is slow and costly. Their framework sends only the most informative pixels, chosen by a Top-K rule. A ground server fuses the sparse images into bird's eye view features for vehicle perception. A diffusion based learning algorithm picks which drones join and how much to sparsify. In simulations, the authors report perception gains over 5 percent with 85 percent less communication overhead.

Catalog record

CREATORYunting Xu, Jiacheng Wang, Ruichen Zhang, Changyuan Zhao, Yinqiu Liu, Dusit Niyato
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73718
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16927

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Leveraging Large Vision Model for Multi-UAV Co-perception in Low-Altitude Wireless Networks Thanks: Y. Xu, J. Wang, R. Zhang, C. Zhao, Y. Liu, and D. Niyato are with the College of Computing and Data Science, Nanyang Technological University, Singapore, 639798 (e-mail: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]). Thanks: L. Yu is with the Alibaba Cloud, Hangzhou, China, 311121 (e-mail: [email protected]). Thanks: H. Zhou is with the School of Electronic Science and Engineering, Nanjing University, Nanjing, China, 210023 (e-mail: [email protected]). Thanks: D. I. Kim is with the Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, South Korea (email: [email protected]). Yunting Xu, Jiacheng Wang, Ruichen Zhang, Changyuan Zhao, Yinqiu Liu, Dusit Niyato Abstract Multi-uncrewed aerial vehicle (UAV) cooperative perception has emerged as a promising paradigm for diverse low-altitude economy applications, where complementary multi-view observations are leveraged to enhance perception performance via wireless communications. However, the massive visual data generated by multiple UAVs poses significant challenges in terms of communication latency and resource efficiency. To address these challenges, this paper proposes a communication-efficient cooperative perception framework, termed Base-Station-Helped UAV (BHU), which reduces communication overhead while enhancing perception performance. Specifically, we employ a Top-K selection mechanism to identify the most informative pixels from UAV-captured RGB images, enabling sparsified visual transmission with reduced data volume and latency. The sparsified images are transmitted to a ground server via multi-user MIMO (MU-MIMO), where a Swin-large-based MaskDINO encoder extracts bird’s-eye-view (BEV) features and performs cooperative feature fusion for ground vehicle perception. Furthermore, we develop a diffusion model–based deep reinforcement learning (DRL) algorithm to jointly select cooperative UAVs, sparsification ratios, and precoding matrices, achieving a balance between communication efficiency and perception utility. Simulation results on the Air-Co-Pred dataset demonstrate that, compared with traditional CNN-based BEV fusion baselines, the proposed BHU framework improves perception performance by over 5% while reducing communication overhead by 85%, providing an effective solution for multi-UAV cooperative perception under resource-constrained wireless environments. I Introduction Multi-unmanned aerial vehicle (UAV) cooperative perception has recently emerged as a promising technology for a variety of low-altitude economy applications, such as intelligent transportation, urban surveillance, and autonomous inspection [ 26 , 30 , 38 ] . By exploiting the complementary multi-view observations from spatially distributed UAVs, cooperative perception systems can substantially enhance perception accuracy and robustness compared with single-UAV solutions [ 32 , 50 ] . However, realizing such performance gains in practical wireless environments remains challenging due to the massive volume of visual data generated by multiple UAVs [ 14 , 40 ] . Additionally, the heterogeneous and time-varying channel conditions between UAVs and base stations (BSs), together with the limited wireless time–frequency resources, make it difficult to satisfy the stringent latency requirements for real-time UAV communications, thereby hindering the efficient data transmission for cooperative multi-UAV perception [ 27 ] . To reduce communication overhead between the cooperative UAVs, instead of transmitting raw UAV images or dense visual features, recent studies have explored feature-level compression or sparsification to alleviate communication bottlenecks [ 12 , 49 ] . By dynamically adjusting the transmission data size of visual images, these approaches can scale effectively in bandwidth-limited and latency-sensitive scenarios [ 29 , 13 ] . Nevertheless, since UAVs operate as resource-constrained edge devices with limited energy and computational capacity, they are typically equipped with lightweight network modules for onboard image processing [ 24 ] . As a result, compression or sparsification approaches often suffer from limited reconstruction and feature extraction capabilities, which significantly degrades the quality of perception. Therefore, effectively balancing perception performance and communication efficiency remains a challenging problem, particularly for multi-UAV cooperative perception tasks. Recent development of large vision models (LVMs), such as Vision Transformers (ViT) [ 7 ] and Self-distillation with No Label (DINO) [ 4 ] , have demonstrated advanced representation and generalization capabilities compared with traditional Convolutional Neural Networks (CNNs). Based on the self-attention mechanism, LVMs can effectively capture long-range dependencies and semantic relationships to extract rich visual information, thus achieving state-of-the-art performance across diverse image tasks such as object classification, instance detection, and semantic segmentation [ 35 ] . As a consequence, LVMs have shown substantial potential for enhancing multi-UAV cooperative perception, enabling more accurate image reconstruction, feature extraction, and scene understanding from information-reduced visual data [ 23 ] . Nevertheless, while LVMs are capable of achieving high perception performance, their large-scale model parameters and substantial computational overhead pose significant challenges to the practical implementation in real-world UAV applications. Motivated by the growing demand to fully exploit the potential of LVMs, this paper proposes a communication-efficient multi-UAV perception framework, named Base-Station-Helped UAV (BHU), aiming to reduce the communication overhead while enhancing perception performance. The core idea of the BHU framework is to transmit sparsified UAV images to a ground server via wireless communication, where an LVM–based encoder extracts bird’s-eye-view (BEV) features from the aerial images and performs cooperative feature fusion for downstream perception tasks. Specifically, we model the communication between cooperative UAVs and the BS as a multi-user multiple-input multiple-output (MU-MIMO) system. To further achieve an effective trade-off between perception utility and communication efficiency, we adaptively select the cooperative UAVs, image sparsification ratios, precoding strategies for each UAV based on the channel state information, establishing a communication-aware multi-UAV cooperative perception mechanism for low-altitude economy scenarios. The main contributions are summarized as follows: • We introduce an importance score–based Top-K mechanism for sparsified visual transmission, which selects the most informative pixels from UAV-captured images and substantially reduces both transmission data volume and communication latency. • We leverage a Swin-large-based MaskDINO encoder [ 17 ] to extract BEV features and perform cooperative feature fusion for vehicle instance and motion-aware perception, thereby significantly enhancing perception performance through the strong representation capability of LVMs. • We propose a diffusion model–based deep reinforcement learning (DRL) framework to jointly optimize cooperative UAV selection, Top-K sparsification ratios, and precoding matrices. A denoising diffusion implicit model (DDIM) is integrated to improve the training stability and efficiency of the DRL framework. • Extensive simulations conducted on Air-Co-Pred dataset [ 31 ] demonstrate that the proposed BHU framework enhances perception utility by over 5% and reduces communication overhead by 85%, offering a communication-efficient solution for multi-UAV cooperative perception in resource-constrained wireless environment. The remainder of this paper is organized as follows. Section II reviews the related work. Section III introduces the system model of multi-UAV cooperative perception. Section IV describes the proposed BHU framework. Section V presents the DDIM-based DRL solution. Simulation results are provided in Section VI. Finally, Section VII concludes this paper. II Related Work II-A Visual Data Reduction and UAV Wireless Transmission The transmission of high-resolution visual data from UAVs to ground infrastructure poses a major challenge due to stringent bandwidth and latency constraints [ 46 , 48 ] . To mitigate communication overhead, extensive research has investigated image and feature compression techniques for efficient transmission [ 39 ] . For example, Giordano et al. [ 11 ] developed an onboard region-of-interest (RoI)-based compression framework for remote sensing images, combining RoI identification with differential compression schemes to reduce transmission volume while preserving target quality. Additionally, sparsification methods adopt token selection mechanisms to transmit only informative content, such as the spatial confidence–guided sparse messaging scheme proposed in Where2comm [ 12 ] and the

Read the complete text