A Camera-Cooperative ISAC Framework for Multimodal Non-Cooperative UAVs Sensing

Wenfeng Wu, Luping Xiang, Kun Yang · 2026

Read A Camera-Cooperative ISAC Framework for Multimodal Non-Cooperative UAVs Sensing by Wenfeng Wu, Luping Xiang, Kun Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22090.txt.

Introduction

Wenfeng Wu, Luping Xiang, and Kun Yang propose a camera-cooperative ISAC framework for sensing non-cooperative drones. Detecting such drones is hard for ISAC systems because single-modal perception is limited and sensing competes with comms for shared resources. Their camera-aided system uses multimodal sensing to enable efficient drone beam steering. Tests show an average reduction of 71 percent in beam steering overhead and 1.69 to 11.15 percent in tracking overhead, while keeping high angular accuracy and freeing room for extra comms tasks.

Catalog record

CREATORWenfeng Wu, Luping Xiang, Kun Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75322
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22090

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Camera-Cooperative ISAC Framework for Multimodal Non-Cooperative UAVs Sensing Wenfeng Wu, Luping Xiang, Kun Yang Abstract The detection of non-cooperative unmanned aerial vehicles (UAVs) presents significant challenges for Integrated Sensing and Communication (ISAC) systems due to the inherent limitations of single-modal perception and the competition for shared communication and sensing resources. To address these challenges, this paper proposes a novel Camera-Cooperative ISAC (CC-ISAC) framework that employs multimodal sensing to enable efficient UAV beam steering and tracking. The proposed framework employs cameras for coarse-grained airspace monitoring and utilizes ISAC for fine-grained, high-precision sensing, forming a complementary perception loop that enhances both sensing accuracy and resource efficiency. Within this framework, two key modules are developed: (1) a Vision-to-Echo Data Alignment (V2EDA) model that aligns visual and echo-domain features through cross-attention mechanisms, and (2) a Multimodal Fusion-Based Estimation (MMFE) model that integrates historical multimodal data with current observations for robust state estimation. Extensive evaluations conducted on the DeepSense6G dataset demonstrate that the proposed framework achieves an average reduction of 71% in beam steering overhead and 1.69–11.15% in tracking overhead while maintaining high angular estimation accuracy. The CC-ISAC framework effectively mitigates resource contention between sensing and communication, enabling reliable UAV surveillance while freeing substantial system resources for additional communication tasks, thereby representing a practical advancement in ISAC system design. I Introduction The low-altitude airspace, a rapidly developing integrated economic ecosystem relying on advanced aerial platforms such as unmanned aerial vehicles (UAVs) and electric vertical take-off and landing vehicles, has garnered significant global attention [ 11 ] . Within this airspace, cooperative UAVs are typically managed by a base station (BS) and transmit real-time location data through cellular networks [ 13 ] . Conversely, non-cooperative UAVs present considerable threats to public safety, necessitating timely detection and tracking for effective countermeasures [ 34 , 43 ] . Integrated Sensing and Communication (ISAC) systems have emerged as a promising solution for establishing wireless communication networks that provide wide coverage and real-time sensing capabilities. In ISAC systems, sensing and communication (S&C) functionalities share finite physical resources while achieving high sensing performance and substantial data throughput [ 19 ] . By allocating orthogonal or non-overlapping resources in the time, frequency, and spatial domains, S&C functions can operate concurrently without interference. As a result, the ISAC BS plays a pivotal role in both communication and sensing tasks, including the detection and tracking of UAVs to mitigate unauthorized intrusions or collisions [ 34 ] . This dual functionality presents significant challenges in terms of resource allocation between S&C. In ISAC systems, motion parameters, such as range, angle, and velocity, of moving objects are estimated through the transmission of ISAC signals and subsequent analysis of the reflected echoes, utilizing techniques such as time-delay measurement, beamforming, and Doppler shift analysis [ 16 ] . However, the inherently low detect ability of non-cooperative UAVs, stemming from their typical low-altitude, low-speed, and small radar cross-section (RCS) characteristics, introduces notable challenges for ISAC systems [ 32 ] . Furthermore, the convergence of ubiquitous wireless sensing and advanced visual sensing technologies has made rapid progress, fueled by the diverse, heterogeneous information provided by various sensors in intelligent systems [ 7 , 5 , 24 ] . This convergence presents a new paradigm for non-cooperative UAV sensing. I-A Related work I-A 1 Vision-Based UAV Sensing Recent advancements in deep learning have significantly improved performance in small-object detection and tracking tasks [ 36 , 15 ] . Among these, the YOLO series has emerged as a prominent solution, offering an optimal balance between real-time processing and robust detection across multiple object scales [ 37 , 20 ] . For instance, in [ 6 ] , a mean Average Precision (mAP) of 93.52% was achieved at 82 frames per second (fps) on a custom UAV dataset under simple scenarios, while [ 9 ] reported 92% accuracy at 23.6 fps using the ARD-MAV dataset under more complex environmental conditions. These results illustrate the trade-off between detection accuracy and real-time processing capabilities in vision-based UAV sensing. However, vision-based approaches remain sensitive to challenges such as illumination variation, occlusion, and background clutter. I-A 2 UAV Sensing in ISAC To mitigate the limitations posed by weak echo signals from small targets, digital beamforming has been employed. This technique dynamically adjusts the phase and amplitude of antenna elements to steer beams, enhancing the Signal-to-Noise Ratio (SNR) and spatial resolution, thus improving the reliability of UAV detection [ 27 , 35 , 10 ] . In practical implementations, codebook-based beam sweeping strategies are commonly used [ 8 ] . While exhaustive sweeping is conceptually simple, it incurs substantial overhead. Alternatively, hierarchical search methods progressively refine beamwidths and are more widely adopted in practice. By employing hierarchical codebook designs [ 38 ] , binary-tree traversal techniques can efficiently sweep the codebook layer by layer, significantly reducing the search complexity compared to exhaustive approaches. I-A 3 Multimodal Fusion-Based Sensing Despite significant progress, research on multimodal fusion for non-cooperative UAV detection is still in its early stages. Sensor fusion has been widely recognized as an effective method to enhance detection accuracy by compensating for the limitations of individual sensing modalities [ 28 ] . As a result, various studies have explored the use of multiple sensors for joint decision-making or the construction of hierarchical frameworks to improve UAV detection [ 33 , 16 ] . Among different fusion paradigms, radar-vision fusion has proven to be particularly promising, enabling seamless integration of wireless sensing with computer vision. By leveraging complementary multi-dimensional features while reducing redundancy, radar-vision fusion improves environmental adaptability, reliability, and fault tolerance [ 44 , 41 , 25 , 26 ] . For example, multimodal fusion at the data-level [ 40 ] , decision-level [ 18 ] , and feature-level [ 14 ] has demonstrated superior performance over single-modal approaches. Moreover, machine learning techniques have gained traction in multimodal beam alignment and prediction tasks [ 39 , 12 ] . These methods leverage auxiliary data to improve wireless environmental awareness and enhance prediction accuracy. For instance, camera-assisted beam tracking at the BS facilitates accurate beam prediction without additional training [ 22 ] , while the fusion of image and GPS data enables precise estimation of UAV position and orientation [ 1 ] . These multimodal approaches underscore the potential of vision-assisted ISAC systems for more robust UAV sensing. TABLE I : Comparison of CC-ISAC with Prior Vision-Assisted ISAC Frameworks Literature Scenario Vision Usage Fusion Strategy Task Partitioning 1 Perception Offloading 2 [ 39 ] V2I Beam selection - Two No Beam tracking - [ 12 ] V2X Transmitter identification Feature-level (with Power Data) One No Transmitter tracking - [ 22 ] V2V Transmitter identification Feature-level (with Power Data) One No Transmitter tracking - [ 1 ] UAV Beam selection Feature-level (with GPS) One No [ 23 ] V2I Beam tracking Feature-level (with GPS) One No This work UAV Airspace detection - Three Yes Beam selection - Beam tracking Decision-level (with echo signal) 1 Task Partitioning represents an explicit hierarchical decomposition where different modalities are assigned distinct and complementary roles (e.g., airspace detection, guided beam selection, and beam tracking), as adopted in CC-ISAC. 2 Perception offloading indicates whether coarse-grained environmental awareness is offloaded from the ISAC sensing module to

Read the complete text