Deep Learning Based Dynamic Environment Reconstruction for Vehicular ISAC Scenarios
Junzhe Song, Ruisi He, Mi Yang · 2025
Read Deep Learning Based Dynamic Environment Reconstruction for Vehicular ISAC Scenarios by Junzhe Song, Ruisi He, Mi Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05226.txt.
Introduction
Junzhe Song and co-authors present a deep learning system that rebuilds a moving street scene from radio signals, in a 2025 paper. Future smart traffic systems aim to sense and communicate through the same wireless links. That reuse could let cars map their surroundings without extra sensors such as laser scanners.
The team first built a dataset that pairs real city street measurements with the matching wireless channel data. Their network then works in stages: it names the scene type, locates the main reflecting points, and fills in fine shapes as a point cloud. Tests show the method tracks changing scenes with better accuracy over time than older tools. The work points toward cheaper sensing for self-driving cars.
Catalog record
| CREATOR | Junzhe Song, Ruisi He, Mi Yang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05226
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Learning Based Dynamic Environment Reconstruction for Vehicular ISAC Scenarios Thanks: J. Song, R. He, M. Yang, Z. Zhang, and B. Ai are with the Schooof Electronics and Information Engineering, Beijing Jiaotong UniversityBeijing 100044, China (email: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]). Bingcheng Liu is with Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China (e-mail: [email protected]). Jiahui Han and Haoxiang Zhang are with China Academy of Industrial Internet, Ministry of Industry and Information Technology, Beijing, China (email: [email protected]; [email protected]). Junzhe Song, Ruisi He, Mi Yang Abstract Integrated Sensing and Communication (ISAC) technology plays a critical role in future intelligent transportation systems, by enabling vehicles to perceive and reconstruct the surrounding environment through reuse of wireless signals, thereby reducing or even eliminating the need for additional sensors such as LiDAR or radar. However, existing ISAC-based reconstruction methods often lack the ability to track dynamic scenes with sufficient accuracy and temporal consistency, limiting the real-world applicability. To address this limitation, we propose a deep learning based framework for vehicular environment reconstruction by using ISAC channels. We first establish a joint channel–environment dataset based on multi-modal measurements from real-world urban street scenarios. Then, a multi-stage deep learning network is developed to reconstruct the environment. Specifically, a scene decoder identifies the environmental context such as buildings, trees and so on; a cluster center decoder predicts coarse spatial layouts by localizing dominant scattering centers; a point cloud decoder recovers fine-grained geometry and structure of surrounding environments. Experimental results demonstrate that the proposed method achieves high-quality dynamic environment reconstruction with a Chamfer Distance of 0.29 and F-Score@1% of 0.87. In addition, complexity analysis demonstrates the efficiency and practical applicability of the method in real-time scenarios. This work provides a pathway toward low-cost environment reconstruction based on ISAC for future intelligent transportation. I Introduction Integrated Sensing and Communication (ISAC) has been identified by International Telecommunication Union Radiocommunication Sector (ITU-R) as a key feature of next-generation networks and is recognized as one of the six core visions of IMT-2030 (6G) [ 1 , 2 , 3 ] . ISAC aims to achieve deep integration of communication and sensing functionalities by leveraging shared spectrum and hardware resources [ 4 , 5 ] . In ISAC-enabled scenarios, network infrastructure is no longer limited to data transmission and connectivity, it also serves as a distributed sensing platform for environmental imaging and three-dimensional (3D) reconstruction tasks. Millimeter-wave (mmWave) frequencies, with their large bandwidth, narrow beamwidth, and high spatial resolution, offer unique advantages in ISAC systems [ 6 , 7 ] . These characteristics make mmWave ISAC particularly effective for target detection, scene modeling, and environmental perception [ 8 ] . In vehicular ISAC scenarios, vehicles can leverage existing communication systems to perceive the surrounding environment, eliminating the need for additional deployment of expensive LiDAR or radar sensors, thereby reducing the cost of environmental perception [ 9 ] . Unlike traditional static ISAC scenarios, vehicular ISAC faces unique challenges such as rapidly changing environments and continuously varying observation angles, which impose stricter requirements on perception accuracy and temporal consistency [ 10 ] . Therefore, developing dynamic environment reconstruction techniques tailored for vehicular ISAC scenarios holds significant practical relevance and technical value. Traditionally, 3D environment reconstruction using wireless signals primarily relies on Frequency-Modulated Continuous Wave (FMCW) signals transmitted by mmWave radar systems, with Fast Fourier Transform (FFT) techniques commonly employed for echo signal processing [ 11 ] . A large of work has focused on leveraging mmWave radar echoes for target detection, point cloud reconstruction, and even scene-level semantic understanding, demonstrating considerable potential in applications such as autonomous driving [ 12 , 13 , 14 ] , security and health monitoring [ 15 ] [ 16 ] , and Internet of Things (IoT) [ 17 ] [ 18 ] . However, such works typically rely on dedicated sensing hardware (e.g., LiDAR, RGB-D cameras), which are costly, power-intensive, and highly sensitive to adverse environmental conditions such as rain, fog, and lighting variations, making them unsuitable for long-term deployment in resource-constrained edge dynamic scenarios [ 16 , 17 , 19 ] . Moreover, sensing tasks are often designed independently and deployed redundantly, resulting in low spectral efficiency and failing to meet the demands of future ubiquitous intelligent systems for low-cost, highly integrated sensing-communication platforms [ 20 ] [ 21 ] . With the advancement of ISAC, communication-based environment reconstruction has emerged as a new research trend. ISAC offers inherent advantages such as spectrum sharing, joint resource scheduling, and hardware co-design, enabling the development of highly integrated and energy-efficient 3D dynamic reconstruction systems. However, FMCW waveforms are not compatible with communication schemes, making existing reconstruction solutions difficult to adapt to ISAC applications [ 22 ] [ 23 ] . Therefore, developing environment reconstruction methods based on ISAC has become critical. In wireless communication systems, the channel refers to the physical medium through which electromagnetic signals propagate from transmitter (Tx) to receiver (Rx). Unlike traditional communication channels that are used exclusively for data transmission, ISAC channels simultaneously carry user signals and embed rich information of surrounding environments [ 24 ] . Channel parameters, such as delay, azimuth angle of arrival (AoA), and elevation angle of arrival (EoA), essentially characterize the physical propagation process of signals in space, rendering the reconstruction of environmental geometry from channel information physically feasible [ 25 ] . Current research on ISAC-enabled environment reconstruction mainly follows two pathways: one relies on communication channel information, and the other on sensing channel feedback. The former often adopts multi-node cooperative architectures, leveraging geometric patterns and statistical variations in received signals to infer the environment. For example, Chang et al. proposed a multi-object reconstruction method by clustering reflection points along communication paths, validated under THz band experiments [ 26 ] , Lu et al. developed a communication channel-driven reconstruction framework enabling multi-user joint modeling [ 27 ] . In contrast, reconstruction approaches based on sensing channels typically extract multi-path characteristics from the observation points to restore environmental structure. For example, Blandino et al. reconstructed UAV 3D positions based on sensing channel observations [ 28 ] , while Yin et al. proposed a dual-base cooperation model that utilizes uplink echoes to achieve 3D imaging [ 29 ] . Although these studies demonstrate the feasibility of using ISAC channels for environment reconstruction, most existing works still rely on simulation data or idealized experimental setups, such as indoor scenes, static targets, or isolated object localization. Consequently, their performance degrades significantly under highly dynamic traffic scenarios. As highlighted in [ 30 ] , when dynamic scatterers vary frequently, the stability and interpretability of mmWave channel parameters deteriorate, severely impacting the continuity and accuracy of 3D reconstruction. Moreover, Ref. [ 31 ] points out that current methods generally lack effective alignment mechanisms for dynamic target features, limiting their applicability in high-mobility tasks such as Vehicle-to-Everything (V2X) communication and autonomous driving. Therefore, developing ISAC channel-based reconstruction methods tailored for dynamic environments has become urgent and critical for future research. Although the ISAC channel inherently contains rich information about surrounding environment, its high-dimensional and abstract features make it challenging to establish an explicit mapping to dynamic environments [ 32 ] . In this context, deep learning excels at modeling such complex and structured