Physically Accurate Differentiable Inverse Rendering for Radio Frequency Digital Twin CCS: Human-centered computing Ubiquitous and mobile computing CCS: Computer systems organization Embedded and cyber-physical systems
Xingyu Chen † , Xinyu Zhang † , Kai Zheng † , Xinmin Fang ‡ , Tzu-Mao Li † , Chris Xiaoxuan Lu § , Zhengxiong Li ‡ · 2026
Read Physically Accurate Differentiable Inverse Rendering for Radio Frequency Digital Twin CCS: Human-centered computing Ubiquitous and mobile computing CCS: Computer systems organization Embedded and cyber-physical systems by Xingyu Chen † , Xinyu Zhang † , Kai Zheng † , Xinmin Fang ‡ , Tzu-Mao Li † , Chris Xiaoxuan Lu § , Zhengxiong Li ‡ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18026.txt.
Introduction
Chen and co-authors present RFDT, a differentiable radio frequency simulator for building digital twins of real scenes. Old RF tools are not differentiable, and path visibility creates sharp breaks. RFDT adds a physics-based edge diffraction step to smooth those breaks. A signal domain surrogate also tames non-convexity from Fourier steps. Tests show it can rebuild twins from real RF measures and aid sensing and network tuning.
Catalog record
| CREATOR | Xingyu Chen † , Xinyu Zhang † , Kai Zheng † , Xinmin Fang ‡ , Tzu-Mao Li † , Chris Xiaoxuan Lu § , Zhengxiong Li ‡ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 115884 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18026
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Physically Accurate Differentiable Inverse Rendering for Radio Frequency Digital Twin CCS: Human-centered computing Ubiquitous and mobile computing CCS: Computer systems organization Embedded and cyber-physical systems Xingyu Chen † , Xinyu Zhang † , Kai Zheng † , Xinmin Fang ‡ , Tzu-Mao Li † , Chris Xiaoxuan Lu § , Zhengxiong Li ‡ Abstract . Digital twins, virtual simulated replicas of physical scenes, are transforming system design across industries. However, their potential in radio frequency (RF) systems has been limited by the non-differentiable nature of conventional RF simulators. The visibility of propagation paths causes severe discontinuities, and differentiable rendering techniques from computer graphics are no longer applicable due to point-source antennas and dominant specular reflections. In this paper, we present RFDT , a physically based differentiable RF simulation framework that enables gradient-based interaction between virtual and physical worlds. RFDT resolves discontinuities with a physically grounded edge-diffraction transition function, and mitigates non-convexity from Fourier-domain processing through a signal-domain transform surrogate. Our implementation demonstrates RFDT ’s ability to accurately reconstruct digital twins from real RF measurements. Moreover, RFDT can augment diverse downstream applications, such as test-time adaptation of machine learning-based RF sensing and physically constrained optimization of communication systems. The supplementary materials and code of RFDT are released at rfdigitaltwin.github.io . 1. Introduction Radio Frequency (RF) signals serve not only as a medium for communication but also interact richly with the physical world. Subtle signal propagation and reflection patterns can encode information about the environment. This property has powered a wave of innovative RF sensing applications, including micro-gesture detection ( 92 ; 46 ; 43 ; 82 ) , vital sign monitoring ( 47 ; 24 ) , and non-line-of-sight (NLOS) imaging ( 91 ; 90 ; 86 ; 84 ) . Likewise, wireless communication systems leverage multipath propagation to enhance reliability, coverage, and spatial multiplexing gain. This is particularly critical in the millimeter-wave (mmWave) bands where performance becomes highly sensitive to ambient environment ( 83 ) . To model and leverage such RF-environment interactions, conventional system design has largely followed two paths. Data-driven approaches can extract complex patterns from large datasets, but often fail to generalize to unseen environments ( 20 ; 86 ; 80 ; 14 ) . Model-driven approaches ( 9 ; 67 ) , rooted in RF propagation theory and signal processing, provide stronger generalization but cannot incorporate the implicit, experiential knowledge about physical environments. Figure 1 . RFDT constructs digital twins by solving the inverse problem of RF simulation. Top: 3D reconstruction through radar simulation. Bottom: Communication system optimization through RF scene rendering. To propel RF system design to its next evolutionary phase, in this paper, we propose a novel framework that casts the problem as constructing digital twins (DTs) of objects in the physical world. As illustrated in Fig. 1 , a DT achieves high-fidelity simulation through continuous, bidirectional, and gradient-based interactions with its physical counterpart. This interaction distinguishes DTs from unidirectional methods such as conventional RF ray tracing simulation (digital-to-physical) or sensing systems (physical-to-digital). Our DT framework, referred to as RFDT , realizes such interaction by incorporating both into a closed-loop optimization framework. Unlike conventional forward simulation techniques ( 14 ) that use ray tracing to predict the RF signal patterns (output) from the physical twin’s characteristic parameters (input), RFDT solves the inverse problem , i.e. , inferring the unknown input from the observable output. Since the DT model is physics grounded and explainable, it overcomes the reliance on training data or labels, and intrinsically handles unseen scenes. On the other hand, its gradient-based optimization enables easy integration of implicit knowledge and constraints ( e.g. , possible geometries for given object categories) represented by pre-trained neural blocks. To enable the closed-loop optimization in DT, the key challenge lies in differentiability of simulation steps that characterize how the RF signals propagate and interact with physical objects. The output of a differentiable RF simulation should vary smoothly and predictably in response to infinitesimal perturbations of its input parameters ( e.g. , the geometry or orientation of a reflecting surface). This property enables accurate computation of gradients between the simulation output and the input parameters, which is fundamental for efficient and robust DT optimization. While differentiable rendering in computer graphics has achieved remarkable progress, we prove in Sec. 3.1 that these methods cannot be directly applied to RF. Graphics rendering heavily relies on Monte Carlo integration, which estimates scene illumination by averaging many randomly chosen light paths. In typical optical settings, light sources are modeled as extended area emitters, and objects’ surfaces scatter rays over many directions. In RF, by contrast, antennas behave as near-point sources, and ray-surface interaction is dominated by specular reflections (Sec. 3.1 ). As a result, the random path sampling in Monte Carlo integration is almost surely to miss the dominant specular path. In addition, many direct differentiation schemes for geometric parameters in ray tracing rely on aggressive approximations ( 49 ) , resulting in inaccurate gradient estimates. This is especially problematic in RF optimization, where wavelength-scale precision is essential. Our RFDT framework represents the first to establish physically consistent gradients for RF simulation. It enables fully differentiable RF ray tracing by identifying and solving two fundamental challenges. (i) Discontinuity with respect to scene parameters. In traditional RF ray tracing, a tiny perturbation in geometry or material parameters can abruptly change which propagation paths contribute to the received signal, resulting in sharp discontinuities in both amplitude and phase. To overcome this challenge, we reformulate the RF ray tracing process into a continuous function. In particular, instead of using the conventional binary path visibility test that either fully includes or fully discards a reflection path, we introduce a physically grounded transition function based on the principle of equivalent edge diffraction. This models how path contributions vary smoothly as reflection points approach or cross object boundaries, making the dominant specular effects differentiable and ensuring stable gradient propagation throughout the simulation. (ii) Local non-convexity caused by the RF waves’ periodic properties and signal-domain transforms. The inherent periodicity of RF waves introduces non-convexity into simulations of signal properties such as strength and angle. This issue is compounded by the signal-domain transforms commonly used in radar and communication systems ( e.g. , FFT for multi-carrier modulation and range/frequency analysis), which further create spurious sidelobes and local minima. As a result, direct application of gradient descent on RF signals, particularly radar waveforms widely used in RF sensing, often fails to converge because the optimization landscape is filled with wavelength-scale ambiguities. To address this challenge, we incorporate a novel surrogate model for transform-domain signal representations, in particular the radar range profile. This surrogate not only provides smooth and accurate gradients but also significantly reduces the computational overhead of the RFDT optimization framework. Table 1 . Comparison of RFDT with related works in Differentiable Rendering Work Spectrum RT Method Point Source Spec.Reflec. Diffraction \partial -Geometry Unbiased \nabla Radar Comm. Direct AD ( 58 ) Optical Monte Carlo No No No No No No No Edge Sampling ( 41 ) Optical Monte Carlo No No No Yes Yes Yes Potential Path Space ( 88 ; 89 ) Optical Monte Carlo Yes No No Yes Yes Potential Potential Warped Area ( 6 ; 51