MM-2FSK: Multimodal Frequency Shift Keying for Ultra-Efficient and Robust High-Resolution MIMO Radar Imaging

Vanessa Wirth, Johanna Bräunig, Martin Vossiek, Tim Weyrich *, Marc Stamminger * · 2025

Read MM-2FSK: Multimodal Frequency Shift Keying for Ultra-Efficient and Robust High-Resolution MIMO Radar Imaging by Vanessa Wirth, Johanna Bräunig, Martin Vossiek, Tim Weyrich *, Marc Stamminger * (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01405.txt.

Introduction

Vanessa Wirth and colleagues speed up high resolution radar imaging of moving targets. Large aperture millimeter wave MIMO radar can reconstruct near field surfaces in three dimensions and works in any light or weather. The reconstruction is so computation heavy that frame rates suffer, and bandwidth and capture rates are squeezed.

Earlier few tone imaging cut the number of transmitted frequencies, at the cost of range ambiguity and a need to already know roughly where the target sits. Their method, MM-2FSK, extends two tone stepped frequency imaging with a per point depth prior from an external source, which need not be radar. Processing the prior resolves the ambiguities, enabling precise near field imaging of geometrically complex surfaces at reduced cost.

Catalog record

CREATORVanessa Wirth, Johanna Bräunig, Martin Vossiek, Tim Weyrich *, Marc Stamminger *
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01405

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MM-2FSK: Multimodal Frequency Shift Keying for Ultra-Efficient and Robust High-Resolution MIMO Radar Imaging Vanessa Wirth, Johanna Bräunig, Martin Vossiek, Tim Weyrich *, Marc Stamminger * Abstract Accurate reconstruction of static and rapidly moving targets demands three-dimensional imaging solutions with high temporal and spatial resolution. Radar sensors are a promising sensing modality because of their fast capture rates and their independence from lighting conditions. To achieve high spatial resolution, MIMO radars with large apertures are required. Yet, they are infrequently used for dynamic scenarios due to significant limitations in signal processing algorithms. These limitations impose substantial hardware constraints due to their computational intensity and reliance on large signal bandwidths, ultimately restricting the sensor’s capture rate. One solution of previous work is to use few frequencies only, which enables faster capture and requires less computation; however, this requires coarse knowledge of the target’s position and works in a limited depth range only. To address these challenges, we extend previous work into the multimodal domain with MM-2FSK, which leverages an assistive optical depth sensing modality to obtain a depth prior, enabling high framerate capture with only few frequencies. We evaluate our method using various target objects with known ground truth geometry that is spatially registered to real millimeter-wave MIMO radar measurements. Our method demonstrates superior performance in terms of depth quality, being able to compete with the time- and resource-intensive measurements with many frequencies. 1 1 footnotetext: The authors contributed equally to this work † † footnotetext: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. I Introduction In recent years, the reconstruction of dynamic targets using contactless sensors has gained significant attention, influencing research in many areas, including entertainment (e.g., computer games, AR/VR), autonomous agents, human-computer interaction, and medical diagnosis [ 1 , 2 ] . Among these, applications involving critical decisions based on complex movements, such as human gait analysis [ 3 , 4 ] and clinical hand function assessments [ 5 , 6 , 7 ] , place a particularly high demand on fast and precise sensing techniques to ensure the reliability of such decisions. Millimeter-wave (mmWave) multiple-input multiple-output (MIMO) radars offer a viable solution, providing spatial resolution beyond the capabilities of traditional monostatic antenna systems, and enabling the distinction of static and dynamic targets. Moreover, radar systems can analyze motion via the Doppler effect, making them well-suited for dynamic environments, compared to other modalities such as conventional LiDAR or RGB-D cameras. Thus, MmWave MIMO radar systems have been utilized for 3D human body reconstruction [ 8 , 9 ] , pose estimation [ 10 , 11 ] , people tracking [ 12 ] , and activity recognition [ 13 ] . To achieve high-resolution three-dimensional imaging, for instance in security screening [ 14 ] , MIMO radars typically employ a large number of transmitting (TX) and receiving (RX) antennas. Traditional radar imaging techniques, such as backprojection [ 15 , 14 ] , rely on these dense antenna arrays to leverage the numerous TX-RX combinations for precise reconstruction; however, this comes at the expense of significant computational resources. Such methods also often require many distinct transmission frequencies, which limits the sensor’s capture rate and renders the algorithm unsuitable for rapidly moving targets. An alternative research direction is to improve resolution capabilities with sparse, low-cost antenna arrays that can operate at high capture rates. Recent advancements in deep learning focus on smooth neural target representations that surpass the spatial resolution limitations of conventional signal processing techniques. One approach is to employ generative methods, such as implicit neural representations [ 16 ] or conditional generative adversarial networks [ 17 ] , to recover the spatially resolved reflective properties of targets at super-resolution. Another line of work uses Neural Radiance Fields (NeRFs) as compact geometric representations to simulate novel views and synthesize raw frequency-space measurements [ 18 ] or range-Doppler maps [ 19 ] , while also incorporating additional modalities such as LiDAR and cameras [ 20 ] . Due to their data-driven learning processes, these approaches typically require a substantially high number of radar measurements, which currently limits their application primarily to autonomous driving scenarios. Moreover, the training process is computationally intensive. Among traditional approaches, Bräunig et al. [ 21 ] introduced an imaging technique for large, densely-packed antenna arrays of high-resolution MIMO radars, which focuses on high-speed pose tracking of both static and dynamic human hands. The proposed 2FSK method operates using just two neighboring frequencies, based on the principle of continuous-wave (CW) Frequency Shift Keying (FSK). This approach significantly speeds up the reconstruction process, making it up to 1000 times faster than backprojection [ 21 ] , while ensuring rapid capture times due to the limited set of transmitted frequencies. A key limitation of the 2FSK method, however, is its assumption that the depth of the captured object is roughly known. Follow-up work [ 22 ] ( 3FSK ) aims to improve robustness by requiring more complex hardware configurations, that is, a larger signal bandwidth and three representative frequencies with specific frequency displacements, allowing for a less accurate depth prior. The initial scalar depth prior, resembling a plane or depth slice in 3D space, makes both methods suitable primarily for flat targets with limited depth extent, as shown for hands in [ 21 , 22 ] . Consequently, their applicability is restricted to a narrow range of target geometries and can lead to inaccuracies in uncertain environments. Our work extends the 2FSK technique into the multimodal domain, aiming towards a broadly applicable method that offers reliable fast-capture and fast-reconstruction performance with respect to unknown static and dynamic environments. We integrate a secondary depth sensing modality, such as an optical RGB-D camera, to obtain a depth prior. Optical depth sensors offer higher spatial resolution compared to existing imaging radars. However, their temporal resolution is significantly lower. The such obtained depth prior allows our method to handle objects with varying geometries in the depth direction. We refer to this approach as multimodal 2FSK (MM-2FSK). We provide a comprehensive evaluation based on a dataset [ 23 ] that provides ground-truth geometry of static objects, spatially aligned with real-world measurements from a mmWave high-resolution MIMO imaging radar with frequency-stepped continuous-wave (FSCW) signal modulation. In this evaluation, we compare our method with 2FSK, 3FSK, and traditional backprojection. In addition, we investigate the influence of signal bandwidth beyond theoretical analysis and provide ablation studies focused on different frequency configurations. In summary, our contributions are the following: 1. A novel multimodal signal processing method that incorporates a mmWave FSCW MIMO radar along with an optical depth camera as an assistive modality; for evaluation, we use an active stereo RGB-D camera. 2. A method for robust, high-speed radar imaging of arbitrary objects without requiring additional knowledge about the capture environment, i.e. the object position and depth variation over surface. 3. A comprehensive evaluation of various static objects: An ablation study over different frequency configurations and comparison to state-of-the-art radar imaging methods, i.e. 2FSK, 3FSK, and backprojection. Fig. 1: In our work, we extend the 2FSK imaging principle to the multimodal domain (MM-2FSK). Given a unified scalar depth prior, \widetilde{d} , for

Read the complete text