Deep Generative Models for Bayesian Inference on High-Rate Sensor Data: Applications in Automotive Radar and Medical Imaging

Tristan S.W. Stevens 1, Jeroen Overdevest 1,2, Oisín Nolan 1, Wessel L. van Nierop 1, Ruud J.G. van Sloun 1, Yonina C. Eldar 3 · 2025

Read Deep Generative Models for Bayesian Inference on High-Rate Sensor Data: Applications in Automotive Radar and Medical Imaging by Tristan S.W. Stevens 1, Jeroen Overdevest 1,2, Oisín Nolan 1, Wessel L. van Nierop 1, Ruud J.G. van Sloun 1, Yonina C. Eldar 3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12154.txt.

Introduction

Tristan Stevens and five co-authors survey how deep generative models serve Bayesian inference on fast sensor streams in this 2025 paper. Such models were built for photos, but radar and medical sensors differ: signals span huge ranges, arrays interfere, and data arrives at high rates. Physics models are also approximate, skewing the error statistics.

The authors discuss these challenges and lay out approaches for each. Applications in car radar and medical imaging anchor the discussion. Real-time use adds strict limits on delay and throughput. The paper maps what must change before image-born generative tricks work on raw sensor data. It aims to guide researchers moving Bayesian methods into live sensing systems.

Catalog record

CREATORTristan S.W. Stevens 1, Jeroen Overdevest 1,2, Oisín Nolan 1, Wessel L. van Nierop 1, Ruud J.G. van Sloun 1, Yonina C. Eldar 3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12154

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deep Generative Models for Bayesian Inference on High-Rate Sensor Data: Applications in Automotive Radar and Medical Imaging Tristan S.W. Stevens 1, Jeroen Overdevest 1,2, Oisín Nolan 1, Wessel L. van Nierop 1, Ruud J.G. van Sloun 1, Yonina C. Eldar 3 Abstract Deep generative models have been studied and developed primarily in the context of natural images and computer vision. This has spurred the development of (Bayesian) methods that use these generative models for inverse problems in image restoration, such as denoising, inpainting, and super-resolution. In recent years, generative modeling for Bayesian inference on sensory data has also gained traction. Nevertheless, the direct application of generative modeling techniques initially designed for natural images on raw sensory data is not straightforward, requiring solutions that deal with high dynamic range signals acquired from multiple sensors or arrays of sensors that interfere with each other, and that typically acquire data at a very high rate. Moreover, the exact physical data-generating process is often complex or unknown. As a consequence, approximate models are used, resulting in discrepancies between model predictions and the observations that are non-Gaussian, in turn complicating the Bayesian inverse problem. Finally, sensor data is often used in real-time processing or decision-making systems, imposing stringent requirements on, e.g., latency and throughput. In this paper, we will discuss some of these challenges and offer approaches to address them, all in the context of high-rate real-time sensing applications in automotive radar and medical imaging. \titlehead Research \subject signal processing, electrical engineering, artificial intelligence \corres Ruud J.G. van Sloun {fmtext} keywords deep generative models, Bayesian inference, inverse modeling, active inference, medical imaging, ultrasound, MRI, radar, diffusion models 1 Introduction Active array sensing techniques are at the core of numerous advanced technologies, playing a critical role in fields such as automotive radar and medical imaging. These sensor arrays consist of multiple individual elements that actively emit signals and capture the reflected waves (sound or light) to obtain accurate representations of the scenery. The culmination of many sensory signals leads to massive data rates and corresponding challenges [ 1 , 2 , 3 ] . These systems are required to provide high-resolution and real-time imaging, all while dealing with interference, noise and a changing environment. Many of the challenges faced in array sensing can be effectively reformulated as inverse problems, which seek to estimate unknown parameters from observations. In the context of array sensing, this typically means reconstructing images or (distance, velocity or angular) measures from the raw data captured by the sensors. However, due to the underdetermined nature of these inverse problems, strong priors and knowledge of the underlying processes at hand are key for reliable reconstructions. This necessitates advanced modeling techniques that extract and capture meaningful information from the raw sensory data. Recent advances in Deep Generative Models (DGMs) have shown great potential in solving problems with high-dimensional data by learning and exploiting the data manifold in domains ranging from medical imaging [ 4 ] , computer vision [ 5 ] and natural language processing [ 6 ] . Specifically, they seek to model the distribution of data and subsequently sample from it, which can serve as a signal prior and aid in the inverse problem solving. However, applying these techniques directly to raw sensory data presents additional challenges due to the high dynamic range (HDR) and rapid data acquisition rates, which impose stringent requirements on latency and throughput, further complicating the use of DGMs . Consequently, these challenges necessitate the development of tailored techniques involving generative models that effectively manage the unique characteristics of sensory data while adhering to the requirements of high-rate real-time sensing applications. In this paper, we review ongoing work in application of deep generative models to sensory data. We start with some background on sensing applications, DGMs and finally Bayesian inference with DGMs in Section 2 . Then, we discuss the two main challenges, namely model mismatches and real-time inference in Sections 3 and 4 , respectively. In both of these sections, we discuss methods that mitigate these challenges, ranging from improved forward models with the use of DGMs , to accelerated inference techniques through approaches such as deep unfolding , temporal inference and active compressed sensing . 2 Background 2.1 Sensing applications Many sensor modalities, such as medical imaging and radar sensing, share a common objective, namely the measurement of an unknown channel impulse response. These channels are often deeply complex, with signals exhibiting significant dynamic range variations in combination with high sample rates. These characteristics of sensory data challenge accurate signal recovery. For instance, in both ultrasound imaging and radar, the generative model of the sensory data is not exactly known due to the highly variable reflectivity of objects within the sensor’s line of sight and the presence of multipath propagation. Extracting a signal-of-interest from raw data is a challenging task for existing model-based techniques, e.g., due to the lack of modeling capacity, non-Gaussian (structured) noise or other undesirable sensing phenomena such as interference or aberration. Another challenge faced when dealing with sensory data are high data rates, for example in real-time imaging applications. Compressed Sensing (CS) [ 7 , 8 , 9 , 10 ] has emerged as a powerful technique for reducing data rates by compressing the size of measurement vector {\mathbf{y}} necessary to recover the signal-of-interest {\mathbf{x}} , through intelligent design of the sensing matrix {\mathbf{A}} and exploitation of known signal statistics. A common application of CS is via subsampling, which aims to recover the full signal-of-interest from a subset of possible measurements, typically fewer than required by the Nyquist-Shannon theorem. This has been successfully applied to reducing data rates in many domains, such as ultrasound imaging [ 11 , 12 , 13 ] , radar [ 14 , 15 ] , and Magnetic Resonance Imaging (MRI) [ 16 ] . Throughout this work, we refer to the following forward model: \displaystyle{\mathbf{y}}={\mathbf{A}}{\mathbf{x}}+{\mathbf{n}}+{\boldsymbol{\epsilon}}\ ,\qquad\qquad{\mathbf{x}}\in{\mathbb{R}}^{M},\,\,\left\{{\mathbf{y}},{\mathbf{n}},{\boldsymbol{\epsilon}}\right\}\in{\mathbb{R}}^{N},\,\,{\mathbf{A}}\in{\mathbb{R}}^{N\times M} (1) where the observations {\mathbf{y}} are contaminated with structured noise {\mathbf{n}} and thermal noise {\boldsymbol{\epsilon}} , respectively. The thermal noise is assumed to be AWGN , i.e., {\boldsymbol{\epsilon}}\sim\mathcal{N}(\mathbf{0},\sigma_{\epsilon}^{2}{\mathbf{I}}) where \sigma_{\epsilon}^{2} represents its variance. In contrast, structured noise encapsulates all model errors (mismatch), including possibly non-linear effects, distortions and multipath components in the observations {\mathbf{y}} . The nature of {\mathbf{n}} depends on the sensing scenario. In some applications, {\mathbf{n}} may be independent of {\mathbf{x}} , such as structured interference from externals sources, commonly encountered in automotive applications of radar. However, in other scenarios, such as diffraction or multipath scattering in ultrasound imaging, {\mathbf{n}} is inherently a function of {\mathbf{x}} , e.g. {\mathbf{n}}({\mathbf{x}}) . Explicitly modeling (and performing inference on) the true forward physics model of {\mathbf{n}} is often challenging. Given the complexity of capturing potential dependencies between {\mathbf{n}} and {\mathbf{x}} , we assume independence, i.e., p({\mathbf{n}}|{\mathbf{x}})\approx p({\mathbf{n}}) and instead learn the marginal distribution p({\mathbf{n}}) in a fully data-driven fashion, as discussed in Section 3 . For our goal of inferring the underlying signal-of-interest {\mathbf{x}} from observations {\mathbf{y}} , we make use of the forward model in ( 1 ) combined with statistical priors. To establish these priors, we resort to Deep Learning (DL) which has been proven to be effective for tasks that require accurate statistical models learned from the data itself.

Read the complete text