A Probabilistic Generative Model for Spectral Speech Enhancement

Marco Hidalgo-Araya [email protected], Raphaël Trésor, Bart van Erp, Wouter W.L. Nuijten, Thijs van de Laar, Bert de Vries · 2026

Read A Probabilistic Generative Model for Spectral Speech Enhancement by Marco Hidalgo-Araya [email protected], Raphaël Trésor, Bart van Erp, Wouter W.L. Nuijten, Thijs van de Laar, Bert de Vries (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28436.txt.

Introduction

Hidalgo and five co-authors build a hearing aid speech cleaner as one Bayesian generative model. Fixed, hand-tuned settings fail when rooms and listeners change. Their framework treats processing, learning, and personal taste as inference in a single state-space model, with uncertainty tracked throughout. It grows out of spectral subtraction and adapts itself to each user and setting. On the VoiceBank+DEMAND corpus it matched strong rivals with only 85 effective parameters, running in real time.

Catalog record

CREATORMarco Hidalgo-Araya [email protected], Raphaël Trésor, Bart van Erp, Wouter W.L. Nuijten, Thijs van de Laar, Bert de Vries
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE93650
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28436

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Probabilistic Generative Model for Spectral Speech Enhancement Marco Hidalgo-Araya [email protected], Raphaël Trésor, Bart van Erp, Wouter W.L. Nuijten, Thijs van de Laar, Bert de Vries Abstract Speech enhancement in hearing aids remains a difficult task in nonstationary acoustic environments, mainly because current signal processing algorithms rely on fixed, manually tuned parameters that cannot adapt in situ to different users or listening contexts. This paper introduces a unified modular framework that formulates signal processing, learning, and personalization as Bayesian inference with explicit uncertainty tracking. The proposed framework replaces ad hoc algorithm design with a single probabilistic generative model that continuously adapts to changing acoustic conditions and user preferences. It extends spectral subtraction with principled mechanisms for in-situ personalization and adaptation to acoustic context. The system is implemented as an interconnected probabilistic state-space model, and inference is performed via variational message passing in the RxInfer.jl probabilistic programming environment, enabling real-time Bayesian processing under hearing-aid constraints. Proof-of-concept experiments on the VoiceBank+DEMAND corpus show competitive speech quality and noise reduction with 85 effective parameters. The framework provides an interpretable, data-efficient foundation for uncertainty-aware, adaptive hearing-aid processing and points toward devices that learn continuously through probabilistic inference. \svgsetup inkscapeexe=inkscape, inkscapeopt=-o, inkscapeversion=1 1 Department of Electrical Engineering, Eindhoven University of Technology, the Netherlands 2 Lazy Dynamics B.V., Utrecht, the Netherlands 3 GN Advanced Science, Eindhoven, the Netherlands K eywords Bayesian Inference \cdot Belief Propagation \cdot Forney-style Factor Graphs \cdot Hearing Aids \cdot Probabilistic Graphical Models \cdot Speech Enhancement \cdot Variational Inference \cdot Variational Message Passing 1 INTRODUCTION Hearing loss affects an estimated 430 million people worldwide, a number projected to exceed 700 million by 2050; nearly one in ten individuals will require rehabilitation for disabling hearing loss [ 62 , Exec. Summary, pp. 4–8] . The global economic impact of unaddressed hearing loss has been estimated at US$ 980 billion annually [ 42 ] , alongside substantial social costs such as increased risk of isolation, cognitive decline, and reduced quality of life [ 26 , 45 , 34 ] . Despite the scale of the problem, Hearing Aid (HA) adoption remains low: only about 17% of those who could benefit obtain them [ 46 ] , and even among owners, approximately 20% report never using their devices, while another 30% use them only occasionally [ 16 , Fig. 3] . One reason for this limited adoption is inadequate speech enhancement quality: HAs may improve audibility in quiet settings but often fall short in noisy, nonstationary environments where users need support most [ 40 , 39 , 20 ] . This gap is especially apparent in everyday acoustic scenes such as restaurants, public transit, and busy public spaces where noise sources are diverse, nonstationary, and highly unpredictable. Conventional one-size-fits-all processing delivers limited benefit in such conditions, causing many users to underutilize their devices. Fine-tuning of a HA’s sound processing algorithm for one listener in a known acoustic environment is feasible, yet developing a solution that performs satisfactorily for many listeners across diverse acoustic scenes without on-the-fly personal adjustments remains extremely challenging. Unfortunately, manual, real-time parameter tuning imposes a cognitive and practical burden that most users are unwilling to accept, particularly when such interactions do not yield immediate, perceivable improvements. In-situ personalization is therefore essential, but it must impose minimal burden on the HA user. Variants of spectral subtraction currently remain the most popular noise reduction algorithm in commercial HAs, due to their simplicity and low computational cost (i.e. [ 25 , 41 ] ). Still, heuristic parameter tuning makes them vulnerable to audible artifacts in real-world conditions [ 2 ] . More recent noise reduction algorithms based on deep learning, such as Conv-TasNet [ 38 ] , Demucs [ 15 ] , and diffusion-based models [ 52 , 36 ] , outperform spectral subtraction on standard benchmark datasets. However, their large model sizes and computational demands strain the strict latency and energy budgets of HAs. Furthermore, the ICASSP 2023 Clarity Challenge highlighted performance degradation when deep learning systems trained on simulated data were evaluated on real acoustic recordings [ 12 ] . Lightweight architectures such as TinyLSTM [ 19 ] , FSPEN [ 63 ] , and Mamba-SEUNet [ 28 ] demonstrate that streaming operations are possible under hardware constraints, yet these solutions lack mechanisms for in-situ personalization. In this paper, we present a novel framework for hearing aid algorithm design that explicitly tackles the challenges described above. Our long-term objective is to establish a framework for developing HA sound processing algorithms that can be trained both offline , like deep neural networks, and online through in-situ appraisals provided by HA users. In essence, we seek to design HA algorithms with a small computational footprint that can nonetheless adapt on-the-spot to unforeseen performance degradations in real-world listening environments. In Section 2 , we identify the root cause of the problem, namely the absence of a principled approach for selecting and adapting tuning parameters in HA algorithms. We address this fitting problem directly by requiring that parameter settings in an HA algorithm depend not on the skills or insights of an audiologist, but rather quantitatively on (input and/or output) audio signals and user appraisals, i.e., signals that can be observed directly in nature. Our approach is based on a Bayesian modeling framework to enable continual (lifelong) learning, and a principled treatment of uncertainty throughout the entire chain from user feedback to HA output. In this framework, all processing, including offline and online learning as well as signal processing itself, is formulated as Bayesian inference in a probabilistic generative model. To keep the study focused, we select speech enhancement as our first application task, noting that other HA processing functions can be addressed within the same framework. In Section 3 , we derive a modular architecture for a generative speech enhancement model. Section 4 details the technical specifications of the generative model. To ensure a small computational footprint, we draw upon well-established mechanisms from the spectral subtraction literature. Once the model is defined, all processing can be expressed as Bayesian inference tasks. In Section 5 , we discuss the state of the art in efficient automated Bayesian inference for dynamic models. We review message passing-based inference in factor graphs and describe how this framework can be used to automate both signal processing and parameter learning, in both offline and online settings, within our generative model. Section 6 presents validation experiments. Our goal is to establish feasibility: can the HA output signal be inferred automatically through message passing in a factor graph representation of a generative speech enhancement model? Where do the current weaknesses lie, and how can they be addressed in future work? We demonstrate that our model achieves performance in the same ballpark as comparable algorithms while requiring significantly fewer tuning parameters. This compact parameterization is essential for future in-situ personalization, and the results motivate continued development along this line of research. A further review of related work is deferred to Section 7 , where we place our approach in the context of existing methods. Section 8 discusses the broader implications for future HA design and outlines our forward-looking research agenda. 2 PROBLEM STATEMENT Consider a sound processing algorithm y_{t}=f_{w}(x^{t}), (1) where t is a discrete time

Read the complete text