Bayesian Modeling and Estimation of Linear Time-Variant Systems using Neural Networks and Gaussian Processes

Yaniv Shulman · 2025

Read Bayesian Modeling and Estimation of Linear Time-Variant Systems using Neural Networks and Gaussian Processes by Yaniv Shulman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-12878.txt.

Introduction

Yaniv Shulman, in a solo paper from twenty twenty-five, framed a Bayesian way to model systems that change over time. The task is to read a linear time varying system from its in and out data. That task is ill posed: many systems fit the same reads. Shulman treats the system's impulse response as a chance process. It splits into a mean part and a random wobble part.

That split gives a clean read of doubt, both from the channel's own swings and from lack of knowledge. It also names a new class: systems that are time invariant in expectation. For the math, he uses Bayesian neural nets and Gaussian processes with scaled inference. Tests on worked cases show the frame in action.

Catalog record

CREATORYaniv Shulman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-12878

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bayesian Modeling and Estimation of Linear Time-Variant Systems using Neural Networks and Gaussian Processes Yaniv Shulman Abstract The identification of Linear Time-Variant (LTV) systems from input-output data is a fundamental yet challenging ill-posed inverse problem. This work introduces a unified Bayesian framework that models the system’s impulse response, h ⁡ ( t , τ ) h(t,\tau) , as a stochastic process. We decompose the response into a posterior mean and a random fluctuation term, a formulation that provides a principled approach for quantifying uncertainty and naturally defines a new, useful system class we term Linear Time-Invariant in Expectation (LTIE). To perform inference, we leverage modern machine learning techniques, including Bayesian neural networks and Gaussian Processes, using scalable variational inference. We demonstrate through a series of experiments that our framework can robustly infer the properties of an LTI system from a single noisy observation, show superior data efficiency compared to classical methods in a simulated ambient noise tomography problem, and successfully track a continuously varying LTV impulse response by using a structured Gaussian Process prior. This work provides a flexible and robust methodology for uncertainty-aware system identification in dynamic environments. 1 Introduction Linear Time-Variant (LTV) systems are fundamental to modeling dynamic processes in fields ranging from geophysics and communications to control theory ( Kozachek et al. 2024 ; Lin et al. 2020 ) . Unlike their time-invariant counterparts, an LTV system’s behavior is described by an impulse response, h(t,\tau) , that changes over time, posing significant challenges for analysis and estimation ( Kailath 1962 ; Bello 1963 ) . The task of identifying h(t,\tau) from input-output data is a severely ill-posed inverse problem, as one must infer a function of two variables from one-dimensional time series ( Aubel and Bölcskei 2015 ) . This work introduces a Bayesian framework for modeling such systems, where the inherent uncertainty and time-varying nature are captured probabilistically. Our central approach is to treat the impulse response itself as a stochastic process. We reparameterize the LTV impulse response into two components: a deterministic posterior mean, which represents the system’s average behavior conditioned on observations, and a zero-mean stochastic process, which captures the random fluctuations or innovations. This decomposition, h=\mu+\mathcal{E} , provides a powerful and generalizable modeling framework. It allows us to unify the description of a wide spectrum of system behaviors, from purely deterministic systems to fully random ones, under a single probabilistic lens. Within this framework, we define a new and useful class of systems, which we term Linear Time-Invariant in Expectation (LTIE). In an LTIE system, the mean impulse response is constant, \mu(t,\tau)=\mu(\tau) , but the system still exhibits stochastic fluctuations. This concept is related to, but more general than, the "mean LTI" or "line-of-sight" component in channel models like Rician fading, as our framework allows for arbitrary, non-stationary covariance structures for the fluctuations around this constant mean. The LTIE concept provides a crucial conceptual bridge, as it perfectly describes the posterior distribution obtained from performing Bayesian inference on a classic, deterministic LTI system. In this context, our uncertainty about the true impulse response is captured precisely by the stochastic term \mathcal{E} . The novelty of this work lies in leveraging this generalized framework with modern machine learning tools, namely Bayesian neural networks and Gaussian processes, to perform robust, uncertainty-aware estimation of the impulse response. In the following sections, we develop this framework for both continuous and discrete-time systems, deriving key statistical properties and discussing their implications. 2 Related Work The conceptual approach of this work, which models the impulse response as a stochastic process, builds upon the seminal statistical characterization of randomly time-variant linear channels by Bello ( Bello 1963 ) . This foundational research led to widely used models such as the Wide-Sense Stationary Uncorrelated Scattering (WSSUS) channel, which describes a purely random, zero-mean channel process corresponding to the case where \mu=0 in our framework ( Yoo and Stark 2005 ) . The Rician fading model is another well-known special case that includes a deterministic, line-of-sight component alongside random, scattered components, which is conceptually analogous to our decomposition of the impulse response h into a non-zero mean component \mu and a stochastic fluctuation \mathcal{E} ( Rice 1945 ; Lindsey 1964 ) . Our framework provides a generalization of these classic models by accommodating a time-varying mean and arbitrary, non-stationary covariance structures. Modern Bayesian system identification has moved towards placing priors directly on the impulse response itself. A significant development in this area is kernel-based regularization, where the impulse response is modeled as a draw from a Gaussian Process (GP) ( Pillonetto and Ljung 2023 ; Darwish 2017 ) . This function-space perspective allows prior knowledge to be encoded via the GP’s covariance function, or kernel. For instance, specialized "stable kernels" have been designed to enforce physical properties like smooth exponential decay, providing a principled form of regularization that ensures the stability of the identified model ( Pillonetto and Ljung 2023 ) . The flexibility of machine learning has also been brought to bear on this problem. Neural networks (NNs) are widely used as universal function approximators for nonlinear systems, often in autoregressive configurations like NARX ( Narendra and Parthasarathy 1990 ; Nelles 2020 ) . To address the fact that standard NNs provide only point estimates, Bayesian Neural Networks (BNNs) have been developed, which place prior distributions on network weights to infer a full posterior distribution over models, thereby capturing model uncertainty ( Jospin et al. 2022 ; Xu et al. 2021 ) . Physics-Informed BNNs (BPINNs) extend this by incorporating governing physical equations into the loss function, a technique that has proven robust for system identification in the presence of noise ( Stock et al. 2024 ) . The synergy between GPs and NNs is an active area of research. While some work focuses on constructing deep hierarchies of GP mappings (Deep GPs) ( Damianou and Lawrence 2013 ) or designing BNNs that replicate GP priors ( Sendera et al. 2025 ) , other approaches use GPs as structured priors within a larger Bayesian model to capture temporal dependencies. For example, recent work has combined GP priors on system states with dynamics governed by Neural Ordinary Differential Equations (ODEs) ( Bhouri and Perdikaris 2022 ) , or used spatio-temporal GP kernels to model time-varying functions for Bayesian optimization ( Li et al. 2025 ) . This work adopts a similar philosophy, using a GP as a structured prior to regularize the estimation of a continuously time-varying impulse response within a BNN framework. The practical application of these advanced Bayesian models is made possible by approximate inference techniques, as the true posterior distribution is almost always intractable. Variational Inference (VI) has emerged as a computationally efficient and scalable alternative to sampling-based methods, and has proven highly effective for deconvolution problems, which are mathematically equivalent to the impulse response estimation task addressed in this paper ( Zhang and Kingsbury 2013 ; Susik and Sbalzarini 2023 ; Jospin et al. 2022 ) . In the following sections, we develop this framework for both continuous and discrete-time systems, validate its performance through a series of increasingly complex estimation tasks, and conclude with a discussion of its implications and avenues for future work. 3 A Bayesian

Read the complete text