Efficient Approximation of Volterra Series for High-Dimensional Systems

Navin Khoshnan, Claudia K Petritsch, Bryce Allen Bagley · 2025

Read Efficient Approximation of Volterra Series for High-Dimensional Systems by Navin Khoshnan, Claudia K Petritsch, Bryce Allen Bagley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-06527.txt.

Introduction

Navin Khoshnan, Claudia Petritsch, and Bryce-Allen Bagley make Volterra series modeling feasible for very high-dimensional nonlinear systems. Volterra series describe a system through layered kernels, but the cost explodes with input dimension. Tensor network solvers eased that burden yet still scale badly in both memory and computation.

Their Tensor Head Averaging algorithm trains an ensemble of small tensor network models, each seeing only a slice of the input space. Theory gives finite-sample error bounds against the full model and an exact split of the squared error. Correlation between included and omitted dynamics turns out to help the ensemble, pushing accuracy beyond what plain truncation of the full model would give.

Catalog record

CREATORNavin Khoshnan, Claudia K Petritsch, Bryce Allen Bagley
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-06527

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Approximation of Volterra Series for High-Dimensional Systems Navin Khoshnan, Claudia K Petritsch, Bryce Allen Bagley Abstract The identification of high-dimensional nonlinear dynamical systems via the Volterra series has significant potential, but has been severely hindered by the curse of dimensionality. Tensor Network (TN) methods such as the Modified Alternating Linear Scheme (MVMALS) have been a breakthrough for the field, offering a tractable approach by exploiting the low-rank structure in Volterra kernels. However, these techniques still encounter prohibitive computational and memory bottlenecks due to high-order polynomial scaling with respect to input dimension. To overcome this barrier, we introduce the Tensor Head Averaging (THA) algorithm, which significantly reduces complexity by constructing an ensemble of localized MVMALS models trained on small subsets of the input space. In this paper, we present a theoretical foundation for the THA algorithm. We establish observable, finite-sample bounds on the error between the THA ensemble and a full MVMALS model, and we derive an exact decomposition of the squared error. This decomposition is used to analyze the manner in which subset models implicitly compensate for omitted dynamics. We quantify this effect, and prove that correlation between the included and omitted dynamics creates an optimization incentive which drives THA’s performance toward accuracy superior to a simple truncation of a full MVMALS model. THA thus offers a scalable and theoretically grounded approach for identifying previously intractable high-dimensional systems. 1 Introduction The identification and modeling of high-dimensional nonlinear dynamical systems represents a significant challenge across numerous domains in science and engineering. From understanding physiological systems to advanced control engineering and signal processing, [ 16 , 22 , 15 , 17 , 26 , 9 , 10 , 25 , 1 , 6 , 14 , 20 ] the ability to characterize and classify input-output relationships without relying on linearization is helpful for capturing various phenomena where memory and nonlinearity are central. For discrete-time, causal, time-invariant systems with fading memory, the Volterra Series (VS) offers a natural extension of linear system theory. [ 23 , 5 , 20 , 4 ] Serving as a functional analogue to the Taylor expansion for static functions, the VS generalizes the convolution integral, representing the system output as a sum of multidimensional convolutions. The characterizing functions of these convolutions, known as Volterra kernels, generalize the linear impulse response of a system to capture higher-order interactions and memory effects. Based on generalizations of the Stone-Weierstrass theorem to functionals, the VS provably acts as a universal approximator. [ 5 , 23 , 20 ] This structure provides a nonparametric representation capable of approximating a variety of nonlinear behaviors, while retaining greater interpretability compared to black-box methods. However, the practical application of the VS has been severely limited by the curse of dimensionality. For a Multiple-Input Multiple-Output (MIMO) system with p inputs, memory M , and system degree d , The number of parameters required to characterize the kernels scales exponentially as O((pM)^{d}) . This explosion in dimensionality creates challenges across the identification process. Computationally, the time complexity of traditional identification algorithms that operate on the full parameter space, such as direct least squares estimation, scales exponentially with degree d . Memory requirements for storing such large kernels quickly become prohibitive, and the resultant exponential complexity renders traditional VS identification methods intractable for all but the smallest systems. The recognition that Volterra kernels are inherently high-order tensors has led to a shift in modeling approaches through the introduction of Tensor Network (TN) methods. [ 13 , 7 ] TN techniques, such as the Tensor Train (TT) format introduced by Oseledets, [ 18 ] exploit the low-rank structure often present within these kernels to achieve significant compression. This approach allows for a parametrization that scales linearly rather than exponentially, with the degree d , effectively mitigating the curse of dimensionality associated with the system’s size and complexity. The Modified Alternating Linear Scheme (MVMALS) algorithm, adapted for Volterra series identification by Batselier et al., [ 3 ] provides an efficient and numerically stable method for identifying VS models of systems within this compressed TN framework. Despite these important advances, a critical bottleneck still remains for systems with high input dimension. While the exponential dependence on d is resolved, the time and memory complexity of MVMALS scales with the input space polynomially as p^{6} . In areas characterized by large input dimensionality such as biological systems or large-scale sensor arrays, this high-order polynomial complexity remains prohibitive. To address this scalability gap, we introduce the Tensor Head Averaging (THA) algorithm. THA utilizes a structure analogous to the MapReduce paradigm [ 8 ] in order to drastically reduce complexity for systems with large input dimensionality. Similar to the “map” phase, THA constructs an ensemble of smaller, localized TN models known as heads, where each head is trained via MVMALS on a distinct small subset of the input variables. The final prediction is obtained by averaging the outputs of these individual heads, corresponding to the “reduce” phase. This approach replaces the dependence on the full input dimension p with one on the much smaller subset size k\ll p , enabling the analysis of systems previously considered to be intractable. A phenomenon inherent to this localized approach is a form of omitted variable bias, which we term “baking.” When a subset model is optimized during training, it does not merely capture the truncated dynamics corresponding to the included variables. Instead, the optimization process inherently adjusts the parameters of the subset model to compensate for the dynamics of the omitted variables, provided there is correlation in the data. This process effectively “bakes” the influence of the global system dynamics into the localized model parameters. Understanding this mechanism is necessary to properly characterize the approximation capabilities of the THA ensemble. This paper establishes the theoretical results for the THA algorithm, focusing on the approximation error of the THA ensemble relative to a full MVMALS model, and the optimization dynamics that induce baking. The primary contributions of this paper are summarized as follows: 1. Observable Error Bound: We establish a numerically computable bound on the error between the THA prediction and that of a full MVMALS model. The bound is in a finite-sample setting without assumptions of convexity or access to global optimizers. This bound is not an a priori error prediction, but rather allows one to understand the sources of error in a THA approximation. 2. Geometric Error Decomposition and Baking Gain: Using the geometry induced by the empirical Frobenius norm, we derive an exact decomposition of the squared error. 3. Optimization Behavior and the Incentive for Baking: We analyze the MVMALS optimization process within the THA context. We prove that under standard regularity conditions aligned with MALS theory, the correlation between included and omitted dynamics creates a gradient incentive that drives the optimization away from a simple truncation, provided this correlation is not orthogonal to the tangent space of the TN manifold. We further show that the condition preventing this incentive is a measure-zero event. 4. Computational Complexity: We analyze the computational complexity of THA, demonstrating a significant reduction in computational cost compared to the full MVMALS approach. Section 2 establishes the necessary notation and background on Volterra series and tensor networks. Section 3 formally defines

Read the complete text