Data-driven identification of nonlinear dynamical systems with LSTM autoencoders and Normalizing Flows

Abdolvahhab Rostamijavanani, Shanwu Li, Yongchao Yang · 2025

Read Data-driven identification of nonlinear dynamical systems with LSTM autoencoders and Normalizing Flows by Abdolvahhab Rostamijavanani, Shanwu Li, Yongchao Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-03977.txt.

Introduction

Abdolvahhab Rostamijavanani, Shanwu Li, and Yongchao Yang present deep learning tools for figuring out the rules of systems that behave in complex, non-linear ways. Straight-line models are simple, but real machines and structures often bend, saturate, or swing in ways such models miss. Learning the true rules from data is key for design, building, and testing.

The study uses two kinds of neural networks. One, built with memory cells, pulls time patterns out of recorded data. The other links those patterns to the unknown settings of the system. Together they form a method that learns a system's behavior from its output alone. The authors frame the work for engineers who must model systems they cannot easily open or probe.

Catalog record

CREATORAbdolvahhab Rostamijavanani, Shanwu Li, Yongchao Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-03977

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Data-driven identification of nonlinear dynamical systems with LSTM autoencoders and Normalizing Flows Abdolvahhab Rostamijavanani, Shanwu Li, Yongchao Yang Abstract While linear systems have been useful in solving problems across different fields, the need for improved performance and efficiency has prompted them to operate in nonlinear modes. As a result, nonlinear models are now essential for the design and control of these systems. However, identifying a nonlinear system is more complicated than identifying a linear one. Therefore, modeling and identifying nonlinear systems are crucial for the design, manufacturing, and testing of complex systems. This study presents using advanced nonlinear methods based on deep learning for system identification. Two deep neural network models, LSTM autoencoder and Normalizing Flows, are explored for their potential to extract temporal features from time series data and relate them to system parameters, respectively. The presented framework offers a nonlinear approach to system identification, enabling it to handle complex systems. As case studies, we consider Duffing and Lorenz systems, as well as fluid flows such as flows over a cylinder and the 2-D lid-driven cavity problem. The results indicate that the presented framework is capable of capturing features and effectively relating them to system parameters, satisfying the identification requirements of nonlinear systems. ∎ 1 Introduction Linear systems have been successfully used to solve various problems across different fields. However, the demand for better performance and efficiency pushes these systems to operate in nonlinear modes, which necessitates the use of nonlinear models for their design and control. Identifying a nonlinear system is a much more intricate process than identifying a linear one. Therefore, it is essential to model and identify nonlinear systems for designing, manufacturing, and testing complex systems. The models for nonlinear systems can be broadly categorized into White Box liu2021dendrite ; giordano2018black ; ljung2010perspectives ; picard2016comparison , Black Box polifke2014black ; gretton2001support ; sjoberg1995nonlinear ; kim2012subspace ; ayala2020nonlinear , and Grey Box kim2012subspace ; pires2023nonlinear ; vizer2013local ; mohammadi2015new ; beneventi2012effective modeling to identify the system schoukens2019nonlinear . White Box modeling is effective but challenging as it requires accurate physics knowledge and is difficult to generalize to various systems. Black Box modeling is an alternative to physics-based modeling and characterizes the model based on experimental data, making it easy to use but less interpretable. Black Box modeling deals with estimating a function by hypothesizing a functional relationship between input and output, making it simpler but difficult to estimate and interpret the parameters. Experiment design becomes more challenging, model selection requires more careful consideration, and parameter estimation is more complex. Therefore, it is important to go beyond traditional linear methods of identification and move towards advanced nonlinear methods of identification. Ideally, a nonlinear system identification approach should be recommended when the function preprocessing step indicates that the distortion levels are excessively high and significantly surpass the data noise floor schoukens2019nonlinear . The Black Box model can be solved using several techniques, both classical petsounis2001parametric and modern juditsky1995nonlinear , such as neural networks. The use of data-driven approaches has become increasingly popular in modeling nonlinear systems due to the advancements in machine learning and deep learning technology. The fundamental architecture of deep learning, known as deep neural networks (DNNs), has proven to be particularly effective in capturing the intrinsic features of complex systems. DNNs offer a remarkable modeling capacity and learning flexibility, allowing them to represent complex relationships in a hierarchical manner. The universal approximation theorem hornik1990universal ; Akhtar2018 is a significant result in the field of deep learning. It states that a DNN with an adequate number of neural units and nonlinear activations can represent a wide range of intricate functions. For example, DNNs have the capability to learn and represent complex nonlinear relationships between input and output variables. DNNs also provide the flexibility of adapting the network architecture to different tasks. This means that the architecture of the DNN can be tailored to the specific characteristics of the problem at hand, such as the identification of nonlinear dynamics. This adaptive design of the network architecture enables DNNs to achieve high accuracy in complex modeling tasks, making them a powerful tool in the field of nonlinear systems. LSTM autoencoder and Normalizing Flows are two deep learning models that can be used for nonlinear system identification, each with its unique features. LSTM autoencoder lazzara2022surrogate ; simpson2021machine ; xu2019multitask can be used to extract temporal features of dynamical systems from time series data, which can then be used for further analysis or modeling. This approach can be useful in various applications such as signal processing, system identification, and anomaly detection. Normalizing Flows urain2020imitationflow ; kobyzev2020normalizing ; khader2021learning ; sun2022probabilistic is a class of generative models that aims to learn the underlying probability distribution of a given dataset. By using Normalizing Flows, it is possible to capture the underlying probability distribution of the data and use it to generate new data samples or infer the latent variables that explain the observed data. By combining these two models (creating a black-box), we can extract meaningful temporal features from the time series data using LSTM autoencoder and use Normalizing Flows to map these features to system parameters. This can facilitate nonlinear system identification in various applications such as control, prediction, and modeling of complex systems Therefore, the primary objective of this study is to create a deep learning framework capable of identifying nonlinear systems. The framework is designed to address two main challenges. The first challenge is the extraction of features from the time series data (i.e., response data) of the dynamical system using a LSTM-encoder. The second challenge involves relating these features to system parameters using a Normalizing Flows framework.To put it differently, the black box consists of two models. The first model is responsible for capturing time-related features of the data (e.g., signal frequency), while the second model establishes a relationship between these features and system parameters. By utilizing deep learning, this process guarantees nonlinear transformations, ensuring that the system identification is based on a nonlinear approach. 2 Methodology: Normalizing Flows concept Normalizing Flows is a generative model urain2020imitationflow ; kobyzev2020normalizing ; khader2021learning ; sun2022probabilistic that efficiently and accurately estimates density by producing tractable distributions. In contrast, GANs goodfellow2020generative ; ogunmolu2016nonlinear ; gedon2021deep and VAEs kingma2019introduction ; ma2020structural ; yan2021deep do not explicitly learn the probability function of training data. Instead, GANs generate similar data to deceive the discriminator in a min-max game until it reaches a saturation point where it cannot differentiate between real and fake samples. VAEs, on the other hand, learn to identify variational inference in latent space and generate data using a decoder. However, neither GANs nor VAEs can learn the real probability density functions (PDFs) of real data. Normalizing Flows, a rigorous generative method, learns the real PDF of a dataset by using invertible and differentiable functions. To create a random variable \mathcal{X} with a complicated distribution \mathcal{P} , Normalizing Flows applies an invertible and differentiable function ( f ) to a random variable \mathcal{Z} with a simple distribution, such as a standard normal distribution \mathcal{Z}\sim N(0,1) using this formula: \mathcal{X}=f(\mathcal{Z})\sim\mathcal{P} and \mathcal{Z}=f^{-1}(\mathcal{X}) . The change of variables method can be used to calculate the transferred distribution \mathcal{P} as follows: log\hskip 2.84544pt\mathcal{P}(\mathcal{X})=log\hskip 2.84544pt\mathcal{P}(\mathcal{Z})-log\hskip 2.84544pt|\frac{\partial f}{\partial\mathcal{Z}}(\mathcal{Z})| (1)

Read the complete text