Generalization Analysis and Method for Domain Generalization for a Family of Recurrent Neural Networks
Atefeh Termehchi, Ekram Hossain, Isaac Woungang · 2026
Read Generalization Analysis and Method for Domain Generalization for a Family of Recurrent Neural Networks by Atefeh Termehchi, Ekram Hossain, Isaac Woungang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08122.txt.
Introduction
Atefeh Termehchi, Ekram Hossain, and Isaac Woungang analyze domain generalization for recurrent neural networks. Their 2026 arXiv paper notes that deep learning models often lack interpretability and fail on new data distributions. That weakness undermines trust, above all in safety-critical uses. Interest has grown in both explaining generalization and building models that hold up off-distribution. The authors give a generalization analysis and a method for a family of recurrent networks. Recurrent models process sequences, so their behavior across domains matters for speech, text, and sensor data.
Catalog record
| CREATOR | Atefeh Termehchi, Ekram Hossain, Isaac Woungang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08122
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalization Analysis and Method for Domain Generalization for a Family of Recurrent Neural Networks Atefeh Termehchi, Ekram Hossain, Isaac Woungang Abstract Deep learning (DL) has driven broad advances across scientific and engineering domains. Despite its success, DL models often exhibit limited interpretability and generalization, which can undermine trust, especially in safety-critical deployments. As a result, there is growing interest in (i) analyzing interpretability and generalization and (ii) developing models that perform robustly under data distributions different from those seen during training (i.e. domain generalization). However, the theoretical analysis of DL remains incomplete. For example, many generalization analyses assume independent samples, which is violated in sequential data with temporal correlations. Motivated by these limitations, this paper proposes a method to analyze interpretability and out-of-domain (OOD) generalization for a family of recurrent neural networks (RNNs). Specifically, the evolution of a trained RNN’s states is modeled as an unknown, discrete-time, nonlinear closed-loop feedback system. Using Koopman operator theory, these nonlinear dynamics are approximated with a linear operator, enabling interpretability. Spectral analysis is then used to quantify the worst-case impact of domain shifts on the generalization error. Building on this analysis, a domain generalization method is proposed that reduces the OOD generalization error and improves the robustness to distribution shifts. Finally, the proposed analysis and domain generalization approach are validated on practical temporal pattern-learning tasks. I Introduction Over the past decade, deep learning (DL) has driven impressive progress across scientific and engineering domains such as physics, robotics, and communications, to name a few. DL advances these areas by providing data-driven solutions when analytical modeling or optimization is difficult to formulate, relies on oversimplified assumptions, or becomes computationally intractable. However, the concerns about DL models have increasingly come to the forefront because they are frequently regarded as “non-transparent” [ 22 , 19 , 27 ] . In particular, they are viewed as non-transparent since, even with full access to the parameters (weights and biases) and activation functions, it remains difficult to determine what knowledge or decision rules the DL model has internalized. This is largely due to the repeated application of nonlinear transformations within the model. This opacity brings two connected challenges: limited interpretability and limited generalization beyond the training data. Interpretability refers to the extent to which a human can understand the reasons behind a model’s predictions, i.e. what information the model uses and how it maps the inputs to the outputs. Although the literature offers many perspectives, a widely cited definition describes interpretability as the ability to provide explanations in human-understandable terms [ 28 ] . On the other hand, generalization refers to a model’s ability to maintain a reliable performance on unseen data beyond the training set. It is often discussed in terms of in-distribution generalization, where the target data follow (approximately) the training distribution, and out-of-distribution (OOD) generalization, where the target data come from different distributions. Moreover, as a closely related concept, domain generalization (DG) aims to learn models whose performance remains robust under such distribution shifts. In contrast to other learning paradigms such as domain adaptation or transfer learning, DG assumes that the target-domain data is unavailable during training [ 29 ] . The limited interpretability and generalization of DL models undermine trust and impede their adoption in high-stakes settings such as healthcare, smart power systems, and security-critical infrastructure. In this context, model failures can lead to serious safety risks and substantial financial losses [ 28 ] . Therefore, many research efforts focus on analyzing interpretability and generalization of DL models, as well as developing DG methods [ 27 , 9 , 29 ] . However, the field is still not mature. For interpretability, some methods face practical limitations (e.g., rule- or logic-based explanations), and many approaches target only one facet of transparency, either by explaining input-output attributions (e.g., perturbation-based tests) or by analyzing internal mechanisms (e.g., information-bottleneck views). For generalization analysis, major gaps exist in settings with dependent data (e.g., sequential data), where temporal correlations and anytime-valid guarantees are required. Although some progress has been made, the area is in its early development, and many challenges remain to be addressed [ 9 , 17 ] . Sequential data is ubiquitous and underpins decision-making in many sectors, including healthcare and energy systems. For modeling such sequences and supporting decisions in tasks with temporal patterns, recurrent neural networks (RNNs) have long been a core architecture. They have been widely used in applications such as power-load forecasting, speech recognition, and video/temporal vision tasks, since they provide a simple and efficient way to process time series. Consequently, this paper introduces an interpretable representation of RNNs (with a focus on Long Short-Term Memory [LSTM]) that captures both input–output attributions and internal mechanisms. Specifically, we model the dynamics of the LSTM internal states and their contributions to the output as an unknown, discrete-time, nonlinear closed-loop feedback system. To analyze these unknown dynamics, we leverage the Koopman operator theory and its data-driven identification capability. We approximate the Koopman spectral characteristics using the dynamic mode decomposition with control (DMDc). Domain shifts are modeled as additive disturbances on the inputs. We then use the \text{H}_{\infty} norm to quantify the worst-case impact of such disturbances on the trained LSTM and to derive generalization-error characterizations under both covariate shift and concept drift. Building on these analyses, we propose a DG method that reduces the OOD generalization error of a trained LSTM under the input distribution shifts. Specifically, we formulate an optimization problem with linear matrix inequality (LMI) constraints that minimizes the effect of domain shift on the LSTM state dynamics, thereby improving the robustness to distribution changes. I-A Motivation and Prior Work Interpretability analyses seek to make the decision-making behavior of DL models more transparent by moving beyond accuracy-focused evaluation toward understanding how and why decisions or predictions are produced. Although a substantial body of work has proposed interpretability analysis methods for DL models (see [ 27 , 28 ] , and the references therein), the field remains immature. This immaturity is reflected in several limitations. First, approaches that produce human-readable explanations (e.g., rule- or decision-tree-based methods such as [ 1 ] ) can become impractical when the explanation complexity is high; thus, complexity must be carefully controlled (e.g., by limiting decision-tree depth). Second, many methods provide only local explanations for a target input (e.g., based on feature values, saliency scores, or gradients), offering limited global insight into the model’s overall behavior. For example, LIME [ 16 ] provides local interpretability. Although Global–LIME [ 11 ] improves the stability and consistency of LIME’s local explanations, it still mainly explains input–output attributions and does not capture internal mechanisms. Third, many methods target only one facet of transparency, focusing either on input–output relationships or internal mechanisms (e.g., mechanistic interpretability [ 2 ] or information-bottleneck-based views [ 8 ] ). In addition, generalization analysis studies how a model’s performance changes on unseen data. It is closely related to interpretability, since understanding how and why a model makes decisions helps assess when that model will remain reliable. Generalization analysis is commonly divided into in-distribution (unseen data follows roughly the training distribution) and out-of-distribution (unseen data comes from a different distribution). While in-distribution generalization analysis (under the independent and identically distributed [i.i.d.] assumption) still faces