On the ability of Deep Neural Networks to Learn Granger Causality in Multi-Variate Time Series Data.

Malik Shahid Sultan, Hernando Ombao · 2025

Read On the ability of Deep Neural Networks to Learn Granger Causality in Multi-Variate Time Series Data. by Malik Shahid Sultan, Hernando Ombao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20347.txt.

Introduction

Malik Shahid Sultan, Hernando Ombao, and Maurizio Filippone asked in 2025 whether a neural network can learn cause-like links in time data on its own. Granger causality marks a link when past values of one series help guess the next values of another. Classic tools for it are straight-line models that miss curved links. Past network work treated the task as picking which inputs to keep.

Their tack is to train one network on all series at once, then drop one series' past and watch how the errors shift. A well-tuned network can find the true link web with no special loss terms, they show. Tests set the single joint model against sparse methods and three network types. It held up as a strong, low-fuss base line.

Catalog record

CREATORMalik Shahid Sultan, Hernando Ombao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-20347

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the ability of Deep Neural Networks to Learn Granger Causality in Multi-Variate Time Series Data. Malik Shahid Sultan, Hernando Ombao Abstract Granger Causality (GC) offers an elegant statistical framework to study the association between multivariate time series data. Linear Vector Autoregressive models (VAR) though have nice interpretation properties but have limited practical application due to underlying assumptions on the kind of associations that can be captured by these models. Numerous attempts have already been made in the literature that exploit the functional approximation power of Deep Neural Networks (DNNs) for the task of GC estimation. These methods however treat GC as a variable selection problem. We present a novel paradigm for approaching GC. We present this idea that GC is essentially linked with prediction and if a deep learning model is used to model the time series collectively or jointly, a well regularized model may learn the true granger causal structure from the data, given that there is enough training data. We propose to uncover the learned GC structure by comparing the model uncertainty or distribution of the residuals when the past of everything is used as compared to the one where a specific time series component is dropped from the model. We also compare the effect of input layer dropout on the ability of a neural network to learn granger causality from the data. We show that a well regularized model infact can learn the true GC structure from the data without explicitly adding terms in the loss function that guide the model to select variables or perform sparse regression. 1 Introduction Neural Networks are power machine learning models that learn data dependent transformations. Deep Neural Networks (DNNs) excel at predictive modeling tasks, recent advances in Natural Language Processing (NLP) and Computer Vision (CV) demonstrate tremendous predictive and representation learning power of these models. Time series data comprise of discrete measurements of stochastic processes. While analyzing multi-variate time series data one maybe interested in predictive modeling or understanding the dependence structure in the data. Understanding the structure in the data is important for various reasons one of which is explainability. Estimation and analysis of dependence structure is of great importance especially in healthcare data where the objective of the study can be to discover a bio-marker that differentiates between groups and subgroups. There are existing tools that take into account specific information while quantifying the dependence structure including cross correlation, mutual information, coherence, and spectral transfer entropy to name a few. Granger Causality (GC) is another measure of dependence in multivariate time series data. GC essentially measures the ability of the past of a time series i to predict or forecast the future of another time series j . This framework proposed by Granger [ 4 ] relies on two main assumptions that cause happens prior to its effect and cause has a unique information about the future of its effect. If X(t) is a P dimensional real valued time series, then to test if time series i is a granger cause of time series j , two nested models are built. One model where the past of everything is used to predict the future of time series j , this is referred to as full model. A second model where the past of everything is used except that of time series i , this is referred to as reduced model. Model fit or residuals are computed under the full and reduced models. A test statistic based on the model fits is computed and compared with a reference distribution under null hypothesis of no granger causality from i to j . Linear and non-linear models have been proposed in literature to estimate GC from observational data. Among linear models Vector Auto regressive (VAR) models are most widely employed for the purpose of GC. Deep Learning (DL) has also been extensively explored for GC estimation where the problem has been essentially dealt as a sparse regression problem. The famous framework introduced in [ 8 ] builds a case for the use of separate neural networks for modeling individual time series components. Sparsity inducing penalties are applied on the input layer of these component wise models to learn the dependence structure from the data. The weights of the input layer are gradually shrink to zero using proximal gradient descent. GC is then estimated from the weights of the input layer of these component wise models. Let X(t) be the recording of a P dimensional multivariate time series data at time instance t. A component wise NN f_{p,\theta} with g_{p,\omega} as the associated input layer, models the dynamics of this time series using the past K lags of the multivariate time series data. X(p,t)=f_{p,\theta}(g_{p,\omega}(X(t-K)))+e(p,t) (1) Group Lasso in applied on g_{p,\omega} to sparsify the input layer weights and GC is estimated from the weights of g_{p,\omega} . As one may think this can become quickly computationally expensive if the number of time series grows or if the number of datasets grow like in healthcare one maybe interested in learning the GC between EEG recording of each subject separately for the purpose of personalized medicine or group level analysis. Authors in [ 8 ] argue that inference on the weights of the neural network can not lead to meaningful conclusion for estimation of GC when a multivariate time series data is modeled using a single auto-regressive model. We however present a different point of view for GC in our work. We agree on the aforementioned statement if a joint model is trained it is not straight forward to estimate GC from the weights of the network. However GC is linked to prediction, if a joint model is trained to predict the future dynamics of a multivariate time series data, it may learn some Granger causal structure which has to be uncovered. This leads towards the motivation of the present work, where we show how to uncover the GC structure learned by the joint model. We in our work show that even if a joint model is being used to model the multivariate time series data, model explanations about GC can still be performed, by comparing the distribution of the residuals from the fit of the joint model under specific input layer configurations. In this work we ask simple questions. 1. Can DNNs jointly model the time series data and learn the GC in the data. 2. How can a trained NN be used to uncover the learned GC from the time series data with a method different from sparse regression, without adding extra tuneable hyper-parameters in the model. The trained model shall be such that explanations can be generated to comment on GC. 3. What is the effect of input layer dropout on the ability of the neural networks to learn granger causal associations in time series context. 4. What shall be the mechanism to quantify the granger causality from a neural network when time series are modeled jointly. The answers to these questions are the novelty of the present work. We propose a method to jointly model the time series using a single neural network, and comparing the errors in the predictions of the model under general and reduced models. We

Read the complete text