Stationary Processes, Wiener-Granger Causality, and Matrix Spectral Factorization
Lasha Ephremidze · 2024
Read Stationary Processes, Wiener-Granger Causality, and Matrix Spectral Factorization by Lasha Ephremidze (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18901.txt.
Introduction
Lasha Ephremidze surveys the math behind Granger causality, a widely used way to judge whether one time series helps predict another. His 2024 paper traces the history of the method and lays out its foundations in detail. The topic links stationary processes, in which the basic patterns stay stable over time, with matrix spectral factorization.
Ephremidze presents full proofs where they are short and clear. For harder theorems that the argument rests on, he points readers to the sources. The survey also covers how recent gains in computing power can widen the method's use. It closes with open directions, with a focus on large-scale data analysis. Readers get a single, careful account of a tool that many fields now rely on to study cause and effect in time-ordered data.
Catalog record
| CREATOR | Lasha Ephremidze |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-18901
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stationary Processes, Wiener-Granger Causality, and Matrix Spectral Factorization Lasha Ephremidze Abstract Abstract. Granger causality has become an indispensable tool for analyzing causal relationships between time series. In this paper, we provide a detailed overview of its mathematical foundations, trace its historical development, and explore how recent computational advancements can enhance its application in various fields. We will not hesitate to present the proofs in full if they are simple and transparent. For more complex theorems on which we rely, we will provide supporting citations. We also discuss potential future directions for the method, particularly in the context of large-scale data analysis. MSC: 60G12, 47A68. Keywords: Stationary processes; Granger causality; matrix spectral factorization. 1. Stationary processes in Hilbert spaces Let H be a Hilbert space with a scalar product \langle\cdot,\cdot\rangle and a norm \|\cdot\| . Usually, we assume that H is the space of square integrable functions f on a probability space (\Omega,\mathcal{F},P) with scalar product \langle f,g\rangle=\int\nolimits f\overline{g}\,dP. A sequence \{x_{n}\}_{n=-\infty}^{\infty} from H is called stationary if \langle x_{n+m},x_{n}\rangle=c_{m}\;\;\text{ for each }n,m\in\mathbb{Z}, i.e., the above scalar product is independent of n . The physical significance of stationarity lies in the invariance of the laws governing the process over time. The sequence c_{m} is positive definite, i.e., \sum_{m=0}^{N}\sum_{n=0}^{N}c_{m-n}z_{m}\overline{z_{n}}\geq 0 for any complex numbers z_{0},z_{1},\ldots,z_{N} . The latter inequality holds since \sum\nolimits_{m=0}^{N}\sum\nolimits_{n=0}^{N}c_{m-n}z_{m}\overline{z_{n}}=\left\|\sum\nolimits_{n=0}^{N}z_{n}x_{n}\right\|^{2} as one can check by direct computation. The positive definiteness of the sequence c_{m} , m\in\mathbb{Z} , guarantees, by the classical theorem of Herglotz-Bochner [ 2 ] , that there exists a positive measure on the unit circle \mathbb{T} , called the spectral measure of the process, such that (1) c_{m}=\frac{1}{2\pi}\int\nolimits_{\mathbb{T}}t^{-m}\,d\mu(t)=\frac{1}{2\pi}\int\nolimits_{0}^{2\pi}e^{-im\theta}\,d\nu(\theta), where \nu is the pullback measure of \mu on [0,2\pi) produced by the exponential map \theta\mapsto e^{i\theta} . In the multidimensional case, we have a sequence (2) \{X_{n}\}_{n\in\mathbb{Z}}=\{(x_{n}^{(1)},x_{n}^{(2)},\ldots,x_{n}^{(d)})^{T}\} of (d\times 1) -columns of vectors from the Hilbert space, i.e, x_{n}^{(j)}\in H , n\in\mathbb{Z} , 1\leq j\leq d , and it is called a (jointly) stationary if (3) \langle x_{n+m}^{(k)},x_{n}^{(l)}\rangle=c_{m}^{kl}. In high dimensions, we can use a matrix of (signed) measures {\mathfrak{m}}=[\mu_{kl}] on \mathbb{T} and it is called a positive definite if the matrix (4) {\mathfrak{m}}(\Delta)=[\mu_{kl}(\Delta)] is positive definite for each Borel set \Delta\in\mathbb{T} , i.e., (z_{1},z_{2},\ldots,z_{d})\cdot\mathfrak{m}(\Delta)\cdot(z_{1},z_{2},\ldots,z_{d})^{T}\geq 0 for each \Delta\in\mathcal{B} and (z_{1},z_{2},\ldots,z_{d})^{T}\in\mathbb{C}^{d\times 1} . A multidimensional extension to Bochner’s theorem assures that for a jointly stationary sequence ( 2 ), there exists a positive definite matrix measure ( 4 ) such that ( 3 ) is equal to \frac{1}{2\pi}\int_{\mathbb{T}}t^{-m}\,\mu_{kl}(t) (see, e.g., [ 10 ] ). The latter equation can be written in the matrix form (5) [X_{n+m},X_{n}^{T}]=\frac{1}{2\pi}\int\nolimits_{\mathbb{T}}t^{-m}\,d\mathfrak{m}(t) if we agree that [X,Y] , for (6) X=(x^{(1)},x^{(2)},\ldots,x^{(d)})^{T}\text{ and }Y=(y^{(1)},y^{(2)},\ldots,y^{(d)}) is the matrix with kl th entry equal to \langle x^{(k)},y^{(l)}\rangle and the integration of a function with respect to a matrix measure happens component-wise. Note that, using the above notation, if X is defined by ( 6 ) and (\alpha_{1},\alpha_{2},\ldots,\alpha_{d})\in\mathbb{C}^{1\times d} is a row vector of scalars, then \left\|\sum_{k=1}^{d}\alpha_{k}x^{(k)}\right\|^{2}=\left\langle\sum_{k=1}^{d}\alpha_{k}x^{(k)},\sum_{k=1}^{d}\alpha_{k}x^{(k)}\right\rangle=(\alpha_{1},\alpha_{2},\ldots,\alpha_{d})[X,X^{T}]({\alpha_{1}},{\alpha_{2}},\ldots,{\alpha_{d}})^{*}. Furthermore, if A_{n}=(\alpha_{n}^{(1)},\alpha_{n}^{(2)},\ldots,\alpha_{n}^{(d)})\in\mathbb{C}^{1\times d} , n=0,1,\ldots,n , and \{X_{n}\}_{n\in\mathbb{Z}} is a stationary process ( 2 ), then \displaystyle\left\|\sum_{n=0}^{N}\sum_{k=1}^{d}\alpha_{n}^{(k)}x_{n}^{(k)}\right\|^{2}\!\!=\!\left\|\sum_{n=0}^{N}A_{n}X_{n}\right\|^{2}\!=\!\left\langle\sum_{n=0}^{N}A_{n}X_{n},\sum_{n=0}^{N}A_{n}X_{n}\right\rangle\!=\!\sum_{n=0}^{N}\sum_{m=0}^{N}A_{n}[X_{n},X_{m}^{T}]A_{m}^{*} (7) \displaystyle=\sum_{n=0}^{N}\sum_{m=0}^{N}A_{n}\frac{1}{2\pi}\int\nolimits_{\mathbb{T}}t^{n-m}\,d\mathfrak{m}(t)A_{m}^{*}=\frac{1}{2\pi}\int\nolimits_{\mathbb{T}}\left(\sum_{n=0}^{N}A_{n}t^{n}\right)\,d\mathfrak{m}(t)\left(\sum_{n=0}^{N}A_{n}^{*}t^{-n}\right). The integration with respect to the matrix measure above should be understood in the usual measure-theoretic sense, with the customary ordering of operations for matrices. These formulas for computing the norms of linear combinations of elements from H will be useful in determining the prediction errors discussed in the next section. 2. Linear prediction theory of stationary processes Let H_{n}\subset H be the minimal closed subspace spanned by \{x_{n},x_{n-1},\ldots\} . Following a linear prediction theory of stationary processes created by Wiener [ 16 ] and Kolmogorov [ 10 ] , projection of x_{L} , L>0 , on H_{0} is called an L -lag ahead prediction of the process, namely \hat{x}_{L}:=Pr(x_{L}|H_{0}), and the norm (8) e_{x}^{\{L\}}:=\|x_{L}-\hat{x}_{L}\| is called the L -lag prediction error. Since the process is stationary, the quantity ( 8 ) is equal to \|x_{n+L}-Pr(x_{n+L}|H_{n})\| for each n\in\mathbb{Z} . In the multidimensional case ( 2 ), H_{n} is the minimal closed subspace spanned by \{x_{n}^{(1)},\ldots,x_{n}^{(d)},x_{n-1}^{(1)},\ldots,x_{n-1}^{(d)},\ldots\} , and the L -lag ahead prediction of X_{L} is taken to be \hat{X}_{L}=(\hat{x}_{L}^{(1)},\hat{x}_{L}^{(2)},\ldots,\hat{x}_{L}^{(d)})^{T},\;\text{ where }\hat{x}_{L}^{(j}=Pr({x}_{L}^{(j)}|H_{0}),\;j=1,2,\ldots,d. The corresponding prediction error ( 8 ) is defined to be e_{X}^{\{L\}}=\|X_{L}-\hat{X}_{L}\|_{H^{d\times 1}}:=\left(\sum\nolimits_{j=1}^{d}\|x_{L}^{(j)}-\hat{x}_{L}^{(j)}\|^{2}\right)^{1/2} (to simplify notation, in what follows, for X=(x^{(1)},\ldots,x^{(d)})^{T} , where x^{(j)}\in H , we assume \|X\|^{2}_{H^{d\times 1}}=\sum_{j=1}^{d}\|x^{(j)}\|^{2} ). 3. Wiener-Granger causality Determining whether two time-varying processes are correlated is of fundamental importance in various branches of applied sciences. In most cases, this conclusion is drawn by analyzing time series, such as \xi_{n} and \zeta_{n} , obtained from measurements of these processes, with several well-known standard techniques available for doing so. However, in many important situations, we are interested in the causal relationships between the processes. Building on the existing theory of predicting stationary processes, Wiener introduced a brilliant new idea for measuring such relationships between time series [ 17 ] . The idea was quite simple. Wiener assumed that \xi_{n}=x_{n}(\omega) and \zeta_{n}=y_{n}(\omega) are realizations of corresponding stationary processes x_{n} and y_{n} , and he proposed that the process generating \zeta_{n} causes the process generating \xi_{n} , viz. \zeta_{n}\to\xi_{n} , if the prediction error (9) e_{xy}=\inf\nolimits_{N,\alpha_{0},\ldots,\alpha_{N},\beta_{0},\ldots,\beta_{N}}\left\|x_{1}-\sum\nolimits_{n=0}^{N}\alpha_{n}x_{-n}-\sum\nolimits_{n=0}^{N}\beta_{n}y_{-n}\right\| is significantly smaller than the error e_{x}=\inf\nolimits_{N,\alpha_{0},\ldots,\alpha_{N}}\left\|x_{1}-\sum\nolimits_{n=0}^{N}\alpha_{n}x_{-n}\right\|. In other words, the inclusion of the second process in the prediction of the first significantly reduces the error norm (denoted as e_{xy}<<e_{x} in such cases). Nevertheless, Wiener’s efforts went largely unnoticed because he described formula ( 9 ) using his own algorithm for matrix spectral factorization—a mathematical tool, which will be presented in the following sections—that appeared formidably complex. Granger [ 8 ] , however, managed to express the same idea more simply using vector autoregressions and demonstrated several practical examples in economics, for which he gained both widespread recognition and a Nobel Prize in 2003. Later, this idea was expanded in several directions. In particular, for vector observations (x_{n}^{(1)},\ldots,x_{n}^{(p)},y_{n}^{(1)},\ldots,y_{n}^{(q)}) that are divided into two groups X_{n} and Y_{n} , one can investigate whether one group causes the other. For this purpose, two quantities should be compared (10) e_{X}=\inf\nolimits_{N,A_{0},\ldots,A_{N}}\left\|X_{1}-\sum\nolimits_{n=0}^{N}A_{n}X_{-n}\right\|_{H^{p\times 1}}\,, where A_{n} are p\times p matrices, A_{n}\in\mathbb{C}^{p\times p} , and (11) e_{XY}=\inf\nolimits_{N,B_{0},\ldots,B_{N}}\left\|X_{1}-\sum\nolimits_{n=0}^{N}B_{n}(X_{-n}^{T},Y_{-n}^{T})^{T}\right\|_{H^{p\times 1}}\,, where B_{n}\in\mathbb{C}^{p\times(p+q)} , and we say that Y_{n}\to X_{n} if e_{XY}<<e_{X}. It may happen that the causal relationship between two processes, represented by the time series x_{n} and y_{n} or vector time series X_{n}=(x_{n}^{(1)},x_{n}^{(2)},\ldots,x_{n}^{(p)}) and Y_{n}=(y_{n}^{(1)},y_{n}^{(2)},\ldots,y_{n}^{(q)}) , is delayed in time. Therefore, there exists an approach [ 1 ] that compares the quantities (12) \displaystyle e_{x}^{\{L\}}=\inf\nolimits_{N,\alpha_{0},\ldots,\alpha_{N}}\left\|x_{L}-\sum\nolimits_{n=0}^{N}\alpha_{n}x_{-n}\right\|, (13) \displaystyle e_{xy}^{\{L\}}=\inf\nolimits_{N,\alpha_{0},\ldots,\alpha_{N},\beta_{0},\ldots,\beta_{N}}\left\|x_{L}-\sum\nolimits_{n=0}^{N}\alpha_{n}x_{-n}-\sum\nolimits_{n=0}^{N}\beta_{n}y_{-n}\right\| and declares that there exists L -lag causal relation between x_{n} and y_{n} , denoted y_{n}\overset{\{L\}}{\to}x_{n} , if e_{xy}^{\{L\}}<<e_{x}^{\{L\}} . Accordingly, this idea can be generalized for vector observations, by taking X_{L} instead of X_{1} in formulas ( 10 ) and ( 11 ). In what follows, we return to Wiener’s approach and examine the role of matrix spectral factorization in the estimation of Granger causality. 4. Spectral factorization Let L_{p}(\mathbb{T}) , p>0 , be the space of p -integrable functions on the unit circle \mathbb{T} in the complex plane, L_{p}(\mathbb{T}):=\{f:\mathbb{T}\to\mathbb{C}\,|\,\int\nolimits_{\mathbb{T}}|f(t)|^{p}\,dt<\infty\}, and let \mathbb{H}_{p}(\mathbb{D}) , p>0 , be the Hardy space of analytic functions f in the unit disc \mathbb{D}=\{z\in\mathbb{C}\,|\,|z|<1\} for which \|f\|_{\mathbb{H}_{p}}^{p}=\sup\nolimits_{r<1}\int\nolimits_{0}^{2\pi}|f(re^{i\theta})|^{p}\,d\theta<\infty (see, e.g., [ 7 ] , [ 11 ] ). The boundary values function of f\in\mathbb{H}_{p} , i.e.