Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning
Masahiro Tanaka · 2025
Read Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning by Masahiro Tanaka (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05050.txt.
Introduction
Masahiro Tanaka builds a scalable online kernel method to estimate causal effects that run in both directions at once. Many real systems have mutual links, where each side shapes the other, yet classic causal work often treats effects as one-way only.
His method joins a likelihood estimator for joint equations with large-scale online kernel learning. Random Fourier features let it fit curved means and spreads, while a step-by-step gradient rule keeps it fast on streaming, high-dimension data. In wide simulations, the method beat rivals on both accuracy and speed for such two-way systems.
Catalog record
| CREATOR | Masahiro Tanaka |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05050
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Estimating Bidirectional Causal Effects with Large Scale Online Kernel Learning Thanks: This study was supported by JSPS KAKENHI Grant Number 25K21168. Masahiro Tanaka Abstract In this study, a scalable online kernel learning framework is proposed for estimating bidirectional causal effects in systems characterized by mutual dependence and heteroskedasticity. Traditional causal inference often focuses on unidirectional effects, overlooking the common bidirectional relationships in real-world phenomena. Building on heteroskedasticity-based identification, the proposed method integrates a quasi-maximum likelihood estimator for simultaneous equation models with large scale online kernel learning. It employs random Fourier feature approximations to flexibly model nonlinear conditional means and variances, while an adaptive online gradient descent algorithm ensures computational efficiency for streaming and high-dimensional data. Results from extensive simulations demonstrate that the proposed method achieves superior accuracy and stability than single equation and polynomial approximation baselines, exhibiting lower bias and root mean squared error across various data-generating processes. These results confirm that the proposed approach effectively captures complex bidirectional causal effects with near-linear computational scaling. By combining econometric identification with modern machine learning techniques, the proposed framework offers a practical, scalable, and theoretically grounded solution for large scale causal inference in natural/social science, policy making, business, and industrial applications. I Introduction The interest in applying big data analytics and machine learning for causal analysis is growing steadily [ 7 , 21 , 15 , 19 ] . The rapid expansion and generation of large datasets present both opportunities and challenges. While large datasets enhance the statistical power, enabling the detection of subtle reciprocal relationships, they require computationally efficient algorithms for handling streaming or high-dimensional inputs without compromising interpretability. A key methodological challenge lies in robustly extracting causal effects from complex data while ensuring tractable estimation and correct identification. Understanding bidirectional causal relationships is fundamental across natural/social science, policy making, business, and industrial applications. Numerous real-world systems exhibit mutual dependence rather than unidirectional causality. For example, interactions between brain activity and behavior, predator-prey populations, policy interventions and public responses, and employee morale and organizational performance are mutually dependent. Despite its importance, recent research on machine-learning-based causal inference has largely overlooked bidirectional causal effects, focusing instead on unidirectional relationships between variables. To address this research gap, we propose a scalable online learning method for bidirectional causal estimation built on heteroskedasticity-based identification [ 17 ] . This identification strategy can be regarded as a variant of the instrumental variable method [ 1 , 3 , 20 ] . In conventional instrumental variable methods, causal parameters are identified through exogenous shifts in the conditional mean of the treatment variable induced by an instrument. For example, in a fish market, the selling price and quantity are jointly determined; thus, regressing the quantity on the price does not reveal the causal effect of price. However, when weather conditions serve as valid instruments—correlated with price but influencing quantity only through price changes—the causal effect is identifiable. In essence, instrumental variable methods exploit exogenous mean shifts in treatment variables. If an instrument shifts the supply curve while leaving the demand curve fixed, or vice versa, the corresponding slope can be estimated. Heteroskedasticity-based identification, on the contrary, relies on exogenous variations in the conditional variance of endogenous variables. This approach estimates the entire simultaneous equation model (SEM) in a single step, enabling the estimation of bidirectional causal relationships. In the fish market example, an SEM comprises two equations that describe supply and demand, respectively. If instruments influence the variability of one equation while leaving the other unchanged, the slopes of the corresponding curves can be identified. Several methods for heteroskedasticity-based identification have been proposed. The approach introduced in [ 17 ] divides the sample into low- and high-variance subsamples. Subsequent studies have developed more flexible and efficient strategies, including the control functional method [ 9 ] and generalized method of moments [ 10 ] . This study builds on the quasi-maximum likelihood (QML) estimator developed in [ 13 ] because it offers the most flexible and powerful framework and can be applied to cases that are unidentifiable under alternative approaches. The primary challenge in QML is specifying the conditional variance. While domain-specific theory or the analyst’s intuition may aid in modeling the conditional mean, they provide limited guidance for modeling the conditional variance. We address this challenge by integrating large scale online kernel learning [ 11 ] with the QML estimator for SEMs [ 13 ] . The proposed algorithm leverages kernel-based functional representations and random Fourier feature approximations to flexibly model nonlinear relationships in both conditional variances and means [ 16 ] . It combines a flexible representation with online optimization for efficient parameter updates as new data arrive. By embedding identification logic within a scalable learning architecture, the proposed method bridges econometric theory and modern machine learning. The resulting estimator captures complex bidirectional causality in the common high-dimensional environments of contemporary empirical research. Contrary to the recent kernel-based instrumental variable methods that estimate unidirectional effects [ 18 , 14 , 12 ] , the proposed approach jointly estimates bidirectional causal effects in a single model. The remainder of this paper is organized as follows. Section II presents the proposed method, including its theoretical properties and local identification conditions. Section III details the simulation experiments conducted to evaluate the practical performance of the proposed framework. Finally, Section IV discusses the findings and concludes the study. II Method II-A Model An SEM is defined as \displaystyle y_{1,i} \displaystyle= \displaystyle\gamma_{1}y_{2,i}+h_{1}\left(\boldsymbol{x}_{i},\boldsymbol{\beta}_{1}\right)+\varepsilon_{1,i}, (1) \displaystyle y_{2,i} \displaystyle= \displaystyle\gamma_{2}y_{1,i}+h_{2}\left(\boldsymbol{x}_{i},\boldsymbol{\beta}_{2}\right)+\varepsilon_{2,i}, for i=1,...,n , where y_{1,i} and y_{2,i} are endogenous variables, \boldsymbol{x}_{i}=\left(x_{1,i},....,x_{d,i}\right)^{\top} represents a d -dimensional vector of exogenous variables, and \varepsilon_{1,i} and \varepsilon_{2,i} are normally distributed error terms. Functions h_{1}\left(\cdot\right) and h_{2}\left(\cdot\right) are assumed to be twice continuously differentiable, and \gamma_{1} , \gamma_{2} , \boldsymbol{\beta}_{1} , and \boldsymbol{\beta}_{2} are unknown parameters. The primary objective is to estimate \gamma_{1} and \gamma_{2} , which capture the causal effects of y_{2,i} on y_{1,i} and vice versa. The conditional variances of the error terms are specified as g_{j,i}=\mathbb{V}\left(\varepsilon_{j,i}|\boldsymbol{x}_{i}\right)=\exp\left(f_{j}\left(\boldsymbol{x}_{i},\boldsymbol{\alpha}_{j}\right)\right),\quad j=1,2, where \boldsymbol{\alpha}_{1} and \boldsymbol{\alpha}_{2} are unknown parameters and every function f_{j}\left(\cdot\right) is twice continuously differentiable with respect to \boldsymbol{\alpha}_{j} . Because y_{1,i} and y_{2,i} are introduced into the model symmetrically, the system can be expressed as \displaystyle y_{1,i} \displaystyle= \displaystyle\left(-h_{2}\left(\boldsymbol{x}_{i}\right)+y_{2,i}-u_{2,i}\right)/\gamma_{2}, \displaystyle y_{2,i} \displaystyle= \displaystyle\left(-h_{1}\left(\boldsymbol{x}_{i}\right)+y_{1,i}+u_{1,i}\right)/\gamma_{1}, for \gamma_{1},\gamma_{2}\neq 0 . The two parameterizations are observationally equivalent, implying the existence of two possible sets of true parameter values. Therefore, the interpretation of each equation and its parameters depends on theoretical reasoning and prior assumptions. The following analysis focuses on local identification. Without additional theoretical structure, the true parameter values of the observationally equivalent representations are treated as distinct and distant from those of the original model. We estimate unknown parameter vector \boldsymbol{\theta}=\left(\gamma_{1},\gamma_{2},\boldsymbol{\beta}_{1}^{\top},\boldsymbol{\beta}_{2}^{\top},\boldsymbol{\alpha}_{1}^{\top},\boldsymbol{\alpha}_{2}^{\top}\right)^{\top} using a loss function derived from log-Gaussian quasi-likelihood. Stacking the equations in ( 1 ) yields \boldsymbol{\Gamma}\boldsymbol{y}_{i}=\boldsymbol{h}_{i}+\boldsymbol{\varepsilon}_{i},\quad\mathbb{V}\left[\boldsymbol{\varepsilon}_{i}|\boldsymbol{x}_{i}\right]=\boldsymbol{G}_{i}=\textrm{diag}\left(g_{1,i},g_{2,i}\right), where \boldsymbol{y}_{i}=\left(y_{1,i},y_{2,i}\right)^{\top},\quad\boldsymbol{\Gamma}=\left(\begin{array}[]{cc}1&-\gamma_{1}\\ -\gamma_{2}&1\end{array}\right), \boldsymbol{h}_{i}=\left(h_{1}\left(\boldsymbol{x}_{i},\boldsymbol{\beta}_{1}\right),h_{2}\left(\boldsymbol{x}_{i},\boldsymbol{\beta}_{2}\right)\right)^{\top}. The log quasi-likelihood is given by \log L_{n}\left(\boldsymbol{\theta}\right)=-n\log\left(2\pi\right)+n\log\det\boldsymbol{\Gamma} -\frac{1}{2}\sum_{i=1}^{n}\left[\log\det\boldsymbol{G}_{i}+\textrm{tr}\left\{\boldsymbol{G}_{i}^{-1}\boldsymbol{\varepsilon}_{i}\left(\boldsymbol{\theta}\right)\boldsymbol{\varepsilon}_{i}\left(\boldsymbol{\theta}\right)^{\top}\right\}\right], \boldsymbol{\varepsilon}_{i}\left(\boldsymbol{\theta}\right)=\left(\varepsilon_{1,i}\left(\boldsymbol{\theta}\right),\varepsilon_{2,i}\left(\boldsymbol{\theta}\right)\right)^{\top}=\boldsymbol{\Gamma}\boldsymbol{y}_{i}-\boldsymbol{h}_{i}. It can be represented as \log L_{n}\left(\boldsymbol{\theta}\right)=-n\log\left(2\pi\right)-\frac{1}{2}\sum_{i=1}^{n}\rho_{i}\left(\boldsymbol{\theta},\mathcal{D}_{i}\right), where \displaystyle\rho_{i}\left(\boldsymbol{\theta},\mathcal{D}_{i}\right) \displaystyle= \displaystyle-2\log\left(1-\gamma_{1}\gamma_{2}\right)+\log\left(g_{1,i}g_{2,i}\right) \displaystyle+\left(\frac{\varepsilon_{1,i}\left(\boldsymbol{\theta}\right)^{2}}{g_{1,i}}+\frac{\varepsilon_{2,i}\left(\boldsymbol{\theta}\right)^{2}}{g_{2,i}}\right), and \mathcal{D}_{i}=\left\{y_{1,i},y_{2,i},\boldsymbol{x}_{i}\right\} denotes the observations for unit i . Function \rho_{i}\left(\boldsymbol{\theta},\mathcal{D}_{i}\right) serves as the loss function for online learning. The gradient of \rho_{i}\left(\boldsymbol{\theta},\mathcal{D}_{i}\right) can be derived analytically, as shown in the Appendix. Point identification in the proposed method relies on the following assumptions. Assumption A For i=1,....,n ,