Distinguishing Cause from Effect with Causal Velocity Models
Johnny Xi, Hugh Dance, Peter Orbanz, Benjamin Bloem-Reddy · 2025
Read Distinguishing Cause from Effect with Causal Velocity Models by Johnny Xi, Hugh Dance, Peter Orbanz, Benjamin Bloem-Reddy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05122.txt.
Introduction
Johnny Xi, Hugh Dance, Peter Orbanz, and Benjamin Bloem-Reddy present a new way to tell cause from effect in two-variable data. Standard causal models judge direction by how well restricted model classes fit. Their approach instead treats the cause variable as time in a moving system.
The speed of that motion, called causal velocity, defines counterfactual curves through each observation. Using measure transport tools, the authors connect these models uniquely to the score function of the data distribution. Their objective compares the velocity directly against that score, needing no assumptions about noise. Simulations and benchmarks show success where many existing methods fail.
Catalog record
| CREATOR | Johnny Xi, Hugh Dance, Peter Orbanz, Benjamin Bloem-Reddy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-05122
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Distinguishing Cause from Effect with Causal Velocity Models Johnny Xi, Hugh Dance, Peter Orbanz, Benjamin Bloem-Reddy Abstract Bivariate structural causal models (SCM) are often used to infer causal direction by examining their goodness-of-fit under restricted model classes. In this paper, we describe a parametrization of bivariate SCMs in terms of a causal velocity by viewing the cause variable as time in a dynamical system. The velocity implicitly defines counterfactual curves via the solution of initial value problems where the observation specifies the initial condition. Using tools from measure transport, we obtain a unique correspondence between SCMs and the score function of the generated distribution via its causal velocity. Based on this, we derive an objective function that directly regresses the velocity against the score function, the latter of which can be estimated nonparametrically from observational data. We use this to develop a method for bivariate causal discovery that extends beyond known model classes such as additive or location-scale noise, and that requires no assumptions on the noise distributions. When the score is estimated well, the objective is also useful for detecting model non–identifiability and misspecification. We present positive results in simulation and benchmark experiments where many existing methods fail, and perform ablation studies to examine the method’s sensitivity to accurate score estimation. † † affiliationnotice: Equal contribution 1 Introduction Distinguishing cause from effect from purely observational data is a challenging task. It is generally impossible to distinguish a causal direction X\to Y from an anti-causal direction Y\to X without intervention, as the corresponding causal graphs are Markov equivalent. To make causal discovery possible, one common approach is to make functional assumptions, for example, non-linear additive noise ( Hoyer et al., 2008 ) or location-scale noise ( Xu et al., 2022 ; Strobl & Lasko, 2023 ; Immer et al., 2023 ) on the underlying causal process, and decide on the causal direction based on model fit and/or complexity in both candidate directions. Likelihood-based approaches seem natural here, as they can be used both for model estimation and evaluation. However, they require the full specification of both the mechanism and the noise distribution. This is undesirable due to the risks of model misspecification, particularly with respect to the noise distribution, which is not directly involved in causality, but can still lead to incorrect causal inference ( Schultheiss & Bühlmann, 2023 ) . To avoid that risk, the most prominent example of a functional method is to fit an additive noise model (ANM) using nonparametric regression, which avoids needing to model the noise distribution. Instead of a likelihood-based measure, an independence score ( Gretton et al., 2005 , e.g., HSIC,) between the estimated residuals and cause variable can be used to determine goodness-of-fit ( Mooij et al., 2009 ; Peters et al., 2014 ) . This approach has been extended to location-scale models (LSNM) by Strobl & Lasko (2023) ; Immer et al. (2023) . Separately, Rolland et al. (2022) ; Montagna et al. (2023b) derived other goodness-of-fit criteria for ANMs that avoid structural model estimation altogether. Instead, those methods extract the signal directly from estimates of the score function \nabla\log p(x,y) under the assumption of Gaussian noise. Subsequent work used score estimation in conjunction with explicit model estimation in the case of non-Gaussian noise ( Montagna et al., 2023a ) . However, we are unaware of any work on functional causal discovery methods beyond the ANM and LSNM classes, which may still be misspecified in many settings. In this work, we develop a new framework that treats bivariate causal models as dynamical systems, and we use it to devise a new estimation procedure based on the score function of the data, applying it to causal discovery. Specifically, we parametrize a causal model via its causal velocity , which describes infinitesimal counterfactuals and can be directly recovered from the score in a simulation-free way analogous to the recent literature on flow-based generative modeling ( Lipman et al., 2023 ; Albergo & Vanden-Eijnden, 2023 ) . Assuming that the marginal and joint score functions can be estimated accurately, our proposed method combines the advantages of various state-of-the-art methods in that it is agnostic to the noise distribution and avoids explicit estimation of the functional model, even for non-Gaussian and non-additive cases. Beyond this, the velocity parametrization allows us to specify model classes that extend beyond ANM and LSNM in terms of dependence on the noise, but are still restrictive enough to be useful for causal discovery. Figure 1: Given a SCM X\to Y and an observation (x,y) , the causal curve represents the implied counterfactual outcomes had x been x^{\prime} (top). The derivative of the causal curve, which we call the causal velocity function v(y,x) , generates all the causal curves and hence characterizes the counterfactuals of the SCM (bottom). In this paper, we show how the velocity can be estimated directly from data using the score function, without the need to evaluate the mechanism itself nor specify the underlying distributions. Contributions We present a new perspective on invertible SCMs as dynamical systems through their implied velocity functions. In particular, counterfactual prediction becomes the solution to an initial value problem in which the factual observation is the initial condition. This allows us to specify novel causal models. It also allows us to use established ideas from measure transport to connect the causal velocity to the score functions of the data distribution. We use this to devise a novel simulation and likelihood-free estimation procedure for bivariate SCMs and a related goodness-of-fit criterion for bivariate causal discovery. To support the methodological contributions, we also examine the causal direction identifiability problem from the dynamical perspective, where we obtain a general result that specializes to those existing in the literature. We also establish statistical consistency of our estimation procedure, and show that the rate of convergence is dictated by the rate at which the score estimators converge. We show through synthetic and benchmark experiments that the added model flexibility is beneficial in situations where ANM and LSNM models can fail, but simple parametric velocity families recover the correct causal direction. 1 1 1 Code supporting our experiments can be found on Github at https://github.com/xijohnny/causal-velocity . The appealing properties of our method depend on an accurate estimate of the score function at the data. The nonparametric score estimators that we use have known consistency properties and convergence rates ( Zhou et al., 2020 ) , which justifies the method asymptotically. However, some distributions, including those seen in common benchmarks, may not have favourable finite-sample behaviour even in the low-dimensional regime. We describe and probe failure modes of our method by way of poor score estimation. Outline In Section 2 we cover the necessary background on bivariate causal discovery and on dynamical systems. Section 3 describes our framework of viewing SCMs as dynamical systems and the velocity parametrization. In Section 4 , we use the continuity equation to derive an identity that we use to recover the causal velocity from the score function. Section 5 applies this to causal discovery and discusses consistency and identifiability. Finally, Section 6 and 7 describe related work and experimental evaluations. 2 Background Table 1: Summary of existing and novel SCMs and their velocities.