Concomitant DAG Learning: On the Roles of Noise Adaptivity, Sparsity, and Non-negativity

Gonzalo Mateos, Samuel Rey, Hamed Ajorlou, Mariano Tepper · 2026

Read Concomitant DAG Learning: On the Roles of Noise Adaptivity, Sparsity, and Non-negativity by Gonzalo Mateos, Samuel Rey, Hamed Ajorlou, Mariano Tepper (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23537.txt.

Introduction

Mateos and colleagues survey concomitant DAG learning, the task of inferring cause-effect maps from observed data alone. Classical methods struggle with the hard task of enforcing no cycles and with hidden confounding or uneven noise. The tutorial recasts structure learning as continuous scoring over link matrices, then covers methods that jointly infer sparse causal structure and noise levels, gaining robustness under uneven noise and distribution shifts. It closes with new directions in online, nonlinear, and neural causal discovery.

Catalog record

CREATORGonzalo Mateos, Samuel Rey, Hamed Ajorlou, Mariano Tepper
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72606
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23537

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Concomitant DAG Learning: On the Roles of Noise Adaptivity, Sparsity, and Non-negativity Gonzalo Mateos, Samuel Rey, Hamed Ajorlou, Mariano Tepper Abstract Introduction Directed acyclic graphs (DAGs) have well-appreciated merits for encoding causal relationships, with a long history in the context of Bayesian networks [ 24 , Ch. 1.2.2] . This graphical modeling paradigm is central to signal processing (SP) and machine learning (ML) applications spanning domains such as biology [ 32 ] , healthcare [ 17 ] , genetics [ 44 ] , and finance [ 33 ] , to name a few. However, since the causal structure underlying a group of variables is often unknown, there is a need to address the task of inferring DAGs from (passive) observational data. While additional interventional data are provably beneficial to the related problem of causal discovery [ 4 , 26 ] , said interventions oftentimes are infeasible or ethically challenging to implement [ 38 ] ; see also [ 25 ] for recent advances involving soft interventions. Learning a DAG solely from observational data poses significant computational challenges, primarily due to the combinatorial acyclicity constraint which is notoriously difficult to enforce [ 6 ] . Moreover, distinguishing between DAGs that generate the same observational data distribution is nontrivial. This fundamental identifiability challenge may arise when data are limited (especially in high-dimensional settings), or, when candidate graphs in the search space belong to the same equivalence class (e.g., they exhibit the so-termed Markov equivalence). In addition, several conditions (sufficiency, faithfulness, minimality) must be satisfied so that causal interpretations can be made from the sought DAG; see e.g., [ 26 , Ch. 7.1] . Recognizing that DAG learning from observational data is in general an NP-hard problem, recent efforts have advocated a continuous relaxation framework which offers an efficient means of exploring the space of DAGs [ 45 , 22 , 1 , 30 ] , albeit without global optimality guarantees. In addition to breakthroughs in differentiable characterizations of acyclicity, the choice of an appropriate score function (usually composed of data fitting and regularization terms) is paramount to guide gradient-based algorithms towards good-quality solutions. While ordinary least squares (OLS) exhibits computational efficiency, robustness, and even consistency [ 15 ] ; it implicitly relies on homoscedasticity, meaning the variances of the exogenous noises in postulated structural equation models (SEMs) are identical across variables. Deviations from this assumption can introduce biases, hindering the accuracy of causal structure identification [ 15 ] . The pursuit of parsimonious DAG representations dictates that score functions often include a sparsity-promoting regularization, for instance an \ell_{1} -norm penalty as in lasso regression. This in turn necessitates careful fine-tuning of the penalty parameter that governs the trade-off between sparsity and data fidelity [ 19 ] . Theoretical insights suggest a regularization weight proportional to the observation noise level [ 3 ] , but the latter is typically unknown. Accordingly, several linear regression studies (unrelated to DAG learning) have proposed convex concomitant estimators based on scaled LS, which jointly estimate the noise level along with the regression coefficients [ 23 , 2 , 21 ] . A noteworthy representative is the smoothed concomitant lasso [ 21 , 19 ] , which not only addresses the aforementioned parameter fine-tuning predicament but also accommodates heteroscedastic scenarios. While these challenges are not extraneous to DAG learning, the impact of concomitant estimators is so far largely unexplored in this timely field that is woven into the natural fabric of the process that entails “going from signals to causes”. In this context, this tutorial aims to delineate the analytical background and SP relevance of innovative statistical inference and optimization tools to tackle the combinatorial problem of learning DAGs from observational data. We start with SEMs as multivariate causal models. Those will be used to formally state the DAG structure learning problem, and to selectively survey key results and assumptions for structure identifiability (i.e., uniqueness). We will then offer a brief historical overview of DAG structure identification methods, with an emphasis on so-termed score-based approaches for linear SEMs. Our didactic exposition will introduce various loss functions used to score candidate DAGs, along with innovative exact acyclicity characterizations. With these foundations in place, we bring to bear ideas from concomitant scale estimation in penalized linear regression and introduce a novel convex score function for sparsity-aware DAG inference [ 30 ] . A key message is that by making the score function noise adaptive , one effectively removes the coupling between the sparsity regularization parameter and the unknown exogenous noise levels, which leads to minimum (or no) recalibration effort across diverse problem instances or distribution shifts. Relative to OLS, we demonstrate that CoLiDE ( Co ncomitant Li near D AG E stimation) is better suited for challenging heteroscedastic scenarios. Finally, we discuss the impact of additional DAG structure on top of sparsity; namely, non-negativity of edge weights. In applications where this assumption is tenable, non-negativity constraints have favorable optimization landscape and statistical implications, which we also bring to light along with algorithmic opportunities [ 29 ] . We conclude with a summarizing discussion as well as an outline of CoLiDE extensions and other frontier topics that offer exciting research opportunities at the confluence of (graph) SP, ML, optimization, and causal inference. Preliminaries and DAG structure learning problem statement We start with graph-theoretic properties of DAGs and then reveal their key role in encoding conditional independencies among observable variables in directed graphical models (Bayesian networks), whose joint distribution satisfies a Markov property with respect to (w.r.t.) the DAG. We introduce SEMs to generate said Markovian distributions, and state the inverse problem of learning multivariate causal models, namely recovering the DAG structure from samples of the joint distribution alone. For concreteness, our focus throughout this tutorial will be (mostly) on linear SEMs, and we close this section by reviewing key results and assumptions guaranteeing structure identifiability for this class of problem instances. DAGs, SEMs, and topology inference problem formulation Consider a directed (di)graph {\mathcal{G}}\left({\mathcal{V}},{\mathcal{E}},{\mathbf{W}}\right) , where {\mathcal{V}}=\{1,\ldots,d\} represents the set of vertices, and {\mathcal{E}}\subseteq{\mathcal{V}}\times{\mathcal{V}} is the set of edges. The adjacency matrix {\mathbf{W}}=[{\mathbf{w}}_{1},\ldots,{\mathbf{w}}_{d}]\in{\mathbb{R}}^{d\times d} collects the edge weights, with W_{ij}\neq 0 indicating a direct link from node i to node j . We henceforth assume that {\mathcal{G}} belongs to the space \mathbb{D} of DAGs, and rely on the graph to represent conditional independencies among entries in the random vector {\mathbf{x}}=[x_{1},\ldots,x_{d}]^{\top}\in{\mathbb{R}}^{d} . Indeed, if the joint distribution p({\mathbf{x}}) satisfies a Markov property w.r.t. {\mathcal{G}}\in\mathbb{D} , each variable x_{i} depends solely on its parents \textrm{PA}_{i}:=\{j\in{\mathcal{V}}:W_{ji}\neq 0\}=\textrm{support}({\mathbf{w}}_{i}) . Partially-ordered sets, DAGs, and graph signal processing (GSP) A partially-ordered set (poset) {\mathcal{V}} with elements i,j,k\in{\mathcal{V}} satisfies: (i) i\leq i (reflexivity); (ii) i\leq j and j\leq i implies i=j (antisymmetry); and (iii) i\leq j and j\leq k implies i\leq k (transitivity). DAGs and posets are intimately related: every DAG induces a unique partial order on its nodes {\mathcal{V}} , where j<i whenever there is a directed path from j to i . Conversely, any poset can be realized by at least one DAG whose edges encode the partial ordering 1 x_{1} 2 x_{2} 3 x_{3} 4 x_{4} 5 x_{5} W_{12} W_{13} W_{14} W_{15} W_{24} W_{34} W_{45} W_{35} relation. In a poset, not all elements are necessarily comparable.

Read the complete text