Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees

Jiahe Lin, Yikai Zhang, George Michailidis · 2025

Read Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees by Jiahe Lin, Yikai Zhang, George Michailidis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16356.txt.

Introduction

Lin, Zhang, and Michailidis fit tie webs whose shape shifts with a side fact in this twenty twenty-five paper. Such webs show which traits link once the rest are held still. In brain and cash data, those links oft bend as the side fact moves.

Inside, a deep net learns the web's shape as a free curve of the side fact. No bell-shape law is forced on the data in their fit route. Proof gives learn pacts with firm odds that the fit stays near true. Fake data runs bench the route gainst past fits in the field. Brain and cash sets then yield webs a plain read can follow.

Catalog record

CREATORJiahe Lin, Yikai Zhang, George Michailidis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE133 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16356

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees Jiahe Lin, Yikai Zhang, George Michailidis Abstract Graphical models are widely used in diverse application domains to model the conditional dependencies amongst a collection of random variables. In this paper, we consider settings where the graph structure is covariate-dependent, and investigate a deep neural network-based approach to estimate it. The method allows for flexible functional dependency on the covariate, and fits the data reasonably well in the absence of a Gaussianity assumption. Theoretical results with PAC guarantees are established for the method, under assumptions commonly used in an Empirical Risk Minimization framework. The performance of the proposed method is evaluated on several synthetic data settings and benchmarked against existing approaches. The method is further illustrated on real datasets involving data from neuroscience and finance, respectively, and produces interpretable results. 0 0 footnotetext: Jiahe Lin and Yikai Zhang contributed equally and are listed alphabetically. 0 0 footnotetext: Correspondence to: George Michailidis. ⟨ [email protected] ⟩. Code is available at https://github.com/GeorgeMichailidis/covariate-dependent-graphical-model . 0 0 footnotetext: This paper is accepted by Transactions on Machine Learning Research (TMLR). 1 Introduction An undirected graphical model captures conditional dependencies amongst a collection of random variables and has been widely used in diverse application areas such as bioinformatics and the social sciences. It is associated with an undirected graph G=(V,E) with node set V:=\{1,\cdots,p\} and edge set E\subseteq V\times V that does not contain self-loops, i.e., (j,j)\not\in E;\forall\,j\in V ; \boldsymbol{x}=(x_{1},x_{2},\cdots,x_{p})^{\prime} is a random real vector indexed by the nodes of V , with probability distribution \mathbb{P} . The connection between the graph G and the probability distribution \mathbb{P}(\boldsymbol{x}) comes through the concept of graph factorization and the properties that G exhibits. The probability distribution \mathbb{P}(\boldsymbol{x}) factorizes with respect to G , if it can be written as \mathbb{P}(\boldsymbol{x})=\tfrac{1}{Z}\prod_{C\in\mathcal{C}(G)}\varphi_{C}(\boldsymbol{x}_{C}) , where \mathcal{C}(G) denotes the set of all cliques of G , \varphi_{C}>0,\forall\,C\in\mathcal{C}(G) potential functions, and Z a normalizing constant. If the underlying graph G in addition exhibits a Markov type property (see, e.g., Lauritzen 1996 and Appendix D.3 ), then the conditional dependence relationships are encoded in the potential functions \varphi_{C} . In particular, for G satisfying the pairwise Markov property, \mathbb{P} can be written as \mathbb{P}(\boldsymbol{x})\propto\prod_{\{j,k\}\in E}\varphi_{\{j,k\}}(\boldsymbol{x}_{\{j,k\}}) , and the absence of an edge in G implies that the corresponding potential function is zero and consequently the nodes are conditional independent given the remaining nodes. The upshot is that the conditional independence relationships for \boldsymbol{x}\sim\mathbb{P} can be read-off from the graph, if \mathbb{P} factorizes w.r.t. G and the latter is Markov. For certain multivariate distributions, the form of the joint distribution \mathbb{P}(\boldsymbol{x}) enables its decomposition into clique-wise potential functions based on G in a fairly straightforward manner; examples include the Gaussian for continuous variables ( Lauritzen 1996 , see also Appendix D.1 ) , and the Ising and the Boltzmann machine models ( Wainwright et al. 2008 ) for binary and discrete random variables, respectively. The corresponding potential functions \varphi_{C}(\cdot) are parameterized by (functions) of the parameters of the underlying distribution (see example in Appendix D.1 ). Consequently, a graphical model and the conditional independence relationships that G encodes can be obtained by estimating the parameters of the underlying distribution, either through maximum likelihood ( Banerjee et al. 2006 ; Friedman et al. 2008 ) or regression procedures using neighborhood selection techniques ( Meinshausen and Bühlmann 2006 ) . For other classes of graphical models beyond those mentioned above, to select the clique-wise potential functions \varphi_{C}(\cdot) becomes more involved. The difficulty stems from specifying potential functions based on which a compatible joint distribution \mathbb{P}(\boldsymbol{x}) with respect to G can be obtained. One popular approach is to define node conditional distributions through pairwise cliques \{j,k\} ; i.e., for any fixed j , \mathbb{P}(x_{j}|\boldsymbol{x}_{-j})\propto\prod_{k\in\text{ne}(j)}\varphi_{\{j,k\}}(x_{j},x_{k}) for k in the ne ighborhood of j . Such a conditional distribution captures pairwise conditional interactions between nodes, however, this construction does not directly define a proper graphical model for the joint distribution \mathbb{P}(\boldsymbol{x}) , unless additional conditions are satisfied (see example in Appendix D.2 ). In general, the question of the compatibility of a graphical model for the joint distribution \mathbb{P}(\boldsymbol{x}) based on a particular specification of all the node conditional distributions in V is rather open, and is generally addressed on a case by case basis ( Berti et al. 2014 ) . In this paper, we adopt the convention that models specified through their node-conditional distributions are also referred to as graphical models, which capture node-wise conditional independence relationships. The presentation thus far focused on a single “static” graphical model, which is suitable for modeling tasks involving homogeneous data sets; i.e., the observed samples \boldsymbol{x}^{i},i=1,\cdots,n are iid from \mathbb{P}(\boldsymbol{x}) . However, in certain applications, the conditional independence relationships may be heterogeneous in that they are modulated by external covariates \boldsymbol{z}\in\mathbb{R}^{q} . An example from biology is that the brain connectivity network—wherein the coordinates of \boldsymbol{x} correspond to different brain regions—varies with age ( \boldsymbol{z}\in\mathbb{R}_{+}) . Similarly, gene expression exhibits individual-level variation affected by single nucleotide polymorphisms (SNPs); see, e.g., Kim et al. 2012 . The SNPs are thereby covariates—potentially of large dimension—that impact the corresponding gene co-expression network. Motivated by such applications, this work introduces a class of covariate dependent, pairwise interaction graphical models, wherein the log-densities of the node conditional distributions are defined by x_{j}=\sum\nolimits_{k\neq j}^{p}\beta_{jk}(\boldsymbol{z})x_{k}+\varepsilon_{j},\quad j\in\{1,\cdots,p\}, (1) where \boldsymbol{z} is a q- dimensional observed covariate, \beta_{jk}(\cdot):\mathbb{R}^{q}\mapsto\mathbb{R} functional regression coefficient dependent on \boldsymbol{z} that impacts the strength of the interaction between nodes j and k . 1 1 1 Note that in the case of a Gaussian distributed error term \varepsilon , a covariate dependent graphical model is defined in terms of the joint distribution of \boldsymbol{x} conditional on \boldsymbol{z} , i.e., \mathbb{P}(\boldsymbol{x}|\boldsymbol{z}) , that factorizes over a graph G(\boldsymbol{z}) , “parameterized” by the external covariate \boldsymbol{z} . 1.1 Related Work We provide a brief overview of approaches aiming to define graphical models for continuous random variables that go beyond the Gaussian case, and also of prior work on covariate dependent graphical models. Non-Gaussian Graphical models. One line of work focuses on graphical models defined by monotone transformations of the Gaussian case, namely the non-paranormal ( Liu et al. 2009 ; Liu et al. 2012 ) . Their estimation entails first learning in a nonparametric fashion the respective transformations that “Gaussianize” the data, then subsequently leveraging approaches used for the Gaussian graphical model. Another broader line of work focuses on defining graphical models through specification of node conditional distributions. Yang et al. 2014 consider pairwise node conditional distributions from the exponential family, wherein the canonical parameter of the distribution is defined as a linear combination of higher order products of univariate functions of neighbors of each node j in G . This construction leads to a family of potential functions that defines a proper graphical model under G for the joint distribution \mathbb{P}(\boldsymbol{x}) (see details in Appendix D.2 ). More recently, Baptista et al. 2024b introduce a framework wherein the conditional dependence/independence between x_{j} and x_{k} is characterized by a score matrix given by \Omega_{jk}:=\mathbb{E}_{p(\boldsymbol{x})}[\partial_{j}\partial_{k}\log p(\boldsymbol{x})]^{2} , where p(\boldsymbol{x}) is the density function of \boldsymbol{x}\sim\mathbb{P}(\boldsymbol{x}) and is further assumed to be continuously differentiable. The authors propose to estimate the density

Read the complete text