Covariate Dependent Mixture of Bayesian Networks

Roman Marchant, Dario Draca, Gilad Francis, Sahand Assadzadeh, Mathew Varidel, Frank Iorfino, Sally Cripps · 2025

Read Covariate Dependent Mixture of Bayesian Networks by Roman Marchant, Dario Draca, Gilad Francis, Sahand Assadzadeh, Mathew Varidel, Frank Iorfino, Sally Cripps (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05745.txt.

Introduction

One map of cause and effect rarely fits a whole population, and a 2025 paper by Roman Marchant and co-authors lets the map split by person. Bayesian networks chart how variables drive each other, but a single chart can mislead when subgroups live by different rules.

Their method fits a mixture of several networks, where each person's traits set the odds of belonging to each network's group. The same run finds the network shapes and the traits that predict group membership. Tests on simulated data and a youth mental health case study show the approach at work. The authors aim at tailored interventions in health, education, and social policy.

Catalog record

CREATORRoman Marchant, Dario Draca, Gilad Francis, Sahand Assadzadeh, Mathew Varidel, Frank Iorfino, Sally Cripps
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05745

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Covariate Dependent Mixture of Bayesian Networks Roman Marchant, Dario Draca, Gilad Francis, Sahand Assadzadeh, Mathew Varidel, Frank Iorfino, Sally Cripps Abstract Learning the structure of Bayesian networks from data provides insights into underlying processes and the causal relationships that generate the data, but its usefulness depends on the homogeneity of the data population—a condition often violated in real-world applications. In such cases, using a single network structure for inference can be misleading, as it may not capture sub-population differences. To address this, we propose a novel approach of modelling a mixture of Bayesian networks where component probabilities depend on individual characteristics. Our method identifies both network structures and demographic predictors of sub-population membership, aiding personalised interventions. We evaluate our method through simulations and a youth mental health case study, demonstrating its potential to improve tailored interventions in health, education, and social policy. \text{*} \text{*} footnotetext: Corresponding author: [email protected] 1 Introduction Probabilistic graphical models, particularly Bayesian Networks (BNs), are powerful tools for modeling statistical dependencies over collections of random variables ( Koller and Friedman 2009 ) . By encoding a set conditional independencies through a Directed Acyclic Graph (DAG), these models provide a succinct and intuitive representations of interactions among locally connected variables. Although conditional independence structures do not uniquely define the causal relationships ( Dawid 2010 ) , they do provide a set of possible causal models that are captured within the same Markov equivalence class. These equivalence classes can be represented by a Completed Partially Directed Acyclic Graph (CPDAG) ( Dawid 2010 ) . Distinguishing between different causal models within an equivalence class typically requires additional assumptions encoded in prior knowledge or interventional data ( Pearl 2009 ) . For an excellent survey and overview of these topics, see Dawid 2024 . Learning the structure of a BN from data is complex; the cardinality of the discrete network search-space grows in a super-exponential fashion with the number of variables, making an exact search implausible outside of moderately sized domains containing 10 - 15 variables. Numerous methods for structure learning have been developed; a recent overview is presented by Vowels et al. 2023 . However the validity of inference tasks is often less dependent on the algorithm used to estimate the network structure and accompanying posterior distribution, and more dependent on the appropriateness of assumptions underpinning the Data Generating Process (DGP) of the network. One example of this, which has been the subject of much research over the years, is the presence of unobserved confounders, potentially giving rise to spurious edges in learned network ( Chobtham and Constantinou 2020 ; Chobtham and Constantinou 2022 ; Häggström 2018 ; Higashigaki et al. 2010 ) . The assumption of the homogeneity of the DGP, has started to receive attention, with some methods using mixture models to relax the assumption of homogeneity in the DGP, ( Saeed et al. 2020 ; Strobl 2023 ; Varıcı et al. 2024 ). However, these methods use constraint-based optimization techniques to obtain only point estimates of component graph structures and probabilities. To our knowledge, with the notable exception of Castelletti et al. 2020 , there are no methods that model the joint posterior probabilities of the graph structure components and the corresponding component probabilities, which is essential for uncertainty quantification and decision-making. Furthermore, the mixture models ( Saeed et al. 2020 ; Strobl 2023 ; Varıcı et al. 2024 ) do not parametrise the mixing probabilities to depend upon covariates. This is unfortunate, as population heterogeneity is likely to arise from particular subsections of the population, where demographic factors such as an individual’s age, gender, and socio-economic status can directly or indirectly affect a wide range of life outcomes. Learning a mixture network without uncertainty quantification or without the ability to specifically encode potential drivers of the heterogeneity scenarios could lead to sub-optimal decision by practitioners who use such models. The purpose of this paper is to propose a fully probabilistic, yet computationally tractable, methodology for estimating and inferring multiple graph structures which result from heterogeneous DGPs. In developing our model we are motivated by two factors. The first is to improve the explainability and interpretability of model outputs by making inference robust to the assumption of a homogeneous DGP. The second is to reduce the computational complexity of the structure learning process for large networks. To these ends we propose that the data are generated from a finite, but unknown number of mixture components where the probability that an individual’s graph structure belongs to a mixture component depends upon an individual’s features. This is a variant of a mixture-of-experts model ( Jacobs 1995 ) , which is receiving renewed interest to model complexity in a parsimonious manner ( Chen et al. 2022 ; Oldfield et al. 2024 ) , where the experts are BNs and the gating or mixing function is a multinomial logistic model. Importantly, our work differentiates from Castelletti et al. 2020 ( Castelletti et al. 2020 ) since we divide the data into those features of an individual which have the potential for an intervention and those which do not. Those features which have the potential for an intervention we term modifiables while those that do not are termed non-modifiables . The former group are the features in the graph while the latter are the features in the mixing function. The motivation for this formulation is that by including features which cannot be subject to an intervention, such as various demographic factors, in the mixture weights, our model can better accommodate the latent class structure in our dataset, and allow for the identification of potential interventions without significantly impacting the computational complexity of the learning process. The paper is structured as follows: Section 2 formally defines our model. Section 3 describes the inference procedure used to obtain a sample estimate of the joint posterior of multiple graph structures, the corresponding parameters and the gating functions. In Section 4 , we evaluate the performance of our mixtures model and inference procedure by fitting both synthetic datasets with known ground truth DAGs, and a real world example involving youth mental health. Finally, Section 5 provides conclusions, discusses the limitations of the methods presented and suggests directions for future work. 2 Mixtures of Bayesian Networks, Model and Priors 2.1 Model Suppose we have variables \boldsymbol{y}\in\mathbb{R}^{M} and \boldsymbol{x}\in\mathbb{R}^{P} , which correspond to M modifiable and P non-modifiable features of an individual, respectively. We model the distribution of \boldsymbol{y} conditional on \boldsymbol{x} , namely p(\boldsymbol{y}|\boldsymbol{x}) as arising from a finite but unknown number of K mixture components. The components in the mixture are DAGs, whose structure, denoted by \mathcal{G} , depends solely on \mathbf{y} . The probability of belonging to a mixture component, denoted by \pi , depends solely on \mathbf{x} , so that p(\boldsymbol{y}|\boldsymbol{x})=\sum_{k=1}^{K}p(\boldsymbol{y}|\mathcal{G}_{k})\pi_{k}(\boldsymbol{x}), (1) where the subscript k indexes the components in the mixture for k=1,\ldots,K . 2.2 Model for Mixture Components The K components in ( 1 ) are modelled as M node BNs. Each of these BNs is defined by its structure \mathcal{G}_{k} and associated parameters, \tilde{\boldsymbol{\theta}}_{k}=(\boldsymbol{\theta}_{1,k},\ldots,\theta_{M,k}) . We assume that the distribution of the nodes in the graph, conditional on its parents are Gaussian and \tilde{\boldsymbol{\theta}}_{k} , contains the conditional means and variances

Read the complete text