Neural Autoregressive Flows for Markov Boundary Learning

Khoa Nguyen 1 , Bao Duong 1 , Viet Huynh 2 , Thin Nguyen 1 · 2026

Read Neural Autoregressive Flows for Markov Boundary Learning by Khoa Nguyen 1 , Bao Duong 1 , Viet Huynh 2 , Thin Nguyen 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20791.txt.

Introduction

Nguyen and co-authors learn Markov boundaries with neural autoregressive flows. A Markov boundary is the smallest variable set that predicts a target best. Past learning methods lean on nonparametric estimates and heuristic search. Their framework scores candidate sets with conditional entropy from information theory. A masked autoregressive network captures complex dependencies in the data. A greedy search then runs in polynomial time with analytic support. Starting graph search from learned boundaries also speeds causal discovery. Tests on real and synthetic data beat prior methods.

Catalog record

CREATORKhoa Nguyen 1 , Bao Duong 1 , Viet Huynh 2 , Thin Nguyen 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70408
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20791

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neural Autoregressive Flows for Markov Boundary Learning Khoa Nguyen 1 , Bao Duong 1 , Viet Huynh 2 , Thin Nguyen 1 Abstract Recovering Markov boundary—the minimal set of variables that maximizes predictive performance for a response variable—is crucial in many applications. While recent advances improve upon traditional constraint-based techniques by scoring local causal structures, they still rely on nonparametric estimators and heuristic searches, lacking theoretical guarantees for reliability. This paper investigates a framework for efficient Markov boundary discovery by integrating conditional entropy from information theory as a scoring criterion. We design a novel masked autoregressive network to capture complex dependencies. A parallelizable greedy search strategy in polynomial time is proposed, supported by analytical evidence. We also discuss how initializing a graph with learned Markov boundaries accelerates the convergence of causal discovery. Comprehensive evaluations on real-world and synthetic datasets demonstrate the scalability and superior performance of our method in both Markov boundary discovery and causal discovery tasks. I Introduction The Markov Blanket, introduced by Pearl [ 31 ] in causal Bayesian networks, is a set \mathbf{S} that captures the local causal mechanisms for a variable T and statistically shields T from the rest of the system \mathbf{V}\backslash\mathbf{S} : T\perp\!\!\!\perp\mathbf{V}\backslash\mathbf{S}\mid\mathbf{S} [ 31 , 18 ] . The Markov boundary (MB) is defined as the minimal Markov blanket. 1 1 1 Some literature refers to this minimal set as the Markov blanket [ 18 , 44 ] . To distinguish it from larger sets, we adopt the Markov boundary definition from [ 31 ] , and use the abbreviation MB for Markov boundary. MBs are crucial in a wide range of domains, including bioinformatics [ 21 ] , neuroscience [ 16 ] , and environmental science [ 33 ] . In the realm of data science and causality, they are prevailingly applied in causal feature selection [ 44 ] , increasingly in missing data imputation [ 22 ] , divide-and-conquer causal discovery [ 11 ] , and LLM reasoning [ 20 ] . Under standard faithfulness assumptions, the MB of a variable is unique without the need for additional experimental data [ 39 ] ; thus, it is theoretically feasible to design a learning algorithm \mathcal{L}^{*} that approximates the optimal solution with sufficient observational data. However, MB learning faces significant challenges in large graphs with nonlinear dependencies and unknown associated data distributions, which complicate accurate and scalable inference [ 43 ] . Traditionally, MB learning in the constraint-based approach indirectly uses pairwise conditional independence (CI) tests. This direction [ 38 , 23 , 32 , 42 , 39 ] has a solid theoretical foundation but relies on the accuracy of these tests, making it susceptible to restricted assumptions about data distribution, insufficient training samples, and the scale of the separating set [ 43 ] . The sequential decision-making process can lead to cascading errors and ignoring true positives [ 43 ] . Score-based approaches, which evolved later but received limited attention, identify local causal structures by optimizing a quantifiable objective [ 43 , 14 ] . However, commonly used scoring criteria, such as BDeu and BDe, impose strict assumptions on the data, mainly designed for discrete or linear Gaussian data [ 43 ] . Recently, KMB [ 43 ] was proposed to address the complexity of data relationships. It, however, employs the conditional covariance operator (CCO), a kernel-based design known for its inefficiency in terms of sample size and dimensionality [ 12 ] . The learning mechanism of KMB relies on a heuristic algorithm lacking theoretical justification and must solve a continuous relaxation-based objective at each step of evolving its predicted MB. This leads to inefficiencies in learning MBs for all variables in the network. Regarding the applicability of MBs in follow-up tasks, we focus on their integration as a warm-start step in causal discovery (CD) [ 1 , 32 , 39 ] . The key advantage lies in guiding CD within a reduced search space by eliminating obviously irrelevant relationships [ 1 , 39 ] . From an overarching technical viewpoint, this group has evolved alongside constraint-based MB learning, which approaches CD by first solving the local causal induction problem and then adopting a local-to-global learning strategy. As a subset of constraint-based CD, a critical component of these techniques is the pairwise CI test, where greater efficiency reflects a reduction in the number of required trials [ 1 , 25 ] . In the emerging field of differentiable causal discovery (DCD), state-of-the-art techniques employ MB learning to construct the moral graph, which serves as a mask for subsequent DAG estimation [ 26 , 2 ] . SDCD [ 26 ] has demonstrated improved stability and scalability, particularly for sparse graphs common in real-world scenarios. However, the moral graphs learned during preselection are not true MB-derived moral graphs but rather sparser approximations that eliminate some false positives while avoiding a complete graph, due to the inherent limitations of DCD [ 41 ] . We address the gaps across these worlds: our method advances a novel class of autoregressive flow-based generative models, aligning them with conditional entropy from classic information theory to derive a theoretically justified scoring objective. We introduce Any-subset Masked Autoregressive Flows , also known as F lows for AN y- S ubset ( FANS ), which efficiently model nonlinear relationships and noise distributions through latent representation learning, thereby transforming intractable universal entropy into an easily estimable counterpart. A single trained FANS can also estimate any conditional entropy H(T|\mathbf{S}) , where T is an arbitrary random variable and \mathbf{S} is any subset of variables. This enables its use in our greedy algorithm, FANS -based I nformation ga T h E ring for MB discovery ( FANSITEMB ), facilitating seamless integration into CD pipelines, unlike previous score-based approaches that require separate models for each variable [ 43 ] . Our main contributions are: • FANS , 2 2 2 https://github.com/khoangdadk/FANS a neural model redesigned from original autoregressive flows [ 17 ] . It combines strengths from both masked autoregressive autoencoder neural networks [ 15 ] and normalizing flows [ 29 ] , enhancing expressiveness and efficacy in approximating underlying distributions. With amortized training, FANS can estimate conditional entropy of arbitrary variables conditioned on any subsets, thereby avoiding the computational expense of learning a separate model for each subset . Its compact design also reduces parameters, optimizing computational efficiency when dealing with sparse Bayesian networks. • FANSITEMB , a greedy and parallelizable algorithm for discovering MBs in polynomial time, supported by a thorough theoretical analysis of error bounds. • FANSITE-DCD , a MB-based differentiable causal discovery method adapted from SDCD, ensuring learning of DAGs from a minimal I-map-based subspace. • Empirical evidence on the effectiveness of the proposed framework across various synthetic settings and four real-world gene regulatory networks. II Related Works A significant portion of MB discovery focuses on constraint-based algorithms that identify MBs by mining conditional independence (CI) relations. As a crucial component of such group, the correctness of every CI test must be assumed, with \lambda^{2} -tests and G^{2} -tests commonly employed for discrete cases, and Fisher’s Z-test used for continuous cases under the assumption of linear relationships and additive Gaussian noise. Growing-Shrink (GS) [ 23 ] was the first

Read the complete text