Nearly Optimal Bayesian Inference for Structural Missingness

Chen Liang, Donghua Yang, Yutong Zhao, Tianle Zhang, Shenghang Zhou, Zhiyu Liang, Hengtong Zhang, Hongzhi Wang, Ziqi Li, Xiyang Zhang, Zheng Liang, Yifei Li · 2026

Read Nearly Optimal Bayesian Inference for Structural Missingness by Chen Liang, Donghua Yang, Yutong Zhao, Tianle Zhang, Shenghang Zhou, Zhiyu Liang, Hengtong Zhang, Hongzhi Wang, Ziqi Li, Xiyang Zhang, Zheng Liang, Yifei Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18500.txt.

Introduction

Chen Liang and eleven colleagues tackle a kind of missing data that breaks the usual fix. In most sets, a blank cell can be filled by a guess from the rest. Here some cells are blank by rule: the value cannot exist at all, given the facts of the case. A child with no job has no wage to fill in. The blank can also hinge on facts that are hid or on blanks in the next cells.

Plain fill-in tools treat such blanks as if the value were just lost. That misread can bend all that comes from the set. The team build a way to learn from such sets that is close to the best one could do. Their work serves fields like health and law, where blanks by rule are the norm. Sound tools for such sets could stop false claims built on bad fills.

Catalog record

CREATORChen Liang, Donghua Yang, Yutong Zhao, Tianle Zhang, Shenghang Zhou, Zhiyu Liang, Hengtong Zhang, Hongzhi Wang, Ziqi Li, Xiyang Zhang, Zheng Liang, Yifei Li
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE87 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18500

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Nearly Optimal Bayesian Inference for Structural Missingness Chen Liang, Donghua Yang, Yutong Zhao, Tianle Zhang, Shenghang Zhou, Zhiyu Liang, Hengtong Zhang, Hongzhi Wang, Ziqi Li, Xiyang Zhang, Zheng Liang, Yifei Li Abstract Structural missingness breaks ’just impute and train’: values can be undefined by causal or logical constraints, and the mask may depend on observed variables, unobserved variables (MNAR), and other missingness indicators. It simultaneously brings (i) a catch-22 situation with causal loop , prediction needs the missing features, yet inferring them depends on the missingness mechanism, (ii) under MNAR, the unseen are different , the missing part can come from a shifted distribution, and (iii) plug-in imputation, a single fill-in can lock in uncertainty and yield overconfident, biased decisions. In the Bayesian view, prediction via the posterior predictive distribution integrates over the full model posterior uncertainty, rather than relying on a single point estimate. This framework decouples (i) learning an in-model missing-value posterior from (ii) label prediction by optimizing the predictive posterior distribution, enabling posterior integration. This decoupling yields an in-model almost-free-lunch : once the posterior is learned, prediction is plug-and-play while preserving uncertainty propagation. It achieves SOTA on 43 classification with missing data and 15 imputation benchmarks, with finite-sample near Bayes-optimality guarantees under our SCM prior. 1 Introduction Structural Missingness. In many real-world applications, data is often incomplete due to various reasons such as sensor failures, terms of data collection, or privacy concerns. Rubin’s theory of missing data ( 32 ) proposed a framework for handling missing data, where the missing data is assumed to be (1) missing completely at random (MCAR), where the missingness is independent of the observed and unobserved variables, (2) missing at random (MAR), where the missingness is dependent on the observed variables, or (3) missing not at random (MNAR), where the missingness is dependent on the unobserved variables. Since MCAR/MAR/MNAR classify missingness by whether \mathbf{M} depends on observed/unobserved entries of X , we focus on structural missingness : some values are logically undefined and masks can exhibit within-mask dependencies (edges M\!\to\!M ) ( 18 ) . In our scope, we model this via a second-order SCM with randomized missingness mechanisms but a fixed Markov structure, in Sec. 4 . Though previous missingness types provides a framework for understanding missing data, reaching a joint solution for fitting the properties of the above missingness types without bias has many challenges as follows: Challenge 1: Alleviating MNAR bias is still an open problem. Bias may be caused only under MNAR by adopting the assumption that observed and unobserved variables have identical distributions, and this issue does not arise under MCAR/MAR ( 23 ) . Under MNAR, the missingness mechanism depends on unobserved values, so the conditional distribution of missing features given observations can be shifted relative to the observed part and is not identifiable without structural assumptions. We make these assumptions explicit via our second-order SCM prior (Sec. 4 , in Remark 6.6 ) ( 23 ; 37 ; 40 ) . We summarize this as: P(x\mid M=1) , values that end up missing, can differ from P(x\mid M=0) , values that are observed. Consequently, standard practice, i.e., training and predicting as if missingness were MAR/ignorable, or filling in a single point estimate (often implicitly treating missing values as drawn from P(x\mid M=0) ), can induce systematic prediction bias, which is named MNAR bias . Challenge 2: Obstacles in Explicit Modeling for Prior Distribution. Even with powerful function approximators, learning from partial observations still requires introducing prior knowledge. This challenge has two parts: (1) Explicit prior modeling is hard : the prior must be both flexible and tractable. Structural causal models can encode Markovian properties ( 24 ; 35 ; 34 ) , but they often rely on independence testing assumptions and scale poorly, so expert knowledge is typically required in highly structured settings ( 24 ) . (2) Uncertainty over prior knowledge is harder : the bottleneck is specifying the right prior, which becomes a near ’chicken-and-egg’ problem when manually encoding what is known vs. unknown ( 37 ) . We show that, despite this need for manual prior design in highly structured settings ( 24 ) , a PFN-style “causal ladder” view ( 7 ) motivates a level-1.5 in-model “almost-free-lunch”: not unconstrained causal discovery, but amortized identification of missingness/causal dependencies for prediction under a second-order SCM prior ( 2 ; 31 ) . Challenge 3: Dealing with Uncertainty for Structural Missingness. Traditional approaches often rely on deterministic imputation to fill in missing values. However, even if one had a reasonable prior/mechanism in mind, plug-in imputation does not fully leverage the prior/posterior support. Instead of integrating over significant possible missing values in the downstream task, it commits to a single fill-in and loses uncertainty propagation. In conclusion, under structural missingness it is difficult to specify or integrate prior knowledge that faithfully captures the dependencies between missing and observed/unobserved variables across arbitrary tasks, which is named plug-in imputation bias , leading to suboptimal performance. The question leaves: How can we address the three challenges in practice in a single, two-birds-with-one-seed unified efficient framework? Our Approach. Bayesian inference is both necessary and sufficient for the above challenges: to be correct under MNAR and avoid plug-in imputation bias, a predictor must reason about the uncertainty of unobserved variables, rather than committing to a single imputed input, after which optimal prediction follows from standard Bayesian principles as demonstrated in Sec. 3 . We make this view practical by constructing a prior distribution over the missingness model and causal mechanisms in Sec. 4 , and learning an implicit posterior-predictive map from incomplete inputs, with an optional missing-value posterior for sampling, in Sec. 5 . We formalize the task and derive the Bayesian predictive formulation in Sec. 2 and Sec. 3 , and discuss near Bayes-optimality via a finite-sample excess-risk bound in Sec. 6 . Finally, we validate the method in two real-world experimental suites for classification and imputation in Sec. 7 . For conceptual background, we provide a comprehensive related-work discussion in Sec. A . Contributions. 1. We formulate classification with structural missingness as posterior predictive inference conditioned on the observed subspace and the mask, unifying MCAR/MAR/MNAR ( addresses Challenge 2 ). 2. We propose posterior integration for prediction via a PFN (nearly optimal under the prior, addresses Challenge 3 ), with an optional learned missing-value posterior for sampling/analysis, propagating uncertainty and avoiding plug-in imputation bias and MNAR bias ( addresses Challenge 1 ). 3. We give a finite-sample risk bound that decomposes posterior-approximation and predictor errors, explaining plug-in bias and MNAR bias and implying lower sample complexity to reach a target error than plug-in/imputation baselines. Code and scripts will be released upon acceptance. 2 Problem Definition We consider a training dataset D=\{(x_{i},y_{i})\}_{i=1}^{N} , where each x_{i}\in\mathbb{R}^{d} represents a feature vector and y_{i}\in\{1,\ldots,C\} is the corresponding class label. The feature vectors are subject to structural missingness, denoted by a binary mask M_{i}\in\{0,1\}^{d} , where M_{ij}=1 indicates that the j -th feature of the i -th sample is missing, and M_{i}=0 indicates it is observed. Our goal is (1) to make tractable inference for the missing variables X_{m} given the observed variables X_{m}^{c} and the mask M and (2) to learn

Read the complete text