To impute or not to impute: How machine learning modelers treat missing data

Wanyi Chen, Mary Cummings · 2025

Read To impute or not to impute: How machine learning modelers treat missing data by Wanyi Chen, Mary Cummings (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16644.txt.

Introduction

Wanyi Chen and Mary Cummings survey how machine learning practitioners handle missing data in a 2025 paper. Gaps in tables are common, and the choice of fix, from filling values to dropping rows, can swing a model's results. Yet how builders actually choose was little known.

They surveyed 70 researchers and engineers who build such models. Most were not making informed choices about missing data treatment. Many picked by habit or tool default rather than by the pattern of the gaps. That gap in practice can quietly undermine the validity of trained models. The authors call for better teaching, standard reporting of missing data handling, and sharper analysis tools.

Catalog record

CREATORWanyi Chen, Mary Cummings
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16644

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

To impute or not to impute: How machine learning modelers treat missing data Wanyi Chen, Mary Cummings Abstract Missing data is prevalent in tabular machine learning (ML) models, and different missing data treatment methods can significantly affect ML model training results. However, little is known about how ML researchers and engineers choose missing data treatment methods and what factors affect their choices. To this end, we conducted a survey of 70 ML researchers and engineers. Our results revealed that most participants were not making informed decisions regarding missing data treatment, which could significantly affect the validity of the ML models trained by these researchers. We advocate for better education on missing data, more standardized missing data reporting, and better missing data analysis tools. 1 Introduction Missing data are prevalent in real-world artificial intelligence datasets across many domains. For instance, a study analyzing over 1.7 million cases in the National Cancer Database found that 54.9% of patients had at least one missing or incomplete variable [ 28 ] . A survey reviewing 1,666 quantitative studies published in 11 educational journals found that 48% of the studies involved missing data [ 19 ] . Missing data treatment is an important step in training machine learning (ML) models. Different missing data treatment methods can significantly affect ML model training results and may lead to different conclusions [ 12 , 23 ] . Moreover, if handled inappropriately, missing data treatment may induce additional biases into ML models [ 8 , 12 ] , as well as call into question their validity. However, despite its importance, the treatment of missing data treatment is often either not discussed in papers or only addressed in a curosry fashion [ 16 ] . Little is known about how ML modelers choose missing data treatment methods and what factors affect their choices. The lack of awareness and transparency regarding missing data treatment makes it difficult to assess the reproducibility and trustworthiness of ML models where missing data is a factor. This problem becomes prominent as ML and artificial intelligence become increasingly popular across various domains. For example, many recognize ML’s potential to help scientists discover more scientific knowledge and do so more efficiently [ 17 ] . However, currently ML does not reach this potential, in part because it can be a very subjective process [ chen2024subjectivity , 5 ] . To investigate the subjectivity involved in missing data treatment, we conducted a survey of 70 ML researchers and engineers regarding their knowledge and practices of missing data treatment. We aim to shed light on the modelers’ preferences in missing data treatment and its implications in responsible model training. 2 Background and related work Missing data occur for different reasons. Statistically, they are classified by three different missing data mechanisms: • Missing Completely at Random (MCAR) : MCAR occurs when the missingness is completely unrelated to any variable and is totally random [ 13 ] . It is the simplest case, yet this strict assumption is rarely satisfied in real-world datasets [ 21 ] . • Missing at Random (MAR) : MAR occurs when the missingness systematically depends on variables with complete information [ 13 ] . • Missing Not at Random (MNAR) : MNAR occurs when the missingness depends on the variable of interest itself or on other unknown variables [ 13 ] . Different missing data mechanisms have different practical implications. These are discussed in conjunction with different missing data treatment methods in the following section. This background section concludes with a discussion of the role of subjectivity in missing data mitigation. 2.1 Missing data treatment There are many available missing data treatment methods [ 6 , 9 , 20 ] . This subsection reviews some common methods, their advantages and drawbacks: • Complete case analysis (CCA) : CCA means discarding all records with missing data and only training models on complete data [ 26 ] . It is also known as available case analysis or listwise deletion. CCA is easy to use as it is the default method in many statistical packages [ 26 ] . However, CCA is criticized for two major drawbacks. First, it could discard a significant amount of data, so it is sometimes considered inefficient [ 26 ] . Second, if the data is not MCAR, CCA is not a random subsample of the original sample and may lead to biases in some cases [ 12 ] . For example, if in a hypothetical medical dataset, women are more likely to have missing data than men, applying CCA to the dataset means discarding most records that belong to women, potentially making model prediction results less accurate for women. • Mean imputation : Mean imputation fills in missing values with the average value of the feature. It is the easiest and one of the most widely used imputation methods [ 7 ] . However, if a feature contains many missing data points, mean imputation could lead to changes in data distribution. For example, the standard deviation could become smaller after imputation [ 7 ] . This change can disturb the relations between variables and induce bias, so mean imputation should not be used when data is not MCAR [ 3 , 24 ] . Similarly, filling in missing values with median, mode, or an arbitrary value (e.g., 0) can also introduce significant bias when data is not MCAR. • Last observation carried forward (LOCF) : This method replaces the missing values with the last observed values. This is a common approach for treating longitudinal data and time series data. However, researchers found that this method introduces significant bias when data is not MCAR [ 11 ] . • Model-based imputation : Recognizing the limitations of mean imputation, some researchers proposed to train models (e.g., regression models, k-nearest neighbors, neural networks, and so on) based on available data to predict missing values [ 8 ] . Model-based imputation methods can improve performance in some cases, but these methods make assumptions regarding relationships among the features [ 8 ] . For example, imputing missing values using regression models assumes a linear relationship between the features [ 8 ] . Regression imputation will introduce biases into the dataset if the assumption is not true [ 8 ] . More generally, all model-based imputation methods assume MAR. By definition, MAR means the missingness systematically depends on variables with complete information, which is why models trained on variables with complete information can be used to predict missing values. However, if the data is not MAR, model-based imputation will introduce biases. • Multiple imputation : Taking the uncertainty of missing data into consideration, many statisticians advocate for multiple imputation [ 10 , 22 ] . Multiple imputation imputes missing values multiple times, producing multiple complete datasets with imputed values. Each imputed dataset is analyzed separately, and the results are pooled together using statistical rules [ 7 , 22 ] . While multiple imputation has long been established in the statistics community, it has only recently gained attention in the ML community. Ensemble learning is one way of incorporating multiple imputation into the ML training process [ 1 ] . One ML model is trained on each dataset produced by

Read the complete text