Chunked Data Shapley: A Scalable Dataset Quality Assessment for Machine Learning Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Informatio

Andreas Loizou, Dimitrios Tsoumakos · 2025

Read Chunked Data Shapley: A Scalable Dataset Quality Assessment for Machine Learning Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Informatio by Andreas Loizou, Dimitrios Tsoumakos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16255.txt.

Introduction

Andreas Loizou and Dimitrios Tsoumakos presented Chunked Data Shapley in 2025, a fast way to grade data quality for machine learning. Data Shapley values score how much each data point helps a model, based on game theory. Exact scores are far too slow for large data sets, and past shortcuts still struggle at scale.

Their method splits a data set into chunks and scores each chunk as a unit. A tuned subset search and a single pass of gradient training keep the cost low. On real classification and regression tasks, the method ran eighty to more than two thousand times faster than past shortcuts. It also found low-quality data regions more accurately, which supports its use on large tables.

Catalog record

CREATORAndreas Loizou, Dimitrios Tsoumakos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16255

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Chunked Data Shapley: A Scalable Dataset Quality Assessment for Machine Learning Conference: Proceedings of the 34th ACM International Conference on Information and Knowledge Management; November 10–14, 2025; Seoul, Republic of Korea Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM ’25), November 10–14, 2025, Seoul, Republic of Korea DOI: 10.1145/3746252.3761305 ISBN: 979-8-4007-2040-6/2025/11 Andreas Loizou, Dimitrios Tsoumakos Abstract . As the volume and diversity of available datasets continue to increase, assessing data quality has become crucial for reliable and efficient Machine Learning analytics. A modern, game-theoretic approach for evaluating data quality is the notion of Data Shapley which quantifies the value of individual data points within a dataset. State-of-the-art methods to scale the NP-hard Shapley computation also face severe challenges when applied to large-scale datasets, limiting their practical use. In this work, we present a Data Shapley approach to identify a dataset’s high-quality data tuples, Chunked Data Shapley (C-DaSh) . C-DaSh scalably divides the dataset into manageable chunks and estimates the contribution of each chunk using optimized subset selection and single-iteration stochastic gradient descent. This approach drastically reduces computation time while preserving high quality results. We empirically benchmark our method on diverse real-world classification and regression tasks, demonstrating that C-DaSh outperforms existing Shapley approximations in both computational efficiency (achieving speedups between 80 × \times – 2300 × \times ) and accuracy in detecting low-quality data regions. Our method enables practical measurement of dataset quality on large tabular datasets, supporting both classification and regression pipelines. † † cc-license: by-sa 1. Introduction Data grows exponentially, in volume as well as variety and velocity ( Hazen et al. 2014 ) . As data increases at this pace in data centres, organisations struggle with the performance of their decision-making algorithms, one promising reason is the data quality. Selecting the “right” data for the used of an data-driven algorithm (such as machine learning) can improve the algorithm’s performance efficiency ( Loizou and Tsoumakos 2025 ) . The plethora of available data within the organisation’s data centres can cause the identification of high-quality data to be a real struggle. Using the human factor to identify high-quality data is time-consuming, as well as drop the performance of the data-driven algorithm. Automating data selection using AI techniques has been gamy tracing the data context is named as Data-Centric Artificial Intelligence (AI) ( Zha et al. 2025 ; Loizou and Tsoumakos 2025 ; Loizou et al. 2025 ) . Data-Centric AI techniques besides automate the data selection, improve the selection of the high quality data and the performance of a data-driven ML algorithm. Also, because data quality is described on different dimensions using Data-Centric AI can focus on the specific dimension’s that matters ( Jakubik et al. 2024 ) . What constitutes Data Quality can vary depending on different contexts and applications. Previous studies have focused on traditional data quality issues, such as low accuracy due to poor data quality, dataset completeness with respect to real-world scenarios, and consistency among the data tuples within a dataset ( Wang and Strong 1996 ; Batini et al. 2009 ) . More recent studies define low data quality with respect to the size, order, and distribution of the data ( Giannakopoulos et al. 2018 ) . Others emphasize on more nuanced dimensions, such as bias and fairness, which can result to discriminatory results in machine learning models trained on suboptimal datasets ( Pitoura 2020 ; Barry et al. 2023 ) . A recent study ( Mohammed et al. 2025a ) categorises data quality in six different dimensions: representation, completeness, feature accuracy, target accuracy, uniqueness and target class balance. Data Quality can be depended on the data tuple features, such as corruption or missing, which is a problem that Data Shapley works try to resolve ( Shapley et al. 1953 ; Roth 1988 ) . As a data-centric AI approach, Data Shapley quantifies the importance of individual data tuples in a data-driven learning algorithm, identifying its quality and value by their contribution to overall model performance ( Wang et al. 2024b ; Zha et al. 2025 ) . However, computing Data Shapley is an NP-hard problem, and various approximations have been designed to improve its computational efficiency ( Ghorbani and Zou 2019 ; Wang et al. 2024a ) . Data Shapley approximations remain computational expensive, often require hours to find each data tuple quality value for a whole dataset with thousands of lines working in parallel mode ( Ghorbani and Zou 2019 ) . In a Data Shapley approximation study ( Mitchell et al. 2022 ) , they mentioned that a critical component in making these approximations practical is the selection of the most informative subset combinations, which can significantly influence both accuracy and efficiency. For instance having a repository like MIMIC-III, which contains daily patient reports from Beth Israel Deaconess Medical Center ( Johnson et al. 2018 ) , it is crucial to identify the missing or corrupted data tuples, as they can negatively impact ML algorithms. Previous studies ( Ghorbani and Zou 2019 ; Mitchell et al. 2022 ) , showed that approximate methods can be used to detect low-quality data tuples. However, these methods still suffer from high computational complexity, often requiring hours to days. This makes their application to large-scale, real-world scenarios impractical. In order to enable practical assessment of dataset quality, we propose a Data Shapley approximation method called Chunked Data Shapley (C-DaSh) . This method aims to close the gap between the computational efficiency and identify the quality of each data tuple accurately. C-DaSh , instead of evaluating the contribution of each individual data tuple, groups data tuples into chunks and computes the contribution of each chunk to the performance of a data-driven ML algorithm. By incorporating a subset selection prior to the Data Shapley computation, we identify the most optimal subsets which enhances the ability of our method to identify low-quality data chunks. Unlike most existing Data Shapley approximation methods that are limited to either classification or regression/time-series tasks, our approximation method is designed to work effectively for both tasks. In comparison to prior baseline work by ( Ghorbani and Zou 2019 ) , which introduces two Data Shapley approximations methods, the Gradient Shapley(G-Shapley) and Truncated Monte Carlo Shapley (TMC-Shapley), our approach demonstrates improved accuracy in detecting low-quality data and achieves considerably improved performance. The main contributions of our work can be summarised as follows: • We introduce a novel Data Shapley approximation algorithm, C-DaSh , that partitions datasets into chunks and computes quality scores per chunk instead of individual data tuples. Using our subset selection process to identify the most informative data chunks, our method effectively reduces computational complexity from exponential to manageable levels with no degradation in accuracy. • We extend the applicability of our method to support both classification and regression/time-series tasks, demonstrating versatility across different domains. • We provide an experimental evaluation across multiple real-world datasets with varied data quality issues (noise, missing values, label corruption), comparing our method against two established Data Shapley approximation techniques. Results show that C-DaSh can identify data quality more accurately, achieving impressive speedups ranging from 80 \times to 2300 \times . We also provides insights into optimal chunk sizes and subset configurations, balancing

Read the complete text