Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets

Tommaso Bendinelli, Artur Dox, Christian Holz · 2025

Read Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets by Tommaso Bendinelli, Artur Dox, Christian Holz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06664.txt.

Introduction

Tommaso Bendinelli, Artur Dox, and Christian Holz set a language model loose on dirty data sets in a 2025 study. Learning models are only as good as their training tables, yet real tables hold sensor slips, typos, and merge scars. Cleaning them takes hand work and field know how, which makes the chore slow and dear.

Their agent pairs the model with Python code and may touch only the training table, not the learning plan. The test tables were known sets spoiled on purpose with set errors. The agent caught plain faults such as absurd values and lone spikes, using hints from the same row and notes from past rounds. It missed faults that hide in trends across rows, such as slow drift and group bias.

Catalog record

CREATORTommaso Bendinelli, Artur Dox, Christian Holz
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-06664

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets Tommaso Bendinelli, Artur Dox, Christian Holz Abstract High-quality, error-free datasets are a key ingredient in building reliable, accurate, and unbiased machine learning (ML) models. However, real world datasets often suffer from errors due to sensor malfunctions, data entry mistakes, or improper data integration across multiple sources that can severely degrade model performance. Detecting and correcting these issues typically require tailor-made solutions and demand extensive domain expertise. Consequently, automation is challenging, rendering the process labor-intensive and tedious. In this study, we investigate whether Large Language Models (LLMs) can help alleviate the burden of manual data cleaning. We set up an experiment in which an LLM, paired with Python, is tasked with cleaning the training dataset to improve the performance of a learning algorithm without having the ability to modify the training pipeline or perform any feature engineering. We run this experiment on multiple Kaggle datasets that have been intentionally corrupted with errors. Our results show that LLMs can identify and correct erroneous entries—such as illogical values or outliers—by leveraging contextual information from other features within the same row, as well as feedback from previous iterations. However, they struggle to detect more complex errors that require understanding data distribution across multiple rows, such as trends and biases. 1 Introduction Despite being essential for achieving high model performance ( Jäger & Biessmann, 2024 ) , data cleaning remains one of the least engaging yet most time-consuming tasks for data scientists ( Sambasivan et al., 2021 ) . While many algorithms and approaches exist to speed up this process, significant challenges remain in efficiently deploying fully automated methods in real world scenarios ( Ni et al., 2023 ) . This is because devising a one-size-fits-all method that works universally for data cleaning is hard. What constitutes an error varies from dataset to dataset. As a result, the standard practice is to manually explore and inspect the dataset using domain knowledge, iteratively formulating hypotheses about potential erroneous entries, correcting these errors, and repeating the process until the dataset is deemed clean ( Ridzuan & Zainon, 2019 ) . In recent years, LLMs have revolutionized various fields, including coding ( Jiang et al., 2024 ) , writing ( Li et al., 2024a ) , information search ( Zhu et al., 2023 ) . Even within the field of ML—the very domain from which LLMs originate—researchers are exploring their potential to automate tasks such as feature selection, model tuning, and optimization ( Huang et al., 2024a ; Küken et al., 2024 ) . Some studies suggest that these models have already surpassed median human performance for certain tasks ( Chan et al., 2024 ) . However, most studies have focused on boosting ML model performance by optimizing model architecture and feature processing, rather than on enhancing the quality of the raw data. Therefore, in this work, we shift the focus to data quality. Specifically, we ask: If we keep the pre-processing and training pipeline fixed, can an LLM improve model performance purely by detecting and correcting errors in the data? Our main contribution is the introduction of a simple framework for the systematic study of LLMs’ ability to interact with datasets to achieve a specific goal—namely, improving the quality of training data for ML downstream tasks and to provide an initial analysis of the LLMs’ performance. 2 Related Work 2.1 Data cleaning methods Data scientists have a wide array of tools available for detecting and correcting errors and inconsistencies. Classical approaches rely on predefined rules and statistical techniques to identify issues in syntax, semantics, and outliers. These methods might involve checking for violations of integrity constraints, identifying duplicate entries, or applying outlier detection algorithms ( Fan & Geerts, 2012 ) . For error correction, problematic data is often replaced using external sources, basic statistical computations, or established integrity constraints ( Ilyas et al., 2015 ) . However, these techniques require careful parameter tuning and rigid rule definitions, making the process both time-consuming and labor-intensive. To alleviate these challenges, several approaches have been developed: Raha ( Mahdavi et al., 2019 ) and HoloDetect ( Heidari et al., 2019 ) focus on error detection, while Baran ( Mahdavi & Abedjan, 2020 ) addresses error correction. These methods leverage ML to reduce or even eliminate the reliance on manually defined rules. Furthermore, these methods tend to also have better overall performance than classical methods, however they struggle in case of rare or complex error types ( Ni et al., 2023 ) . With the advent of LLMs, many works have begun exploring their utilisation for dataset cleaning motivated by the common sense reasoning of these models. Narayan et al. (2022) has explored using the GPT-3.5 architecture for various data wrangling tasks including error detection and data imputation, showing state of the art performance. Other works, such as Qi & Wang (2024) and Li et al. (2024b) , also explore the use of LLMs for value standardization. Recently, IterClean ( Ni et al., 2024 ) has claimed state of the art performance for entire detection-correction pipeline while using fewer supervised examples than previous methods. However, while the results are impressive, current evaluation has been restricted only to established benchmarks of data cleaning, and it still unclear to which extent LLM can automate the entire data cleaning pipeline ( Majumder et al., 2024 ) . 2.2 Benchmarking LLM powered agents on data science tasks LLMs have been extensively studied for a variety of data science applications. The seminal work of Lai et al. (2023) benchmarked LLMs for code generation in data science using thousands of problems specifically targeting data science tasks. Other studies have expanded the range of tasks and improved evaluation methodologies ( Zhang et al., 2024 ; Huang et al., 2024b ; Galimzyanov et al., 2024 ) . Closely related to our work, several benchmarks move beyond simple question answering to enable function tool calling and multi-turn interactions. For example, Liu et al. (2023) evaluates LLMs on question answering tasks that involve tabular data. Similarly, Hu et al. (2024) proposes a framework for evaluating LLM-based agents on data analysis tasks—including data exploration, summary statistics, regression analysis, and feature engineering. Moreover, Majumder et al. (2024) introduces a comprehensive benchmark that formalizes the entire multi-step process of data-driven discovery—from hypothesis formulation and verification using both real-world and synthetic tasks across diverse domains—to systematically assess and improve LLMs’ capabilities in automating scientific discovery. Likewise, Chen et al. (2024) presents a benchmark that assesses LLMs ability to autonomously generate executable Python programs for diverse, expert-validated data-driven scientific discovery tasks. Finally, approaches such as Huang et al. (2024a) and Chan et al. (2024) benchmark LLMs on ML competitions with explicit target of improving performance on a test set. However, the focus in these works is on feature engineering and machine learning, rather than on improving the quality of the training data. 3 Method Our goal is to evaluate how LLMs can effectively detect and correct errors in dirty training datasets that negatively impact model performance on a held-out dataset. In our setup, the LLM has access to two tools. The first is the interactive Python shell IPython ( Pérez & Granger, 2007 ) , which executes

Read the complete text