SoftPipe: A Soft-Guided Reinforcement Learning Framework for Automated Data Preparation

Jing Chang [email protected] Chang Liu [email protected] Jinbin Huang [email protected] Shuyuan Zheng [email protected] Rui Mao [email protected], Jianbin Qin [email protected] College of Computer Science and Software Engineering, Shenzhen UniversitySchool of Artifi · 2025

Read SoftPipe: A Soft-Guided Reinforcement Learning Framework for Automated Data Preparation by Jing Chang [email protected] Chang Liu [email protected] Jinbin Huang [email protected] Shuyuan Zheng [email protected] Rui Mao [email protected], Jianbin Qin [email protected] College of Computer Science and Software Engineering, Shenzhen UniversitySchool of Artifi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13710.txt.

Introduction

Jing Chang and co-authors present in 2025 a learning system that builds data preparation pipelines on its own. Getting raw data ready is a core but hard step in machine learning, because the choices combine in a vast number of ways. Learning-based searches often fence off parts of that space with rigid rules, which can cut away the best pipeline.

Their system, SoftPipe, swaps rigid rules for soft guidance. A large language model supplies a high-level strategy that steers the search without locking it. That guidance is blended with a learned ranking score for each step and with the agent's own long-term value estimates. Tests on eighteen data sets showed pipeline quality up to about fourteen percent better than rival methods. The system also reached good pipelines almost three times faster.

Catalog record

CREATORJing Chang [email protected] Chang Liu [email protected] Jinbin Huang [email protected] Shuyuan Zheng [email protected] Rui Mao [email protected], Jianbin Qin [email protected] College of Computer Science and Software Engineering, Shenzhen UniversitySchool of Artifi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13710

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SoftPipe: A Soft-Guided Reinforcement Learning Framework for Automated Data Preparation Jing Chang [email protected] Chang Liu [email protected] Jinbin Huang [email protected] Shuyuan Zheng [email protected] Rui Mao [email protected], Jianbin Qin [email protected] College of Computer Science and Software Engineering, Shenzhen UniversitySchool of Artificial Intelligence, Shenzhen UniversityGraduate School of Information Science and Technology, Osaka University Abstract Data preparation is a foundational yet notoriously challenging component of the machine learning lifecycle, characterized by a vast combinatorial search space. While reinforcement learning (RL) offers a promising direction, state-of-the-art methods suffer from a critical limitation: to manage the search space, they rely on rigid “hard constraints” that prematurely prune the search space and often preclude optimal solutions. To address this, we introduce SoftPipe, a novel RL framework that replaces these constraints with a flexible “soft guidance” paradigm. SoftPipe formulates action selection as a Bayesian inference problem. A high-level strategic prior, generated by a Large Language Model (LLM), probabilistically guides exploration. This prior is combined with empirical estimators from two sources through a collaborative process: a fine-grained quality score from a supervised Learning-to-Rank (LTR) model and a long-term value estimate from the agent’s Q-function. Through extensive experiments on 18 diverse datasets, we demonstrate that SoftPipe achieves up to a 13.9% improvement in pipeline quality and 2.8 × \times faster convergence compared to existing methods. 1 Introduction Data preparation is a foundational, yet notoriously labor-intensive, component of the machine learning (ML) lifecycle [ 1 ] . The process of transforming raw, often messy data into a clean, well-structured format suitable for model training is not a single action but a multi-stage workflow encompassing tasks like data cleaning, integration, transformation, feature engineering, and feature selection. The quality of the final ML model is profoundly dependent on the quality of this preparatory pipeline [ 2 , 3 ] . The task’s primary difficulty lies in discovering an optimal sequence of operations from a vast library of potential operators, subject to a severe combinatorial explosion. To illustrate, with merely 24 available operators and pipelines of a modest length of 8, the search space contains over O(24^{8})\approx 1.1\times 10^{11} possible configurations. Reinforcement Learning (RL) has emerged as a particularly promising paradigm for this challenge, as its formulation as a sequential decision-making process naturally aligns with pipeline construction [ 4 , 5 , 6 , 7 , 8 , 9 ] . However, a close examination of existing state-of-the-art methods reveals a critical, unaddressed limitation: to manage the vast search space, they implicitly rely on “hard constraints” that rigidly structure the pipeline construction process. These constraints, while simplifying the search, prematurely prune the solution space and often preclude optimal pipelines. A prime example is CtxPipe [ 10 ] , a leading RL-based framework. While claiming flexibility, its implementation imposes a crucial hard constraint: It prevents the selection of multiple operators belonging to the same semantic type. These types, detailed in Table 1 , span the entire data preparation workflow. This seemingly minor restriction has major consequences. On the wall-robot-nav dataset as Table 2 listed, CtxPipe’s constrained search achieves an accuracy of 0.946. Similarly, HAI-AI [ 11 ] and TPOT [ 12 ] , also fall short, achieving 0.896 and 0.941 respectively. Compared to an approximated optimal solution (ES*) discovers a superior pipeline - [QuantileTransformer, StandardScaler, PolynomialFeatures]-reaching 0.962 accuracy. This pipeline’s effectiveness hinges on combining two ‘Feature Preprocessing’ operators, a structure inaccessible to CtxPipe’s search strategy, thus demonstrating the weakness of such hard constraints 1 1 1 All data presented are directly drawn from original paper. . Table 1: Selected data preparation operators and their types Type Operator (ID) Imputer Mean(1), Median(2), Most Frequent(3) Encoder Label(4), One-hot(5) Feature Preprocessing Min Max Scaler(6), Max Abs Scaler(7), Robust Scaler(8), Standard Scale(9), Quantile Transformer(10), Power Transformer(12), Log Transformer(11), Normalizer(13), K-bins Discretizer(14) Feature Engineering Polynomial Features(15), Interaction Features(16), PCA AUTO(17), PCA LAPACK(18), PCA ARPACK(19), Incremental PCA(20), Kernel PCA(21), Truncated SVD(22), Random Trees Embedding(23) Selection Variance Threshold(24) Table 2: wall-robot-nav dataset performance Comparison Approach Accuracy Pipelines 1 CtxPipe 0.946 [6, 23] HAI-AI 0.896 [6, 15, 24] TPOT 0.941 [14, 15] ES * 0.962 [10, 9, 15] 1 The pipelines are represented as sequences of operator id, and (id, operator) mappings are listed in Table 1 . In summary, the existing methods discussed above with hard constraints reveals several limitations that block the search of optimal data preparation pipelines: (1) Rigid search space pruning: Hard constraints restrictions exclude potentially optimal solutions from consideration; (2) Premature convergence: By enforcing strict rules early in the search process, these methods may converge to suboptimal local minima without exploring promising alternative pathways.(3) Limited adaptability: Hard constriants lack the flexibility to adapt based on changing data characteristics during the search process. To address these limitations, we introduce SoftPipe, a framework that replaces hard constraints with “soft guidance”. Our approach formulates action selection as a Bayesian inference problem. Instead of rigid rules, we use a Large Language Model (LLM) to generate a high-level strategic prior, which probabilistically guides the exploration. This prior is then synergistically combined with empirical evidence from a fine-grained Learning-to-Rank (LTR) model and the agent’s own long-term value estimates (Q-function). This principled fusion allows SoftPipe to benefit from strategic guidance while maintaining the flexibility to override it when evidence warrants, enabling the discovery of unconventional yet optimal pipelines. In summary, our contributions are: • We identify and expose a critical limitation in current AutoDP methods: their reliance on rigid “hard constraints” that prematurely prune the search space and prevent the discovery of optimal pipelines. • We propose SoftPipe, a novel RL framework that enables flexible exploration via a “soft guidance” mechanism grounded in Bayesian inference that facilitates the collaboration of a strategic LLM prior, a fine-grained LTR score, and an empirical RL value function. • We demonstrate significant improvements, achieving up to 13.9% higher pipeline quality and 2.8× faster convergence compared to state-of-the-art methods across 18 diverse datasets. 2 Related Works Automated Data Preparation (AutoDP) AutoDP has emerged as a critical component in AutoML systems aiming to streamline preprocessing workflows. Early approaches like Auto-Sklearn [ 13 ] integrated rule-based preprocessings along side hyperparameter optimization, while TPOT [ 12 ] extended the process by combining genetic algorithms to evolve feature engineering and transformation steps. Interactive systems like Data Civilizer [ 14 ] and DynaML [ 15 ] provide visual platforms for data curation. Commercial systems like H2O.ai [ 16 ] further automate data preparation by incorporating automated feature engineering templates. These cases demostrate the awareness that data preparation strongly impacts tasks in data science. Reinforcement Learning for AutoDP Recent studies exploreReinforcement learning (RL) and neural networks (NN) for AutoDP. RL-based models, such as Learn2Clean [ 4 ] , model data as a sequential decision process [ 17 , 18 , 19 ] . Meanwhile, MetaPrep [ 20 ] combines meta-learning with RL to adaptively select preprocessing steps for new tasks, reducing search space via learned priors. In the context of AutoDP, RL has been applied to specific sub-problems like feature selection [ 11 , 10 , 5 ] . However, the RL agent may face scalability challenges and lack of inherent structure guidance, causing possible overfitting risks. LLMs for Code and Planning Recently, Large Language Models (LLMs) have shown remarkable capabilities in generating code and plans

Read the complete text