fastml: Guarded Resampling Workflows for Safer Automated Machine Learning in R
Selcuk Korkmaz, Dincer Goksuluk, Eda Karaismailoglu · 2026
Read fastml: Guarded Resampling Workflows for Safer Automated Machine Learning in R by Selcuk Korkmaz, Dincer Goksuluk, Eda Karaismailoglu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05225.txt.
Introduction
Korkmaz, Goksuluk, and Karaismailoglu present fastml, an R package through redo, redoing prep inside each redo. Leakage arises when scaling, filling, or other changes are estimated before redo, lifting seen scores. It supports redo, blocks risky setups, audits recipes, and has boxed runs and notes. A test pitting global against fold-local scaling shows prep inflates seen scores versus guarded redo. fastml matched tidymodels on held-out scores while cutting work, and ran lasting models through one panel.
Catalog record
| CREATOR | Selcuk Korkmaz, Dincer Goksuluk, Eda Karaismailoglu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78024 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.05225
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
fastml: Guarded Resampling Workflows for Safer Automated Machine Learning in R Selcuk Korkmaz, Dincer Goksuluk, Eda Karaismailoglu Abstract Preprocessing leakage arises when scaling, imputation, or other data-dependent transformations are estimated before resampling, inflating apparent performance while remaining hard to detect. We present fastml, an R package that provides a single-call interface for leakage-aware machine learning through guarded resampling, where preprocessing is re-estimated inside each resample and applied to the corresponding assessment data. The package supports grouped and time-ordered resampling, blocks high-risk configurations, audits recipes for external dependencies, and includes sandboxed execution and integrated model explanation. We evaluate fastml with a Monte Carlo simulation contrasting global and fold-local normalization, a usability comparison with tidymodels under matched specifications, and survival benchmarks across datasets of different sizes. The simulation demonstrates that global preprocessing substantially inflates apparent performance relative to guarded resampling. fastml matched held-out performance obtained with tidymodels while reducing workflow orchestration, and it supported consistent benchmarking of multiple survival model classes through a unified interface. fastml: Guarded Resampling Workflows for Safer Automated Machine Learning in R Selcuk Korkmaz 1 1 1 1 Corresponding author: [email protected] , ORCID: 0000-0003-4632-6850 , Dincer Goksuluk 2 2 2 2 ORCID: 0000-0002-2752-7668 , Eda Karaismailoglu 3 3 3 3 ORCID: 0000-0003-3085-7809 1 Faculty of Medicine, Department of Biostatistics, Trakya University, Edirne 22030, Türkiye 2 Faculty of Medicine, Department of Biostatistics, Sakarya University, Sakarya 54290, Türkiye 3 Faculty of Medicine, Department of Biostatistics, University of Health Sciences, Ankara 06010, Türkiye Keywords: machine learning, preprocessing leakage, cross-validation, resampling, survival analysis 1 Introduction Over the past decade, machine learning workflows in R have evolved from monolithic interfaces, such as caret ( Kuhn 2008 ) , toward modular ecosystems including tidymodels ( Kuhn and Wickham 2020 ) and mlr3 ( Lang et al. 2019 ) . These frameworks decompose modeling into explicit steps for resampling, preprocessing, model specification, and evaluation, providing flexibility and methodological transparency. However, this modularity also shifts responsibility to the user to correctly assemble and coordinate multiple components into a valid pipeline. This shift increases the risk of methodological error by expanding the number of decisions that must be coordinated correctly. Users must ensure that preprocessing, resampling, and model fitting are applied in the proper order and with strict separation between training and evaluation data. In complex workflows, particularly those involving grouped or nested resampling, even experienced practitioners may inadvertently introduce errors that are not detected by standard software checks. Among these errors, data leakage remains one of the most consequential. In particular, preprocessing leakage occurs when transformations such as scaling, imputation, or feature construction are estimated using information that should be unavailable at evaluation time, for example by learning transformations on the full dataset prior to resampling. This violates training-test isolation and can substantially inflate performance estimates while remaining difficult to detect, because models may still appear well-validated under standard cross-validation ( Hastie et al. 2009 ) . Recent studies report that leakage and related evaluation flaws are widespread in applied machine learning and contribute to irreproducible and overly optimistic results across multiple scientific domains ( Kapoor and Narayanan 2023 ; Rosenblatt et al. 2024 ; Tampu et al. 2022 ) . Modern R frameworks provide tools that can prevent preprocessing leakage, most notably by coupling transformations to resampling-aware workflows. However, these safeguards are typically optional rather than enforced. Correct usage requires users to explicitly define recipes or workflows, ensure that preprocessing is trained within each resampling split, and avoid global data-dependent transformations prior to model evaluation. When these conditions are not met, incorrect pipelines can run without warnings while yielding biased estimates. As a result, methodological validity often depends more on user discipline than on guarantees provided by the software. For example, tidymodels’ fit_resamples() and tune_grid() correctly re-estimate preprocessing within each fold when a recipe is embedded in a workflow; however, users must assemble this workflow correctly, and nothing prevents them from applying global preprocessing before invoking these functions. The fastml package ( Korkmaz et al. 2025 ) addresses this gap through a safety-by-design interface that makes fold-local preprocessing the default execution path while retaining access to established modeling engines. A central mechanism is its guarded resampling path, in which preprocessing is re-estimated within each resample on the analysis split and the fold-trained transformations are then applied to the corresponding assessment split. fastml also includes checks intended to surface common configuration errors, including aborting a resampling run when a full-analysis (no-holdout) split is detected and rejecting user recipes that appear to depend on external data sources. When hyperparameter tuning is requested under this guarded path, tuning is performed inside the same resampling loop. More broadly, fastml provides resampling options and diagnostics that target additional evaluation pitfalls. Grouped resampling can keep related observations together when repeated measures or clustered records exist. For time-ordered settings, blocked or rolling resampling relies on an explicit ordering variable and fastml warns when the ordering required by the chosen design is not provided. An optional audit mode can flag risky patterns in custom preprocessing, such as references to the global environment or code that reads data from files or writes outputs to files, which can create hidden dependencies and undermine reproducibility. These mechanisms are best understood as leakage-risk mitigation under supported configurations rather than universal guarantees; users can still introduce leakage upstream, for example by supplying preprocessed inputs or externally constructed resamples. The contribution of fastml is therefore not a new learning algorithm, but an execution model that makes leakage-aware evaluation easier to carry out correctly in practice by packaging common patterns, defaults, and checks into a single interface. This paper describes the design and implementation of fastml , details how guarded resampling and related checks are realized when those execution paths are used, and demonstrates via simulation how preprocessing leakage can inflate performance estimates and how fold-local workflow fitting mitigates this failure mode under the evaluated configurations. The remainder of this paper is organized as follows. Section 2 presents the guarded resampling architecture and its implementation, introduces a Monte Carlo simulation designed to quantify the impact of preprocessing leakage, describes the native survival analysis interface, outlines the software architecture and audit utilities, and compares fastml with existing frameworks. Section 3 reports the simulation results, followed by applied benchmarks and case studies in classification, regression, and survival analysis. Section 4 discusses limitations and future directions. 2 Materials and methods 2.1 Guarded resampling As introduced in the previous section, preprocessing leakage occurs when data-dependent transformation parameters are estimated on the full dataset before resampling, so that assessment-fold observations influence training-fold transformations ( Kapoor and Narayanan 2023 ; Kaufman et al. 2012 ; Vabalas et al. 2019 ) . This breaks training-test separation and tends to yield optimistic performance estimates, even for unsupervised transformations that never access outcome labels ( Hastie et al. 2009 ; Kaufman et al. 2012 ) . Although existing R frameworks can avoid this problem by embedding transformations within resampling-aware workflows, such isolation is optional, and leaky pipelines can execute without warnings. Figure 1 illustrates the distinction. In a leaky workflow, transformation parameters are estimated on the full dataset before resampling, so each fold’s assessment data influences the transformation applied to that fold’s training data. In a properly specified workflow,