Beyond algorithm hyperparameters: on preprocessing hyperparameters and associated pitfalls in machine learning applications Christina Sauer, Anne-Laure Boulesteix, Luzia Hanßum, Farina Hodiamont, Claudia Bausewein, Theresa Ullmann Abstract Adequately generating and evaluating prediction models based on supervised machine learning (ML) is often challenging, especially for less experienced users in applied research areas. Special attention is required in settings where the model generation process involves hyperparameter tuning, i.e. data-driven optimization of different types of hyperparameters to improve the predictive performance of the resulting model. Discussions about tuning typically focus on the hyperparameters of the ML algorithm (e.g., the minimum number of observations in each terminal node for a tree-based algorithm). In this context, it is often neglected that hyperparameters also exist for the preprocessing steps that are applied to the data before it is provided to the algorithm (e.g., how to handle missing feature values in the data). As a consequence, users experimenting with different preprocessing options to improve model performance may be unaware that this constitutes a form of hyperparameter tuning, albeit informal and unsystematic, and thus may fail to report or account for this optimization. To illuminate this issue, this paper reviews and empirically illustrates different procedures for generating and evaluating prediction models, explicitly addressing the different ways algorithm and preprocessing hyperparameters are typically handled by applied ML users. By highlighting potential pitfalls, especially those that may lead to exaggerated performance claims, this review aims to further improve the quality of predictive modeling in ML applications. Keywords : predictive modeling, machine learning, preprocessing, hyperparameter optimization, tuning 1 Introduction Many applied research areas have recently seen an increase in the development of prediction models based on supervised machine learning (ML) algorithms. However, after initially generating widespread enthusiasm—partly due to the availability of user-friendly software that enables model development without requiring extensive expertise—ML-based prediction models are now undergoing critical reexamination ( Ball, 2023a ; Kapoor & Narayanan, 2023a ; Pfob et al., 2022a ) . Among other concerns, such as insufficient reporting of relevant aspects of the model development process, it has been found that the claimed predictive performance of many models is considerably exaggerated ( Kapoor & Narayanan, 2023a ; Dhiman et al., 2022b ; Dhiman et al., 2022c ; Andaur et al., 2021a ) . While some of the pitfalls leading to such optimistically biased performance claims (e.g., using the exact same observations for model generation and evaluation) typically occur only among very inexperienced applied ML users and are well known within the ML research community, others arise more subtly ( Domingos, 2012a ; Kapoor & Narayanan, 2023a ; Poldrack et al., 2020a ; Hofman et al., 2023a ) . This is particularly true when the model generation process involves data-driven hyperparameter optimization, which is also referred to as hyperparameter tuning and is commonly employed in ML applications. The most prominent type of hyperparameters (HPs) are those associated with the learning algorithm, which specify its configuration (e.g., the minimum number of observations in each terminal node for tree-based algorithms). If selected by an adequate (and ideally automated) tuning procedure, HPs can substantially enhance the performance of the resulting prediction model. However, HP tuning also complicates model evaluation, as common procedures such as simple k -fold cross-validation no longer guarantee an unbiased assessment ( Bischl et al., 2023a ; Hosseini et al., 2020a ) . An additional challenge comes from the fact that, beyond algorithm HPs, there are also preprocessing HPs, which specify the steps applied to the data before it is fed into the learning algorithm (e.g., selecting the set of features for prediction or determining how missing feature values are handled; Binder & Pfisterer, 2024a ; Bischl et al., 2023a ). While the tuning of algorithm HPs is rightfully considered important for model performance, the relevance of tuning preprocessing HPs should not be overlooked. Preprocessing steps can make or break a model’s predictive performance, and solely relying on user expertise to specify these steps (which is the alternative to tuning) is often impractical and may result in arbitrary decisions ( Kuhn & Johnson, 2013a ) . Despite this, reports of tuning preprocessing HPs aside from feature selection are relatively rare. This could be because integrating preprocessing HPs into automated tuning workflows typically requires advanced programming expertise, which not all applied ML users have, or because this possibility is not widely recognized. Importantly, the limited use of automated tuning procedures for preprocessing HPs does not mean that these HPs are not being tuned at all. In fact, it appears fairly common for applied ML users to experiment informally with different preprocessing options ( Hosseini et al., 2020a ; Hofman et al., 2023a ; Lones, 2024a ) , often without realizing that this constitutes a form of (manual) HP tuning. If this type of tuning is indeed conducted subconsciously, it will also remain unaccounted for during model evaluation, thereby increasing the risk of drawing overly optimistic conclusions about the model’s performance. To avoid such issues, it is essential to educate users in applied settings about the different types of HPs, the different forms of HP tuning, and how tuning can impact both the true and estimated performance of prediction models. Although valuable literature already exists describing the concept of HP tuning and various automated procedures (e.g., Bischl et al., 2023a ; Bartz et al., 2023a ; Feurer & Hutter, 2019a ) , this research primarily adopts the perspective of ML methods researchers who are concerned with evaluating the overall performance of ML algorithms used to generate prediction models. This focus does not align with the perspective of applied ML users, who are more interested in the performance of a specific prediction model. Although this literature is still useful for them—since the general principles described there essentially hold for all types of audiences—applied ML users additionally need specific guidance for developing their “final model” (a notion that does not exist in the methodological context). Moreover, they may find it challenging to extract the relevant insights from literature aimed at a different audience with partly different needs. In contrast, literature explicitly directed toward applied ML users tends to either focus on general guidelines for ML-based predictive modeling, lacking detailed coverage of HP tuning (e.g., Kuhn & Johnson, 2013a ; Pfob et al., 2022a ; Lones, 2024a ; Kapoor et al., 2024a ; Poldrack et al., 2020a ; Collins et al., 2024b ; Royen et al., 2023a ) , or addresses HP tuning only within specific research areas (e.g., Hosseini et al., 2020a ; Dunias et al., 2024a ) . Additionally, much of the existing HP tuning literature does not consider preprocessing HPs. Exceptions include the review by Bischl et al., 2023a , which, however, touches on this topic only briefly. This lack of detail is reasonable, given that preprocessing HPs can, in principle, be tuned using the same automated procedures as algorithm HPs. However, this perspective overlooks that preprocessing HPs are often tuned manually in applied settings, which carries implications different from those associated with automated tuning. This paper aims to complement the existing literature by reviewing the implications and pitfalls of HP tuning in the generation and evaluation of prediction models from the perspective of applied ML users with varying levels of expertise. It explicitly distinguishes between preprocessing and algorithm HPs, as well as the different procedures commonly used to tune them in practice. A particular focus is placed on the potential for optimistically biased performance estimation, which is also illustrated using a real-world prediction problem from palliative care medicine. The paper is structured as follows. Section 2 introduces the key concepts related to predictive modeling using ML, including the two types of HPs. In the next two sections, the challenges and pitfalls that arise in the generation and evaluation of prediction models are described, differentiating between the setting where all HPs are pre-specified (Section 3 ) and the setting where one or more HPs are selected through tuning (Section 4 ). Section 5 empirically illustrates the impact of different tuning and evaluation procedures on the estimated model performance. Section 6 summarizes the key insights, discusses the limitations of the empirical study, and outlines future research directions. 2 General concepts of predictive modeling using supervised ML 2.1 Terminology and notation The following terminology and notation is adapted from Bischl et al. (2023a) . Let \mathcal{D}_{\text{train}} be a labeled data set with n_{\text{train}} observations. Accordingly, each observation i ( i=1,\dots,n_{\text{train}} ) consists of an outcome y^{(i)} (i.e. the variable to be predicted, also referred to as label or target) and a p -dimensional feature vector \boldsymbol{x}^{(i)} (i.e. the p variables used to predict y^{(i)} , also referred to as predictors), where y^{(i)} and \boldsymbol{x}^{(i)} can take any value from the outcome space \mathcal{Y} and feature space \mathcal{X} , respectively. Two common types of prediction problems are regression, for which y^{(i)} can be any real number (i.e. \mathcal{Y}=\mathds{R} ), and classification, for which y^{(i)} can be one of g classes (i.e. \mathcal{Y} is finite and categorical with |\mathcal{Y}|=g ). We assume that the observations in \mathcal{D}_{\text{train}} are independent and have been sampled from the same (unknown) probability distribution \mathds{P}_{xy} . The general aim of supervised ML is to “learn” a model from the data set \mathcal{D}_{\text{train}} that is able to predict the outcome values of new observations. Essentially, a prediction model is a function \hat{f}:\mathcal{X}\rightarrow\mathds{R}^{g} that maps any observed feature vector \boldsymbol{x} to a prediction vector \hat{f}(\boldsymbol{x}) in \mathds{R}^{g} . The prediction vector \hat{f}(\boldsymbol{x}) either directly corresponds to the predicted outcome value (e.g., for regression, where g=1 ) or can be transformed accordingly (e.g., for classification, where \hat{f}(\boldsymbol{x}) corresponds to predicted probabilities for each class and the predicted class could be the class with the highest probability). The prediction model results from a learning pipeline \mathcal{I} , which uses the data set \mathcal{D}_{\text{train}} to find the function \hat{f} that yields the best predictions for the true outcome values in \mathcal{D}_{\text{train}} . To stress that a prediction model \hat{f} is based on learning pipeline \mathcal{I} and data set \mathcal{D}_{\text{train}} , we write \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} . The prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} can usually be parameterized, meaning that it is defined by a set of parameters \boldsymbol{\hat{\theta}}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} (simply denoted as \boldsymbol{\hat{\theta}} when data set and learning pipeline are clear from context and \boldsymbol{\theta} when referring to the parameters prior to estimation). There are two key processes associated with \mathcal{I} and \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} , which we will explore in more detail throughout the paper: (i) the training process, in which the learning pipeline \mathcal{I} is applied to \mathcal{D}_{\text{train}} and estimates the parameters \boldsymbol{\hat{\theta}}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} and thus the prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} , and (ii) the prediction process, in which \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} is used to make predictions for an observation (whether from \mathcal{D}_{\text{train}} or from a new data set) with feature vector \boldsymbol{x} , resulting in \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}}(\boldsymbol{x}) . Note that to make predictions on a new data set, the outcome does not need to be observed (it would only be necessary for evaluating those predictions). The training and prediction processes serve as the foundation for more complex processes related to the development of prediction models, which we will address in Section 2.4 . 2.2 Learning pipeline Each learning pipeline \mathcal{I} contains a learning algorithm as a central component but can also include several preprocessing steps that are performed before the algorithm is applied to the data. Since preprocessing steps are a particular focus of this paper, we use the term “learning pipeline” instead of the more common term “learner” to emphasize that \mathcal{I} can consist of several components. Note that for now, we consider all components of \mathcal{I} as fixed, but we will discuss the case in which they can be modified in Section 2.3 . 2.2.1 Learning algorithm The choice of learning algorithm usually depends on the specific prediction problem. For example, if the desired prediction model is a decision tree (which is the case for the real-world prediction problem considered in Section 5 ), a possible algorithm choice is the well-known Classification and Regression Tree algorithm (CART), which partitions the feature space \mathcal{X} by a sequence of binary splits into terminal nodes and assigns a prediction value to each terminal node ( Breiman et al., 1984a ) . In this case, the parameters of the learning algorithm contained in \boldsymbol{\hat{\theta}}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} are the splitting rules that generate the tree structure (i.e. which features are used with which threshold value) and the prediction values at each terminal node. The learning algorithm can also consist of multiple individual algorithms that are combined into one overall algorithm (e.g., random forests). These types of algorithms are referred to as ensemble methods, but will not be discussed further in this paper. In general, the choice of algorithm has a large impact on the hypothesis space of the learning pipeline, i.e. the set of prediction models the learning pipeline can generate. For example, selecting a standard linear regression as algorithm (with \boldsymbol{\hat{\theta}}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} containing the regression coefficients) would imply that the corresponding learning pipeline would not be able to learn prediction models that do not correspond to linear combinations of the features (e.g., polynomials). 2.2.2 Preprocessing While a data set can, in theory, be fed directly into the algorithm (i.e. the algorithm is the only component of the learning pipeline), it typically undergoes some modification first. This process can be referred to as data preprocessing and encompasses all the steps taken to transform the data set from its rawest available form into the final form provided as input to the learning algorithm ( Kapoor et al., 2024a ) . Data preprocessing steps are usually performed to improve the performance of the resulting prediction model, to enable the data to be (better) handled by the learning algorithm ( Thomas, 2024a ) , or to improve the interpretability of the resulting prediction model. To better illustrate the different characteristics of preprocessing steps and their implications on the training and prediction process, we consider a simple learning pipeline as an example, which is also depicted in Figure 1 (middle panel). Figure 1 : Example of a learning pipeline \mathcal{I} consisting of two preprocessing steps and one learning algorithm. Left panel: HPs of the learning pipeline, with each HP set to an example value. Middle panel: Training process, where the learning pipeline is applied to the data set \mathcal{D}_{\text{train}} to generate the prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} . Right panel: Prediction process, where a prediction for an observation with feature vector \boldsymbol{x} is obtained by reapplying all preprocessing steps, followed by the prediction model resulting from the learning algorithm (here: a decision tree). It consists of two preprocessing steps, which are followed by the CART algorithm. The first preprocessing step is the replacement of missing feature values using mean imputation, and the second preprocessing step is the log-transformation of features. Parameterized vs. parameterless steps Based on this example learning pipeline, we can make a first distinction between preprocessing steps. This distinction concerns whether the steps have parameters estimated from \mathcal{D}_{\text{train}} (with these parameters included in \boldsymbol{\theta} ) or whether they are parameterless and are carried out independently for each observation ( Kapoor et al., 2024a ; Binder & Pfisterer, 2024a ) . In the example, the replacement of missing feature values is a parameterized preprocessing step, as it involves the parameter \theta_{impute} , representing the mean of all non-missing values estimated from \mathcal{D}_{\text{train}} . In contrast, the log-transformation of features does not involve any parameters. Other examples of preprocessing steps with parameters include centering or scaling of features, where parameters such as the mean or standard deviation are estimated from \mathcal{D}_{\text{train}} . On the other hand, creating a new feature by summing multiple features serves as another example of a parameterless preprocessing step. Application during prediction vs. training only The second key distinction in preprocessing steps concerns whether they are applied only during the training process as part of the learning pipeline or also during the prediction process. This distinction is closely related to whether a preprocessing step modifies only the feature distribution or also affects the outcome distribution. More formally, let \boldsymbol{y} denote the outcome vector in \mathcal{D}_{\text{train}} . If, after applying all preprocessing steps in the learning pipeline during training, \boldsymbol{y} remains unchanged, we classify the step as affecting only the feature distribution. Otherwise, the step affects the outcome distribution, for example, by removing or adding observations or transforming outcome values. We first consider preprocessing steps that affect only the feature distribution. These comprise all preprocessing steps mentioned above, including those in the example learning pipeline. Additional examples are dimensionality reduction techniques (e.g., principal component analysis), feature selection, or data cleaning steps that do not alter the outcome distribution (e.g., correction of errors in features) ( Kuhn & Johnson, 2013a ; Thomas, 2024a ) . Preprocessing steps of this type must be applied not only during training but also during prediction, in the same sequence as in the learning pipeline. This ensures that the model produced by the learning algorithm receives the data in the same format during prediction as it did during training, preserving the validity of the model ( Binder & Pfisterer, 2024a ) . This requirement implies that these steps are not only components of the learning pipeline \mathcal{I} but also part of the resulting prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} . Consequently, if a learning pipeline \mathcal{I} includes h preprocessing steps that only affect the feature distribution, the prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} is not a single function but a function composition of h+1 functions (omitting \mathcal{D}_{\text{train}} and \mathcal{I} for simplicity of notation): \hat{f}_{h+1}(\hat{f}_{h}(\dots(\hat{f}_{1}(\boldsymbol{x})))), (1) where \hat{f}_{h+1} corresponds to the model resulting from the learning algorithm, and \hat{f}_{h},\dots,\hat{f}_{1} reflect the h preprocessing steps. Accordingly, a more accurate name for a prediction model would be prediction model pipeline , but for brevity, we will continue to use the former. Returning to the example learning pipeline, the resulting prediction model is a composition of three functions, \hat{f}_{3}(\hat{f}_{2}(\hat{f}_{1}(\boldsymbol{x}))) , where \hat{f}_{1} , \hat{f}_{2} , and \hat{f}_{3} correspond to the imputation step, the log-transformation step, and the decision tree model, respectively. When making a prediction for one or more observations, all three functions must be applied (see Figure 1 , right panel). Importantly, if any functions constituting the prediction model are omitted during the prediction process, or if any preprocessing or algorithm parameters are re-estimated on a new data set for which predictions are to be made, the validity of the prediction model may be compromised. However, in practice, this pitfall is often unavoidable for users who wish to apply a model but were not involved in its development, as studies introducing new prediction models frequently fail to report the preprocessing steps performed prior to applying the learning algorithm ( Kapoor et al., 2024a ) . In contrast to preprocessing steps that only affect the feature distribution, preprocessing steps that modify the outcome distribution are not necessarily applied during prediction. Here, we must distinguish between steps aimed at improving compatibility with the learning algorithm and those intended to alter the scope or interpretation of the prediction model. An example of the first type is (invertible) transformations applied to the outcome during training, such as a log-transformation to reduce skewness. To ensure predictions are returned on the correct scale, these transformations must be reversed during prediction ( Thomas, 2024a ) . For instance, if the outcome was log-transformed during training, the model will output \mathrm{log}(\hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}}(\boldsymbol{x})) , which must then be exponentiated to restore the prediction to its original scale. Note that some other compatibility-focused steps are not applied at all during prediction. In the context of classification problems, this includes class-balancing steps such as oversampling, where observations from the least prevalent class are randomly resampled to overcome class imbalance effects during the training process (see, e.g., Kuhn & Johnson, 2013a , for more details). In the notation of the prediction model as a function composition introduced above, preprocessing steps that are applied only in their inverted form or not at all during prediction are represented as inversion function or identity function, respectively. In contrast, preprocessing steps that modify the outcome to alter the scope or interpretation of the prediction model should be consistently applied during prediction. For example, if a continuous outcome is discretized to convert a regression problem into a classification problem ( Hofman et al., 2023a ) , this (irreversible) transformation must also be applied to the true outcome during prediction in order to enable a meaningful comparison between the predictions and the actual outcome values. Such transformations of the outcome are not part of the prediction model itself (which maps \boldsymbol{x} to predictions, not y ), but must be performed alongside the prediction process. Moreover, since the outcome values are generally unknown when making predictions for observations from a new data set that does not correspond to \mathcal{D}_{\text{train}} , these transformations are typically not actual steps executed when making predictions but instead determine how the predictions are interpreted. 2.3 Hyperparameters Until now, we have assumed that the learning pipeline \mathcal{I} is fixed. However, individual components of \mathcal{I} usually have several hyperparameters (HPs), which determine their specific configuration and thus substantially influence the resulting prediction model. This also applies to the learning pipeline example considered in the previous section, for which possible HPs are shown in the left panel of Figure 1 (see below for further explanation). In contrast to the parameters \boldsymbol{\theta} , which are estimated as outputs of the learning pipeline, the HPs serve as inputs. This means that they must be specified before the learning pipeline is applied to the data set ( Bischl et al., 2023a ) . 2.3.1 Additional notation for HPs The following notation is based on Feurer & Hutter (2019a) . We denote the j th HP of a learning pipeline as \lambda_{j} , which is selected from its domain \Lambda_{j} (i.e. \lambda_{j}\in\Lambda_{j} ). The domain of \lambda_{j} can generally be real-valued, integer-valued, binary, or categorical, as we will see in the examples given below. All J HPs of a learning pipeline can be summarized as a vector \boldsymbol{\lambda}=(\lambda_{1},\ldots,\lambda_{J}) and their overall configuration space as \boldsymbol{\Lambda}=\Lambda_{1}\times\Lambda_{2}\dots\times\Lambda_{J} (with \boldsymbol{\lambda}\in\boldsymbol{\Lambda} ). Note that \boldsymbol{\Lambda} may contain conditionality, meaning that some HPs might only be relevant when one or more other HPs are set to a certain value (see below for examples). As described in Section 2.2 , the learning pipeline consists of several preprocessing steps and a learning algorithm. We can consequently differentiate between preprocessing and algorithm HPs, which we denote as \boldsymbol{\lambda}_{P} and \boldsymbol{\lambda}_{A} (i.e. \boldsymbol{\lambda}=(\boldsymbol{\lambda}_{P},\boldsymbol{\lambda}_{A})) . 2.3.2 Algorithm HPs Each learning algorithm usually has several HPs, which are specified by the software package used and can have a large impact on its complexity, speed, and other important properties of the algorithm ( Bischl et al., 2023a ) . For example, the HPs of the CART algorithm include the minimum number of observations in any terminal node ( \lambda_{minbucket}) , the maximum tree depth, with the root node counted as depth 0 ( \lambda_{maxdepth} ), and the factor by which a split needs to decrease the overall lack of fit to be attempted ( \lambda_{cp} ) ( Therneau & Atkinson, 2022a ) . In the CART implementation of the R package mlr3 ( Lang et al., 2019a ) , the respective HP domains are \Lambda_{minbucket}=\{1,\ldots,n_{\text{train}}\} , \Lambda_{maxdepth}=\{1,\ldots,30\} (both being integer-valued domains), and \Lambda_{cp}=[0,1] (real-valued domain). Most algorithm HPs have default values that are specified by the software in which they are implemented (e.g., in mlr3 , \lambda_{minbucket}=7 per default). Note that since there is usually more than one algorithm suitable for a given prediction problem, the choice of algorithm can also be seen as an HP of the learning pipeline (with the HPs associated with each algorithm representing conditional HPs that are only relevant when the respective algorithm is used; Bischl et al., 2023a ). This creates an even more flexible but also complex learning pipeline, which is why, in this paper, we assume that the algorithm has already been selected. 2.3.3 Preprocessing HPs As mentioned above, it is not only possible to specify learning algorithm HPs but also preprocessing HPs ( Binder & Pfisterer, 2024a ; Bischl et al., 2023a ) . In principle, whenever multiple options exist for performing a preprocessing step, these options can be considered as different HP values of the respective preprocessing step. First, the choice of whether a preprocessing step PS is applied at all can be considered as a binary HP \lambda_{PS} with \Lambda_{PS}=\{\mathrm{yes,no}\} (e.g., whether features should be log-transformed or not). Second, there is often more than one possible option for performing a preprocessing step. For example, the influence of outliers in features can be reduced by replacing all values that are outside the range [x_{min},x_{max}] by x_{min} and x_{max} , respectively (“winsorizing”; Steyerberg, 2019a ). There are different options to specify x_{min} and x_{max} , which means that \lambda_{x_{min}} and \lambda_{x_{max}} are HPs of the winsorizing preprocessing step (e.g., Steyerberg, 2019a , suggests percentiles such as \lambda_{x_{min}}=1\text{st percentile} and \lambda_{x_{max}}=99\text{th percentile} ). Several possible options also exist for the imputation of missing feature values. For example, imputation can be based on the feature’s mean or median, or on a sampled value from its empirical distribution (as illustrated in Thomas, 2024a ) . This constitutes a (categorical) preprocessing HP \lambda_{impute} with \Lambda_{impute}=\{\text{mean, median, sample},\ldots\} . Another typical example of a preprocessing step with many possible options is feature selection. To define HPs in this context, we have to differentiate between filter and wrapper methods (the following explanations are based on Wright, 2024a , who also provides more details and additional examples). Filter methods are preprocessing steps that assign a numeric score to each feature (e.g., the correlation coefficient \rho between each feature and the outcome) and select a set of features according to this score (e.g., all features with \rho>0.2 ). Consequently, the set of selected features is the parameter of the filter (i.e. \theta_{filter} , with, e.g., \hat{\theta}_{filter}=\{x_{6},x_{8},x_{21},x_{25}\} ), while its specific configuration can be modified by its HPs. For example, there are different options to define the score ( \lambda_{filter_{1}} , with \Lambda_{filter_{1}}=\{\text{correlation, variance, importance score},\ldots\} ) and to select the features based on their score ( \lambda_{filter_{2}} , with \Lambda_{filter_{2}}=\{\text{top $r$ features, all features with a score}\geq\tau,\dots\} , where r and \tau themselves are HPs that are conditional on \lambda_{filter_{2}} ). Instead of using filter methods, it is also possible to directly specify the set of features that should be selected. In this case, the set of selected features is an input rather than an output of the learning pipeline and is therefore the HP ( \lambda_{features} ) of the feature selection step. For example, if only the features x_{6},x_{9} , and x_{21} should be used by the learning algorithm, then \lambda_{features}=\{x_{6},x_{9},x_{21}\} . In many applications, \lambda_{features} is not specified once by the user, but different values of \lambda_{features} are tried and evaluated on \mathcal{D}_{\text{train}} . This process is referred to as a wrapper method but is, in fact, a special case of HP tuning, which will be discussed in Section 4.1 . Note that the individual HP values can also be application-specific. For example, in the real-world prediction problem considered in Section 5 , several options for aggregating 17 individual features covering physical symptoms, psycho-social burden, family needs, and practical problems of palliative care patients to a sum score are reasonable (see Section 5.2.2 ). In addition to specifying the preprocessing steps, the order in which they appear in the learning pipeline can technically be considered an HP as well. For instance, in the learning pipeline shown in Figure 1 , the log-transformation step could also be applied before the imputation step, resulting in a different \hat{\theta}_{impute} and, therefore, potentially a different prediction model. However, we will not consider this type of preprocessing HP further in the remainder of this paper. As already indicated by the examples above, many preprocessing HPs are conditional on other preprocessing HPs (e.g., the winsorizing HPs \lambda_{x_{min}} and \lambda_{x_{max}} are only relevant when winsorizing is the chosen method to reduce the influence of feature outliers, which could also be implemented by transforming the features instead). Moreover, in contrast to algorithm HPs, preprocessing HPs often cannot be set by a single software function argument (for example, all HPs of the CART algorithm named in the previous section can be specified within a single R function, using, e.g., the argument minbucket for \lambda_{minbucket} ); instead, in many cases, the different options for a specific preprocessing step are implemented by different software packages. Consequently, there is often no formal HP domain, and defining the domain such that it contains all possible HP values may not even be feasible (e.g., for \lambda_{impute} , defining \Lambda_{impute} would require collecting all available methods for imputing missing values). Moreover, many preprocessing HPs do not have a formal default value, although the option of not applying a preprocessing step (if applicable and not leading to an error) seems to be a reasonable default value that we will adopt in the following. In contrast to algorithm HPs, it seems that preprocessing HPs—apart from those related to feature selection—are rarely discussed or referred to as such in ML applications (see, e.g., the systematic reviews of Dhiman et al., 2022b , and Andaur et al., 2023a , where such terms were not mentioned). ML methods research usually also focuses on algorithm HPs rather than preprocessing HPs. An exception is the benchmark study by Stüber et al. (2023a) , which, among other factors, examines the impact of using principal component analysis in radiomics-based survival analysis. 2.3.4 Selection of HPs While it is usually possible to leave all HPs at their respective default value, it is common to modify them in an attempt to optimize the prediction model generated by the learning pipeline. This can also be necessary if there is no specified default value. The term “optimization” here often refers to the predictive performance of the model but can also take into account other criteria such as simplicity, interpretability, or runtime to generate the model ( Domingos, 2012a ; Pfob et al., 2022a ; de Hond et al., 2022a ; Bischl et al., 2023a ) . Note that the selection of HPs can be considered a “researcher degree of freedom” ( Simmons et al., 2011a ) , as it is one of many choices that users must make throughout the model development process (other choices are, e.g., how predictive performance is assessed; Hofman et al., 2017a ; Klau et al., 2020a ; Hosseini et al., 2020a ). We can distinguish between two primary types of HP selection: data-independent and data-dependent procedures. Data-independent HP selection does not make use of the data set \mathcal{D}_{\text{train}} and is ideally based on the user’s knowledge about the data set and learning algorithm. For example, sensible algorithm HPs can be selected when users are experienced with the learning algorithm or when corresponding recommendations from the literature (e.g., previous benchmark studies) are available ( Bartz et al., 2023a ; Bischl et al., 2023a ) . Similarly, some preprocessing HPs may be inferred from substantive knowledge about the data set (e.g., which set of features should be selected) or knowledge about how the learning algorithm is affected by certain data set characteristics (e.g., whether the algorithm is sensitive to outliers in features, which requires some form of transformation; Kuhn & Johnson, 2013a ). An example of data-independent HP selection on the basis of model simplicity is the specification of the maximum tree depth in the real-world prediction problem considered in Section 5 , where the project team set the HP to \lambda_{maxdepth}=4 to ensure that the resulting decision tree can be implemented in clinical practice. In cases where users have insufficient knowledge about the data and learning algorithm to ensure a reasonable HP selection but wish to avoid arbitrary or default HP values, it is possible to use the data set \mathcal{D}_{\text{train}} to select optimal HP values. This process corresponds to a data-dependent HP selection, but terms such as HP tuning and (data-driven) HP optimization are more common (e.g., Bischl et al., 2023a ; Probst et al., 2019a ; Bartz et al., 2023a ) . We will accordingly use the term HP tuning in the remainder of this paper. Note that HP tuning implies that not only the parameters \boldsymbol{\theta} are estimated from the data set \mathcal{D}_{\text{train}} but also one or more HPs in \boldsymbol{\lambda} . HP tuning thus generally complicates model generation and evaluation, which will be described in more detail in Section 4 . Importantly, there are HPs that should not be selected through tuning. For learning algorithms, this includes, for example, the number of trees ( \lambda_{num.trees} ) in the random forest algorithm for classification problems: Due to the monotonous relation between \lambda_{num.trees} and model performance in most cases, the largest computationally feasible number of trees should be chosen ( Probst & Boulesteix, 2018a ) . Regarding preprocessing HPs, this typically applies to those associated with steps that alter the scope or interpretation of the prediction model (see Section 2.2.2 ). As such steps require careful specification, the corresponding HPs should be set based on user expertise (i.e. data-independently) rather than determined through tuning. To indicate how the value of a HP \lambda_{j} has been specified, we write \lambda_{j}^{\mathrm{I}} if the value is left at default value or selected independently of the data, and \lambda_{j}^{\mathrm{II}} if the value was chosen through tuning. 2.4 Model development processes The development of ML-based prediction models generally involves two key processes: (i) the generation of the prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} (model generation) and (ii) the evaluation of its predictive performance (model evaluation). Given our focus on HPs and their selection, we distinguish between two settings in the remainder of this paper. In Setting I, all HPs of the learning pipeline are pre-specified (i.e. either set to default values or selected independently of the data). In Setting II, one or more HPs are selected through tuning. Before explaining the principles and potential pitfalls of model generation and evaluation for both settings in Sections 3 and 4 , we first clarify their general concepts. 2.4.1 Model generation We refer to the model generation process as the set of processes required to obtain the final prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} . In Setting I, the model generation process consists of a single training process, where the parameters that define the final prediction model are estimated from \mathcal{D}_{\text{train}} using the learning pipeline \mathcal{I} with pre-specified HPs. In Setting II, where one or more HPs are selected through tuning, the model generation process consists of a tuning process conducted on \mathcal{D}_{\text{train}} (which yields the tuned HPs), followed by a training process, where, similar to Setting I, the parameters of the final prediction model are estimated from \mathcal{D}_{\text{train}} using the learning pipeline \mathcal{I} with tuned HPs. 2.4.2 Model evaluation Once the final prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} has been generated, the next important step is its evaluation. Since many algorithms yield black-box models that cannot be easily interpreted, and are thus difficult to assess for plausibility without additional tools (see, e.g., Molnar, 2022a ), a key quantity in the evaluation of a model is its prediction error. In the context of this work, we will accordingly use the term “model evaluation” synonymously with determining a model’s prediction error. The prediction error indicates how well a model performs on new observations that are independently drawn from the same distribution as the observations in \mathcal{D}_{\text{train}} (i.e. from \mathds{P}_{xy} ). It is specified with respect to a loss function L , which assesses the discrepancy between true outcomes and predictions and constitutes the performance measure. Formally, the prediction error of \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} can be defined as \mathrm{PE}(\hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}})=\mathrm{E}_{(\boldsymbol{x},y)\sim\mathds{P}_{xy}}[L(\hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}}(\boldsymbol{x}),y)] (2) ( Hastie et al., 2009a ; Boulesteix et al., 2015a ; Bischl et al., 2023a ) . The loss function L can be chosen according to the prediction problem being addressed. For instance, a common choice for L in regression problems is the squared loss. In this case, the prediction error reflects the well-known mean squared error (MSE). Note that in equation ( 2 ), we assume for simplicity that L corresponds to a point-wise loss function, although many commonly used performance measures (e.g., the area under the receiver operating characteristic curve, AUC) would necessitate a more general definition (provided in Bischl et al., 2023a ). Nonetheless, all following statements regarding the prediction error hold regardless of this simplified (and more common) representation. An estimate of the prediction error in equation ( 2 ) can be obtained by using \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} to make predictions for an additional data set with new observations drawn from \mathds{P}_{xy} (referred to as test data set \mathcal{D}_{\text{test}} ). The prediction error can then be estimated by evaluating the loss function L for each observation and calculating the average across all observations (again, assuming a point-wise loss; Bischl et al., 2023a ; Hastie et al., 2009a ). The resulting prediction error estimate for \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}} can be denoted as \widehat{\mathrm{PE}}(\hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}},\mathcal{D}_{\text{test}}) . Note that the outcome values for \mathcal{D}_{\text{test}} must be observed; otherwise, the loss function L cannot be evaluated. The requirement for an additional data set, \mathcal{D}_{\text{test}} , for model evaluation can be challenging in applications where data resources are limited. Denoting \mathcal{D} as the only available data set at the time of model generation and evaluation, there are two general approaches for defining \mathcal{D}_{\text{train}} and \mathcal{D}_{\text{test}} : (i) all available data are used for model generation, in which case \mathcal{D}_{\text{test}} is inevitably a subset of \mathcal{D}_{\text{train}} (i.e. \mathcal{D}_{\text{train}}=\mathcal{D} and \mathcal{D}_{\text{test}}\subseteq\mathcal{D}_{\text{train}} ), or (ii) the model is generated on a (proper) subset of the available data, with the remaining subset held back for model evaluation (i.e. \mathcal{D}_{\text{train}}\subset\mathcal{D} and \mathcal{D}_{\text{test}}=\mathcal{D}\setminus\mathcal{D}_{\text{train}} ). For the first approach, there are several ways to define \mathcal{D}_{\text{test}} , each leading to a different evaluation procedure, which will be detailed in Section 3.2 (Setting I) and Section 4.2 (Setting II). Depending on the chosen evaluation procedure, a potential issue can be data leakage, which occurs whenever information about the designated \mathcal{D}_{\text{test}} is improperly available during the generation of the model to be evaluated ( Kapoor & Narayanan, 2023a ; Kapoor et al., 2024a ; Kaufman et al., 2012a ; Rosenblatt et al., 2024a ; Hornung et al., 2023a ) . Since, in this case, the observations in \mathcal{D}_{\text{test}} no longer truly represent new observations to which the model will be applied, and the model thus has an unfair advantage when predicting these observations, the resulting prediction error estimate can be optimistically biased. Kapoor & Narayanan, 2023a identify three general types of data leakage, which may arise from: (i) overlap between the data used for model generation and evaluation, (ii) violation of the assumption that all observations are independently drawn from the same distribution, or (iii) use of illegitimate features. In this paper, we will focus on overlap-induced data leakage but provide additional information on the other two types in Supplementary Section A. Furthermore, we encounter an example of one of the other types in our empirical illustration in Section 5 . Finally, note that in some applications of ML (e.g., in the context of healthcare research), the process of assessing a model’s performance on observations from \mathds{P}_{xy} is referred to as internal validation. This is in contrast to external validation, which evaluates how well the model predicts observations from different distributions (e.g., different time points or healthcare settings; de Hond et al., 2022a ; Collins et al., 2024b ; Royen et al., 2023a ; Van et al., 2023a ). As external validation is recommended to be performed in subsequent research only after successful internal validation ( Collins et al., 2024b ) , we will focus on internal validation in this paper. Note that, in general, the term “evaluation” should be preferred over “validation” as the latter suggests that a “validated model” has a low prediction error, which is not necessarily the case ( Collins et al., 2024b ) . 3 Setting I: Pre-specified HPs In this section, we describe the model generation and evaluation process for Setting I. We accordingly assume that the learning pipeline \mathcal{I} is configured by HP values that are either set to their default values or selected independently of the data, i.e. \boldsymbol{\lambda}=\boldsymbol{\lambda}^{\mathrm{I}} . This aspect is emphasized by denoting the learning pipeline as \mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}} . 3.1 Model generation As stated in Section 2.4 , the model generation process in Setting I consists of a single training process. Moreover, as already outlined, “training” refers to the learning pipeline estimating the parameters \boldsymbol{\theta} (which constitute the prediction model) from \mathcal{D}_{\text{train}} . For brevity, we will also refer to this process as “training the prediction model” although it is the learning pipeline that is being trained and subsequently yields the prediction model. Importantly, all parameters in \boldsymbol{\theta} must be estimated, including those from preprocessing steps. The estimation of preprocessing parameters follows the sequence of their corresponding steps in the learning pipeline \mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}} . This process is specified by the respective preprocessing step. For example, in the case of mean imputation, the corresponding parameter estimate is found by calculating the mean of all non-missing observations of the corresponding feature. The parameters of the learning algorithm are usually estimated based on a loss function l that measures the discrepancy between the true outcome and a prediction vector for each observation i , i.e. l(y^{(i)},f(\boldsymbol{x}^{(i)})) . The algorithm parameters are then found by minimizing \sum_{i=1}^{n_{\text{train}}}l(y^{(i)},f(\boldsymbol{x}^{(i)})) (see, e.g., Bischl et al., 2023a , or Bartz et al., 2023a , for more details). For example, in a regression problem where the learning algorithm corresponds to the CART algorithm, the splitting rules are found by minimizing the sum of squared errors and the prediction value for each terminal node corresponds to the mean of all outcome values in the respective node ( Breiman et al., 1984a ) . Note that the loss function l may, but does not necessarily have to, align with the loss function L from Section 2.4.2 , which is used to estimate the prediction error. When estimating the parameters, the learning pipeline may not only capture the signal in \mathcal{D}_{\text{train}} which represents the true underlying data-generating mechanism \mathds{P}_{xy} , but it may also erroneously learn the specific pattern of noise (i.e. unexplained variation) in \mathcal{D}_{\text{train}} . The resulting prediction model is too adapted to \mathcal{D}_{\text{train}} and will perform worse on new observations (drawn from \mathds{P}_{xy} ) than on the observations in \mathcal{D}_{\text{train}} . This is a well-known problem in prediction model training and is commonly referred to as overfitting (e.g., Hastie et al., 2009a ; Kuhn & Johnson, 2013a ; Steyerberg, 2019a ; Bischl et al., 2023a ; Poldrack et al., 2020a ; de Hond et al., 2022a ) . The risk of obtaining an overfitted prediction model depends on both the data set \mathcal{D}_{\text{train}} (specifically on its signal-to-noise ratio, which tends to decrease as the number of observations decreases) and on the learning pipeline \mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}} used to train the model ( Poldrack et al., 2020a ; Lones, 2024a ) . The association between the characteristics of a learning pipeline and its tendency to overfit is not straightforward, but it is related to factors such as the size of its hypothesis space (i.e. the number of prediction models that can be trained by \mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}} ) and the procedure by which the model is chosen from the hypothesis space (e.g., whether the hypothesis space is searched exhaustively; Domingos, 2012a ). These factors can vary greatly between learning pipelines, especially depending on the type of learning algorithm and the chosen HP values. Note that the learning pipeline may also suffer from underfitting rather than overfitting, which occurs if it is not flexible enough to adequately model the underlying data-generating mechanism ( Hastie et al., 2009a ) . As mentioned above, after training the learning pipeline once (and only once) on \mathcal{D}_{\text{train}} , the generation of the final prediction model is completed. This implies that if the model is found to have a poor predictive performance in the subsequent evaluation (e.g., due to over- or underfitting), the result either has to be accepted or the HPs of the learning pipeline have to be modified based on the evaluation result. However, users should be aware that the latter approach corresponds to Setting II, which has different implications for model evaluation (Section 4 ). We denote the final prediction model as \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}}} to emphasize that it is the result of training a learning pipeline configured with HP values \boldsymbol{\lambda}^{\mathrm{I}} . 3.2 Model evaluation As outlined in Section 2.4.2 , evaluating the prediction model \hat{f}^{\mathcal{D}_{\text{train}}}_{\mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}}} requires a test data set \mathcal{D}_{\text{test}} , which is used to estimate the model’s prediction error. In that section, it was also stated that evaluation procedures can be differentiated based on whether model generation (which corresponds to model training in Setting I) has been performed on all available data (with \mathcal{D}_{\text{train}}=\mathcal{D} and \mathcal{D}_{\text{test}}\subseteq\mathcal{D}_{\text{train}} ) or only on a (proper) subset of the available data (with \mathcal{D}_{\text{train}}\subset\mathcal{D} and \mathcal{D}_{\text{test}}=\mathcal{D}\setminus\mathcal{D}_{\text{train}} ). In the following sections, we examine the implications for model evaluation in more detail for both approaches. An additional graphical overview is provided in Figure 2 . Figure 2 : Overview of different model evaluation procedures and their relation to the model generation process if all HPs are pre-specified. Data leakage is present if any subset of \mathcal{D}_{\text{test}} used for prediction error estimation has also been employed to generate the evaluated prediction model (which is not necessarily the final model). In the figure, the point at which data “leaks” into the model evaluation is marked by the red caution symbol. 3.2.1 Evaluation of a model generated on all available data Apparent error A straightforward way to evaluate a prediction model trained on all available data is to estimate its prediction error using the same data set, i.e. \mathcal{D}_{\text{train}}=\mathcal{D}_{\text{test}}=\mathcal{D} . The resulting prediction error estimate is referred to as apparent error (see Figure 2 , model evaluation a). As explained in Section 2.4.2 , data leakage is present when information about the designated \mathcal{D}_{\text{test}} is present during model generation. For the apparent error, this is clearly the case, as \mathcal{D}_{\text{test}} is equal to \mathcal{D}_{\text{train}} . As a consequence, the apparent error is not able to detect any overfitting of the model (since the specific pattern of noise in \mathcal{D}_{\text{train}} exactly corresponds to that in \mathcal{D}_{\text{test}} ) and will therefore be affected by a (possibly substantial) optimistic bias. Although this evaluation procedure is well-known to be flawed and has been frequently warned against in literature (e.g., Efron, 1986a ; Kuhn & Johnson, 2013a ; Hastie et al., 2009a ; Poldrack et al., 2020a ; Collins et al., 2024b ) , it is often still the only prediction error estimate that is reported in studies presenting new prediction models ( Kapoor & Narayanan, 2023a ; Poldrack et al., 2020a ) . Resampling error To avoid the optimistic bias caused by the overlap between \mathcal{D}_{\text{train}} and \mathcal{D}_{\text{test}} , several procedures exist that partition \mathcal{D}_{\text{train}} one or multiple times into two subsets for evaluation purposes while still training the final prediction model on the full data set. These procedures can be referred to as resampling methods and the resulting estimate as the resampling error (see Figure 2 , model evaluation b). The following description is based on Simon, 2007a , Kuhn & Johnson, 2013a , Bischl et al., 2023a , and Casalicchio & Burk, 2024a ; see their work for more details. The simplest resampling method is the holdout or split-sample method, where \mathcal{D}_{\text{train}} is randomly split into two subsets with different purposes: One subset, denoted as \mathcal{D}^{\prime}_{\text{train}} , is used to retrain the same learning pipeline \mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}} that has been used to obtain the final prediction model. This results in an additional prediction model \hat{f}^{\mathcal{D}^{\prime}_{\text{train}}}_{\mathcal{I}_{\boldsymbol{\lambda}^{\mathrm{I}}}} , whose prediction error is then estimated on the second subset, which serves as \mathcal{D}_{\text{test}} . The holdout method essentially has two drawbacks, whose impact on the prediction error varies according to the split ratio and the absolute number of observations in \mathcal{D}^{\prime}_{\text{train}} and \mathcal{D}_{\text{test}} (denoted as n^{\prime}_{\text{train}} and n_{\text{test}} ). First, while the holdout method ensures a clean separation between \mathcal{D}^{\prime}_{\text{train}} and \mathcal{D}_{\text{test}} , it does not evaluate the actual prediction model trained on \mathcal{D}_{\text{train}} but the additional prediction model trained on \mathcal{D}^{\prime}_{\text{train}} , which does not necessarily coincide with the former. Since the additional prediction model is trained on fewer observations (i.e. n^{\prime}_{\text{train}}