Imbalanced Regression Pipeline Recommendation
Juscimara G. Avelino, George D. C. Cavalcanti, Rafael M. O. Cruz · 2025
Read Imbalanced Regression Pipeline Recommendation by Juscimara G. Avelino, George D. C. Cavalcanti, Rafael M. O. Cruz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-11901.txt.
Introduction
Juscimara Avelino, George Cavalcanti, and Rafael Cruz recommend pipelines in twenty twenty-five. Imbalanced data is common, and regression suffers from it too, not just classification. Rare target values are hard to learn. Balancing algorithms in preprocessing are a standard remedy, but choices abound.
The best resampling method and model pair depends on the data set and metric. Testing every combination is costly. Their framework, Meta-IR, trains meta-classifiers to recommend a full pipeline per task. It works zero-shot, with no trials on the new task. Meta-features describe each task and are mapped to the best pipeline class. Readers get a recommender that turns pipeline search into a single prediction.
Catalog record
| CREATOR | Juscimara G. Avelino, George D. C. Cavalcanti, Rafael M. O. Cruz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 99 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-11901
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Imbalanced Regression Pipeline Recommendation Juscimara G. Avelino, George D. C. Cavalcanti, Rafael M. O. Cruz Abstract Imbalanced problems are prevalent in various real-world scenarios and are extensively explored in classification tasks. However, they also present challenges for regression tasks due to the rarity of certain target values. A common alternative is to employ balancing algorithms in preprocessing to address dataset imbalance. However, due to the variety of resampling methods and learning models, determining the optimal solution requires testing many combinations. Furthermore, the learning model, dataset, and evaluation metric affect the best strategies. This work proposes the Meta-learning for Imbalanced Regression (Meta-IR) framework, which diverges from existing literature by training meta-classifiers to recommend the best pipeline composed of the resampling strategy and learning model per task in a zero-shot fashion. The meta-classifiers are trained using a set of meta-features to learn how to map the meta-features to the classes indicating the best pipeline. We propose two formulations: Independent and Chained. Independent trains the meta-classifiers to separately indicate the best learning algorithm and resampling strategy. Chained involves a sequential procedure where the output of one meta-classifier is used as input for another to model intrinsic relationship factors. The Chained scenario showed superior performance, suggesting a relationship between the learning algorithm and the resampling strategy per task. Compared with AutoML frameworks, Meta-IR obtained better results. Moreover, compared with baselines of six learning algorithms and six resampling algorithms plus no resampling, totaling 42 (6 × \times 7) configurations, Meta-IR outperformed all of them. The code, data, and further information of the experiments can be found on GitHub: https://github.com/JusciAvelino/Meta-IR . keywords Imbalance domain learning, Regression, Resampling Strategies, Meta-Learning. 1 Introduction Imbalanced problems are characterized by the low representation of some target values. In classification tasks, an imbalanced dataset is determined by the presence of a class with low representation (minority class) compared to another (majority class) ( Ganganwar 2012 ; Haixiang et al 2017 ) . However, in regression problems, the target value is continuous, representing a complex definition. In this context, Ribeiro 2011 proposes the concept of relevance function that determines the relevance of continuous target values to define which examples are rare and normal. Such classification allows verifying the imbalance between the instances called rare and normal. Figure 1 represents the distribution and frequency of examples from an imbalanced dataset (FuelCons). The values on the edges of the graph have low frequency and are considered rare examples. Figure 1: Distribution and frequency of the target value Y from the FuelCons dataset. Stronger colors indicate less represented examples (rare examples), while lighter colors indicate more represented examples (normal examples). In many cases, low-represented values hold significant importance, both to users and prediction processes. For instance, in software engineering, prediction mistakes in large projects are associated with higher development costs ( Rathore and Kumar 2017a ; Rathore and Kumar 2017b ) . Similarly, forecasting errors become significantly more costly in meteorological applications when dealing with extreme conditions such as extremely high temperatures ( Ribeiro and Moniz 2020 ) . Learning algorithms often struggle with these scenarios, prioritizing frequent target ranges while neglecting rare cases. This limitation can lead to suboptimal predictions for these specific instances, which are sometimes misinterpreted as outliers. Outliers, while typically rare and often considered unwanted, can actually hold significant value in some situations. In imbalanced regression problems, these cases can provide critical insights that are crucial for accurate predictions. In such contexts, ignoring these rare occurrences could lead to severe prediction errors, demonstrating that, far from being irrelevant, outliers can offer essential information for achieving more accurate and valuable outcomes. Few studies explore solutions for handling imbalanced regression problems. Commonly, such solutions employ resampling strategies such as the Synthetic Minority Oversampling TEchnique for Regression (SmoteR) ( Torgo et al 2013 ) and WEighted Relevance-based Combination Strategy (WERCS) ( Branco et al 2019 ) to balance the training data distributions prior to the training process. Although there is limited research exploring solutions for handling imbalanced regression problems, there are various resampling strategies available to choose from ( Wu et al 2022 ) . Recent findings in Avelino et al 2024 indicate that no single solution — a learning model and a resampling strategy — performs consistently well in all cases. Furthermore, there is a significant dependency between the blocks in the pipeline. The application of a resampling algorithm alters the optimal learning model in nearly 50% of the datasets studied. This interdependence means that the choice of the resampling strategy can influence the optimal choice for the learning model and vice versa. These findings underscore that approaches focusing solely on selecting and optimizing the best model per dataset, without accounting for preprocessing algorithms like Naive AutoML ( Mohr and Wever 2023 ) , are likely to underperform. It may overlook the significant impact of resampling techniques on model performance, missing potentially more effective models during the initial selection. Thus, identifying the optimal pipeline is crucial for accurate imbalanced regression results. In a landscape with numerous resampling strategies and regression algorithms, identifying the most fitting pipeline requires potentially evaluating all or a significant subset, thereby leading to a resource-intensive endeavor. To enhance optimal solution selection, meta-learning (MtL) ( Vanschoren 2019 ; Khan et al 2020 ) models prove effective. These models are based on meta-features, i.e., problem characteristics extracted from data, to learn new tasks more quickly. These meta-features — such as number of examples, number of attributes, number of rare cases, percentage of rare cases and data complexity measures ( Lorena et al 2019 ) — enable a meta-classifier to correlate them with algorithm performance (target attribute). Once trained, the meta-learning model works in a zero-shot fashion, suggesting algorithms and techniques that best fit the new task’s meta-features. In light of this, we propose a meta-learning-based model — Meta-learning for Imbalanced Regression (Meta-IR)— as an alternative solution for imbalanced regression problems. Meta-IR, illustrated in Figure 2 (a), is designed to suggest optimal learning models and resampling strategies based on problem meta-features, leveraging metrics tailored for optimization in imbalanced regression problems, such as F1-score for Regression (F1-scoreR) and Squared error-relevance area (SERA). The meta-classifier recommends resampling strategy and learning algorithm — a pipeline — customized to the new problem’s meta-features. For example, it could recommend the Random Over-sampling strategy with the Decision Tree learning model for a given dataset. Alternatively, for a different dataset, it might be more appropriate to use the WEighted Relevance-based Combination strategy with the Random Forest regression model, demonstrating the need to adjust the strategy and model according to the specific characteristics of the dataset. We propose two ways to train the meta-models: Independent and Chained. The Independent approach involves training the meta-classifiers to indicate the best learning model and resample strategy separately. In contrast, the Chained model is proposed based on the hypothesis that a dependency exists between the resampling algorithm and the regressor. In this way, Chained performs a sequential procedure where the output of one meta-classifier is used as input for another. In other words, the information of the best resampling algorithm is fed into the meta-classifier that predicts the best regressor, and vice versa. Figure 2: Comparison between our proposed meta-learning-based model Meta-IR and traditional