Tree Boosting Methods for Balanced and Imbalanced Classification and their Robustness Over Time in Risk Assessment

Gissel Velarde, Michael Weichert, Anuj Deshmunkh, Sanjay Deshmane, Anindya Sudhir, Khushboo Sharma, Vaibhav Joshi · 2025

Read Tree Boosting Methods for Balanced and Imbalanced Classification and their Robustness Over Time in Risk Assessment by Gissel Velarde, Michael Weichert, Anuj Deshmunkh, Sanjay Deshmane, Anindya Sudhir, Khushboo Sharma, Vaibhav Joshi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18133.txt.

Introduction

Seven researchers test tree boosting methods on skewed data sets in a 2025 paper. Many real tasks, like risk scoring, hinge on a rare class that models often miss. Tree boosting is fast and strong on table data, so the team put two variants through a full grid of tests.

They varied set size from one thousand to one hundred thousand rows and the positive share from half to five percent. Scores fell as skew grew, yet stayed well past a simple base rate. Balancing the training set by sampling did not help in a steady way. Tuning the model settings, by contrast, lifted results in a reliable way across the runs.

Catalog record

CREATORGissel Velarde, Michael Weichert, Anuj Deshmunkh, Sanjay Deshmane, Anindya Sudhir, Khushboo Sharma, Vaibhav Joshi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-18133

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tree Boosting Methods for Balanced and Imbalanced Classification and their Robustness Over Time in Risk Assessment Gissel Velarde, Michael Weichert, Anuj Deshmunkh, Sanjay Deshmane, Anindya Sudhir, Khushboo Sharma, Vaibhav Joshi Abstract Most real-world classification problems deal with imbalanced datasets, posing a challenge for Artificial Intelligence (AI), i.e., machine learning algorithms, because the minority class, which is of extreme interest, often proves difficult to be detected. This paper empirically evaluates tree boosting methods’ performance given different dataset sizes and class distributions, from perfectly balanced to highly imbalanced. For tabular data, tree-based methods such as XGBoost, stand out in several benchmarks due to detection performance and speed. Therefore, XGBoost and Imbalance-XGBoost are evaluated. After introducing the motivation to address risk assessment with machine learning, the paper reviews evaluation metrics for detection systems or binary classifiers. It proposes a method for data preparation followed by tree boosting methods including hyper-parameter optimization. The method is evaluated on private datasets of 1 thousand (K), 10K and 100K samples on distributions with 50, 45, 25, and 5 percent positive samples. As expected, the developed method increases its recognition performance as more data is given for training and the F1 score decreases as the data distribution becomes more imbalanced, but it is still significantly superior to the baseline of precision-recall determined by the ratio of positives divided by positives and negatives. Sampling to balance the training set does not provide consistent improvement and deteriorates detection. In contrast, classifier hyper-parameter optimization improves recognition, but should be applied carefully depending on data volume and distribution. Finally, the developed method is robust to data variation over time up to some point. Retraining can be used when performance starts deteriorating. [orcid=0000-0001-5392-9540] {graphicalabstract} † † highlights: It describes a Tree Boosting-based method evaluated empirically on balanced and imbalanced datasets of different volumes. It provides examples to illustrate how different evaluation measures are to be interpreted for detection systems or binary classifiers. On private datasets, it demonstrates empirically that the method increases its detection performance as the data volume increases. Besides, the F1 score decreases as the data distribution becomes more imbalanced, but it is still significantly superior to the baseline of precision-recall or ratio between positives divided by positives and negatives. It shows that sampling to balance the training set to deal with imbalance does not consistently improve recognition. More generally, it worsens detection. It discusses the care that should be taken when considering hyper-parameter optimization depending on data volume and class distribution. It shows that the developed method is robust to data variation over time, up to some point. Retraining can be used when performance starts deteriorating. keywords Balanced & Imbalanced Classification ,XGBoost ,Machine Learning ,AI ,Risk Assessment ,Performance Evaluation 1 Introduction Classification is a widely applied machine learning task in industrial setups. Outside laboratories, there might be a few cases where class distribution is balanced, since most real-world problems deal with imbalanced datasets. Binary classification systems are evaluated on their ability to correctly identify negative and positive samples. Detecting positive samples is critical, and often challenging when the datasets are imbalanced ( Lemaître et al. 2017 ; Wang et al. 2020 ; Kim and Hwang 2022 ; Hajek et al. 2022 ; Li et al. 2023 ; Yang et al. 2021 ; Wang et al. 2020 ) . In risk assessment, positive class samples may represent substantial business losses. At the same time, negative samples are essential, and therefore, flagging a negative sample as positive, is a lost business opportunity. Furthermore, the challenges are the following: • positive examples may represent rare cases, they can be anomalous and continuously change their behavior, • patterns may even be unseen during training, and • there might be a considerable delay until an abnormal activity is identified. Sometimes, realizing that an activity was anomalous can take months. In 2021, estimations in the telecommunications sector report that loss due to anomalous activity accounts for around USD 40 Billion, representing over two percent of the global revenue of USD 1.8 Trillion ( Howell 2021 ) . These activities include equipment theft, illegitimate commissions, and device reselling, in which global losses were estimated at USD 3.1 Billion, USD 2.2 Billion, and USD 1.7 Billion, respectively ( Howell 2021 ; Yang et al. 2021 ) . In recent years, eXtreme Gradient Boosting (XGBoost) has gained attention since it stands out as a highly competitive approach in machine learning contests for its recognition performance and speed ( Chen and Guestrin 2016 ) . In this study, XGBoost is systematically evaluated on small, medium, and large datasets presenting different class distributions. The contributions of this paper are the following: • It provides examples to illustrate how different evaluation measures can be interpreted for detection systems or binary classifiers. • It reviews the principles of Boosting Trees and the advantages of XGBoost as the selected boosting system. • It describes a method for a Vanilla XGBoost and a Random Search-Tuned XGBoost. • On private datasets, it demonstrates empirically that the method increases its detection performance as the data volume increases. Besides, the F1 score decreases as the data distribution becomes more imbalanced, but it is still significantly superior to the baseline of precision-recall or ratio between positives divided by positives and negatives. • It shows that sampling to balance the training set to deal with imbalance does not consistently improve recognition. More generally, it worsens detection. • It discusses the care that should be taken when considering hyper-parameter optimization depending on data volume and class distribution. • It shows that the developed method is robust to data variation over time, up to some point. Retraining can be used when performance starts deteriorating. The following section motivates the automation and decision-making support of risk assessment with AI, i.e., machine learning. Section 3 reviews evaluation in binary detection systems or binary classifiers. Section 4 , reviews XGBoost. The method is described in section 5 . The experiments are presented in section 6 . Finally, conclusions are drawn in section 7 . This paper revisits and extends the content, method, and experiments presented in ( Velarde et al. 2023 ) . 2 On the motivation to automate and support risk assessment with AI Figure 1: AI can simulate human decisions in a shorter time, helping human inspectors save time and focus on critical cases. From ( Velarde 2023 ) . Figure 2: Consider that each decision takes a human expert 5 minutes. Therefore, for 1000 requests, the time to execute will be 5000 minutes, 10000 minutes for 2000 requests, and so on. AI Machines execute each request in milliseconds and, therefore, can help save time, increasing productivity. In addition, AI can easily scale as the number of requests increases. From ( Velarde 2023 ) . AI applications have proven to solve a variety of business use cases ( World Intellectual Property Organization 2019 ) . The question is not whether to apply AI, but where to start to maximize return on investment. In risk assessment, expert inspectors develop, through years and experience, a sense of which customers should be granted a credit or loan. At the same time, they have to carefully screen each new customer and score them to approve or deny

Read the complete text