Disentangled Deep Smoothed Bootstrap for Fair Imbalanced Regression

Samuel Stocksieker, Denys Pommeret, Arthur Charpentier · 2025

Read Disentangled Deep Smoothed Bootstrap for Fair Imbalanced Regression by Samuel Stocksieker, Denys Pommeret, Arthur Charpentier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-13829.txt.

Introduction

Samuel Stocksieker, Denys Pommeret, and Arthur Charpentier fight skewed data in regression in a 2025 paper. Most imbalance research targets classification, where rare classes get ignored. Regression has the same disease: rare outcome ranges are learned worst, and fairness suffers.

Their method first squeezes data into a latent space with a disentangled variational autoencoder. A smoothed bootstrap then breeds new samples inside that space, fattening the thin regions. The disentangling keeps the synthetic points honest about which factors vary. Benchmark contests against rival methods on imbalanced regression data sets grade the idea. The authors pitch the pipeline for tabular data, where the problem is common and tools are few.

Catalog record

CREATORSamuel Stocksieker, Denys Pommeret, Arthur Charpentier
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-13829

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Disentangled Deep Smoothed Bootstrap for Fair Imbalanced Regression Samuel Stocksieker, Denys Pommeret, Arthur Charpentier Abstract Imbalanced distribution learning is a common and significant challenge in predictive modeling, often reducing the performance of standard algorithms. Although various approaches address this issue, most are tailored to classification problems, with a limited focus on regression. This paper introduces a novel method to improve learning on tabular data within the Imbalanced Regression (IR) framework, which is a critical problem. We propose using Variational Autoencoders (VAEs) to model and define a latent representation of data distributions. However, VAEs can be inefficient with imbalanced data like other standard approaches. To address this, we develop an innovative data generation method that combines a disentangled VAE with a Smoothed Bootstrap applied in the latent space. We evaluate the efficiency of this method through numerical comparisons with competitors on benchmark datasets for IR. K eywords Imbalanced Regression \cdot Smoothed Bootstrap \cdot Variational Autoencoders \cdot Synthetic Data 1 Introduction In various fields such as finance, economics, medicine, and engineering, regression techniques are frequently employed to establish relationships between features and a continuous target variable. However, real-world datasets often exhibit a significant imbalance in the distribution of the target variable, which poses specific challenges for traditional modeling methods [ 1 , 2 ] . This notion refers to certain ranges of values appearing more frequently than others, leading to an imbalance in learning. Typically, standard approaches treat all values as equally important and focus on minimizing the average error. As a result, rare and/or extreme values can have minimal influence on error reduction during the training phase due to their scarcity in the training set. This kind of modeling is unfair and biased: rare values are neglected in comparison with the majority values. Nevertheless, such values are often highly relevant for studies. The problem of learning from imbalanced data has been extensively studied in the context of classification (e.g. [ 3 , 4 ] ). However, the issue of Imbalanced Regression (IR) has received less attention, despite its significance to many real-world applications [ 1 ] . Indeed, regression introduces additional challenges compared to classification: i) identifying rare/minority values is difficult due to the continuous nature of the distribution; ii) determining the degree of imbalance is not straightforward; iii) measuring the extent of rebalancing is also difficult; and iv) generating synthetic data requires the creation of a new value for the target variable, associated with the new synthetic features. Three categories of non-exclusive solutions address this issue: pre-processing, in-processing, and post-processing techniques [ 5 ] . Pre-processing techniques aim to balance the target variable’s distribution before model training, while in-processing techniques modify the learning process itself to better manage data imbalances. Preprocessing solutions are often the preferred choice for tabular data due to their universality, allowing the use of any standard regression technique after resampling [ 6 ] . Preprocessing techniques primarily rely on resampling by generating synthetic data. Various methods have been suggested to create synthetic observations from the original data space, such as the widely recognized SMOTE algorithm [ 7 ] . Another approach involves using a method that embeds observations into a latent space from which data can be generated, like Variational Autoencoders (VAEs). VAEs are a data generation method that employs a neural network to encode observations into a continuous latent space. The probabilistic structure of this latent space can then be utilized to simulate new data. The key idea of this paper is to propose a new data representation space to enable more relevant synthetic data generation, especially by accounting for categorical data and nonlinear correlations. In particular, we propose a suitable framework for the application of the Smoothed Bootstrap, which is generally not the case in the initial data space. We aim to combine a Smoothed Bootstrap with a VAE to address the potential lack of robustness of the VAE in inferring rare values, as underlined by [ 8 ] . Furthermore, generations based on VAE are efficient as long as the latent distributions are well-adapted. In this paper, we propose a simple and effective adaptation of disentangled \beta- VAEs to tailor their training and generation specifically for the problem of IR on structured data. Our main contributions can be outlined as follows: • Introducing a novel loss function for training Disentangled VAEs in the context of IR • Using a free-parameter weighting scheme to manage the sampling of rare values when the target variable is continuous (regression) • Presenting a new synthetic data generator that combines the learning power of disentangled VAEs with a non-parametric generator: the (Disentangled) Deep Smoothed Bootstrap. To our knowledge, disentangled VAEs have never been used for synthetic data generation applied to tabular datasets. This application highlights a new use case for disentangled VAEs. Moreover, unlike the imbalanced regression techniques in the benchmark, our approach can handle mixed data types and account for nonlinear correlations, reproducing them during generation. This is made possible by the transformation into a latent representation space. The paper is organized as follows. Section 2 presents various state-of-the-art works, highlighting their shortcomings and how they differ from our proposal. Section 3 introduces our DSB (for Deep Smoothed Bootstrap) approach, which combines a disentangled VAE adapted to IR with an alternative method generation: the Smoothed Bootstrap. A scheme of our approach is presented in Figure 1 . Section 4 presents real-world applications. Finally, Section 5 discusses the proposed method and presents some perspectives. 2 Related Work Tabular Imbalanced Regression The initial efforts on IR were based on a utility function that enabled the binarization of the problem. Indeed, [ 9 ] initially suggested using a relevance function to assign a value to each instance of the target variable Y . Subsequently, the majority and minority values are identified using a user-defined threshold. This initial approach facilitates the adaptation of existing solutions within such as the well-known SMOTE algorithm to regression [ 10 ] , Gaussian Noise [ 11 , 12 , 13 ] or SMOTE extensions [ 14 , 15 ] . While this approach has paved the way for the first solutions to IR, it has the drawback of losing information about the target variable. Moreover, these utility-theoretic approaches are highly sensitive to the arbitrarily chosen utility function and associated threshold. A more recent work suggested an alternative approach without binarizing the problem: GOLIATH algorithm [ 16 ] . Lastly, as for classification, standard performance metrics are also biased and need to be reevaluated in the case of an imbalanced distribution [ 5 , 17 ] . Generating data from the initial space involves two major challenges: 1) the need to identify existing correlations in order to respect and reproduce them, and 2) the handling of categorical data. Finally, existing methods seek to generate synthetic data in the data space, with varying degrees of complexity. However, these techniques are subject to two important conditions: i) being able to identify potentially complex dependencies in the data, and ii) being able to reproduce them during generation. Deep Imbalanced Regression More recently, the concept of IR has been extended to image and text (NLP) data, such as estimating the age of individuals from photos, leading to the development of

Read the complete text