M²FGB: A Min-Max Gradient Boosting Framework for Subgroup Fairness
Jansen S. B. Pereira, Giovani Valdrighi, Marcos Medeiros Raimundo · 2025
Read M²FGB: A Min-Max Gradient Boosting Framework for Subgroup Fairness by Jansen S. B. Pereira, Giovani Valdrighi, Marcos Medeiros Raimundo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12458.txt.
Introduction
Jansen Pereira, Giovani Valdrighi, and Marcos Medeiros Raimundo teach gradient boosting to mind subgroup fairness while it learns. Predictive models can quietly disadvantage marginalized groups defined by traits like gender and race. The team folds fairness directly into the boosting objective. Standard losses for classification and regression share the stage with a min-max fairness term that guards the worst-off subgroup.
Each boosting round solves a primal-dual pair for that combined objective. The theory shows the algorithm converges under mild conditions. The generic frame adapts to several fairness concepts, binary and subgroup alike. Experiments back the claim of a powerful, flexible approach. Rather than auditing a finished model for bias, the method bakes the fairness pressure into every tree it grows. Justice, here, is an optimization term, not a postscript.
Catalog record
| CREATOR | Jansen S. B. Pereira, Giovani Valdrighi, Marcos Medeiros Raimundo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-12458
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
M²FGB: A Min-Max Gradient Boosting Framework for Subgroup Fairness Jansen S. B. Pereira, Giovani Valdrighi, Marcos Medeiros Raimundo Abstract In recent years, fairness in machine learning has emerged as a critical concern to ensure that developed and deployed predictive models do not have disadvantageous predictions for marginalized groups. It is essential to mitigate discrimination against individuals based on protected attributes such as gender and race. In this work, we consider applying subgroup justice concepts to gradient-boosting machines designed for supervised learning problems. Our approach expanded gradient-boosting methodologies to explore a broader range of objective functions, which combines conventional losses such as the ones from classification and regression and a min-max fairness term. We study relevant theoretical properties of the solution of the min-max optimization problem. The optimization process explored the primal-dual problems at each boosting round. This generic framework can be adapted to diverse fairness concepts. The proposed min-max primal-dual gradient boosting algorithm was theoretically shown to converge under mild conditions and empirically shown to be a powerful and flexible approach to address binary and subgroup fairness. A Preprint Keywords Fairness, Gradient Boosting, Min-max optimization 1 Introduction Fairness has been a recurring study topic in recent years due to the numerous examples of biased decision-making algorithms identified in high-impact applications, such as crime recidivism [ 3 ] , recruiting process [ 10 ] , and others. Group justice is the most utilized definition of fairness, which demands the independence of the predictions and a sensitive attribute, such as gender or race [ 30 ] . This definition can also be interpreted as requiring no disparity in the allocation of resources among privileged and unprivileged groups [ 30 ] or no disparity in error rates between groups [ 19 ] . In that sense, most fairness algorithms consider minimizing a prediction loss with a constraint on the difference of utility measures. However, this may achieve fairness by only reducing the quality of the predictions for a group, which is not feasible in some applications. Based on Rawlsian ethics [ 35 ] , min-max fairness aims to increase the well-being of the marginalized group. This concept becomes more complex when considering multiple demographic groups [ 26 ] . Considering an example in the scenario of health diagnostics, where fairness is evaluated with combined attributes of race (white and Black) and gender (man and woman) in four subgroups. Fairness principles based on disparity have the objective of reducing the gap between the highest and lowest group-level errors. If there is an initial solution where the errors are e_{1}<e_{2}<e_{3}<e_{4};e_{i}\in\mathbb{R} , the equalized loss metric is \max_{i\neq j}|e_{j}-e_{i}|=e_{4}-e_{1} . However, another classifier with losses e_{1}+\epsilon<e_{2}<e_{3}<e_{4},\epsilon>0\in\mathbb{R} , would have the equalized loss metric equal to \max_{i\neq j}|e_{j}-e_{i}|=e_{4}-e_{1}-\epsilon<e_{4}-e_{1} . Although this new classifier has equal or worse performance for all individuals, it is evaluated as better by the equalized loss metric. This was obtained by harming a subgroup without improving the condition of another one. In the min-max setting, the second classifier will not be evaluated as an improvement from the initial one. This is a common situation where e_{4} might be the best loss possible atainable for the worst-performing group under the available data. Previous studies have already considered min-max fairness algorithms. Martinez et al. [28] presented an initial work using this definition, which considered a multi-objective optimization problem where each group-level error was an objective. The proposed algorithm searches for a weighted sum of the objectives that results in a solution to the original min-max problem. Diana et al. [11] proposed a two-player game formulation where one player minimizes the group-level losses at each round while the other finds a maximum weighted sum of the losses. However, both algorithms encompass fitting multiple logistic models, which can have high computational costs, particularly with large datasets. The computing and accuracy costs of such fairness solutions can impede decision-makers from utilizing them. Considering the scenario of tabular data, a common data type for social domains, gradient-boosting algorithms have excellent results in accuracy and computing cost [ 37 ] . Despite these algorithms being already considered for fairness [ 34 , 9 ] , there is still no adaptation considering min-max fairness. In this setting of min-max fairness, we propose M²FGB , an algorithm that leverages the competitive accuracy and efficiency of gradient-boosting algorithms to maximize the worst-group (WG) utility, i.e., min-max fairness. By considering the Lagrangian function, the algorithm performs primal-dual optimization within the gradient-boosting rounds by first performing a gradient ascent on dual variables and then a gradient descent to build the ensemble model. Although our algorithm asks for differentiable measures of fairness, we present diverse proxy functions for different fairness criteria. The empirical results showed that optimizing for the WG utility with the proxy differentiable fairness measure also improves upon the non differentiable measure of interest. M²FGB is versatile, working with different loss functions and fairness metrics. It can also optimize fairness metrics defined for sensitive attributes with more than two categories. In summary, the main contributions of this paper are: • M²FGB, a novel and versatile framework for optimizing min-max fairness through gradient-boosting; • The derivation of our method at three different fairness criteria: equalized loss (for classification and regression), true positive rate (related to equal opportunity difference), and positive rate (related to statistical parity difference); • The validation of our framework in different settings and datasets, including German Credit [ 21 ] , COMPAS [ 3 ] , ENEM [ 2 ] , ACSIncome [ 12 ] . M²FGB showed a competitive performance with compared fairness algorithms; • An open-source implementation of M²FGB available at https://github.com/hiaac-finance/m2fgb . 2 Preliminaries and Problem Statement We consider access to a dataset (x_{i},y_{i})^{n}_{i=1} with n observations, in which x_{i}\in\mathbb{R}^{d} corresponds to the input variables and y_{i} represents the dependent variable. y_{i}\in\{0,1\} if we consider a classification task or y_{i}\in\mathbb{R} if it is a regression task, but the following discussions are valid for both. We also have access to a categorical sensitive attribute z_{i}\in\mathcal{Z} for each observation that indicates belonging to a specific demographic group, for example z_{i}\in\{\text{male, female, non-binary}\} or z_{i}\in\{\text{(male, white), (male, Black), (female, white), (female, Black), (non-binary, white), (non-binary, Black)}\} . The initial objective is to reconstruct the unknown mapping x\overset{\mathcal{F}}{\longrightarrow}y using an estimate f:\mathbb{R}^{d}\to\mathbb{R} . 2.1 Gradient-Boosting Gradient-Boosting Algorithms are ensemble methods that create a strong predictor that minimizes a loss function \mathcal{L} by iteratively adding new weak models to the predictor. The weak model is a step in a functional gradient descent at each boosting round. In more detail, at boosting round t with a predictor f^{(t-1)}:\mathbb{R}^{d}\to\mathbb{R} , the gradient-boosting algorithm adds a new weak learner h^{(t)}:\mathbb{R}^{d}\to\mathbb{R} from an hypothesis space \mathcal{H} to approximate the gradient of the loss \mathcal{L} w.r.t. the predictions f^{(t-1)}(x) . All of the gradient-boosting methods – from seminal [ 14 ] to quadratic [ 5 ] , efficient [ 25 ] , and categorical [ 32 ] adaptations – are grounded on the same basis. We have three main components: (1) the loss function, which evaluates how well the model fits the observed data; (2) the weak learner, which makes the predictions; and (3) the additive model, a sequential approach to adding models. This framework has flexibility in the