A Weighted-likelihood framework for class imbalance in Bayesian prediction models
Stanley E. Lazic · 2025
Read A Weighted-likelihood framework for class imbalance in Bayesian prediction models by Stanley E. Lazic (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17013.txt.
Introduction
Lazic tackles class imbalance in Bayesian prediction models in a paper from 2025, using drug safety as the test case. In toxicology data, harmless compounds far outnumber harmful ones. Models trained on such data call most things safe and miss the rare toxic cases that matter most for safety checks.
His fix raises each observation's likelihood to a power set by how rare its class is, scaled so total information stays the same. That builds cost-sensitive learning into the Bayesian update itself. Tests on fake data and on a liver injury model improved balance and caught more toxic cases. The change is small in standard tools.
Catalog record
| CREATOR | Stanley E. Lazic |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-17013
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Weighted-likelihood framework for class imbalance in Bayesian prediction models Stanley E. Lazic Abstract Class imbalance occurs when data used for training classification models has a different number of observations or samples within each category or class. Models built on such data can be biased towards the majority class and have poor predictive performance and generalisation for the minority class. We propose a Bayesian weighted-likelihood (power-likelihood) approach to deal with class imbalance: each observation’s likelihood is raised to a weight inversely proportional to its class proportion, with weights normalized to sum to the number of samples. This embeds cost-sensitive learning directly into Bayesian updating and is applicable to binary, multinomial and ordered logistic prediction models. Example models are implemented in Stan , PyMC , and Turing.jl , and all code and reproducible scripts are archived on Github ( https://github.com/stanlazic/weighted_likelihoods ). This approach is simple to implement and extends naturally to arbitrary error-cost matrices. K eywords Bayesian updating \cdot Class imbalance \cdot Cost-sensitive learning \cdot Power likelihood \cdot Calibration 1 Introduction Class imbalance occurs when classes differ in the number of observations or samples they have. As a result, classification models may favour the majority class over the class with the fewest samples. Consequently, some standard metrics may overestimate the performance of the model. For example, if the majority class contains 90% of the samples, a model that always predicts the majority class will achieve 90% accuracy. The greater the imbalance, the more severe the problem. Two basic approaches have been developed to address class imbalance ( Krawczyk 2016 ) . The first is sampling-based methods, which alter the data by (1) oversampling the minority class, (2) undersampling the majority class, or (3) more sophisticated methods of synthetic data generation such as Synthetic Minority Over-sampling Technique (SMOTE; Chawla et al. 2002 ) and Adaptive Synthetic Sampling (ADASYN; He et al. 2008 ). The second approach modifies the loss function so that samples from the minority class get a higher weight and therefore have more influence on the results. Recently, a third approach has been developed by Nassiri et al. 2024 for Bayesian classification models that adjusts the posterior predictive distribution of class probabilities based on the proportion of samples within each class in the training data. Although this approach adjusts the predictive distributions for class imbalance, parameter estimates, decision boundaries, and standard classification metrics remain unchanged. Depending on the purpose of the model, it may be desirable to adjust more than the predictive distributions. Here, we describe how to account for class imbalance by modifying the loss function, which is the log-likelihood of a Bayesian prediction model. This is consistent with the approach taken in many popular frequentist machine learning models and is easy to implement within a Bayesian framework. Given N independent samples or observations x , a prior on the model parameters p(\theta) , and the information contained in the likelihood or data model p(x|\theta) , Bayesian updating can be used to obtain the posterior for the parameters given the data p(\theta|x) . Information on class imbalance can be incorporated by raising the likelihood to a power ( w ), giving p(\theta|x_{1:N})\propto p(\theta)\prod_{i=1}^{N}p(x_{i}|\theta)^{w_{i}}, (1) where w_{i}>0 and \sum_{i}w_{i}=N . Power-likelihoods are not a new idea in Bayesian inference. This approach is used to incorporate and downweight historical data ( Chen and Ibrahim 2000 ; Ibrahim et al. 2003 ) , account for misspecified likelihoods ( Holmes and Walker 2017 ; Aitchison 2021 ) , make the model robust to outliers ( Ghosh and Basu 2015 ) , update distributions over parameters with a general loss function instead of a traditional likelihood function ( Bissiri et al. 2016 ; Lyddon et al. 2019 ) , and even to account for biased judgements in Bayesian models of human decision making ( Matsumori et al. 2018 ) . Research, and debate, in this area often focuses on how to determine suitable values for w , and depending on the model, w could be a scalar or a vector of length N . When accounting for class imbalance, each sample gets its own weight, which is objectively set by the inverse proportion of its class size, and with the restriction that the weights sum to the sample size ( N ). This simple approach removes the bias due to class imbalance from the start, rather than counteracting it as a post-processing step. Modifying the likelihood will result in different parameter estimates, and therefore different predictions. This approach has the advantage of being applicable to general cost-sensitive predictions where errors have different costs. For example, when developing a clinical prediction model, misdiagnosing a person as healthy when they have a disease (false negative) is typically considered a more serious error than incorrectly classifying a healthy person as having the disease (false positive). 2 Methods 2.1 Data Two datasets will be used to demonstrate the effects of accounting for class imbalance. The first is simulated data for a binary outcome with a large class imbalance. The class proportions are Class 0 = 0.87 and Class 1 = 0.13. The data consists of 100 samples with two continuous predictor variables. The second data set is the drug-induced liver injury (DILI) data from Williams et al. 2020 . The data includes 96 compounds and the outcome is an ordered three-level categorical variable, where the categories represent increasing levels of liver toxicity. Class 1 represents the safest compounds, Class 2 has mild hepatotoxicity, and Class 3 are the most toxic. Class proportions are 0.34, 0.42, and 0.24, respectively. The predictors are the results from several in vitro toxicity assays, the degree to which a compound favours a lipid versus an aqueous environment, and the maximum clinical concentration in the blood (see the original paper for details). 2.2 Models A Bayesian binary classification model can be represented as \displaystyle y_{i} \displaystyle\sim \displaystyle\mathrm{Bernoulli}(\eta_{i})^{w_{i}}\quad i=1\dots N \displaystyle\mathrm{logit}(\eta_{i}) \displaystyle= \displaystyle X_{ij}\beta_{j} \displaystyle\beta_{j} \displaystyle= \displaystyle\mathrm{Normal}(0,\sigma) where y is a vector of N of binary values indicating the class, and i indexes the sample (e.g. patient or compound). \eta is the linear predictor, defined by an i -by- j matrix of predictor variables ( X ) and the model parameters \beta estimated from the data. A Bayesian model also includes a prior over the \beta parameters, which is given here as a Normal distribution with hyperparameters mean 0 and standard deviation \sigma . The likelihood can be modified to account for the class imbalance by raising it to a power that is inversely proportional to the proportion of samples in each class ( w ). Each sample i gets a weight that is the same for all samples in that class. It is crucial that the weights sum to the sample size, N , otherwise the amount of information in the data will be artificially increased or decreased. For example, if 75% of the samples are Class 0 and 25% are Class 1, the inverse proportions are 1/0.75 = 1.33 and 1/0.25 = 4. Each sample is assigned its respective value, and the weights are then normalised by dividing by their sum and multiplying by N . If there are 8 samples, the unnormalised weights are {1.33, 1.33, 1.33, 1.33, 1.33, 1.33, 4, 4},