Robust Classification with Noisy Labels Based on Posterior Maximization
Nicola Novello, Andrea M. Tonello · 2025
Read Robust Classification with Noisy Labels Based on Posterior Maximization by Nicola Novello, Andrea M. Tonello (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06805.txt.
Introduction
Nicola Novello and Andrea Tonello study training classifiers when some labels are wrong in 2025. Real data sets often carry noisy labels, which can wreck a model. The team examines a family of loss functions based on f-divergence, called f-PML, and asks how each member handles label noise.
They show any member can be corrected so the trained network matches the one clean data would give. A second fix refines the network outputs at test time instead. Even without fixes, the whole family resists symmetric label noise, where errors spread evenly. Cross-entropy sits inside the family, so it gains the same proof. Paired with refined training, the losses match top rival methods.
Catalog record
| CREATOR | Nicola Novello, Andrea M. Tonello |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06805
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Robust Classification with Noisy Labels Based on Posterior Maximization Nicola Novello, Andrea M. Tonello Abstract Designing objective functions robust to label noise is crucial for real-world classification algorithms. In this paper, we investigate the robustness to label noise of an f f -divergence-based class of objective functions recently proposed for supervised classification, herein referred to as f f -PML. We show that, in the presence of label noise, any of the f f -PML objective functions can be corrected to obtain a neural network that is equal to the one learned with the clean dataset. Additionally, we propose an alternative and novel correction approach that, during the test phase, refines the posterior estimated by the neural network trained in the presence of label noise. Then, we demonstrate that, even if the considered f f -PML objective functions are not symmetric, they are robust to symmetric label noise for any choice of f f -divergence, without the need for any correction approach. This allows us to prove that the cross-entropy, which belongs to the f f -PML class, is robust to symmetric label noise. Finally, we show that such a class of objective functions can be used together with refined training strategies, achieving competitive performance against state-of-the-art techniques of classification with label noise. I Introduction The success of large deep neural networks is highly dependent on the availability of large labeled datasets. However, the labeling process is often expensive and sometimes imprecise, either if it is done by human operators or by automatic labeling tools. On average, datasets contain from 8\% to 38.5\% of samples that are corrupted with label noise [ 1 , 2 , 3 , 4 , 5 ] . For classification tasks, different lines of research focused on the architecture and training strategy development or on the objective function design. For supervised classification tasks, various objective functions have been proposed with the goal of replacing the cross-entropy (CE) [ 6 , 7 , 8 , 9 ] , achieving promising results. Meanwhile, in the weakly-supervised scenario of classification with label noise, various evidence showed that the standard CE minimization is not the best option [ 10 , 11 , 12 ] . In this paper, we show that the class of objective functions relying on the maximum a posteriori probability (MAP) approach proposed in [ 9 ] for supervised classification (referred to as f -divergence based Posterior Maximization Learning ( f -PML) in this paper), is an effective option also in the presence of label noise. We propose two correction techniques to make f -PML robust to label noise. The first has to be applied during the training phase, similarly to other approaches, to learn a neural network that is equal to the neural network learned with the clean dataset. For the second, instead, we show that the MAP-based formulation of the classification problem allows to express the posterior in the presence of label noise as a function of the neural network’s output and the noise transition probabilities. This allows the design of a novel correction method applied during the test phase to correct the posterior estimate, making it tolerant to label noise. Moreover, we show that, although f -PML objective functions are not symmetric, they are robust to symmetric label noise for any f -divergence without requiring the estimation of the noise transition probabilities, for mild conditions on the noise rates. As a side (but fundamental) outcome, we demonstrate that the CE is robust to symmetric label noise, correcting many previous papers that affirmed the contrary by relying on the fact that it is not symmetric. In addition, we observe that f -PML can be seen as a specific case of active passive losses (APLs) [ 13 ] , and attains significantly higher accuracy than other APL-like losses. Finally, we combine the robust losses with refined training strategies to demonstrate that f -PML can also be used with complex training strategies to achieve a competitive performance with state-of-the-art techniques. The key contributions of this paper are: • We prove the robustness of f -PML to symmetric label noise for any f -divergence, without requiring the class of objective functions to be symmetric. As a key byproduct, we demonstrate the robustness of the CE. • For label noise models where f -PML is not robust to label noise, we provide novel approaches to correct either the objective function or the posterior estimator, to achieve robustness. • Our experimental results show that f -PML can be used jointly with refined training strategies to achieve performance competitive with state-of-the-art techniques. II Related Work In this section, we provide a brief summary of the existing approaches for classification in the presence of label noise. Objective function correction These methods all rely on the idea of modifying the objective function to improve the classifier’s label noise robustness. These algorithms require to know the matrix of transition probabilities from true labels to fake labels (i.e., transition matrix). When the transition matrix is not known, it can be estimated, as studied in [ 14 , 15 , 16 , 17 , 18 ] . In [ 19 ] , the authors propose a weighted loss function for binary classification in the presence of class-conditional noise. In [ 20 ] , the authors utilize the transition matrix to employ reweighting, which utilizes importance sampling to ensure robustness. Forward and Backward [ 14 ] are two algorithms for loss correction given the transition matrix, which is estimated finding the dataset anchor points. In [ 21 ] , the authors propose a loss correction approach to avoid the overfitting of noisy labels during the dimensionality expansion phase of the training process. In [ 22 ] , the authors propose a resampling technique that works better than reweighting in the label noise scenario. Shifted Gaussian Noise (SGN) [ 23 ] provides a method combining loss reweighting and label correction. Robust objective functions These algorithms utilize objective functions that are theoretically robust to label noise without the need of estimating the transition probabilities. In [ 10 ] , the authors prove the robustness of symmetric objective functions. In particular, they show that the CE is not symmetric, while proving that the mean absolute error (MAE) is a robust loss. In [ 11 ] , the authors show that MAE performs poorly for challenging datasets and propose the generalized cross entropy (GCE), which is a trade-off between MAE and categorical CE, leveraging the negative Box-Cox transformation. Symmetric Cross Entropy (SCE) [ 24 ] combines the CE loss with a Reverse Cross Entropy (RCE) loss robust to label noise, to avoid overfitting to noisy labels. In [ 25 ] , the authors propose a robust loss function based on the determinant based mutual information. In [ 13 ] , the authors prove that all the objective functions can be made robust to label noise with a normalization. However, they show that robust losses can tend to underfit. Therefore, they propose a class of objective functions, referred to as active passive losses (APLs), that mitigate the underfitting problem. Peer Loss functions [ 26 ] are a class of robust loss functions inspired by correlated agreement. In [ 27 ] , the