Mitigating Negative Flips via Margin Preserving Training
Simone Ricci, Niccolò Biondi, Federico Pernici, Alberto Del Bimbo · 2025
Read Mitigating Negative Flips via Margin Preserving Training by Simone Ricci, Niccolò Biondi, Federico Pernici, Alberto Del Bimbo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08322.txt.
Introduction
Simone Ricci, Niccolo Biondi, Federico Pernici, and Alberto Del Bimbo tackle negative flips, where an updated image classifier starts failing on pictures it once got right. As new classes arrive, margins between categories shrink and conflicting patterns appear, so overall progress can quietly damage older skills. Their training method preserves the margins of the original model while the new model learns.
An explicit margin calibration term on the output scores widens the gap between old and new classes in a controlled way. A double source focal distillation loss learns from both the old model and a freshly trained one, avoiding over constraint. Benchmarks show consistently lower negative flip rates with high overall accuracy.
Catalog record
| CREATOR | Simone Ricci, Niccolò Biondi, Federico Pernici, Alberto Del Bimbo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-08322
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Mitigating Negative Flips via Margin Preserving Training Simone Ricci, Niccolò Biondi, Federico Pernici, Alberto Del Bimbo Abstract Minimizing inconsistencies across successive versions of an AI system is as crucial as reducing the overall error. In image classification, such inconsistencies manifest as negative flips, where an updated model misclassifies test samples that were previously classified correctly. This issue becomes increasingly pronounced as the number of training classes grows over time, since adding new categories reduces the margin of each class and may introduce conflicting patterns that undermine their learning process, thereby degrading performance on the original subset. To mitigate negative flips, we propose a novel approach that preserves the margins of the original model while learning an improved one. Our method encourages a larger relative margin between the previously learned and newly introduced classes by introducing an explicit margin-calibration term on the logits. However, overly constraining the logit margin for the new classes can significantly degrade their accuracy compared to a new independently trained model. To address this, we integrate a double-source focal distillation loss with the previous model and a new independently trained model, learning an appropriate decision margin from both old and new data, even under a logit margin calibration. Extensive experiments on image classification benchmarks demonstrate that our approach consistently reduces the negative flip rate with high overall accuracy. Code — https://github.com/miccunifi/negative_flip_MPT Introduction Recent advances in machine learning have required the frequent deployment of updated models in production systems ( Raffel 2023 ; Yadav et al. 2025 ) . These updates often introduce new models that leverage more expressive network architectures ( Touvron et al. 2023 ) , novel training techniques or paradigms ( Biondi et al. 2024 ; Echterhoff et al. 2024 ; Shen et al. 2020 ; Zhou et al. 2023 ) , and additional data ( Gunasekar et al. 2023 ) . However, replacing an existing model requires balancing the potential reduction in overall error rates against the risk of introducing new errors that can disrupt downstream processes ( Milani Fard et al. 2016 ; Yan et al. 2021 ) or lead to unexpected system behaviors for human users ( Bansal et al. 2019 ) . This regression 1 1 1 The term regression —from the software industry—denotes a decline in system performance following an update. Although an updated model may demonstrate improved average performance, any instances of regression can potentially disrupt downstream post-processing workflows. phenomenon, observed in both visual and natural language tasks ( Yan et al. 2021 ; Abdul Samadh et al. 2024 ) , arises from the occurrence of negative flips—instances that are correctly classified by the old model but misclassified by the new one (see Figure 1 ). Figure 1: Illustration of possible two prediction changes following a model update. Each row shows the predictions of the old model (left) and the new model (right) for a given image, showing two possible cases: (1) a Positive Flip, where the old model was wrong and the new model is correct—the desirable outcome after a model update; (2) a Negative Flip, where the old model was correct but the new model is wrong, which can lead to unexpected system behaviors. In this paper, we focus on reducing Negative Flips in a classification task. Negative flips in classification tasks are most likely to occur when the number of classes increases to accommodate new categories ( Yan et al. 2021 ; Zhao et al. 2024 ; Zhang et al. 2021 ) . From a margin-based perspective, increasing the number of classes at each model update reduces the margin between adjacent classes, due to class means pushing themselves to arrange in a way that maximizes the minimum one-vs-rest margin ( Jiang et al. 2024a ; Papyan, Han, and Donoho 2020 ; Yaras et al. 2022 ; Tirer, Huang, and Niles-Weed 2023 ; Pernici et al. 2019 ; Pernici et al. 2021 ) . In addition to the reduction imposed by the optimization process and geometric constraints ( Jiang et al. 2024a ) , when new categories are introduced, the decision margins are subject to further challenges arising from negative transfer ( Li, Nguyen, and Zhang 2023 ; Zhang et al. 2022 ) . As the model learns to encode features that discriminate among an expanded set of classes, the incorporation of additional categories may lead to the emergence of interfering features, thereby adversely impacting the performance on previously acquired tasks or classes ( Tirer, Huang, and Niles-Weed 2023 ; Liu et al. 2021 ; Torralba, Murphy, and Freeman 2007 ) . In extreme cases, negative flips involve not only ambiguous samples near the decision boundaries but also high-confidence samples influenced, for example, by spurious correlations ( Wang et al. 2021 ) . Despite these challenges, most existing approaches for reducing negative flips ( Yan et al. 2021 ; Zhao et al. 2024 ; Zhang et al. 2021 ) overlook a key factor: the progressive reduction of class margins as new categories are added . Smaller margins make the model more susceptible to negative flips, since even confident, previously correct predictions can become incorrect due to slight shifts in decision boundaries. Therefore, preserving or restoring these margins is critical for maintaining consistent performance across model updates—especially for previously learned classes. This observation motivates us to focus on margin preservation as a central component of our approach. To this end, we preserve the logit decision margins 2 2 2 The logit decision margin is the difference between the logit of the ground-truth class and the largest logit among all other classes. Following Pleiss et al. 2020 notation, the logit margin for an input sample of class y is defined as \gamma(\mathbf{z},y)=\mathbf{z}_{y}-\max_{j\neq y}\mathbf{z}_{j} , where \mathbf{z} denotes the logit vector from the final (pre-softmax) layer. associated with previously learned classes, thereby addressing the challenge of negative flips when new classes are incrementally introduced. Specifically, we introduce a positive logit bias for the new classes as a margin-calibration term within the softmax cross-entropy loss, thereby modifying the optimization process such that the decision margins are biased towards the previously learned classes. Although this adjustment effectively reduces negative flips for previously learned classes, it may also result in underfitting of the newly introduced ones, as the model is encouraged to learn a smaller margin for them. To overcome this limitation and ensure appropriate margin learning for both previously learned and newly introduced classes, we propose a double-source focal distillation strategy. We distill knowledge corresponding to the correct predictions on the newly introduced categories from a model independently trained on the complete set of classes, while at the same time leveraging the previous model to preserve its correct behavior on the original categories. Our main contributions are as follows: • We present a novel approach, called Margin Preserving Training, to address the reduction of negative flips by focusing on the preservation of decision margins for previously learned classes. • We propose a double-source focal distillation strategy to mitigate the negative impact of the margin-calibration term introduced in the training loss on newly added classes, thereby ensuring balanced and robust classification performance for both new and old categories. • We conduct extensive experiments on image classification benchmarks, demonstrating that our method consistently reduces