CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss

Dileepa Pitawela, Gustavo Carneiro, Hsiang-Ting Chen · 2025

Read CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss by Dileepa Pitawela, Gustavo Carneiro, Hsiang-Ting Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17813.txt.

Introduction

Some classification errors cost more than others. Mistaking one benign tumor grade for the next matters little; missing the step into cancer changes a life. Dileepa Pitawela, Gustavo Carneiro, and Hsiang-Ting Chen argue in 2025 that ordinal classifiers should respect such uneven stakes. Their method, CLOC, learns ordered representations by contrastive learning with a new multi-margin loss that lets each boundary between neighboring ranks carry its own importance.

Flexible margins smooth the transitions between classes and reduce overfitting to quirks in training data. The team tested CLOC on five real image datasets — age estimation, photo dating, knee arthritis, diabetic eye disease, and breast cancer subtyping — plus a synthetic set simulating biased clinical decisions. It beat existing ordinal methods across the board. The authors stress that the learned order is interpretable and controllable, matching how clinicians actually weigh mistakes.

Catalog record

CREATORDileepa Pitawela, Gustavo Carneiro, Hsiang-Ting Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-17813

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss Dileepa Pitawela, Gustavo Carneiro, Hsiang-Ting Chen Abstract In ordinal classification, misclassifying neighboring ranks is common, yet the consequences of these errors are not the same. For example, misclassifying benign tumor categories is less consequential, compared to an error at the pre-cancerous to cancerous threshold, which could profoundly influence treatment choices. Despite this, existing ordinal classification methods do not account for the varying importance of these margins, treating all neighboring classes as equally significant. To address this limitation, we propose CLOC, a new margin-based contrastive learning method for ordinal classification that learns an ordered representation based on the optimization of multiple margins with a novel multi-margin n-pair loss (MMNP). CLOC enables flexible decision boundaries across key adjacent categories, facilitating smooth transitions between classes and reducing the risk of overfitting to biases present in the training data. We provide empirical discussion regarding the properties of MMNP and show experimental results on five real-world image datasets (Adience, Historical Colour Image Dating, Knee Osteoarthritis, Indian Diabetic Retinopathy Image, and Breast Carcinoma Subtyping) and one synthetic dataset simulating clinical decision bias. Our results demonstrate that CLOC outperforms existing ordinal classification methods and show the interpretability and controllability of CLOC in learning meaningful, ordered representations that align with clinical and practical needs. † † Code is available at https://github.com/dpitawela/CLOC Figure 1 : An illustration of the impact of classification errors in ordinal contexts. Standard classification treats all errors equally, while ordinal classification penalizes larger rank errors more heavily. CLOC further accounts for the varying impact of adjacent errors, particularly across critical boundaries like benign versus malignant categories or minor vs threshold age of majority. This figure shows samples from IDRID (cancer grading) [ 43 ] and Adience (age estimation) [ 16 ] datasets. Longer arrow indicates a greater impact from adjacent errors. 1 Introduction Ordinal classification, the assignment of data to discrete, ordered categories, is essential across fields where ranking impacts key decisions. In healthcare, assessments such as TNM for cancer staging [ 22 ] , APACHE for intensive care [ 28 ] , and the Glasgow Coma Scale for brain injury [ 54 ] are essential for guiding treatment and predicting outcomes. This approach is also widely employed in psychology (e.g., Likert scales), education (grading proficiency levels), and finance (credit scoring), where maintaining ordinal relationships is key to deriving meaningful insights that inform actionable decisions [ 24 ] . Figure 1 illustrates the impact of various types of classification errors in ordinal contexts. Unlike standard classification problems, in ordinal classification or ranking problems, not all label mistakes are equal [ 58 , 8 , 13 ] ; misclassifications with larger rank differences carry more significant consequences. For example, they often represent greater discrepancies in disease severity or treatment needs. Moreover, not all adjacent label mistakes are equally significant ; for instance, an error between two benign tumor categories may have less impact compared to a misclassification at the boundary between pre-cancerous and cancerous stages, where the shift in diagnosis could significantly alter treatment and management [ 2 , 52 ] . Similarly, a misclassification between minors and adults would have a larger impact than other adjacent ordinal categories [ 19 ] , as it could lead to more substantial legal or social implications. Existing methods have not been designed to address both of these challenges. Regression attempts to model ordered categories, but its single continuous output often loses the ordinal structure and lacks clear boundaries between ranks [ 62 , 35 , 14 ] . Additionally, its error penalties are less flexible, as typical regression or models, such as those minimizing mean squared error, do not inherently account for the ordinal relationships between classes nor allow modification. Ordinal classification methods [ 40 , 8 ] are well-suited for tasks involving ordered categories, as they preserve the ranking structure and focus on maintaining the correct order, rather than merely predicting label distributions. However, most existing approaches focus on preserving order and minimizing label overlap in adjacent ranks, without considering the varying significance of adjacent label errors or examining the impact of margins between ranked labels. We propose the new Contrastive Learning for Ordinal Classification (CLOC) method that is based on an optimization that minimizes the novel Multi-Margin N-Pair (MMNP) loss. The new MMNP loss is a central contribution of this paper that introduces multiple learnable margins that allow flexible decision boundaries between adjacent labels, while its cumulative property preserves class ordering. These multiple learnable margins improve generalization by preventing overfitting to specific biases or minor variations in the training data, while also enabling more robust transitions between classes than previous ordinal classifiers that rely on fixed, hard decision boundaries. Additionally, CLOC enables nuanced ordinal classifications, making it particularly suited for challenging tasks like medical image analysis, where subtle distinctions between some categories are crucial. Furthermore, these learnable margins also offer interpretability of the importance between ordinal categories and provide an option for manual adjustment, allowing flexible control over critical decision boundaries. To summarize, our key contributions are: • Novel Contrastive Learning for Ordinal Classification (CLOC) method that is based on an optimization that minimizes the novel Multi-Margin N-Pair (MMNP) loss, which provides learnable margins for flexible decision boundaries while preserving the ordinal structure. • A novel method allowing users to control the model behavior through controlled margins, with experiments showing its effectiveness and robustness to biases. • Enhanced interpretability via multi-margin learning, offering a new tool for interpretable AI and insights into class separability and decision boundaries in ordinal classification tasks. We conducted experiments on five real-world image datasets (Adience, Historical Colour Image Dating, Knee Osteoarthritis, Indian Diabetic Retinopathy Image, and Breast Carcinoma Subtyping), as well as a synthetic dataset designed to simulate clinical decision biases. Our results show that CLOC outperforms existing ordinal classification methods. We also show empirical evidence of both CLOC’s interpretability and controllability in learning meaningful, ordered representations that align closely with clinical and practical needs. 2 Related Work Ordinal Classification: Early ordinal classification approaches [ 4 , 23 ] relied on standard classification or regression, missing ordinal relationships and non-uniform label separations. More recent models like ORCNN [ 40 ] and RankingCNN [ 8 ] use multiple binary classifiers to predict rank order, while CNNPOR [ 37 ] introduced pairwise ordinal constraints to preserve structure. Methods like SORD [ 13 ] and POEs [ 34 ] use soft labels and probabilistic embeddings, respectively, to better capture ordinal information. Unlike these, which assume fixed class-pair error distributions that limit prioritization of critical boundaries, CLOC enables class-pair-specific distributions and allows direct, meaningful adjustments over critical decision boundaries. Order learning approaches such as pairwise comparators [ 36 ] and Deep Repulsive Clustering (DRC) [ 30 ] compare instances directly to infer rank but are sensitive to reference quality. Moving Window Regression (MWR) [ 49 ] refines ranks iteratively with global and local regressors, though it also depends on good reference selection. Vision-language models like OrdinalCLIP [ 35 ] , Wang et al. [ 58 ] , and NumCLIP [ 14 ] utilise CLIP for ordinal regression with rank-specific prompts and regularization, but these large models (190M+ parameters) are data-intensive. In contrast, our method efficiently

Read the complete text