Information Maximization for Long-Tailed Semi-Supervised Domain Generalization

Leo Fillioux 1*, Omprakash Chakraborty 2, Quentin Gopée 1, Pierre Marza 1, Paul-Henry Cournède 1, Stergios Christodoulidis 1, Maria Vakalopoulou 1, Ismail Ben Ayed 2 and Jose Dolz 2 · 2026

Read Information Maximization for Long-Tailed Semi-Supervised Domain Generalization by Leo Fillioux 1*, Omprakash Chakraborty 2, Quentin Gopée 1, Pierre Marza 1, Paul-Henry Cournède 1, Stergios Christodoulidis 1, Maria Vakalopoulou 1, Ismail Ben Ayed 2 and Jose Dolz 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08434.txt.

Introduction

Fillioux and colleagues tackle semi-supervised domain generalization when class counts are long-tailed, a common real-world shape where leading methods falter. Their fix, IMaX, adapts the InfoMax principle to this setting, maximizing the mutual information between learned features and latent labels under the guidance of the labeled samples. An alpha-entropic objective tempers the class-balance bias of the usual marginal entropy term, so skewed class mixes are handled better. The objective plugs into recent leading methods and lifts their scores across two image modalities in tests.

Catalog record

CREATORLeo Fillioux 1*, Omprakash Chakraborty 2, Quentin Gopée 1, Pierre Marza 1, Paul-Henry Cournède 1, Stergios Christodoulidis 1, Maria Vakalopoulou 1, Ismail Ben Ayed 2 and Jose Dolz 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26162
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08434

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Information Maximization for Long-Tailed Semi-Supervised Domain Generalization Leo Fillioux 1*, Omprakash Chakraborty 2, Quentin Gopée 1, Pierre Marza 1, Paul-Henry Cournède 1, Stergios Christodoulidis 1, Maria Vakalopoulou 1, Ismail Ben Ayed 2 and Jose Dolz 2 Abstract Semi-supervised domain generalization (SSDG) has recently emerged as an appealing alternative to tackle domain generalization when labeled data is scarce but unlabeled samples across domains are abundant. In this work, we identify an important limitation that hampers the deployment of state-of-the-art methods on more challenging but practical scenarios. In particular, state-of-the-art SSDG severely suffers in the presence of long-tailed class distributions, an arguably common situation in real-world settings. To alleviate this limitation, we propose IMaX , a simple yet effective objective based on the well-known InfoMax principle adapted to the SSDG scenario, where the Mutual Information (MI) between the learned features and latent labels is maximized, constrained by the supervision from the labeled samples. Our formulation integrates an α \alpha -entropic objective, which mitigates the class-balance bias encoded in the standard marginal entropy term of the MI, thereby better handling arbitrary class distributions. IMaX can be seamlessly plugged into recent state-of-the-art SSDG, consistently enhancing their performance, as demonstrated empirically across two different image modalities. † † * Work done during an internship at ILLS. 1 Introduction Deep learning approaches have dominated the computer vision field in the last decade, with unprecedented performances in a plethora of visual recognition problems [ 18 ] . However, these models are typically trained under the assumption that training and testing samples follow the same distribution. When distributional drifts violate this i.i.d. (independent and identically distributed) assumption, their performance can degrade substantially [ 11 , 24 ] . To address this problem, Domain Generalization (DG) aims to learn models that remain robust to unseen target domains [ 19 , 22 , 34 ] . In particular, in DG scenarios we assume that labeled data from multiple source domains is available, and the primary objective is to train a model that can adequately generalize to novel unseen domains. However, in real-world scenarios, such as healthcare, where labeling samples for all the domains may become impractical, the assumption that the data from multiple source domains are fully labeled is unrealistic, and generally unmet. Thus, to reduce the burden of the annotation process, semi-supervised domain generalization (SSDG) has been recently explored [ 35 , 9 ] , which unifies the problems of semi-supervised learning and domain generalization. In this setting, each source domain has access to a few labeled samples and a larger amount of unlabeled data. Thus, beyond striving for cross-domain generalization, the model should achieve this objective efficiently with respect to labeled data requirements. Very recently, [ 9 ] exposed that several existing semi-supervised learning (SSL) approaches outperformed methods tailored to DG in the SSDG scenario. Based on these observations, authors proposed two plug-and-play strategies (i.e., FBCSA [ 9 ] and DGWM [ 8 ] ) that can be integrated into different SSL methods, setting a novel state-of-the-art for this task. Nevertheless, after exploring the proposed frameworks we identified an important limitation that hampers their application in real-life scenarios. In particular, these approaches assume uniform class distributions across the different source domains. However, in real-world applications, many tasks involve inherently imbalanced data, e.g. , rare diseases in medical imaging. Indeed, we empirically observed that if we assume long-tailed class distributions, the performance of both FBCSA [ 9 ] and DGWM [ 8 ] is substantially degraded across several tasks (Fig. 1 ), underscoring the importance of considering this more realistic scenario. Figure 1 : Performance degradation under class imbalance. Impact on the accuracy of having long-tailed class distribution in the training data using the FBCSA [ 9 ] trainer ( left ). Impact of the imbalance factor \gamma (see Section 4.1 ) on the performance of FBCSA and our proposed approach ( right ). In light of the observation presented above, in this paper we investigate the problem of domain generalization in the context of semi-supervised learning with class-imbalanced datasets, which combines the problems of domain generalization and label-efficient learning in the presence of class imbalance, a more realistic scenario than prior works [ 9 , 8 ] . Thus, our contributions can be summarized as: • We introduce a more realistic SSDG setting where, in addition to the standard labeled and unlabeled data across multiple source domains, labeled samples are exposed to imbalanced class distributions. • We present IMaX , an information theoretic approach to tackle this challenging yet realistic scenario. In particular, the proposed solution accommodates the specificities of this challenging setting by deriving a semi-supervised view of the standard Mutual Information (MI). • To further account for class imbalance, we replace the less flexible marginal entropy term in the modified MI by a learning objective derived from Tsallis divergences [ 30 ] , which tolerates better variations on class distributions. • Experiments across several datasets, SSL and SSDG approaches demonstrate that IMaX consistently enhances existing methods in the studied scenario, showcasing its model-agnosticity and plug - and - play versatility. 2 Related Work Semi-supervised Domain Generalization (SSDG) focuses on improving generalization across domains when labeled data is scarce but substantial unlabeled data is accessible. Within the broader computer vision community, state-of-the-art SSDG [ 9 , 8 ] leverage semi-supervised learning. These methods consist in generating pseudo-labels from weakly augmented samples, and training the model to enforce that predictions from strongly augmented versions to align with these pseudo-labels. In particular, FixMatch [ 26 ] , FreeMatch [ 32 ] , and StyleMatch [ 35 ] are typically employed as SSL frameworks in SSDG [ 9 , 8 ] . The SSDG setting is particularly well suited to medical imaging, where acquisition conditions can vary widely. In this scenario, different approaches have been developed specifically for medical imaging problems [ 33 , 28 , 27 ] . However, these techniques typically rely on restrictive, modality - dependent assumptions, making them applicable only to a subset of medical imaging modalities. For example, [ 33 ] assumes that the distribution shift results in a modification of the hue saturation and contrast, while [ 27 ] assumes that structural similarity between classes is preserved across domains. Both conditions do not hold consistently across diverse and real clinical settings, e.g., anatomical variability and pathology - dependent morphology can substantially alter structural patterns. Mutual information (MI) in deep learning. Our approach is grounded in the well - established InfoMax principle [ 20 ] , which advocates maximizing the MI between a system’s inputs and outputs. Several variants of this general principle, which accommodate the specific requirements of different scenarios, have been recently used in machine learning and computer vision tasks, including deep clustering [ 13 , 16 , 21 ] , few-shot learning [ 2 , 3 , 10 ] , representation learning [ 1 , 12 , 15 ] , or generalized category discovery [ 5 ] . To the best of our knowledge, however, addressing SSDG from an information-theoretic standpoint remains unexplored. 3 Methodology Problem definition and notation. We now formally define the semi-supervised domain generalization (SSDG) problem. Let \mathcal{X} and \mathcal{Y} define the input and label space, respectively. A domain

Read the complete text