A Language Anchor-Guided Method for Robust Noisy Domain Generalization
Zilin Dai, Lehong Wang, Fangzhou Lin, Yidong Wang, Zhigang Li, Kazunori D Yamada, Ziming Zhang, and Wang Lu* · 2025
Read A Language Anchor-Guided Method for Robust Noisy Domain Generalization by Zilin Dai, Lehong Wang, Fangzhou Lin, Yidong Wang, Zhigang Li, Kazunori D Yamada, Ziming Zhang, and Wang Lu* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17211.txt.
Introduction
Zilin Dai and co-authors harden image models against both domain shift and label noise in a 2025 paper. Models love to lean on spare, flashy traits that fail in new places, and wrong labels deepen the trap.
Their method anchors each training image to a word based sketch of its class from a language model. Such sketches hold steady across domains. A weighted loss then trims the say of images that sit far from their anchor. Bench tests across sets and noise rates beat lead domain methods on accuracy and toughness. The language anchor acts as a fixed star for noisy training.
Catalog record
| CREATOR | Zilin Dai, Lehong Wang, Fangzhou Lin, Yidong Wang, Zhigang Li, Kazunori D Yamada, Ziming Zhang, and Wang Lu* |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17211
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Language Anchor-Guided Method for Robust Noisy Domain Generalization Zilin Dai, Lehong Wang, Fangzhou Lin, Yidong Wang, Zhigang Li, Kazunori D Yamada, Ziming Zhang, and Wang Lu* Abstract Real-world machine learning applications are often hindered by two critical challenges: distribution shift and label noise. Networks inherently tend to overfit to redundant, uninformative features present in the training distribution, which undermines their ability to generalize effectively to the target domain’s distribution. The presence of noisy data further exacerbates this issue by inducing additional overfitting to noise, causing existing domain generalization methods to fail in effectively distinguishing invariant features from spurious ones. To address these challenges, we propose A nchor A lignment and A daptive W eighting ( A 3 W A^{3}W ), a novel algorithm based on sample reweighting guided by natural language processing (NLP) anchors that seeks to extract representative features. In particular, A 3 W A^{3}W leverages semantic representations derived from natural language models to serve as a source of domain-invariant prior knowledge. We also introduce a weighted loss function that dynamically adjusts the contribution of each sample based on its distance to the corresponding NLP anchor, thereby improving the model’s resilience to noisy labels. Extensive experiments on benchmark datasets demonstrate that A 3 W A^{3}W outperforms state-of-the-art domain generalization methods, yielding significant improvements in both accuracy and robustness across various datasets and noise levels. I Introduction Domain Generalization (DG) has emerged as a pivotal algorithm in machine learning, aiming to develop models that can maintain high performance on previously unseen environments—or domains . Traditional methods often assume that training and test data share the same distribution, yet in real-world scenarios, there is frequently a substantial shift between these distributions. This phenomenon, widely referred to as domain shift , can cause severe performance degradation in tasks spanning computer vision, natural language processing, and medical image analysis [ 1 ] . As shown in Figure 1 (a)(b), even within the same class label, the distribution of feature representations can vary considerably. This variation may stem from differences in image acquisition conditions—such as lighting variations, changes in pose, or complex background environments—and even from more subtle domain-specific factors like sensor noise or camera calibration differences. Such intra-class variability poses a significant challenge for developing accurate and adaptable models, which must learn to extract invariant features that capture the true semantic essence of the class while ignoring irrelevant variations. In response, DG task has been proposed to enable robust model behavior without further fine-tuning, thereby facilitating broader applicability in diverse domains [ 2 , 3 ] . Over time, researchers have explored various ways to align or unify source and target domains, including strategies that learn domain-invariant features through multi-task autoencoders, augmented architecture, or adversarial alignment [ 4 , 5 , 6 ] . Fig. 1: Illustration of domain shift using four distinct domains—painting, photo, cartoon, and sketch—for the same object class. In (a), the visual appearance of “elephant” varies substantially across domains, underscoring significant style discrepancies. In (b), the t-SNE projection shows that even for the same class, the distribution of features differs across domains, highlighting the inherent challenges of domain generalization. In (c), unclear or mislabeled samples introduces additional noise, further exacerbating the difficulty of achieving robust generalization. In (d), we show the accuracy of a network trained on noisy data: ideally, the model should resist overfitting to noise, but the graph indicates a steady increase in noise accuracy over time, suggesting progressive overfitting to noisy labels. One persistent obstacle in DG is the prevalence of spurious correlations in deep neural networks, where models inadvertently link target labels to irrelevant, domain-specific features [ 7 , 8 ] . For instance, a classifier tasked with recognizing animals might rely on particular backgrounds for prediction, causing it to misclassify animals photographed in unfamiliar settings [ 9 ] . Such spurious correlations frequently arise because modern networks have the capacity to overfit even low-level noise, thereby anchoring their decisions on non-causal cues [ 10 ] . To address these pitfalls, researchers have proposed a variety of techniques, from causal inference algorithms like Invariant Risk Minimization (IRM) [ 7 ] to methods specifically designed to disentangle spurious and invariant features [ 11 ] . For example, Wu et al. [ 12 ] introduced the Discover and Cure (DISC) strategy, which detects suspicious features and systematically mitigates their impact by leveraging interpretable domain knowledge. The development of methods rooted in causal representation learning also arises to mitigate the reliance on spurious correlations [ 13 ] . Despite these advances, achieving robust generalization under significant domain shift and considerable label noise persists as an open research challenge that necessitates more comprehensive methods and better theoretical insights. Existing domain generalization (DG) approaches have predominantly focused on aligning feature distributions or disentangling spurious factors using data-driven methods [ 1 ] . While these techniques have paved the way for significant advances, they often encounter considerable difficulties when label noise is prevalent or when data lack distinct domain-invariant cues [ 14 ] , leading to rapid degradation in network performance as noise intensity increases. This challenge is further exacerbated in large, overparameterized networks that are prone to memorizing spurious correlations, thereby amplifying errors. Indeed, estimates suggest that 8% to 38.5% of real-world data may suffer from label corruption, undermining the generalizability of models that fail to distinguish genuine signals from incidental patterns [ 15 ] . Although some DG algorithms inherently exhibit robustness to label noise, in general cases they lack performance superiority compared to empirical risk minimization (ERM) baseline [ 8 ] . Consequently, understanding how to boost robustness against both domain shifts and noisy labels remains a pressing goal in building truly reliable artificial intelligence systems. In light of these challenges, we consider an alternative approach that focuses on introducing novel sources of features from diverse perspectives to guide the learning pipeline. An emerging line of research has investigated the role of external knowledge , which can guide models toward more semantically grounded representations [ 16 ] . Some studies incorporate domain knowledge directly into the learning pipeline, for instance by embedding specialized scientific insights into feature extraction or loss design [ 17 ] . In other cases, researchers have shown that domain knowledge can significantly enhance model interpretability and robustness—for example, in laser-induced breakdown spectroscopy, where integrated expert knowledge improved quantification performance [ 18 ] . Another promising direction involves multi-modal alignment, in which semantic text descriptions serve as stable anchors that help networks transcend purely visual or sensor-driven biases [ 19 ] . Models like CLIP [ 20 ] exemplify this principle by projecting both text and images into a shared embedding space, improving generalization across dissimilar environments. Yet critical barriers remain regarding how to ensure that the guidance remains valid across diverse domains and how to structure knowledge encoding effectively to align with modern deep learning architectures [ 16 ] . Inspired by this recent trend of knowledge integration, we propose a new algorithm called A nchor A lignment and A daptive W eighting ( A^{3}W ), which integrates external knowledge—specifically, linguistic cues derived from large-scale language models—into the DG pipeline. Our central insight is that NLP anchors , such as