DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization
Xudong Han, Senkang Hu, Yihang Tao, Yu Guo, Philip Birch, Sam Tak Wu Kwong, Yuguang Fang · 2026
Read DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization by Xudong Han, Senkang Hu, Yihang Tao, Yu Guo, Philip Birch, Sam Tak Wu Kwong, Yuguang Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19394.txt.
Introduction
Xudong Han and seven co-authors regularize model parameters by domain so models travel to new data safely. Domain generalization asks a model to do well on distributions it never saw, a must for real use. Most methods chase features that stay the same across source domains.
Their method, DSP-Reg, instead regularizes the parameters themselves with domain sensitivity in mind. Not all weights should be held equally loose or tight, and domains hint at which. Better travel to unseen data matters anywhere the world shifts after training, from hospitals to highways.
Catalog record
| CREATOR | Xudong Han, Senkang Hu, Yihang Tao, Yu Guo, Philip Birch, Sam Tak Wu Kwong, Yuguang Fang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-19394
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization Xudong Han, Senkang Hu, Yihang Tao, Yu Guo, Philip Birch, Sam Tak Wu Kwong, Yuguang Fang Abstract Domain Generalization (DG) is a critical area that focuses on developing models capable of performing well on data from unseen distributions, which is essential for real-world applications. Existing approaches primarily concentrate on learning domain-invariant features, which assume that a model robust to variations in the source domains will generalize well to unseen target domains. However, these approaches neglect a deeper analysis at the parameter level, which makes the model hard to explicitly differentiate between parameters sensitive to domain shifts and those robust, potentially hindering its overall ability to generalize. In order to address these limitations, we first build a covariance-based parameter sensitivity analysis framework to quantify the sensitivity of each parameter in a model to domain shifts. By computing the covariance of parameter gradients across multiple source domains, we can identify parameters that are more susceptible to domain variations, which serves as our theoretical foundation. Based on this, we propose Domain-Sensitive Parameter Regularization (DSP-Reg), a principled framework that guides model optimization by a soft regularization technique that encourages the model to rely more on domain-invariant parameters while suppressing those that are domain-specific. This approach provides a more granular control over the model’s learning process, leading to improved robustness and generalization to unseen domains. Extensive experiments on benchmarks, such as PACS, VLCS, OfficeHome, and DomainNet, demonstrate that DSP-Reg outperforms state-of-the-art approaches, achieving an average accuracy of 66.7% and surpassing all baselines. 1 Introduction Domain Generalization (DG) remains a fundamental yet challenging problem in machine learning, addressing the critical task of training models that maintain robust predictive capabilities across unseen, out-of-distribution target domains. This capability is crucial for real-world applications such as autonomous driving Hu et al. 2024c ; Hu et al. 2024a ; Hu et al. 2024b ; Hu et al. 2025a ; Hu et al. 2024d ; Hu et al. 2024c , medical imaging Liu et al. 2020 , face recognition Wen et al. 2016 , and AIGC Guo et al. 2025 ; Li et al. 2025 ; Guo et al. 2024 ; Hu et al. 2025b , where even slight changes in environments can induce substantial distributional shifts. In addition, standard models typically suffer significant performance degradation under such shifts, underscoring limitations to their reliability in practice. Currently, most DG methods focus on learning domain-invariant feature representations from multiple source domains Hu et al. 2024c ; Hu et al. 2024a ; Yang and Soatto ; Zhou et al. 2022 ; Nguyen et al. 2022 . Representative strategies include domain-adversarial training to promote feature invariance Ganin et al. 2016b , reducing distributional discrepancies using metrics such as Maximum Mean Discrepancy Li et al. 2018b , and enforcing invariant predictive relationships as in Invariant Risk Minimization (IRM) Arjovsky et al. 2019 . Although these methods have achieved notable progress, they primarily operate at the feature level and lack direct, fine-grained control over individual model parameters. As a result, these methods are unable to explicitly differentiate between domain-invariant and domain-sensitive components, which may limit its overall generalization capability. To address this limitation, recent research has begun examining model parameters directly through sensitivity analysis. For example, Fisher Information matrix Kirkpatrick et al. 2017 provides valuable insights into how model parameters respond differently under domain shifts. Approaches like Fisher regularization (Fishr) Rame et al. 2022 utilize gradient-based analyses to enforce invariance in gradient distributions across domains, thus partially addressing parameter-level domain sensitivity. Nevertheless, these approaches remain limited because of their reliance on static, local curvature estimates and are often ineffective under dynamically evolving or non-stationary domain shifts. Motivated by the observation that existing approaches underutilize parameter-level information, we begin our investigation from the perspective of model parameter sensitivity . Specifically, we propose a covariance-based parameter sensitivity analysis framework to quantify the sensitivity of each parameter to domain shifts. First, we introduce a linear perturbation model to locally approximate the change in output resulting from small changes in the input and parameters. Then, by applying the law of covariance propagation, we analyze how these uncertainties propagate through the network and affect the final output variance. Finally, we derive a single-parameter sensitivity index s_{k} to measure the contribution of the variance of parameter \theta_{k} to the variance of the final output; a larger s_{k} indicates that small fluctuations in \theta_{k} significantly affect the output, making \theta_{k} sensitive to inputs. Building on this theoretical foundation, we propose a novel approach named Domain-Sensitive Parameter Regularization (DSP-Reg). The key idea is that if a parameter exhibits significant sensitivity differences across different source domains, it is likely to have learned domain-specific features rather than generalizable knowledge . We then design a soft regularization scheme that penalizes domain-sensitive parameters, encouraging the model to rely on domain-invariant parameters that perform consistently across all domains. Our method’s granular control enhances model robustness and generalization, potentially extending to other DG strategies as a complementary mechanism. In summary, the primary contributions of this work are as follows. • Introducing a principled covariance-based parameter sensitivity framework to systematically quantify parameter sensitivity to domain shifts. • Proposing DSP-Reg, which can adaptively penalize domain-sensitive parameters, substantially improving model generalization. • Demonstrating through extensive experiments on standard DG benchmarks (PACS, VLCS, OfficeHome, TerraIncognita, DomainNet) that DSP-Reg significantly outperforms state-of-the-art approaches. 2 Background and Related Work 2.1 Domain Generalization Domain Generalization (DG) aims to train models capable of generalizing to unseen, out-of-distribution target domains by learning from one or multiple source datasets, which may exhibit differing data distributions. DG methods can be broadly distinguished based on the nature of the source data: single-source approaches typically operate without leveraging information about distinct domains within the training set, effectively treating all source data as a single unit, whereas multi-source algorithms explicitly utilize domain labels and exploit the statistical differences between the available source domain distributions to inform the learning process. A predominant strategy involves focusing on domain-invariant representations, where models strive to capture underlying data characteristics that are robust to domain shifts. This has been pursued through various means, including adversarial training Ganin et al. 2016a , distribution discrepancy minimization using metrics like Maximum Mean Discrepancy (MMD) Li et al. 2018b ; Long et al. 2015 ; Muandet et al. 2013 , and invariance principles such as Invariant Risk Minimization (IRM) Arjovsky et al. 2019 ; Krueger et al. 2021 . Other common approaches include extensive data augmentation Carlucci et al. 2019 ; Zhou et al. 2021 to expose models to wider variations, and meta-learning Balaji et al. 2018 ; Dou et al. 2019 ; Li et al. 2018a , which trains models to adapt to new domains rapidly. Additionally, domain alignment Ganin et al. 2016a ; Muandet et al. 2013 and self-supervised learning He et al. 2020 have also been proposed to focus on learning domain-invariant features by minimizing discrepancies between source domains. Recently, FisherTune ( Zhao et al. 2025 ) has leveraged Vision Foundation Models (VFMs) for DG tasks, particularly in semantic segmentation, capitalizing on their robust pre-trained representations. 2.2 Parameter Sensitivity Analysis Parameter Sensitivity Analysis in DG focuses on understanding how model parameters respond to domain shifts and using this insight to enhance generalization.