Multi-Granularity Feature Calibration via VFM for Domain Generalized Semantic Segmentation

Xinhui Li, Xiaojie Guo · 2025

Read Multi-Granularity Feature Calibration via VFM for Domain Generalized Semantic Segmentation by Xinhui Li, Xiaojie Guo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03007.txt.

Introduction

Xinhui Li and Xiaojie Guo present a method in a 2025 paper to make image labeling work on scenes it has never seen. The task, semantic segmentation, paints each pixel with a class label, like road or person. Models trained in one city or weather often fail in another, and the new place's data is not available during training.

Their method builds on large pretrained vision models and tunes them in layers. It first aligns broad scene features, then sharpens class-level features, then refines fine detail. This coarse-to-fine calibration aims to keep labels right when lighting, place, or style shifts. Robust labeling matters for self-driving cars and robots.

Catalog record

CREATORXinhui Li, Xiaojie Guo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-03007

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Granularity Feature Calibration via VFM for Domain Generalized Semantic Segmentation Xinhui Li, Xiaojie Guo Abstract Domain Generalized Semantic Segmentation (DGSS) aims to improve the generalization ability of models across unseen domains without access to target data during training. Recent advances in DGSS have increasingly exploited vision foundation models (VFMs) via parameter-efficient fine-tuning strategies. However, most existing approaches concentrate on global feature fine-tuning, while overlooking hierarchical adaptation across feature levels, which is crucial for precise dense prediction. In this paper, we propose Multi-Granularity Feature Calibration (MGFC), a novel framework that performs coarse-to-fine alignment of VFM features to enhance robustness under domain shifts. Specifically, MGFC first calibrates coarse-grained features to capture global contextual semantics and scene-level structure. Then, it refines medium-grained features by promoting category-level feature discriminability. Finally, fine-grained features are calibrated through high-frequency spatial detail enhancement. By performing hierarchical and granularity-aware calibration, MGFC effectively transfers the generalization strengths of VFMs to the domain-specific task of DGSS. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art DGSS approaches, highlighting the effectiveness of multi-granularity adaptation for the semantic segmentation task of domain generalization. Introduction Benefiting from large-scale supervised data, deep neural networks have achieved remarkable performance in a wide range of computer vision tasks, such as object detection ( He et al. 2017 ; Cheng et al. 2018 ) and semantic segmentation ( Strudel et al. 2021 ; Ding et al. 2020 ) . A core premise behind these advances is the assumption that training and test data are independently and identically distributed. However, in realistic scenarios, this assumption often breaks down due to unpredictable distribution shifts, especially since test data is typically unknown and inaccessible during training. To address these challenges, domain generalized semantic segmentation (DGSS) has emerged as a crucial research direction ( Yue et al. 2019 ; Matsuura and Harada 2020 ; Li et al. 2021 ; Wu et al. 2022 ) . DGSS aims to train segmentation models solely on labeled source domains while ensuring robust generalization performance across diverse and unseen target domains during inference. Figure 1: Illustration of the existing DGSS methods and Ours. Left: Previous methods focus on feature fine-tuning at the same granularity. Right: Our method achieves multi-granularity feature calibration at different feature levels. Recent advancements in DGSS researches ( Wei et al. 2024 ; Yi et al. 2024 ; Bi et al. 2024 ) have been significantly influenced by the emergence of Vision Foundation Models (VFMs), which offer substantial advantages over earlier methods ( Li et al. 2018 ; Peng et al. 2019 ; Matsuura and Harada 2020 ; Zhu and Li 2021 ) . Pretrained on massive and diverse datasets spanning various modalities and tasks, VFMs such as DINO ( Oquab et al. 2023 ) , SAM ( Kirillov et al. 2023 ) , and CLIP ( Radford et al. 2021a ) demonstrate impressive zero-shot and few-shot generalization capabilities. These models inherently capture rich visual semantics and exhibit strong robustness to distribution shifts. Notably, recent work ( Wei et al. 2024 ) shows that even frozen VFMs, when directly applied to DGSS without any task-specific fine-tuning, can outperform many existing domain generalization methods. This finding underscores the unique generalization potential embedded in VFMs and reveals their capacity to serve as strong inductive priors for dense prediction tasks under domain shift. In parallel, the emergence of parameter-efficient fine-tuning techniques, such as Low-Rank Adaptation (LoRA) ( Hu et al. 2022 ) , has further enhanced the application of VFMs in DGSS. Recent methods combining frozen VFMs with modular fine-tuning ( Wei et al. 2024 ; Yi et al. 2024 ; Bi et al. 2024 ) have achieved notable success. These approaches leverage pre-trained VFMs to achieve domain generalization while requiring minimal updates to their network parameters. However, current methods focus primarily on global feature adaptation, even when decomposing feature representations ( Yi et al. 2024 ; Bi et al. 2024 ) within specific layers. As shown in Fig. 1 , most existing methods, whether based on fine-tuning or multi-branch adaptation, primarily operate at a global level. While global adaptation plays an important role in DGSS, fine-grained calibration is equally essential for achieving robust performance. In particular, pixel-wise segmentation accuracy is highly dependent on object-level semantic consistency and precise boundary delineation, which are often overlooked by global alignment strategies. This raises a critical question: How can fine-tuning methodologies address the granularity required for the pixel-level prediction task in DGSS? To tackle this challenge, we propose a novel Multi-Granularity Feature Calibration (MGFC) framework for DGSS. As shown in Fig. 1 , MGFC performs hierarchical feature alignment across three granularity levels enabling comprehensive fine-tuning from global semantics to local details. Specifically, Coarse-grained calibration preserves global contextual representations and captures scene-specific semantics critical for global feature alignment. Medium-grained calibration guides the generalization power of VFMs to concentrate on category-level semantics, enhancing task-specific feature representation. Fine-grained calibration emphasizes spatial refinement by reinforcing high-frequency details, facilitating precise edge localization and pixel-level segmentation accuracy. By explicitly modeling feature adaptation across multiple granularities, MGFC bridges the gap between global robustness and local precision. To further integrate multi-granularity representations, we propose a query fusion module that combines contextual global patterns with precise local details. This module enhances performance on downstream DGSS tasks while preserving the generalization capacity of the frozen VFMs. Overall, the major contributions of this work can be summarized as follows: • We propose a novel multi-granularity feature calibration framework for DGSS, which enables hierarchical fine-tuning of vision foundation model features across coarse, medium, and fine levels. • A query fusion module is desgined to effectively integrate multi-granularity representations, guiding the network to jointly capture both global semantics and fine-grained structural cues. • Extensive experiments and ablation studies demonstrate the effectiveness and superior performance of our method over state-of-the-art approaches. Related Work Domain Generalization To address the challenge of performance degradation when models encounter data from unseen domains, domain generalization (DG) has emerged as a key strategy. DG methods ( Su, Wang, and Yang 2022 ; Li et al. 2022 ) are fundamentally designed to enhance the generalization of models under diverse or unknown distribution conditions. Specifically, DG concentrates on enhancing generalization in entirely unseen domains through multi-source training ( Gan, Yang, and Gong 2016 ; Zhao et al. 2021 ) , feature normalization ( Lee, Bae, and Kim 2023 ; Qi et al. 2024 ) , and data augmentation ( Su et al. 2023 ; Wang et al. 2024 ; Zhou et al. 2024 ) . The aforementioned DG methods have been extensively applied to various downstream tasks, such as image classification tasks ( Zhou et al. 2021 ; Kang et al. 2022 ; Zhang et al. 2022 ) and semantic segmentation ( Yue et al. 2019 ; Choi et al. 2021 ; Wu et al. 2022 ) . Most DG methods ( Li et al. 2022 ; Jiang et al. 2023 ; Fahes et al. 2024 ) are built on classical network backbones, such as ResNet ( He et al. 2016 ) , VGGNet ( Simonyan and Zisserman 2014 ) , and ShuffleNetV2 ( Ma et al. 2018 ) . However, with the advent of vision foundation models(VFM) ( He et al. 2022 ; Radford et al. 2021b

Read the complete text