LoGoSeg: Integrating Local and Global Features for Open-Vocabulary Semantic Segmentation

Junyang Chen, Xiangbo Lv, Zhiqiang Kou, Xingdong Sheng, Ning Xu, Yiguo Qiao · 2026

Read LoGoSeg: Integrating Local and Global Features for Open-Vocabulary Semantic Segmentation by Junyang Chen, Xiangbo Lv, Zhiqiang Kou, Xingdong Sheng, Ning Xu, Yiguo Qiao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05578.txt.

Introduction

Segmenting an image into labeled regions for any category a user names in text is called open-vocabulary semantic segmentation. Junyang Chen, Xiangbo Lv, and 4 co-authors propose LoGoSeg, a single-stage framework that joins local detail with global context for this task. Vision-language models like CLIP give the text-image link, but their image-level training often yields sloppy spatial alignment and invented or missed objects.

LoGoSeg adds three pieces: a prior that weights likely categories by global image-text similarity, and a region-aware module that ties regions to text precisely. A dual-stream fusion mixes local structure with global meaning. It needs no outside mask proposals, extra backbones, or added data sets. Across six standard benchmarks, it showed competitive scores and strong generalization to unseen categories. The paper targets computer vision researchers who need segmentation beyond a fixed label list.

Catalog record

CREATORJunyang Chen, Xiangbo Lv, Zhiqiang Kou, Xingdong Sheng, Ning Xu, Yiguo Qiao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05578

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LoGoSeg: Integrating Local and Global Features for Open-Vocabulary Semantic Segmentation Junyang Chen, Xiangbo Lv, Zhiqiang Kou, Xingdong Sheng, Ning Xu, Yiguo Qiao Abstract Open-vocabulary semantic segmentation (OVSS) extends traditional closed-set segmentation by enabling pixel-wise annotation for both seen and unseen categories using arbitrary textual descriptions. While existing methods leverage vision-language models (VLMs) like CLIP, their reliance on image-level pretraining often results in imprecise spatial alignment, leading to mismatched segmentations in ambiguous or cluttered scenes. However, most existing approaches lack strong object priors and region-level constraints, which can lead to object hallucination or missed detections, further degrading performance. To address these challenges, we propose LoGoSeg, an efficient single-stage framework that integrates three key innovations: (i) an object existence prior that dynamically weights relevant categories through global image-text similarity, effectively reducing hallucinations; (ii) a region-aware alignment module that establishes precise region-level visual-textual correspondences; and (iii) a dual-stream fusion mechanism that optimally combines local structural information with global semantic context. Unlike prior works, LoGoSeg eliminates the need for external mask proposals, additional backbones, or extra datasets, ensuring efficiency. Extensive experiments on six benchmarks (A-847, PC-459, A-150, PC-59, PAS-20, and PAS-20 b ) demonstrate its competitive performance and strong generalization in open-vocabulary settings. Figure 1: Comparison of open-vocabulary segmentation frameworks. (Top) Two-stage methods rely on external mask proposals, often causing hallucinations. (Middle) One-stage methods are more efficient but struggle with pixel-level grounding. (Bottom) LoGoSeg integrates object priors, region alignment, and dual fusion to improve cross-modal consistency and segmentation quality. 1 Introduction Semantic segmentation, a fundamental computer vision task that assigns semantic labels to each image pixel, achieves high accuracy on fixed-category benchmarks ( 4 ; 2 ; 29 ) . However, these conventional methods suffer from overfitting, are constrained by predefined label sets, and fail to generalize to unseen classes. To address these limitations, open-vocabulary semantic segmentation (OVSS) has recently emerged, enabling pixel-level prediction for arbitrary categories specified via textual prompts ( 48 ; 37 ; 22 ) . This paradigm significantly broadens the applicability of semantic segmentation beyond closed-set constraints. The core challenge of OVSS lies in its requirement to handle open-domain categories during inference—a capability fundamentally absent in traditional vision-only backbone networks. To address this, current research predominantly employs vision-language models (VLMs) such as CLIP ( 26 ) , ALIGN ( 13 ) , EVA-CLIP ( 31 ) and the SigLIP series ( 33 ; 46 ) , leveraging their zero-shot recognition capabilities acquired through large-scale image-text pretraining ( 30 ; 44 ) . However, these models’ image-level supervision paradigm creates inherent difficulties in adapting to pixel-level segmentation tasks ( 50 ; 49 ) . Specifically, the pretraining process lacks explicit constraints for aligning textual concepts with visual regions, resulting in blurred segmentation boundaries and uncertain category predictions—limitations that become particularly pronounced when dealing with unseen or rare categories in open-vocabulary scenarios. Furthermore, the absence of object priors and region-aware constraints may lead to false positives or missed detections in complex scenes, significantly compromising segmentation accuracy. To address these challenges, researchers have developed two distinct methodological frameworks: two-stage ( 17 ; 7 ; 43 ; 6 ; 42 ; 10 ; 40 ; 45 ; 14 ; 35 ) and single-stage ( 8 ; 5 ; 51 ; 41 ; 38 ; 28 ; 30 ) approaches. The two-stage paradigm, exemplified by mask proposal generators such as SAM ( 15 ; 27 ; 47 ) and MaskFormer ( 4 ) , predict category-agnostic mask proposals and then classify them using the zero-shot capability of frozen CLIP models. While achieving competitive accuracy, the efficiency is hindered by separate mask generation and classification stages. As shown in Fig. 1 (a), they fail to fully exploit contextual information and depend heavily on mask quality, which limits their generalization. Single-stage approaches, by contrast, directly adapt VLMs for pixel-level segmentation, simultaneously enhancing efficiency while leveraging the models’ inherent segmentation capabilities. While recent advances like SAN ( 41 ) , SED ( 38 ) , and CAT-Seg ( 5 ) have demonstrated promising results, Fig. 1 (b) reveals critical limitations: an imbalance between local and global feature integration, insufficient region-level alignment, and unaddressed classification hallucinations. To overcome the limitations of current methods, we propose LoGoSeg, a unified single-stage framework that addresses three core challenges in OVSS: classification hallucinations, misaligned pixel-level supervision, and insufficient fusion of local and global cues. The proposed architecture incorporates three novel components: first, an adaptive object existence prior computed from global image-text similarity dynamically adjusts category weights to suppress false positives; second, our region-aware alignment module establishes precise correspondences between visual regions and textual concepts through localized similarity computation; third, a dual-stream fusion mechanism jointly encodes fine-grained spatial structures and high-level semantic context in a unified representation space. Our contributions are summarized below: (i) Region-aware alignment with object prior : Our strategy significantly reduces category hallucinations while improving pixel-level text-visual alignment through explicit region-level constraints. (ii) Dual-stream feature fusion : The proposed framework effectively combines detailed local spatial context with comprehensive global semantics, overcoming the limitations of previous single-stream approaches. (iii) Unified single-stage framework : LoGoSeg integrates these innovations into a single-stage pipeline that demonstrates superior performance across six standard benchmarks, achieving state-of-the-art results in both accuracy and computational efficiency. Figure 2: Overview of LoGoSeg. CLIP encoders extract multi-level visual and textual embeddings. A prior-guided alignment module uses a lightweight MLP to score regions for category-aware guidance and a Region Aligner to tightly align visual and textual features. A dual-branch fusion integrates local directional self-attention with global state-space modeling. A transformer with learnable queries then performs fine-grained cross-modal fusion via linear attention, and a hierarchical decoder with learnable upsampling and guidance-modulated convolutions yields the final segmentation map. 2 Method We propose LoGoSeg, a region-aware framework for OVSS, as illustrated in Fig. 2 . Given an input image and a set of category prompts, the model first extracts image-level visual and textual features using CLIP. The prior-guided regional alignment module ( Section 2.2 ) then measures similarity between regional visual features and adjusted text embeddings to provide category-aware guidance. These aligned features are subsequently passed to the cross-modal fusion module ( Section 2.3 ), which integrates local and global cues via parallel architectures capturing directional spatial details and long-range contextual dependencies. Finally, the transformer-based decoder ( Section 2.4 ) aggregates the fused features and predicts the final segmentation masks using learnable queries. Comprehensive details of each module are provided in the following subsections. 2.1 Preliminary Open‐vocabulary semantic segmentation (OVSS) performs dense prediction of text-defined categories at the pixel level. Given an input image \mathbf{I}\in\mathbb{R}^{H\times W\times 3} and a set of N text prompts \mathcal{T}=\{\mathbf{t}_{1},\dots,\mathbf{t}_{N}\} , the task outputs a segmentation map \mathbf{Y}\in\{1,\dots,N\}^{H\times W} , where each pixel is assigned the index of its corresponding prompt. We use CLIP as the backbone, with image encoder f_{\mathrm{img}}(\cdot) producing \mathbf{V}=f_{\mathrm{img}}(\mathbf{I})\in\mathbb{R}^{C\times H\times W} and text encoder f_{\mathrm{text}}(\cdot) producing prompt embeddings \mathbf{T}=f_{\mathrm{text}}(\mathcal{T})\in\mathbb{R}^{N\times P\times C} . However, as CLIP is trained only at the image level, these embeddings lack direct pixel correspondence. Notation. Let B denote batch size, C feature channels, H\times W spatial dimensions, N the number of categories, P prompts per category, K number of regions, D guidance dimension, T number of queries, and L_{t} pooled text tokens. The

Read the complete text