From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs

Boyong Wu, Sanghwan Kim, Zeynep Akata · 2026

Read From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs by Boyong Wu, Sanghwan Kim, Zeynep Akata (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17228.txt.

Introduction

Wu, Kim, and Akata trace how segmentation ability moves through a multimodal language model, part by part. They probe each stage with linear probes: the vision encoder, the adapter, and the language model layers. They find the adapter causes a drop off in segmentation quality. The language model layers then recover it, as attention lets correctly classified tokens steer wrong neighbors toward the right label. Early image tokens recover less, bounded by causal attention. Letting image tokens attend both ways eases that bound.

Catalog record

CREATORBoyong Wu, Sanghwan Kim, Zeynep Akata
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45998
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17228

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs Boyong Wu, Sanghwan Kim, Zeynep Akata Abstract Multimodal Large Language Models (MLLMs) are increasingly applied to pixel-level vision tasks, yet their intrinsic capacity for spatial understanding remains poorly understood. We investigate segmentation capacity through a layerwise linear probing evaluation across the entire MLLM pipeline: vision encoder, adapter, and LLM. We further conduct an intervention based attention knockout analysis to test whether cross-token attention progressively refines visual representations, and an evaluation of bidirectional attention among image tokens on spatial consistency. Our analysis reveals that the adapter introduces a segmentation representation drop-off, but LLM layers progressively recover through attention-mediated refinement, where correctly classified tokens steer misclassified neighbors toward the correct label. At early image token positions, this recovery is bounded by causal attention, which bidirectional attention among image tokens alleviates. These findings provide a mechanistic account of how MLLMs process visual information for segmentation, informing the design of future segmentation-capable models. Figure 1 : Overview of main findings. (a) Layerwise linear probing on ADE20K: the adapter introduces a representation drop-off, but LLM layers progressively recover segmentation quality. (b) Attention knockout on conflicting class pairs: knocking out attention from correctly classified tokens degrades segmentation, confirming that cross-token self-refinement is driven by semantic anchors. (c) Per-token pixel accuracy at an intermediate LLM layer: causal attention starves early position tokens of semantic anchors, while bidirectional attention among image tokens alleviates this bottleneck. 1 Introduction Detecting and segmenting content in visual scenes is foundational for applications in robotics, AR/VR, autonomous driving, and medical imaging [ 16 , 9 , 39 ] . Vision Transformers pretrained with contrastive image-text objectives or self-supervised distillation have emerged as powerful feature extractors: their patch-level embeddings capture rich dense semantics and transfer strongly to segmentation when paired with lightweight decoders or linear probes [ 3 , 28 , 27 ] . More recently, large-scale pretraining has yielded specialized vision encoders such as SAM [ 18 , 30 , 5 ] that further push segmentation performance. In parallel, Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in multimodal reasoning, instruction following, and language conditioned control [ 2 , 8 , 23 ] . Among MLLMs, adapter style architectures are notable for their simplicity and have demonstrated state-of-the-art performance by combining a pretrained Vision Encoder with a pretrained Large Language Model (LLM) through a learned adapter network, that maps vision embeddings into the LLM’s token space [ 25 , 24 , 23 ] . These capabilities have motivated a growing body of work that adapts MLLMs for pixel-level semantic segmentation tasks, arguing that language supervision can improve interpretability and enable compositional segmentation driven by Referring Expression Segmentation, which allows segmentation through instruction following beyond what traditional segmentation models support [ 17 , 19 ] . While recent works have proposed MLLM-based methods specialized for semantic segmentation [ 36 , 7 , 31 , 40 ] , it remains unclear whether the MLLM as a whole is actually better at image segmentation than its underlying vision encoder under a controlled probing setup. Recent analyses suggest that, without task-specific tuning, MLLMs may underperform on classical vision tasks and may overlook visual evidence that is already present in their vision backbones [ 42 , 13 , 33 , 14 ] . To address this gap in the literature, we propose a systematic, intervention-driven framework to dissect where segmentation competence arises or degrades across the MLLM stack. First, we perform linear probing of frozen embeddings at the vision encoder, the adapter, and every intermediate LLM layer ( Section 3 ). This reveals a representation drop-off at the adapter: projecting into the LLM embedding space trades fine-grained spatial fidelity for cross-modal alignment, degrading token-level separability. However, downstream LLM layers progressively self-refine these visual representations and gradually recover segmentation quality. This recovery raises the question of whether it is actively driven by cross-token attention or is merely a byproduct of residual connections and normalization. To further study these mechanisms, we design attention knockout interventions on conflicting class pairs (e.g., ceiling vs. sky), which suppress specific attention pathways while leaving all other operations unchanged ( Section 4 ). We find that correctly classified tokens act as semantic anchors whose attention signals pull misclassified neighbors toward the correct label, indicating that cross-token attention drives the observed self-refinement. Finally, under standard causal attention, early image tokens can only attend to preceding tokens in sequence order and thus lack global spatial context, limiting self-refinement at those positions. We ablate this context starvation by introducing bidirectional attention exclusively among image tokens and by varying the vision encoder ( Section 5 ). Bidirectional attention eliminates the positional bias at early patches, and the degree of recovery depends on the compatibility between the vision encoder’s representation space and the LLM’s embedding space, with text-aligned encoders (e.g., CLIP, SigLIP) benefiting more than vision-only ones (e.g., DINOv2). Our contribution can be summarized as follows ( From Drop-off to Recovery: A Mechanistic Analysis of Segmentation in MLLMs ): • We reveal a representation drop-off at the adapter and progressive self-refinement across LLM layers by performing layerwise linear probing that analyzes segmentation competence across the MLLM stack. • Our attention knockout experiments show that self - refinement is driven by cross - token attention, with correctly classified tokens acting as semantic anchors that guide their misclassified neighbors. • We further find that applying bidirectional attention alleviates context starvation at early image tokens and that text-aligned encoders benefit more from LLM-layer refinement than vision-only encoders. Across experiments, we observe that LLM layers provide structure- and constraint-aware refinement that improves local consistency and resolves certain class conflicts, but they do not recover fine-grained spatial details absent from the encoder. These findings clarify when and where MLLMs aid semantic segmentation and where they fall short, offering practical guidance for the design of segmentation-capable MLLMs. 2 Related Work Vision Encoders for Segmentation. Vision Transformers pretrained at scale with supervised, contrastive, or self-supervised objectives have become powerful feature extractors for dense prediction tasks [ 11 , 28 , 30 , 6 , 38 ] . Notably, patch-level features from these models already transfer strongly to segmentation and other dense tasks when decoded with lightweight heads or linear probes [ 3 , 27 ] , establishing that rich spatial information is present in the encoder representations themselves. Our work builds on this probing paradigm but extends it beyond the vision encoder in isolation: we probe representations at every stage of the full MLLM pipeline, from the encoder through the adapter and into each LLM layer, to understand whether the downstream components preserve, degrade, or enhance the spatial information already present in the encoder features. MLLMs for Segmentation. Adapter-style MLLMs connect a pretrained vision encoder to a pretrained LLM through a learned projection [ 25 , 24 , 37 ] , enabling multimodal reasoning without training either component from scratch. A growing line of work adapts this architecture for segmentation, leveraging instruction following and language-conditioned control to produce pixel-level outputs. A common strategy introduces special tokens that bridge language reasoning with mask prediction. LISA [ 19 ] pioneered this by connecting an MLLM to SAM via a learned [SEG] token, enabling reasoning segmentation from complex

Read the complete text