Source-Modality Monitoring in Vision-Language Models
Etha Tianze Hua, Tian Yun, Ellie Pavlick · 2026
Read Source-Modality Monitoring in Vision-Language Models by Etha Tianze Hua, Tian Yun, Ellie Pavlick (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22038.txt.
Introduction
Hua, Yun, and Pavlick define and study source-modality monitoring, the ability of multimodal models to track and share the input source of pieces of information. They evaluate how far models exploit syntactic versus semantic signals to bind words like image in a prompt to exact input parts, that is, actual images. Across experiments spanning 11 vision-language models on target-modality retrieval tasks, both syntactic and semantic signals play an important role. The latter tend to outweigh the former when modalities are highly distinct distributionally.
Catalog record
| CREATOR | Etha Tianze Hua, Tian Yun, Ellie Pavlick |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66433 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.22038
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Source-Modality Monitoring in Vision-Language Models Etha Tianze Hua, Tian Yun, Ellie Pavlick Abstract We define and investigate source-modality monitoring –the ability of multimodal models to track and communicate the input source from which pieces of information originate. We consider source-modality monitoring as an instance of the more general binding problem , and evaluate the extent to which models exploit syntactic vs. semantic signals in order to bind words like image in a user-provided prompt to specific components of their input and context (i.e., actual images). Across experiments spanning 11 vision-language models (VLMs) performing target-modality information retrieval tasks, we find that both syntactic and semantic signals play an important role, but that the latter tend to outweigh the former in cases when modalities are highly distinct distributionally. We discuss the implications of these findings for model robustness, and in the context of increasingly multimodal agentic systems. 1 1 1 All resources will be available at https://github.com/ethahtz/source-modality-monitoring . 1 Introduction We are moving toward a world of multimodal AI agents that must build and update world models from many sources of information and interaction ( 14 ; 25 ; 6 ; 22 ) . In order for humans to interact effectively with such systems, AI models need to be able to track and communicate where (i.e., from which modality) information comes from. Borrowing terminology from cognitive psychology ( 11 ; 8 ) , we refer to this ability as source-modality monitoring . We consider source-modality monitoring as an instance of the more general binding problem ( 20 ) : a model must associate particular content with an abstract role, such as image or text , and retrieve the content corresponding to the queried role. Recent work has shown that language models solve related binding problems through abstract, symbolic-like mechanisms, for example by associating entities with their attributes ( 5 ; 7 ) or binding values to variables ( 24 ) . Given this, we might expect modern multimodal models to invoke a similar mechanism for binding inputs to modalities, especially since their inputs are often explicitly marked with modality-specific wrapper tokens such as <image> and </image> . However, viewing source-modality monitoring as a case of purely symbolic binding misses many of the key properties. Namely, unlike arbitrary role labels, words such as image and text are grounded in content with distinct distributional and semantic structure. Some kinds of inputs just are images, regardless of whether or not they are tagged as such. As a result, when a multimodal model is asked to report “what is in the image,” it need not rely only on explicit modality markers; it should also be able to infer from the nature of the input itself whether or not a piece of information should bind to a word like image . Moreover, as AI becomes increasingly multimodal and agentic, it becomes unrealistic to assume that all the “modalities” to which a human might refer will be explicitly tagged. Humans might ask AI for information from “yesterday’s meeting”, or “the discussion about dinner plans”. In such cases, AI will need to rely on contextual and distributional information about the content being bound, instead or in addition to information available in explicit syntactic markers. In this paper, we ask if and how modern vision-language models (VLMs) accomplish source-modality monitoring. Source-modality monitoring can be studied at multiple levels, including mechanisms acquired over training ( 12 ) and mechanisms deployed in context at inference time. We focus on the latter, and investigate how it is accomplished in a variety of VLMs across model sizes and preprocessing choices. We first show that many of these models perform well on a task designed to test in-context source-modality monitoring, and that models with explicit marker tokens (e.g., <image> & </image> ) generally outperform those without them. We then ask what kind of signal supports this performance. By structurally perturbing symbolic modality markers, we find that models’ binding behavior is driven primarily by distributional semantic signals rather than purely symbolic ones, although explicit markers still improve selectivity. Finally, by applying optimized transformations to marker and content representations, we show that symbolic cues can nonetheless be exploited to reverse models’ modality attribution. Together, these results provide a more nuanced picture of how syntactic and semantic signals combine to support binding in complex multimodal settings. 2 How Well Do VLMs Bind Words to Their Inputs? 2.1 Target-Modality Retrieval Task Figure 1: Example task instance. We first ask simply whether VLMs are capable of associating different parts of their input (i.e., images, text) with the words that refer to those inputs ( image , caption )? We define the following task whose successful performance depends on a model’s ability to monitor source modality. Each instance consists of a single in-context inconsistent image-caption pair, and the model is prompted to retrieve information from one specified input source (e.g., “what is in the image” or “what is in the caption”). We refer to the queried source as the target modality and the other source as the non-target modality . Figure 1 shows an example, illustrating the correct answers when a model is asked to retrieve image or caption information from its input pair. We exclude the possibility that VLMs rely on input order to retrieve information by evaluating them under both input orderings -- image tokens first or text tokens first -- and report the average result across the two conditions. 2 2 2 We adapt this approach to all models except InstructBLIP, which has a fixed order of having image tokens at the beginning of the input sequence. Because the visual and textual inputs convey mutually exclusive information, answering correctly requires the model to retrieve content from the specified target modality. As the target modality is explicitly indicated, this task does not merely measure a model’s overall modality bias, but instead targets at the model’s ability to associate retrieved information with its input modality thus making it suitable for assessing source-modality monitoring. 2.2 Datasets and Models To construct inconsistent image–caption pairs, we leverage existing image captioning datasets flickr30k ( 26 ) and MSCOCO ( 15 ) , and systematically sample captions that are inconsistent with the visual content of each image. We use a simple heuristic procedure to ensure that the caption paired with the image is genuinely mismatched at the semantic level. To do this, we require that the sampled caption have cosine similarity below a threshold of 0.2 with any of the five image’s ground-truth associated captions, where similarity is computed using a BERT-style sentence embedding model ( sentence-transformers/all-mpnet-base-v2 ( 21 ) ). We evaluate a range of 11 open-source VLMs on this task. Beyond differences in model size, these systems also vary along an architectural dimension that is especially relevant to our setting: whether the input preprocessing pipeline inserts explicit image wrapper tokens that mark the span of image tokens. Qwen-2.5-VL, InternVL3, and Gemma-3 models ( 1 ; 28 ; 23 ) are trained with these wrapper tokens, while earlier models such as LLaVA-1.5 ( 16 ) , InstructBLIP ( 2 ) , and LLaVA-OneVision ( 13 ) do not use such markers. See Table 3 for the image marker tokens used across model families. This