Prompt the Unseen: Evaluating Visual-Language Alignment Beyond Supervision

Raehyuk Jung, Seungjun Yu, Hyunjung Shim · 2025

Read Prompt the Unseen: Evaluating Visual-Language Alignment Beyond Supervision by Raehyuk Jung, Seungjun Yu, Hyunjung Shim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00700.txt.

Introduction

Raehyuk Jung, with two co-authors, Seungjun Yu and Hyunjung Shim, evaluates whether vision-language models can handle visual concepts they were never aligned on. Such models join a vision encoder with a large language model through a projection layer, which maps image features into the language model's space.

Jung and his co-authors adapt object detection datasets into a prompting format and split labels into disjoint seen and unseen sets. The projection layer keeps about seventy-nine to eighty-eight percent of seen-class performance on unseen classes. A mechanistic analysis suggests its feed-forward network acts like a key-value memory, processing seen and unseen tokens alike. The benchmark offers a new way to test alignment generalization and train with limited aligned data.

Catalog record

CREATORRaehyuk Jung, Seungjun Yu, Hyunjung Shim
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-00700

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Prompt the Unseen: Evaluating Visual-Language Alignment Beyond Supervision Raehyuk Jung, Seungjun Yu, Hyunjung Shim Abstract Vision-Language Models (VLMs) combine a vision encoder and a large language model (LLM) through alignment training, showing strong performance on multimodal tasks. A central component in this architecture is the projection layer, which maps visual features into the LLM’s embedding space. Despite its importance, its ability to generalize to unseen visual concepts has not been systematically evaluated. To address this, we propose a benchmark for evaluating projection layer generalization. We adapt object detection datasets—rich in fine-grained annotations—into a prompting format, and design train/test splits with disjoint label sets. This setup enables precise control over seen and unseen concept separation. Experimental results show that the projection layer retains over 79-88% of the performance on unseen classes compared to seen ones, across various settings. This suggests a non-trivial level of generalization, even without explicit alignment supervision on those concepts. We further analyze this behavior through a mechanistic interpretability lens. Our findings suggest that the feed-forward network in the projection layer operates like a key-value memory, processing seen and unseen tokens in similar ways. This study introduces a new evaluation framework for alignment generalization and highlights the potential for efficient VLM training with limited aligned data. 1 1 footnotetext: Code available at: https://github.com/raemoi93/PromptTheUnseen 1 Introduction The rise of large language models (LLMs) has transformed natural language processing, enabling strong generalization across a wide range of linguistic tasks. As these models expand to support visual inputs, Vision-Language Models (VLMs) are becoming key to building general-purpose multimodal systems. From image captioning to visual assistants, VLMs now drive many real-world applications, with models such as GPT-4V and Gemini leading this transition. At the core of modern VLM design lies a simple yet powerful idea: instead of training large multimodal architectures from scratch, one can connect a pretrained vision encoder and a pretrained LLM using a lightweight projection layer. This approach offers two major advantages—computational efficiency and modular flexibility. It allows the system to inherit the generalization capabilities of both components while requiring only modest alignment training to bridge the modalities. This paradigm has been widely adopted in recent models such as Qwen-VL ( Bai et al. 2025 ) and InternVL-2.5 ( Chen et al. 2024 ) . Figure 1: This figure illustrates our overall research question: Can the projection layer effectively transfer visual information of unseen concepts? At the top, the VLM is shown as consisting of three modules: the vision encoder, the projection layer, and the LLM. The vision encoder and LLM are frozen, while only the projection layer is trainable. In the middle, we provide simplified, illustrative training examples (e.g., grape, car, mug) showing how the projection layer learns to map fixed visual embeddings into tokens that the frozen LLM can recognize. At the bottom, we depict hypothetical test examples (sunglasses, keyboard), demonstrating the model’s ability to recognize novel objects, even though the projection layer was never trained on them. These examples are schematic and meant to aid intuition, rather than drawn from the actual datasets used in our experiments. However, this modular approach introduces a critical dependency: the projection layer becomes the sole pathway through which visual information reaches the language model. Despite its functional importance, the projection layer is orders of magnitude smaller in both training data and parameter count. For example, in LLaVA ( Liu et al. 2023 ) , the vision encoder (CLIP-Large ( Radford et al. 2021 ) ) is trained on 400 million image-text pairs, and the LLM (Vicuna-13B ( Chiang et al. 2023 ) ) is trained on over two trillion tokens. The projection layer connecting the two, by contrast, is trained on only 750,000 alignment examples and consists of just 4.2 million parameters. This sharp asymmetry raises an important but underexplored question: Can such a small module generalize to visual concepts it has never seen during alignment training? As illustrated in figure 1 , we present this research question with a visual aid that clarifies our setting and motivation. This question is critical in scalable and efficient VLM design. If the projection layer fails to generalize, one must rely on increasingly large and diverse alignment datasets, which would erode the efficiency and modularity benefits of current VLM architectures. Although projection-based models report high overall performance as reported in various studies ( Bai et al. 2025 ; Li et al. 2024b ) , they are typically trained and evaluated on datasets that lack structured annotations distinguishing seen from unseen visual concepts, making it difficult to draw conclusions about generalization behavior. To properly assess this generalizibility in unseen categories, a carefully curated benchmark is necessary where a clear separation between seen and unseen visual concepts is guaranteed, providing controlled, prompt-based evaluation. However, no such dataset currently exists. Large-scale multimodal datasets such as LAION ( Schuhmann et al. 2022 ) offer scale, but lack structural annotation to isolate concept-level generalization to construct a suituable test set with the desired properties. Figure 2: This figure illustrates how an object detection annotation is converted into a VLM-style prompt. Each annotation consists of an image, bounding box coordinates, and a class label. We convert these into a prompt where the image, bounding box, and class label appear sequentially. The model is trained to predict the class label conditioned on the image and bounding box, with the language modeling loss applied only to the class label. For clarity, we show a hypothetical bounding box and object in the exemplar image, but such boxes are not explicitly drawn during training or evaluation. In this work, we propose a new evaluation framework to address this gap. Our key idea is to leverage object detection datasets—particularly Visual Genome ( Krishna et al. 2017 ) , which provides the largest number of fine-grained unique class labels among commonly used datasets and includes explicit bounding boxes and class labels. Unlike classification datasets, detection datasets allow multiple labeled regions per image and mitigate label ambiguity often found in image-level annotations ( Yun et al. 2021 ) . We partition the label space into disjoint seen and unseen sets, and convert bounding box annotations into prompt-style training samples to construct a vision-language alignment dataset with fine-grained control over visual concepts. Figure 2 illustrates an exemplar annotation and its conversion into a VLM-style prompt. We then train only the projection layer while keeping the vision encoder and LLM frozen, ensuring that any observed generalization arises solely from the alignment process. To evaluate the resulting models, we first design a multiple-choice QA benchmark with separate splits for seen and unseen labels. We design a series of experiments to rigorously evaluate the model’s generalization capability across diverse settings, including challenging scenarios where the language model is as small as 0.6B. Even under limited capacity, the model achieves over 79%—and up to 88%—of its seen-class accuracy on the unseen split. These results demonstrate that even with limited supervision, the projection layer learns a effective general mapping between vision and language. Then, we perform dataset ablation to further investigate which aspects of the dataset contribute to enhancing generalization. To understand how this generalization is possible, we further conduct a mechanistic interpretability analysis of the transformer layers of the frozen LLM.

Read the complete text