The Power of One: A Single Example is All it Takes for Segmentation in VLMs

Mir Rayat Imtiaz Hossain, Mennatullah Siam, Leonid Sigal, James J. Little · 2025

Read The Power of One: A Single Example is All it Takes for Segmentation in VLMs by Mir Rayat Imtiaz Hossain, Mennatullah Siam, Leonid Sigal, James J. Little (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10779.txt.

Description

Mir Rayat Imtiaz Hossain and co-authors show in a 2025 paper that one visual example can greatly improve image segmentation by vision language models. Such models can find objects from text alone, but results hinge on prompt wording and hand picked internal layers. The authors replace much of that guesswork with a single example image for each class.

Fine tuning the text to image attention with that one example lifted scores a lot. A ranking step based on entropy picks the best layers on its own, with no need for labeled masks. Grouping tuned layers or prompts helped further. The method worked across several model families, which makes it a flexible tool for open ended image tasks.

Catalog record

CREATORMir Rayat Imtiaz Hossain, Mennatullah Siam, Leonid Sigal, James J. Little
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10779

Browse the full library · About the Tower of Babel