Chain-of-caption: training-free improvement of multimodal large language model on referring expression comprehension

Yik Lung Pang, Changjae Oh · 2026

Read Chain-of-caption: training-free improvement of multimodal large language model on referring expression comprehension by Yik Lung Pang, Changjae Oh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08211.txt.

Description

Referring expression comprehension asks a model to locate the object a text description points to in an image. Yik Lung Pang and Changjae Oh study how extra visual and textual context, supplied through tools, affects multimodal language models on this task, then propose a training-free framework called Chain-of-Caption.

Individual textual or visual contexts already improve performance without fine-tuning, and combining multiple contexts yields gains between 5 and 30 percent over baselines on RefCOCO, RefCOCOg, RefCOCO+, and Ref-L4 benchmarks. The approach shows that careful context provision can rival heavier training for grounding language in images. Vision-language researchers gain a cheap, training-free recipe for sharper object localization.

Catalog record

CREATORYik Lung Pang, Changjae Oh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08211

Browse the full library · About the Tower of Babel