OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models

Monika Wysoczańska Shyamal Buch Anurag Arnab Cordelia Schmid · 2025

Read OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models by Monika Wysoczańska Shyamal Buch Anurag Arnab Cordelia Schmid (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19262.txt.

Description

Monika Wysoczańska, Shyamal Buch, Anurag Arnab, and Cordelia Schmid measure truthfulness in long image captions in a 2025 paper. Vision-language models can write rich, long captions, but they often invent details. Old factuality scores were built for short captions and need human-written references, which many settings lack. The team built a reference-free check instead.

Their method, OVFact, grounds caption claims in the image with open-vocabulary tools and verifies them piece by piece. It scores both how much of the scene the caption covers and how much of it is right, in one number. Agreement with human judges improved over older scores. The team then used the check to filter a huge noisy training set. Models trained on the smaller, cleaner subset wrote more factual captions without losing richness.

Catalog record

CREATORMonika Wysoczańska Shyamal Buch Anurag Arnab Cordelia Schmid
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-19262

Browse the full library · About the Tower of Babel