ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation

Cihang Peng, Qiming Hou, Zhong Ren, Kun Zhou · 2025

Read ROVI: A VLM-LLM Re-Captioned Dataset for Open-Vocabulary Instance-Grounded Text-to-Image Generation by Cihang Peng, Qiming Hou, Zhong Ren, Kun Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01008.txt.

Description

Cihang Peng and co-authors present ROVI, a dataset for text to image generation with labeled objects, in this 2025 paper. The dataset holds one million curated web images with instance level labels. Making such labels at scale is the hard part. Their trick is to re-caption each image before detection begins.

A vision language model first writes a rich description of each image. A language model then extracts a flat list of candidate categories from that text. An open vocabulary detector finds those objects in the image. The prompts stay tied to the labels and catch details humans often miss. A model trained on ROVI beat rivals on grounding accuracy and image quality. The dataset and pipeline were released publicly.

Catalog record

CREATORCihang Peng, Qiming Hou, Zhong Ren, Kun Zhou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01008

Browse the full library · About the Tower of Babel