SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosis

Rishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship · 2026

Read SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosis by Rishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00249.txt.

Description

Rishav Pramanik and coauthors present SANEval, an open vocabulary benchmark for compositional text to image generation. Current models can be creative, yet they often fail prompts with several objects, attributes, and spatial relations. SANEval joins a large language model for deep prompt understanding to an LLM enhanced, open vocabulary object detector, checking compositional adherence without a fixed word list.

Catalog record

CREATORRishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52853
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00249

Browse the full library · About the Tower of Babel