SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosis
Rishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship · 2026
Read SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosis by Rishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00249.txt.
Description
Rishav Pramanik and coauthors present SANEval, an open vocabulary benchmark for compositional text to image generation. Current models can be creative, yet they often fail prompts with several objects, attributes, and spatial relations. SANEval joins a large language model for deep prompt understanding to an LLM enhanced, open vocabulary object detector, checking compositional adherence without a fixed word list.
Catalog record
| CREATOR | Rishav Pramanik, Ian E. Nielsen, Jeff Smith, Saurav Pandit, Ravi P. Ramachandran, Zhaozheng Yin, [1em] Stony Brook University, NY, USA, [0.5em] Equal contribution Work done during internship |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52853 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.00249