SLVMEval: Synthetic Meta Evaluation Benchmark for Text-to-Long Video Generation

Ryosuke Matsuda, Keito Kudo, Haruto Yoshida, Nobuyuki Shimizu, Jun Suzuki · 2026

Read SLVMEval: Synthetic Meta Evaluation Benchmark for Text-to-Long Video Generation by Ryosuke Matsuda, Keito Kudo, Haruto Yoshida, Nobuyuki Shimizu, Jun Suzuki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29186.txt.

Description

Matsuda and co-authors propose SLVMEval, a benchmark for judging text-to-video evaluators. The benchmark checks evaluators on videos up to about three hours long. The team degrades source videos in controlled ways across ten quality aspects. That creates clear high-versus-low quality pairs. Crowd workers kept only pairs where the drop was easy to see. Humans picked the better video with 84.7 to 96.8 percent accuracy. Evaluator systems fell short of humans on nine of ten aspects.

Catalog record

CREATORRyosuke Matsuda, Keito Kudo, Haruto Yoshida, Nobuyuki Shimizu, Jun Suzuki
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84551
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.29186

Browse the full library · About the Tower of Babel