SF20K Competition 2025: Summary and findings

Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev · 2026

Read SF20K Competition 2025: Summary and findings by Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01496.txt.

Description

Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, and Ivan Laptev report the first Short-Films 20K competition, a contest testing whether AI can follow a whole story on film. Short-clip action spotting is too easy a proxy, so the task is open question answering over amateur short films, which models cannot have memorized.

The test set holds 95 films and nearly a thousand question-answer pairs, graded by an automated judge. Twenty-two teams sent 286 entries across a free track and a small-model track for systems under eight billion parameters. Winners reached about two thirds accuracy on the free track and about half on the small track, far below the human mark above ninety percent. Shot-aware processing beat plain frame sampling. Small staged pipelines matched giant single models, and subtitle quality swung scores hard. The bottleneck is picking and reasoning over the right moments, not raw size.

Catalog record

CREATORRidouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01496

Browse the full library · About the Tower of Babel