SF20K Competition 2025: Summary and findings
Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev · 2026
Read SF20K Competition 2025: Summary and findings by Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01496.txt.
Description
Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, and Ivan Laptev report the first Short-Films 20K competition, a contest testing whether AI can follow a whole story on film. Short-clip action spotting is too easy a proxy, so the task is open question answering over amateur short films, which models cannot have memorized.
The test set holds 95 films and nearly a thousand question-answer pairs, graded by an automated judge. Twenty-two teams sent 286 entries across a free track and a small-model track for systems under eight billion parameters. Winners reached about two thirds accuracy on the free track and about half on the small track, far below the human mark above ninety percent. Shot-aware processing beat plain frame sampling. Small staged pipelines matched giant single models, and subtitle quality swung scores hard. The bottleneck is picking and reasoning over the right moments, not raw size.
Catalog record
| CREATOR | Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01496