Your VLM is not good at counting pushups
Shengzhi Li, Jiarun Chen, Karun Sharma, Jiaqi Su, Shichao Pei · 2026
Read Your VLM is not good at counting pushups by Shengzhi Li, Jiarun Chen, Karun Sharma, Jiaqi Su, Shichao Pei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23407.txt.
Description
Li and four colleagues introduce PushupBench, 446 long-form clips averaging about 37 seconds for evaluating repetition counting in video. Vision-language models recognise what happens but fail to count how many times: the best frontier model reaches 42.1 percent exact accuracy, while open 4B models score about 6 percent, matching supervised baselines. Accuracy alone misleads, since weaker models lean on the modal count instead of reasoning over time. Fine-tuning on counting with 1,000 samples transfers to general video understanding, lifting three benchmarks, suggesting counting proxies broader temporal reasoning.
Catalog record
| CREATOR | Shengzhi Li, Jiarun Chen, Karun Sharma, Jiaqi Su, Shichao Pei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35777 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23407