How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos Conference: Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26); June 5–9, 2026; Montreal, Canada Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26), June 5–9, 2026, Montreal, Canada CCS: Software and its engineering Software defect analysis
Wentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer · 2026
Read How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos Conference: Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26); June 5–9, 2026; Montreal, Canada Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26), June 5–9, 2026, Montreal, Canada CCS: Software and its engineering Software defect analysis by Wentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22706.txt.
Description
Lu and co-authors test how well vision-language models find visual bugs in gameplay video. They draw keyframes from 100 industrial QA videos: 41 hours and 19,738 frames in all. A single-prompt model reaches 0.50 precision and 0.72 accuracy. Two add-ons are then tried: a second judge model, and prompts enriched with past bug reports. Both give only small gains. They also add cost and output variance. The authors suggest future gains need hybrid methods that split text and visual anomaly checks.
Catalog record
| CREATOR | Wentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25716 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.22706