How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos Conference: Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26); June 5–9, 2026; Montreal, Canada Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26), June 5–9, 2026, Montreal, Canada CCS: Software and its engineering Software defect analysis

Wentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer · 2026

Read How Far Can VLMs Go for Visual Bug Detection? Studying 19,738 Keyframes from 41 Hours of Gameplay Videos Conference: Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26); June 5–9, 2026; Montreal, Canada Companion Proceedings of the 34th ACM Symposium on the Foundations of Software Engineering (FSE ’26), June 5–9, 2026, Montreal, Canada CCS: Software and its engineering Software defect analysis by Wentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22706.txt.

Description

Lu and co-authors test how well vision-language models find visual bugs in gameplay video. They draw keyframes from 100 industrial QA videos: 41 hours and 19,738 frames in all. A single-prompt model reaches 0.50 precision and 0.72 accuracy. Two add-ons are then tried: a second judge model, and prompts enriched with past bug reports. Both give only small gains. They also add cost and output variance. The authors suggest future gains need hybrid methods that split text and visual anomaly checks.

Catalog record

CREATORWentao Lu, Alexander Senchenko, Alan Sayle, Abram Hindle, Cor-Paul Bezemer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25716
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22706

Browse the full library · About the Tower of Babel