Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction

Sunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang 🖂 · 2026

Read Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction by Sunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang 🖂 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29445.txt.

Description

Fan and four colleagues connect two skills: answering questions about videos, and learning procedures from video tutorials well enough to act on them. Existing benchmarks mostly test whether multimodal large language models spot shallow visual cues. They rarely check whether a model can follow a tutorial and finish a long graphical interface task.

The team releases VG-GUI-Bench, a benchmark for video-guided GUI agents, where GUI means the on-screen interface of buttons and menus. They also propose TASKER, a keyframe extraction algorithm that picks informative frames using both task relevance and scene change. Good keyframes, the few frames that carry the content, matter for both question answering and agent tasks. TASKER beat the best baseline on the EgoSchema and NExT-QA datasets. The authors argue that generalized keyframe extraction is a shared bottleneck for video understanding, and they release their code and data openly.

Catalog record

CREATORSunqi Fan, Qingle Liu, Runqi Yin, Meng-Hao Guo, Shuojin Yang 🖂
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29445

Browse the full library · About the Tower of Babel