VideoBrain: Learning Adaptive Frame Sampling for Long Video Understanding
Junbo Zou, Ziheng Huang, Shengjie Zhang, Liwen Zhang, Weining Shen · 2026
Read VideoBrain: Learning Adaptive Frame Sampling for Long Video Understanding by Junbo Zou, Ziheng Huang, Shengjie Zhang, Liwen Zhang, Weining Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-04094.txt.
Description
Long videos overflow a vision model's memory, so most systems sample frames blindly. Junbo Zou and 4 co-authors built VideoBrain, which learns where to look. Their 2026 framework sends two agents over the video: one retrieves frames by semantic match, the other samples densely inside promising intervals.
The vision-language model itself judges whether it has seen enough, rather than obeying a text-only controller. A behavior-aware reward stops the model from calling agents wastefully. Training data teaches when each agent helps. Adaptive sampling with recovery from bad choices beats both uniform sampling and one-pass keyframe selection on long-form understanding.
Catalog record
| CREATOR | Junbo Zou, Ziheng Huang, Shengjie Zhang, Liwen Zhang, Weining Shen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-04094