Unified Spatio-Temporal Token Scoring for Efficient Video VLMs

2026

Read Unified Spatio-Temporal Token Scoring for Efficient Video VLMs (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18004.txt.

Description

Zhang and co-authors introduce STTS, a module that prunes vision tokens across both the vision transformer and the language model. It learns time scores from an added loss and space scores from downstream gradients. No text conditioning or token merging is needed, and training stays end to end. STTS cuts half the vision tokens and speeds training and inference by 62%. Mean scores drop just 0.7% across thirteen video question tasks.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46962
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.18004

Browse the full library · About the Tower of Babel