Multi-Scale Contrastive Learning for Video Temporal Grounding

Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu · 2024

Read Multi-Scale Contrastive Learning for Video Temporal Grounding by Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07157.txt.

Description

Thong Thanh Nguyen and co-authors improve video search by description, in a paper published in 2024. The task, temporal grounding, finds the clip segment that matches a spoken or written query. Systems encode clips at several scales, short ones in fine detail and long ones squeezed down, which dulls the long segments' content.

Their fix trains the encoder by contrast: segments that match the same query are pulled together in feature space, across all scales. They draw several candidate segments per query from the encoder's own stages, so no extra data or memory banks are needed. Short and long views of the same moment thus teach each other. Tests on both long and short video grounding improved. The code is public for other teams to build on.

Catalog record

CREATORThong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07157

Browse the full library · About the Tower of Babel