Multi-Scale Contrastive Learning for Video Temporal Grounding
Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu · 2024
Read Multi-Scale Contrastive Learning for Video Temporal Grounding by Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07157.txt.
Description
Thong Thanh Nguyen and co-authors improve video search by description, in a paper published in 2024. The task, temporal grounding, finds the clip segment that matches a spoken or written query. Systems encode clips at several scales, short ones in fine detail and long ones squeezed down, which dulls the long segments' content.
Their fix trains the encoder by contrast: segments that match the same query are pulled together in feature space, across all scales. They draw several candidate segments per query from the encoder's own stages, so no extra data or memory banks are needed. Short and long views of the same moment thus teach each other. Tests on both long and short video grounding improved. The code is public for other teams to build on.
Catalog record
| CREATOR | Thong Thanh Nguyen, Yi Bin, Xiaobao Wu, Zhiyuan Hu, Cong-Duy Nguyen, See-Kiong Ng, Anh Tuan Luu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-07157