Adversarial Video Promotion Against Text-to-Video Retrieval
Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Shuai Liu, Qian Li, Chao Shen · 2025
Read Adversarial Video Promotion Against Text-to-Video Retrieval by Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Shuai Liu, Qian Li, Chao Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-06964.txt.
Introduction
Qiwei Tian and five co-authors attack video search engines by boosting a video's rank instead of burying it, in a 2025 paper. Text-to-video retrieval matches typed queries to clips. Earlier attacks pushed target videos down the results, a nuisance; pushing a video up, toward chosen queries, pays attackers in views and spreads whatever the clip promotes.
Their attack perturbs a video subtly so retrieval models rank it for many target queries at once. A refinement step models the fine interplay between the video's frames and the query text, helping the attack transfer to models it never saw. Tests spanned three leading systems and datasets of over ten thousand videos, plus defense and visibility checks. The attack beat existing baselines, exposing a profit-shaped hole in retrieval security.
Catalog record
| CREATOR | Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Shuai Liu, Qian Li, Chao Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-06964
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adversarial Video Promotion Against Text-to-Video Retrieval Qiwei Tian, Chenhao Lin, Zhengyu Zhao, Shuai Liu, Qian Li, Chao Shen Abstract Thanks to the development of cross-modal models, text-to-video retrieval (T2VR) is advancing rapidly, but its robustness remains largely unexamined. Existing attacks against T2VR are designed to push videos away from queries, i.e., suppressing the ranks of videos, while the attacks that pull videos towards selected queries, i.e., promoting the ranks of videos, remain largely unexplored. These attacks can be more impactful as attackers may gain more views/clicks for financial benefits and widespread (mis)information. To this end, we pioneer the first attack against T2VR to promote videos adversarially, dubbed the Video Promotion attack (ViPro). We further propose Modal Refinement (MoRe) to capture the finer-grained, intricate interaction between visual and textual modalities to enhance black-box transferability. Comprehensive experiments cover 2 existing baselines, 3 leading T2VR models, 3 prevailing datasets with over 10k videos, evaluated under 3 scenarios. All experiments are conducted in a multi-target setting to reflect realistic scenarios where attackers seek to promote the video regarding multiple queries simultaneously. We also evaluated our attacks for defences and imperceptibility. Overall, ViPro surpasses other baselines by over 30 / 10 / 4 % 30/10/4\% for white/grey/black-box settings on average. Our work highlights an overlooked vulnerability, provides a qualitative analysis on the upper/lower bound of our attacks, and offers insights into potential counterplays. Our manually trained weights are available at https://github.com/michaeltian108/ViPro . We will release full code after acceptance. 1 Introduction Text-to-video retrieval (T2VR) has thrived in recent years, driven by advances in vision–language models (VLM) Gao et al. 2021 ; Radford et al. 2021 ; Xue et al. 2022 ; Li et al. 2022 , making it a vital tool for users to search, discover, and consume video content across. While many T2VR models Jin et al. 2023 ; Lei et al. 2023 ; Wang et al. 2022 ; Wu et al. 2023 have achieved strong performance on prevailing datasets such as MSR-VTT Xu et al. 2016 , DiDeMo Hendricks et al. 2017 , and ActivityNet Yu et al. 2019 , their robustness to adversarial attacks remains largely unexplored. For example, as the only existing attack against T2VR, Yang et al. 2024 proposed an adversarial attack against T2VR models to suppress video ranks by pushing videos away from queries and vice versa under black-box and white-box settings. However, attacks that promote videos to top positions can be more hazardous, as they allow attackers to gain more views/clicks for financial benefits and widespread (mis)information. Additionally, such manipulation can trigger a snowball effect: once the manipulated videos gain initial popularity through adversarial promotion, the platform’s recommendation/retrieval algorithm may further amplify their exposure. Recognizing this overlooked threat, we investigate this vulnerability to video promotion attacks as a critical step towards building robust and trustworthy T2VR systems. Figure 1 : A simplified illustration for video suppression (left) and video promotion (right). The circle represents the retrieval boundary in which videos can be retrieved by the query (text). The yellow shape indicates the targeted area in which the perturbed videos must reach for successful attacks. Video promotion is more challenging, as it seeks an intersection of the retrieval boundaries for all target queries, while video suppression only pushes videos outside the retrieval boundary. To this end, we propose the first attack against T2VR to promote videos adversarially for the selected queries, dubbed the Video Promotion attack (ViPro). Furthermore, to reflect realistic user scenarios, we restrict the attack to candidate videos only, without modifying the text queries, as attackers cannot interfere with queries from other users. In Figure. 1 , we provide a hypothesized illustration to highlight the difference between existing attacks (video suppression) and ours (video promotion) intuitively. In the left figure, video suppression attacks succeed if the perturbation \delta pushes the video \mathbf{X} out of the retrieval boundary of q_{1} (depicted as the blue circle), which is determined by the retrieval list length and the videos scattered around the query. Thus, the desired perturbation can be obtained by simply maximizing the distance between the video \mathbf{X} and the query q_{1} , as illustrated by the red arrows ( \delta_{1},\delta_{2},\delta_{3} ). In contrast, video promotion attacks (right figure) aim to push \mathbf{X} into the small overlapping area (yellow shape) of the retrieval boundaries for all target queries (e.g., q_{1},q_{2},q_{3} ). Thus, finding the optimal \hat{\delta} requires pre-knowledge of model embeddings and the distribution of target queries. To overcome the above challenges for better transferability, we propose Mo dality Re finement (MoRe) to guide perturbation towards target areas through finer-grained optimization. MoRe first conducts Temporal Clipping to group temporally similar frames into clips, and then applies Semantic Weighting to resolve potential gradient conflicts for intra-modality and inter-modality through frame-to-frame and frame-to-query similarities, respectively. Consequently, MoRe provides focused guidance to push the video towards the targeted queries for better transferability. We validate the effectiveness of MoRe through ablation studies in Section. 4.3 . Overall, we conduct experiments on 3 open-sourced leading models for MSR-VTT-1k Xu et al. 2016 , DRL Wang et al. 2022 , Cap4Video Wu et al. 2023 , and Singularity Lei et al. 2023 (ranked #11/#17/#30 1 1 1 Using the leaderboard from paperwithcode.com . ). We define the white-box, grey-box, and black-box scenarios for ViPro and evaluate its effectiveness and transferability across all three models. For dataset-wise evaluations, we test on three prevailing datasets using the Singularity model—ActivityNet Yu et al. 2019 , DiDeMo Hendricks et al. 2017 , and MSR-VTT. As for baselines, we adopt Co-Attack Zhang et al. 2022a and SGA Lu et al. 2023 from the image-to-text domain for comparison, due to the lack of reproducible T2VR-specific attacks. Lastly, we further assess robustness under defences using JPEG compression Dziugaite et al. 2016 (image-based) and Temporal Shuffling Hwang et al. 2024 (video-based). A user study with 17 experts on 43 groups of videos is also conducted for human imperceptibility. Across all baselines across all datasets, models, and settings, ViPro consistently outperforms other baselines. Finally, we validate our theoretical motivation through thorough experiments and provide detailed analysis in Section 4.5 . In sum, ViPro exhibits superior effectiveness, generalizability, and transferability over existing baselines, achieving an averaged 30/10/4\% lead for white/grey/black-box settings. Our main contributions are listed as follows: • We introduce a realistic and impactful attacking paradigm for text-to-video retrieval (T2VR) models where attackers promote video ranks for selected queries adversarially. We thus pioneer the Video Promotion Attack (ViPro) as the first attack targeting such vulnerability. • We propose Modality Refinement (MoRe) to group temporally similar frames into clips and apply semantical weighting using frame-to-frame and frame-to-query similarities for enhanced black-box transferability. • We conduct thorough experiments and validate the superiority of ViPro over baselines and provide a qualitative analysis of the upper and lower bounds for all attacks. 2 Related Work Vision-Language Models. Vision-Language models (VLMs), as suggested by the name, require training in both textual and visual encoders. Textual encoders have witnessed the advancement in natural language processing, which progressed from Word2Vec Mikolov et al. 2013 to Bert-base Kenton and Toutanova 2019 ; Liu et al. 2019c and GPT-based Radford et al. 2018 encoders. Besides, visual encoders Dosovitskiy et al. 2020 ; Anderson et