MobileViCLIP: An Efficient Video-Text Model for Mobile Devices

Min Yang, Zihan Jia, Zhilin Dai, Sheng Guo, Limin Wang · 2025

Read MobileViCLIP: An Efficient Video-Text Model for Mobile Devices by Min Yang, Zihan Jia, Zhilin Dai, Sheng Guo, Limin Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-07312.txt.

Introduction

Min Yang and co-authors build a video-text model that runs well on phones in this 2025 paper. Light models get much attention for their speed, yet most video models still use a heavy design with high delay. Few teams have built a video model that truly fits mobile devices.

Their model, called MobileViCLIP, adds a time-aware step to a light image-text model and trains it on a large, high-grade video-text set. The small version runs far faster on phones than two larger rival models. On a search test where a text query must find the right video, its scores matched one large rival and beat a smaller one. Code is shared in public.

Catalog record

CREATORMin Yang, Zihan Jia, Zhilin Dai, Sheng Guo, Limin Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-07312

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MobileViCLIP: An Efficient Video-Text Model for Mobile Devices Min Yang, Zihan Jia, Zhilin Dai, Sheng Guo, Limin Wang Abstract Efficient lightweight neural networks are with increasing attention due to their faster reasoning speed and easier deployment on mobile devices. However, existing video pre-trained models still focus on the common ViT architecture with high latency, and few works attempt to build efficient architecture on mobile devices. This paper bridges this gap by introducing temporal structural reparameterization into an efficient image-text model and training it on a large-scale high-quality video-text dataset, resulting in an efficient video-text model that can run on mobile devices with strong zero-shot classification and retrieval capabilities, termed as MobileViCLIP . In particular, in terms of inference speed on mobile devices, our MobileViCLIP-Small is 55.4 × \times times faster than InternVideo2-L14 and 6.7 × \times faster than InternVideo2-S14. In terms of zero-shot retrieval performance, our MobileViCLIP-Small obtains similar performance as InternVideo2-L14 and obtains 6.9% better than InternVideo2-S14 on MSR-VTT. The code is available at https://github.com/MCG-NJU/MobileViCLIP . 0 0 footnotetext: ✉ Corresponding author. 1 Introduction Large vision-text models [ 9 , 72 , 27 , 51 ] have demonstrated excellent zero-shot performance in a wide range of downstream tasks. Learning transferable video-text representations is becoming a fundamental task in video understanding. With the advent of high-quality video-language datasets [ 2 , 64 , 42 , 8 ] for large-scale pre-training, several powerful video-text foundation models [ 63 , 65 , 28 ] have emerged. They are typically trained from scratch and have achieved excellent results on various downstream tasks including video-text retrieval and action recognition. However, to ensure performance, most of these works use ViT-L [ 15 ] or larger backbones. The model size limits its possibility of deployment on mobile devices with limited storage capability. Also, excessive computing resource requirements (tens of A100 GPUs are required), too long training time (thousands of hours of GPU training time), and a large amount of pre-training datasets (ensemble of datasets larger than 100M) increase the training cost and make these works difficult to reproduce for a university lab. Figure 1 : Comparison of latency and the R@1 scores for zero-shot text-to-video retrieval on video retrieval dataset MSR-VTT [ 69 ] . The latency is measured on iPad Air 2020 with iOS 18. MobileViCLIP achieves the best trade-off of recall and latency. Parameter-efficient transfer learning (PETL) [ 30 , 44 , 71 , 33 ] is becoming a promising and efficient paradigm for adapting large image-text models for video understanding due to their huge numbers of parameters and the high computational cost of full fine-tuning. These methods have successfully transferred image representations to video recognition tasks and obtained good zero-shot recognition results on several action classification datasets [ 5 , 50 , 24 , 19 ] . However, these transfer learning methods often focus on the simple video classification task and fail to directly investigate the visual-textual representations on more complex video-text tasks. We argue that video-text models are expected to be more flexible and can handle a more diverse of downstream video understanding tasks. Furthermore, despite the extremely small number of trainable parameters in these transfer learning works, their used image-text models are still too large (ViT-B or ViT-L) to be deployed on mobile devices. It still remains a challenge that how to develop an efficient video-text model for mobile devices. For image-text models, there have been several works [ 74 , 14 , 60 , 57 , 66 , 56 ] exploring efficient neural network architectures for mobile devices. Most of them adopt a hybrid architecture, by combining ViTs to capture global context through self-attention mechanisms and lightweight CNNs for their parameter efficiency and low latency. With the help of high-quality and large-scale image-text datasets [ 18 , 49 , 47 ] , these models [ 41 , 12 , 58 ] have achieved good results on multiple image classification and retrieval tasks. However, the domain gap between image and video and the lack of temporal modeling limits their capabilities in video-text understanding. We aim to bridge this gap by adapting these image clip models to enable video understanding on mobile devices. Specifically, we design an efficient video-text model that is able to run on a mobile device, coined as MobileViCLIP . The number of parameters in MobileViCLIP is smaller than all of these existing video-text models, which makes it possible to be deployed on mobile devices. In addition, the latency of MobileViCLIP is comparable with other efficient image-text models, which is significantly faster than the existing video-text models. To avoid the complex design and multi-stage training process of previous video-text models [ 65 , 10 ] , we resort to the PETL paradigm by leveraging the image-text understanding capability of an existing model [ 58 ] and retraining on the additional high-quality and large-scale video-text datasets [ 64 ] to learn powerful and transferable video-text representations. To endow MobileViCLIP with the temporal modeling power, we replace the original block with Spatiotemporal RepMixer and Spatiotemporal Attention to capture both temporal dynamic and spatial representation. We only use 8 NVIDIA GeForce RTX 3090 GPUs to train our MobileViCLIP by fully fine-tuning the video branch and freezing the text branch. It only takes 2 days for training on the InternVid dataset. As shown in Fig 1 , the resulting MobileViCLIP-Small obtains comparable zero-shot text-to-video retrieval results to the ViT-L model from InternVideo2 [ 65 ] (coined as InternVideo2-L14) with much lower latency and fewer parameters. In summary, our contributions are as follows: • We introduce MobileViCLIP , the first video-text model with efficient structure that is able to be efficiently deployed on mobile devices. Our MobileViCLIP has learned good video-text representations and achieved excellent zero-shot evaluation results on multiple datasets. • We are the first to present an in-depth analysis on the latency of each basic module in current video-text models on the mobile devices, trying to figure out the key ingredients for future design of efficient video-text models. • We verify the generalization performance of MobileViCLIP on downstream tasks including text-video retrieval and temporal grounding, demonstrating stronger generalization performance compared to the baseline. 2 Related Work Vision-Language Pretraining. By utilizing web-scale paired image-text data for training, CLIP [ 46 ] shows robust zero-shot object recognition abilities. Also, some advanced works [ 52 , 9 , 53 , 11 ] have explored the upper bound of scaling up contrastive language image pre-training. These models have billions of parameters, requiring billions of pre-training datasets and tens of thousands of hours of GPU training time. These works have achieved results comparable to fully supervised models on widely used recognition and retrieval datasets. The success of contrastive language-image pre-training has also inspired pre-training tasks for other modalities like video and audio. Video-Language Pretraining. The success of CLIP inspired the video domain. The study of video-language pretraining is becoming increasingly crucial considering its wide applications. Following the idea of image-text pre-training, several foundation models in the video domain have been born [ 29 , 68 , 28 , 7 , 73 , 17 , 61 ] . These works explore the relationship between video modality and other modalities from different perspectives, achieving leading results across various video

Read the complete text