AI-Boosted Video Annotation: Assessing the Process Enhancement
Juan Gutiérrez, Ángel Mora, Pablo Regodón, Silvia Rodriguez, José Luis Blanco · 2025
Read AI-Boosted Video Annotation: Assessing the Process Enhancement by Juan Gutiérrez, Ángel Mora, Pablo Regodón, Silvia Rodriguez, José Luis Blanco (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21798.txt.
Introduction
Juan Gutiérrez, Victor Gutiérrez, Ángel Mora, Silvia Rodriguez, and José Luis Blanco test whether machine-made draft labels can speed up the slow hand-labeling of video. Careful manual annotation is still the trusted standard for dense video datasets, but it consumes large amounts of annotator time, especially when events in the footage are ambiguous.
In a single-round study, eighteen volunteers labeled difficult spatiotemporal footage with and without pre-annotations from a tuned vision-language encoder. Most participants finished about thirty-five percent faster with the machine drafts in place. Beyond the time savings, the authors offer a careful framework for judging assisted labeling workflows, weighing raw speed against whether human checking still catches machine mistakes.
Catalog record
| CREATOR | Juan Gutiérrez, Ángel Mora, Pablo Regodón, Silvia Rodriguez, José Luis Blanco |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21798
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AI-Boosted Video Annotation: Assessing the Process Enhancement Juan Gutiérrez, Ángel Mora, Pablo Regodón, Silvia Rodriguez, José Luis Blanco Abstract We explore the enhancement of Human-in-the-Loop video annotation by integrating automatic capabilities to ease the task for annotators and assess their performance. The research delves into the practical implications of the annotation processes, the integration of AI components, and the evaluation of its outcomes. We analyze their impact on efficiency, accuracy, and overall annotation quality. Focusing on the Human-in-the-Loop for video annotation tasks, we implemented a single-iteration scheme using Label Studio and AI-powered zero-shot pre-annotations. Using this framework, we designed a test based on the annotation of the UCF-Crime dataset to discriminate between normal and abnormal activities in video footage. Our results evidence how automatic AI-based pre-annotation can streamline the video annotation workflow, empowering human annotators and optimizing the overall pipeline. Using the pre-annotated data, we observed a 35% reduction in the annotation time for 70% of the annotators with similar quality annotations, compared to the traditional manual annotation task. Results are consistent with asset duration and complexity. We also observed that while annotators rapidly learned to use the tool, the produced annotations are more coherent among annotators and better match the natural clustering of the video frames. 1 1 1 The collected data is accessible on https://github.com/jgnav/AI-Boosted-Video-Annotation/ . 1 Introduction Video annotation has been the cornerstone of data-driven learning, particularly in scenarios demanding highly reliable annotated data. Alleviating the burden for annotators, expediting the process, and improving the annotations are recurrent goals to which we want to contribute. By incorporating Artificial Intelligence (AI) tools into the annotation process within the Human-in-the-Loop (HITL) paradigm, the aim is to reduce the number of frames to annotate and make each human input more valuable, reducing the work time and minimizing the probability of an error or inconsistency. We are leveraging human intelligence to handle the nuances and complex decisions, which are the weak points of fully automated annotation, and harnessing the machines’ ability to process large amounts of data. We devised a comprehensive annotation test to assess pre-annotations’ impact on the video annotation flow. We aim to explore the advantages and disadvantages of this approach and its impact on final data outcomes. We have decided to conduct the test using real-world videos primarily captured from surveillance cameras. Surveillance videos are characterized by their typically low resolution and poor quality, recorded from a stationary point amidst varying lighting and environmental conditions. This choice is because it presents a challenging use case for pre-annotating automatically with AI models and is a widely valued scenario in today’s context. Additionally, we chose a group of annotators for data labeling instead of relying on a limited set of “experts”. This approach leverages the collective intelligence of a diverse pool of individuals for tasks such as labeling, classification, and data annotation. Nonetheless, it would be interesting to evaluate the results of a similar test on professional annotators to compute performance figures and evaluate their proficiency level. For the time being, this was out of the scope of our investigation. Crowdsourcing annotations for discrete labeling has become a major trend these days. The task involves collecting multiple judgments from annotators, which are then aggregated to establish a single “ground truth” published in the dataset [ 6 ] . However, it is crucial to recognize that disagreements among annotators may contain valuable nuances worth exploring. Our test follow a crowdsourcing methodology, implementing stringent quality control measures to ensure the generation of high-quality annotations. These measures include mandatory training on the annotation tool before the test, removing outliers (annotations that deviate logically from the expected average), and assigning each video for annotation with a 6-vote overlap. This ensures that each task receives input from six different annotators, promoting diversity in perspectives. Therefore, our primary objective is to design a hypothetical annotation test that affords annotators significant freedom, allowing for the consideration of diverse criteria. Subsequently, we conducted a study on and analyzed the impact of our pre-annotations on annotators’ criteria and efficiency and evaluated the consistency of the results obtained. 2 Related work The literature describes several different annotation tasks and platforms. Reviewing previous initiatives incorporating AI-based pre-annotations to streamline the annotation process, we observe the inherent complexity and diversity, adding a layer of intricacy to the evaluation. However, let us first clarify the concept of data annotation: Data annotation involves the labeling or tagging data with specific labels or tags, rendering it more comprehensible for machines. This crucial activity is pivotal in developing supervised Machine Learning (ML) algorithms and models and validating their unsupervised and self-supervised counterparts [ 26 ] . These annotations vary significantly in terms of the task at hand and the tools used, and these aspects must be considered in the annotation process. 2.1 Common annotation tasks Video assets represent rich sources of information, prevalent across a diverse range of applications and tasks associated with information management. The extensive variety of video annotation tasks contributes to a broad spectrum of annotation tasks. The granularity required for video annotation gives rise to many annotation cases. The annotation may encompass the entire video [ 21 , 27 ] , temporal segments (shots) [ 27 , 1 ] , frames (temporal segments with zero duration) [ 9 , 32 ] , regions within frames [ 20 , 19 ] , or even moving regions—referring to a region tracked across a sequence of frames [ 4 ] . Another important use case relates to the Video Moment Retrieval (VMR). VMR retrieves a segment described by a given natural language query from an untrimmed video. This task, alternatively known as natural language video localization and video temporal grounding, is a fundamental challenge in comprehending computer vision and visual information retrieval. VMR finds wide applications in scenarios such as video browsing websites and semantics-based video search engines [ 30 , 3 , 2 ] . Interestingly, the annotations introduced for each use case vary significantly, and therefore, we must consider the task while looking for AI tools that may boost the annotation process. 2.2 Tools for video annotation We have reviewed the characteristics of the primary video annotation platforms. CVAT, an open-source annotation platform developed by Intel, offers a comprehensive set of annotation tools, including object detection, image segmentation, and video annotation. CVAT supports multiple annotation formats and provides APIs for easy integration with other computer vision applications and workflows. Being a web-based tool, CVAT can be downloaded and installed on custom local or cloud servers, making it customizable and extensible. It also includes tools for medical image conversion and annotation [ 16 ] . ANU CVML VIDeo Annotation Tool (VIDAT) is a high-quality, simple, and efficient open-source video annotation tool developed by ANU Computer Vision Machine Learning. It is user-friendly, supporting multiple annotation types, including temporal segments, object bounding boxes, semantic and instance regions, tracklets, and human pose (skeleton). VIDAT runs in a browser without external libraries or the need for server-side processing. However, it is easy to integrate with a back-end “Human-in-the-Loop” processing. It ensures data security as the data does not need to leave the local machine, and it is open-source [ 14 ] . The Video and Image Annotation (VIA)