Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge

Kirill Trapeznikov 1 , Gabriel Mancino-Ball 1 , Jonathan Li 1 , Paul Cummer 1 , Jai Aslam 1 , Danial Samadi Vahdati 2 , Tai Nguyen 2 , Matthew C. Stamm 2 , Peter Bautista 3 , Michael Davinroy 3 , Laura Cassani 3 , Jill Crisman 4 · 2026

Read Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge by Kirill Trapeznikov 1 , Gabriel Mancino-Ball 1 , Jonathan Li 1 , Paul Cummer 1 , Jai Aslam 1 , Danial Samadi Vahdati 2 , Tai Nguyen 2 , Matthew C. Stamm 2 , Peter Bautista 3 , Michael Davinroy 3 , Laura Cassani 3 , Jill Crisman 4 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06912.txt.

Introduction

Kirill Trapeznikov and eleven co-authors report on the SAFE challenge, a blind competition for detecting synthetic videos. As generative video improves, telling real footage from fabricated clips becomes urgent. The challenge, held alongside an ICCV 2025 workshop, drew over six hundred submissions from twelve teams over ninety days on the Hugging Face platform.

Two tasks were scored: spotting videos from thirteen modern generators matched against real clips from twenty-one sources, and spotting synthetic content after common post-processing like resizing, recompression, and motion blur. The dataset totals twenty hours across six thousand videos. Results show measurable progress in generalizing across generators, but post-processing artifacts remain a persistent weakness for detectors.

Catalog record

CREATORKirill Trapeznikov 1 , Gabriel Mancino-Ball 1 , Jonathan Li 1 , Paul Cummer 1 , Jai Aslam 1 , Danial Samadi Vahdati 2 , Tai Nguyen 2 , Matthew C. Stamm 2 , Peter Bautista 3 , Michael Davinroy 3 , Laura Cassani 3 , Jill Crisman 4
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06912

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advancing Reliable Synthetic Video Detection: Insights from the SAFE Challenge Kirill Trapeznikov 1 , Gabriel Mancino-Ball 1 , Jonathan Li 1 , Paul Cummer 1 , Jai Aslam 1 , Danial Samadi Vahdati 2 , Tai Nguyen 2 , Matthew C. Stamm 2 , Peter Bautista 3 , Michael Davinroy 3 , Laura Cassani 3 , Jill Crisman 4 Abstract The proliferation of generative video technologies has intensified the need for reliable methods to detect and characterize synthetic media. To address this challenge, we organized the SAFE: Synthetic Video Detection Challenge , co-located with the Authenticity and Provenance in the Age of Generative AI (APAI) Workshop at ICCV 2025. The competition invited participants to develop and evaluate algorithms capable of distinguishing real from synthetic videos under fully blind evaluation conditions with over 600 submissions from 12 teams over a 90 day span. Hosted on the Hugging Face platform, the challenge comprised two primary tasks: (1) detection of synthetic video content generated by diverse state-of-the-art models, and (2) detection of synthetic content following common post-processing operations such as resizing, re-compression, motion blur and others. The challenge data consisted of 13 modern high quality synthetic video models with generated content matched to real videos from 21 diverse and challenge sources, all adding up to 20 hours of 6,000 video samples. This paper describes the challenge design, dataset construction, evaluation methodology, and outcomes, offering insights into the generalization and robustness of contemporary synthetic video detection methods. Our findings highlight measurable progress in cross-generator generalization but also persistent vulnerabilities to post-processing artifacts. https://safe-video-2025.dsri.org 1 Introduction The emergence of large-scale generative video models spanning diffusion, transformer, and adversarial architectures, has rapidly blurred the boundary between authentic and synthetic media. Modern text-to-video and image-to-video systems can synthesize human motion, environmental dynamics, and complex scenes with heightened realism, creating both opportunities for creativity and new challenges for verifying media authenticity. As generative systems evolve, they increasingly produce content that defeats traditional semantic inconsistency markers. Ensuring video authenticity and provenance has therefore become a central problem across journalism, digital safety, fraud prevention, and scientific integrity. Detecting synthetically generated or manipulated videos requires analytics that are robust to visual variability, resilient to post-processing, and generalizable across unseen generation techniques. However, most existing benchmarks provide fixed and openly available datasets, which makes it difficult to keep pace with the rapidly changing generative landscape. These datasets are also typically limited to facial or human-centric content, leaving open questions about robustness across broader semantic domains. The SAFE: Synthetic Video Detection Challenge was designed to fill this gap. Executed using Hugging Face infrastructure, SAFE introduced a fully blind, model-submission-based framework where all evaluations were run against a sequestered dataset unavailable to participants. This approach ensured fairness and a closer approximation to real-world forensic workflows, where detectors must operate on unseen sources and cannot rely on metadata or file-format cues. The challenge received over 600 submissions from more than twelve teams over a 90-day period. Two tasks were released: (Task 1) Synthetic Video Detection: baseline evaluation of generalization across modern high quality generators and diverse real-world content sources; and (Task 2) Post-Processed Video Detection: assessment of robustness to realistic transformations such as resizing, compression and other changes common during online dissemination. Together, these tasks provided a structured, multi-factor test of both core detection capability and operational robustness, offering a view into how current synthetic-video detectors perform under controlled yet realistic blind conditions. This paper makes the following key contributions: 1. Evaluation on realistic synthetic and real content The challenge evaluated detection performance (Task 1) across 13 modern high quality text+image-to-video (TI2V) and other models with generated content matched to videos from 21 diverse and challenging real sources. The data comprised 20 hours of video across 6,000 samples of content designed to match the distribution of videos found in the wild. 2. A dual-task design measuring generalization and robustness. While Task 1 established baseline performance, Task 2 simultaneously evaluated post-processing degradations to reveal detector vulnerabilities. Task 2 data consisted of videos with 14 post-processing techniques commonly applied to videos when shared online (i.e. resizing, recompression, etc.) 3. Fully blind benchmark for synthetic video detection. Unlike prior open-dataset efforts, SAFE required participants to submit their models to be evaluated on sequestered data using the same computational platform. Reproducing real world settings, participants had no knowledge of video content and sourcing. The competition also reduced the risk of over-training by restricting the number of daily submissions and providing metrics only on the public subset of the data. 4. Comprehensive analysis of state-of-the-art detectors. This paper reports aggregated findings from top submissions, identifying several key trends such as (i) strong detection performance of unmodified synthetic video content in the fully blind competition setup (ii) non-trivial detection degradation after common post-processing, (iii) emerging good practice for detector designs such as using large pre-trained vision backbones and autoencoding augmentation (encoding input data into a latent representation and subsequently reconstructing the original input via a decoder) and (iv) surprising transferability of detectors to novel synthetic sources. 2 Background and Related Work Early manipulated media benchmarks primarily targeted faces and controlled capture environments. FaceForensics++ introduced one of the first large-scale benchmarks for facial manipulation detection, encompassing four automated generation methods (Face2Face, FaceSwap, DeepFakes, and NeuralTextures). The study reported pronounced performance degradation under compression, underscoring the sensitivity of detectors to video quality and encoding conditions [ 21 ] . Building on this foundation, the DeepFake Detection Challenge (DFDC) dramatically expanded scale and diversity, releasing over 100,000 videos featuring 3,426 paid actors and multiple pipelines to facilitate community-wide benchmarking [ 10 ] . Subsequent efforts have focused on improving robustness and representativeness. DeeperForensics-1.0 introduced 50k real and 10k forged videos accompanied by extensive “in-the-wild” perturbations, providing a framework for evaluating models under real-world noise and transformations [ 13 ] . Broader demographic coverage was addressed by KoDF, a large-scale dataset focused on Korean subjects with 175k fake and 62k real clips generated by six synthesis models, highlighting distribution shifts relative to prior corpora [ 16 ] . Moving beyond highly curated settings, the WildDeepfake dataset collected 7k face sequences from 707 deepfake videos gathered entirely from the internet; baseline detectors trained on curated datasets degrade substantially on this challenging distribution [ 32 ] . As image-to-video and text-to-video generators matured, recent work broadened scope beyond face-centric manipulations to general (non-face) synthetic video. The GenVideo benchmark reports large-scale real and generated videos spanning diverse categories and generation techniques to study cross-source generalization in AI-generated video detection [ 6 ] . DF40 aggregates content from 40 distinct deepfake techniques to facilitate next-generation cross-technique evaluation at scale [ 29 ] . Complementary evaluations such as VANE-Bench probe large multimodal models on anomaly and inconsistency detection (including synthetic videos), illuminating sensitivity to subtle generative artifacts even when the task is framed as question answering rather than binary forensics [ 4 ] . Recent datasets have further emphasized open-set, multilingual, and multimodal evaluation. Deepfake-Eval-2024 curates deepfakes from 88 websites across 52 languages, demonstrating substantial performance drops for existing detectors when tested on contemporary content compared to prior academic benchmarks [ 5 ] . SocialDF focuses on social-media contexts, compiling 2k real and synthetic videos and proposing multimodal detection approaches

Read the complete text