Fair and Interpretable Deepfake Detection in Videos

Akihito Yoshii, Ryosuke Sonoda, Ramya Srinivasan Fujitsu Limited, Japan Fujitsu Research of America, Inc., USA · 2025

Read Fair and Interpretable Deepfake Detection in Videos by Akihito Yoshii, Ryosuke Sonoda, Ramya Srinivasan Fujitsu Limited, Japan Fujitsu Research of America, Inc., USA (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17264.txt.

Introduction

Akihito Yoshii, Ryosuke Sonoda, and Ramya Srinivasan build a fair deepfake spotter for videos in this 2025 paper. Past spotters can favor some groups, hide their logic, and miss time cues. Those flaws make calls uneven across faces and hard to trust. The team adds time traits and group-aware training data.

Their frame groups frame traits in time order to catch fakes. Concept notes turn the calls into terms lay users can grasp. A boost step pads rare groups and twists signals in tone space. Tests on four shared sets give the best mix of truth and fairness. Gains hold across two top image model backbones.

Catalog record

CREATORAkihito Yoshii, Ryosuke Sonoda, Ramya Srinivasan Fujitsu Limited, Japan Fujitsu Research of America, Inc., USA
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-17264

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fair and Interpretable Deepfake Detection in Videos Akihito Yoshii, Ryosuke Sonoda, Ramya Srinivasan Fujitsu Limited, Japan Fujitsu Research of America, Inc., USA Abstract Existing deepfake detection methods often exhibit bias, lack transparency, and fail to capture temporal information, leading to biased decisions and unreliable results across different demographic groups. In this paper, we propose a fairness-aware deepfake detection framework that integrates temporal feature learning and demographic-aware data augmentation to enhance fairness and interpretability. Our method leverages sequence-based clustering for temporal modeling of deepfake videos and concept extraction to improve detection reliability while also facilitating interpretable decisions for non-expert users. Additionally, we introduce a demography -aware data augmentation method that balances underrepresented groups and applies frequency-domain transformations to preserve deepfake artifacts, thereby mitigating bias and improving generalization. Extensive experiments on FaceForensics++, DFD, Celeb-DF, and DFDC datasets using state-of-the-art (SoTA) architectures (Xception, ResNet) demonstrate the efficacy of the proposed method in obtaining the best tradeoff between fairness and accuracy when compared to SoTA. I INTRODUCTION The rise of deepfakes has posed a major threat to the safety and privacy of individuals, institutions, societies, and nations [ 31 , 12 ] . Scholars posit that with the rapid proliferation of deepfakes, we are heading towards an “infopocalypse” where we cannot tell what is real from what is not [ 11 ] . To add to this threat is the fact that the very technologies that enable innovation can be manipulated for creation of deepfakes, resulting in malicious content that undermine privacy and promote disinformation [ 45 ] . In response to these growing concerns, researchers and practitioners have developed a suite of deepfake detection methods that also generalize to out of distribution datasets, taking into account the multiple manipulations that deepfakes may undergo [ 4 , 44 , 33 ] . In parallel, there have also been efforts to develop standardized, unified, and comprehensive benchmarks that enable fair comparison of various deepfake detection methods [ 45 , 24 ] . Despite these efforts, challenges remain. Preserving fairness in deepfake detection across demographic groups is one prominent challenge [ 2 ] . Although recent methods such as [ 27 ] have investigated this problem by proposing disentanglement learning to extract demographic and domain-agnostic forgery features to encourage fair learning, the method does not take into account spatio-temporal changes which can affect both accuracy and fairness of deepfake detectors. Another challenge concerns effectively learning dynamic changes to uncover spatio-temporal manipulations. Although existing works such as [ 14 ] consider spacial manipulation cues and temporal inconsistency , such features are still prone to biases owing to their latent correlations with sensitive attributes such as race or gender. A third challenge is to enable transparent deep fake detection and mitigation, whereby lay-users can understand the rationale behind fake identification. This requirement has become more important than before given the new regulations around AI such as the EU AI act among others [ 9 ] . Although explainable AI methods have been developed for applications such as facial affect detection [ 25 , 16 ] , their feasibility in deepfake detection remains limited. Fig. 1 : Illustration of system outputs for an example input video. Five frames with heat maps highlighting the likelihood of fake regions and Concept Sensitivity Score (CSS) providing human-interpretable explanations of model’s decision. Contributions: Towards addressing the aforementioned challenges, in this work, we develop a novel deep-fake detection method that can uncover subtle manipulations in videos while mitigating biases. The proposed framework leverages spatio-temporal cues to effectively detect minute manipulations across video frames. The proposed method offers fine-grained analysis by highlighting the fake regions in each frame in terms of human-interpretable concepts (e.g., facial mole, spectacle shape, etc.), thereby providing a user-friendly explanation and visualization [ 38 ] (Fig. 1 ). Furthermore, the proposed framework also includes a novel frequency-aware data augmentation method that mitigates bias in deepfake detection across sensitive attributes such as gender and race. The proposed method takes into account high-frequency components of video frames where deepfake-specific artifacts are most prominent, ensuring no negative impact on model performance while promoting fairness in deepfake detection. Extensive experiments on state-of-the-art face datasets demonstrate the effectiveness of the proposed methods. Fig. 2 provides an overview of the overall system. (a) Overview of our framework. Our core contributions lie in the concept extraction and data augmentation modules, which can enhance model performance. (b) Illustration of the proposed concept extraction method. (c) Illustration of the proposed data augmentation method. Fig. 2 : System diagram. Our framework first extracts proxy attributes for demographic attributes from the training data. Next, it applies frequency-aware data mixing to mitigate biases associated with these attributes. Finally, the model is re-trained on the de-biased dataset. II Related work In this section, we review recent works related to deepfake detection in videos. We also situate our work in the context of recent methods related to bias mitigation and transparency in deepfake detection. II-A Deepfake Detection in Videos A significant number of techniques to detect deepfakes in videos are based on deep learning methods. In [ 35 ] , a deep convolutional neural network, known as XceptionNet, has demonstrated high accuracy in detecting deepfake videos. It was submitted to the DeepFake Detection Challenge (DFDC), receiving a score of 0.9965 for its AUC-ROC. In [ 1 ] , the authors proposed a deep learning architecture called Mesonet to identify manipulated facial expressions. EfficientNet and ResNet based architectures have also proven to be effective in deepfake detection [ 39 , 15 , 3 ] . More recently, transformer based models are also being employed for deepfake detection [ 49 ] [ 5 ] . On the other hand, [ 46 ] leverages spatiotemporal features introducing an adapter applicable to existing models. As adopted by most state of the art techniques, we compare our method using ResNet and XceptionNet based architectures across multiple datasets to demonstrate the efficacy of the proposed methods. Further, unlike most of the existing works, the proposed method leverages both spatial and temporal information in not only detecting deepfakes but also in terms of enhancing stakeholders’ understanding of the results. II-B Bias Mitigation in Deepfake Detection Deepfake detection methods have shown varied performance across different genders and races, markedly showing higher false positive rates on certain minority groups [ 43 ] . The results from [ 43 ] showed that deepfake detection methods trained on such imbalanced/biased datasets result in incorrect detection results leading to generalizability, fairness, and security issues. In order to make detection results statistically independent of demographic factors and thereby improve fairness, the authors in [ 19 ] propose novel loss functions that handle both the setting where demographic information is available as well as the case where this information is absent. Other methods include learning demographic-agnostic features [ 27 ] , but their utility across datasets needs investigation. Beyond bias mitigation, data augmentation is widely used to enhance model generalization and robustness. While traditional methods such as MixUp [ 48 ] and CutMix [ 47 ] improve generalization by blending training samples, these methods overlook frequency-specific deepfake artifacts, which can be crucial for detection. Recent work explores frequency-aware augmentation, which modifies representations in the frequency domain rather than relying

Read the complete text