Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection

Jianfeng Liao, Yichen Wei ( 🖂 ), Raymond Chan Ching Bon, Shulan Wang, Kam-Pui Chow, Kwok-Yan Lam · 2026

Read Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection by Jianfeng Liao, Yichen Wei ( 🖂 ), Raymond Chan Ching Bon, Shulan Wang, Kam-Pui Chow, Kwok-Yan Lam (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-01450.txt.

Introduction

Jianfeng Liao, Yichen Wei, Raymond Chan Ching Bon, and colleagues wrote this 2026 study on how to spot fake face videos and images. Fake media made by AI now looks very real, and it can harm public trust and safety. Older tools often fail when a new kind of fake appears.

The team builds a system called DFA on top of CLIP, a large vision and language model, without changing CLIP itself. One part checks the whole image for odd signs, while a second part looks for small flaws in the face. A third part joins both views to make the final call. Tests on still frames and on video show strong results, with a clear gain on the hard DFDC test set.

Catalog record

CREATORJianfeng Liao, Yichen Wei ( 🖂 ), Raymond Chan Ching Bon, Shulan Wang, Kam-Pui Chow, Kwok-Yan Lam
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37996
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.01450

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection Jianfeng Liao, Yichen Wei ( 🖂 ), Raymond Chan Ching Bon, Shulan Wang, Kam-Pui Chow, Kwok-Yan Lam Abstract The rapid advancement of deepfake generation techniques poses significant threats to public safety and causes societal harm through the creation of highly realistic synthetic facial media. While existing detection methods demonstrate limitations in generalizing to emerging forgery patterns, this paper presents Deepfake Forensics Adapter (DFA), a novel dual-stream framework that synergizes vision-language foundation models with targeted forensics analysis. Our approach integrates a pre-trained CLIP model with three core components to achieve specialized deepfake detection by leveraging the powerful general capabilities of CLIP without changing CLIP parameters: 1) A Global Feature Adapter is used to identify global inconsistencies in image content that may indicate forgery, 2) A Local Anomaly Stream enhances the model’s ability to perceive local facial forgery cues by explicitly leveraging facial structure priors, and 3) An Interactive Fusion Classifier promotes deep interaction and fusion between global and local features using a transformer encoder. Extensive evaluations of frame-level and video-level benchmarks demonstrate the superior generalization capabilities of DFA, particularly achieving state-of-the-art performance in the challenging DFDC dataset with frame-level AUC/EER of 0.816/0.256 and video-level AUC/EER of 0.836/0.251, representing a 4.8% video AUC improvement over previous methods. Our framework not only demonstrates state-of-the-art performance, but also points out a feasible and effective direction for developing a robust deepfake detection system with enhanced generalization capabilities against the evolving deepfake threats. Our code is available at https://github.com/Liao330/DFA.git † † footnotetext: This work was supported by the Natural Science Foundation of Top Talent of SZTU(grant no. GDRC202414) and the Guangdong Provincial Engineering and Technology Research Center for Universities (grant no. 2024GCZX004). 1 Introduction The rapid advancement of deepfake techniques, such as Generative Adversarial Networks (GANs) [ 1 , 2 , 3 ] and Diffusion Models [ 4 , 5 ] , has enabled the generation of synthetic images and videos with unprecedented realism, often indistinguishable from authentic content to human observers. Historical precedents underscore these dangers: since late 2017, when an anonymous Reddit user (”deepfakes”) pioneered the illicit insertion of celebrity faces into pornographic content, misuse cases have increased in sophistication and impact [ 36 ] . Recent incidents, such as the 2024 Hong Kong corporate fraud involving deepfake video impersonation (resulting in HK$200 million losses) [ 35 ] and South Korea’s ”Nth Room 2.0” scandal, exemplify the misuse of the technology for large-scale harm [ 37 ] . Such synthetic media, often featuring manipulated facial attributes, now permeate illegal activities ranging from disinformation campaigns and digital evidence tampering to non-consensual pornography. Law enforcement agencies, including the FBI [ 6 ] and Europol [ 45 ] , have issued urgent warnings, with Europol characterizing deepfakes as “one of the most direct, tangible and destructive applications of Artificial Intelligence.” Despite these concerns, conventional forensics methods, whether analyzing signal-level artifacts (e.g., double JPEG compression), physical inconsistencies, or semantic anomalies (e.g., metadata discrepancies) [ 7 , 8 , 9 , 10 , 11 ] , demonstrate diminishing reliability against evolving deepfake techniques. Contemporary deepfake detection predominantly employs deep learning-based binary classifiers operating at image or video levels. Image-centric approaches leverage convolutional neural networks (CNNs), including ResNeXt [ 38 ] , ConvNet [ 39 ] , and Xception [ 12 ] , to assess frame-level authenticity by capturing subtle spatial artifacts, with Xception excelling in trace detection through separable depthwise convolutions. Video-level methods extend this by integrating temporal analysis: spatial features extracted via CNNs are processed sequentially using architectures such as Vision Transformers (ViT) to identify cross-frame inconsistencies [ 13 , 14 , 15 ] . However, despite progress, these methods struggle with generalization when encountered in synthetic media from novel generators (e.g., advanced GAN variants or emerging diffusion models). This limitation highlights the necessity for detection systems with enhanced cross-domain adaptability. Recent breakthroughs in Foundation Models [ 16 , 17 ] particularly Contrastive Language-Image Pre-training (CLIP) model [ 17 ] , offer promising solutions through their zero-shot inference capabilities and task adaptation potential. While CLIP-based linear probing achieves state-of-the-art performance in general synthetic image detection [ 18 ] , its application to facial deepfake detection (a domain requiring precise localization of facial anomalies) remains underexplored, which presents both methodological challenges and novel opportunities. Building on recent breakthroughs in adapting large foundation models, we propose the Deepfake Forensics Adapter (DFA), a dual-stream framework that leverages CLIP’s pre-trained knowledge through innovative adapter modules. Realizing that facial forgery often introduces anomalies in specific semantic regions, such as the eyes and mouth, we employ a skillfully designed Global feature Adapter that interacts with CLIP. This interaction generates attention biases to guide the focus of CLIP’S attention toward discriminative features. Simultaneously, a Local Anomaly Stream uses facial structural information to extract local features oriented towards key regions. Finally, an Interactive Fusion Classifier effectively integrates features derived from both the global context and the local feature, enabling robust detection of complex forgery patterns. The DFA framework operates through three interconnected components: (1) A Global Feature Adapter (Global) integrates the visual features of the pre-trained model into the shallow layers of the adaptation process. Through an attention mechanism, it feeds knowledge acquired during adaptation back to the pre-trained model, guiding its focus onto forgery traces without altering the original parameters. (2) A Local Anomaly Stream (Local) leverages anatomical priors derived from facial landmarks to isolate and amplify inconsistencies in critical regions (e.g., irregular pupil geometries, asymmetric lip textures), addressing the limited localized perception of conventional methods. (3) An Interactive Fusion Classifier (IFC) Module receives feature tensors from both the Global and Local stream. It performs feature fusion to form a comprehensive forgery representation. Leveraging the interaction enhancement capabilities of Transformers, it captures dependencies between features, thereby enhancing sensitivity to forgery cues. Finally, the fused information is transformed into a binary classification output, providing an accurate real versus forged judgment. Through extensive experiments and ablation studies, the proposed method demonstrates superior performance compared to existing state-of-the-art methods under unified dataset evaluation protocols. The results, illustrated in Figure 1 , validate the effectiveness of our approach. Our main contributions are summarized as follows: 1. We propose a novel CLIP-based dual-stream adapter framework (DFA) that enhances generalization in deepfake detection. By integrating the pre-trained CLIP model with customized adapter modules, our architecture preserves CLIP’s original parameters while adapting its capabilities to facial forgery tasks through a dual-stream interaction strategy. This approach effectively leverages CLIP’s semantic-visual knowledge while improving sensitivity to diverse manipulation artifacts. As a result, it achieves superior cross-dataset performance compared to existing methods. 2. We develop a novel local anomaly stream and interactive fusion mechanism to address localized forgery detection. The Local stream utilizes facial structural priors to focus on critical regions (e.g., eyes, mouth), while the IFC employs transformer-based feature interaction to model dependencies between local anomalies and global context. This design overcomes the limitations of traditional methods in capturing subtle regional inconsistencies, significantly improving detection robustness. 3. We conduct extensive evaluations on multiple benchmarks to validate the effectiveness of DFA, including the challenging DFDC dataset. Our framework achieves the state-of-the-art performance in the DFDC dataset, not only reaching 0.816 and 0.836 AUC at the

Read the complete text