Multi-modal Deepfake Detection and Localization with FPN-Transformer

Chende Zheng, Ruiqi Suo, Zhoulin Ji, Jingyi Deng, Fangbin Yi, Chenhao Lin, Chao Shen · 2025

Read Multi-modal Deepfake Detection and Localization with FPN-Transformer by Chende Zheng, Ruiqi Suo, Zhoulin Ji, Jingyi Deng, Fangbin Yi, Chenhao Lin, Chao Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08031.txt.

Description

Chende Zheng and co-authors build a system that both detects deepfakes and locates the forged parts in time. Modern generators make audio and video fakes realistic enough to threaten digital trust. Methods using only one channel miss cross channel clues and cannot pinpoint which segments were altered.

Their framework pairs a feature pyramid with a transformer, using pretrained WavLM for audio and CLIP for video. Multi scale blocks with local attention join temporal patterns across both channels. A dual head predicts forgery probability and refines segment boundaries, reaching a score near zero point seven five on a challenging benchmark test set.

Catalog record

CREATORChende Zheng, Ruiqi Suo, Zhoulin Ji, Jingyi Deng, Fangbin Yi, Chenhao Lin, Chao Shen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08031

Browse the full library · About the Tower of Babel