Accepted author manuscript, 3.53 MB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Research output: Contribution to conference - Without ISBN/ISSN › Conference paper › peer-review
Research output: Contribution to conference - Without ISBN/ISSN › Conference paper › peer-review
}
TY - CONF
T1 - PUDD: Towards Robust Multi-modal Prototype-based Deepfake Detection
AU - Lopez Pellicer, Alvaro
AU - Li, Yi
AU - Angelov, Plamen
PY - 2024/3/6
Y1 - 2024/3/6
N2 - Deepfake techniques generate highly realistic data, mak- ing it challenging for humans to discern between actual and artificially generated images. Recent advancements in deep learning-based deepfake detection methods, particularly with diffusion models, have shown remarkable progress. However, there is a growing demand for real-world appli- cations to detect unseen individuals, deepfake techniques, and scenarios. To address this limitation, we propose a Prototype-based Unified Framework for Deepfake Detec- tion (PUDD). PUDD offers a detection system based on similarity, comparing input data against known prototypes for video classification and identifying potential deepfakes or previously unseen classes by analyzing drops in similar- ity. Our extensive experiments reveal three key findings: (1) PUDD achieves an accuracy of 95.1% on Celeb-DF, out- performing state-of-the-art deepfake detection methods; (2) PUDD leverages image classification as the upstream task during training, demonstrating promising performance in both image classification and deepfake detection tasks dur- ing inference; (3) PUDD requires only 2.7 seconds for re- training on new data and emits 105 times less carbon com- pared to the state-of-the-art model, making it significantly more environmentally friendly.
AB - Deepfake techniques generate highly realistic data, mak- ing it challenging for humans to discern between actual and artificially generated images. Recent advancements in deep learning-based deepfake detection methods, particularly with diffusion models, have shown remarkable progress. However, there is a growing demand for real-world appli- cations to detect unseen individuals, deepfake techniques, and scenarios. To address this limitation, we propose a Prototype-based Unified Framework for Deepfake Detec- tion (PUDD). PUDD offers a detection system based on similarity, comparing input data against known prototypes for video classification and identifying potential deepfakes or previously unseen classes by analyzing drops in similar- ity. Our extensive experiments reveal three key findings: (1) PUDD achieves an accuracy of 95.1% on Celeb-DF, out- performing state-of-the-art deepfake detection methods; (2) PUDD leverages image classification as the upstream task during training, demonstrating promising performance in both image classification and deepfake detection tasks dur- ing inference; (3) PUDD requires only 2.7 seconds for re- training on new data and emits 105 times less carbon com- pared to the state-of-the-art model, making it significantly more environmentally friendly.
M3 - Conference paper
T2 - IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024
Y2 - 19 June 2024 through 21 June 2024
ER -