Deepfake Detection Beyond Benchmark Accuracy: Generalization, Robustness, and Reproducibility

1. Introduction

1.1. Motivation and Background

The rapid advancement of generative artificial intelligence has fundamentally altered the landscape of digital media creation. Technologies such as generative adversarial networks (GANs), variational autoencoders (VAEs), and more recently diffusion models have made it possible to synthesize highly realistic images, videos, and audio with minimal expertise. While these capabilities unlock creative and productive applications—from film production to accessibility tools—they simultaneously introduce profound threats to information integrity, personal privacy, and national security.

Deepfakes, a term originally coined to describe AI-generated face-swapped videos, have evolved into a broad class of synthetic-media manipulations encompassing face reenactment, attribute editing, full-face synthesis, and multimodal forgeries across visual and auditory domains; the broader AI-generated-content landscape increasingly includes text as well. The consequences of undetected manipulations are far-reaching: fabricated videos of public figures can distort democratic processes, synthetic identity documents enable financial fraud, non-consensual intimate imagery causes severe individual harm, and manipulated evidence can undermine judicial proceedings. Detector reliability, uncertainty quantification, and error-rate characterization therefore affect the evidentiary value of forensic outputs, particularly when decisions carry legal or institutional consequences. Table 1 summarizes representative real-world incidents and deployments whose synthetic-media component was publicly reported; it is illustrative rather than a forensic verification record.

The resulting research effort has been extraordinary in both scale and pace. From a handful of early works in 2018, the field has grown rapidly, with hundreds of publications per year by 2025. Detection methods have evolved from simple binary classifiers built on convolutional neural networks to sophisticated architectures incorporating transformers, frequency-domain analysis, multimodal fusion, and—most recently—foundation models, with 121 papers (8.6% of the corpus) adopting foundation model-based approaches; the year-annotated series includes 28 such papers in 2024 and 50 in 2025. Concurrently, the resources used for training and evaluation have expanded from a few curated benchmarks to a heterogeneous landscape of deepfake benchmarks, source/pretraining datasets, and generator models or services. Our resource audit identifies 926 distinct surface resource labels among papers reporting dataset or resource usage, illustrating both the scale of the literature and the need for resource-type normalization before making benchmark claims. This growth, while indicative of the field’s importance, has also created fragmentation: inconsistent evaluation protocols, limited cross-dataset generalization, and significant gaps in reproducibility hinder the translation of research advances into deployable systems.

Our analysis reveals a striking paradox. On one hand, detection methods have grown increasingly powerful: CNN-based architectures (266 papers), transformer-based systems (207 papers), and the emerging foundation model paradigm demonstrate genuine technical progress. On the other hand, systemic weaknesses undercut the reliability of reported advances. 72.6% of papers (1,024) conduct zero robustness testing of any kind. 73.1% of the 1,091 papers with evaluation metadata do not specify their train-test split strategy. 667 papers (47.3%) explicitly claim state-of-the-art performance; because protocols, splits, and metrics differ, many of these claims are not directly comparable. Generalization—across generators, datasets, manipulation types, modalities, and deployment conditions—remains the single most frequently mentioned open challenge, with 436 mentions across the 401 papers that discuss open challenges. These findings point to an evaluation crisis that limits the field’s ability to measure its own progress and deploy systems that perform reliably outside laboratory conditions. The main thesis of this review is therefore stronger than a catalog of methods: deepfake detection is no longer primarily bottlenecked by model capacity, but by evidence quality. The field’s central task has shifted from inventing yet another detector to proving, under standardized and reproducible conditions, which detectors generalize across generators, datasets, modalities, and deployment environments.

1.2. Related Surveys

The deepfake detection field has attracted numerous survey efforts, each contributing valuable perspectives but inevitably limited in scope, recency, or analytical depth. Table 2 provides a structured comparison of nine recent surveys against the eleven research questions addressed in this work.

Several patterns emerge from this comparison. First, formal systematic methods remain uncommon, but they are not unique to a single prior review: both CH et al. [11] and Khan et al. [18] document structured search and selection procedures. Table 2 marks a review as systematic when it reports named information sources, explicit inclusion/exclusion rules, and a reproducible selection process. Second, quantitative analysis of field-wide trends remains rare: Wang et al. [13] provides a survey-level quantitative reliability analysis, while Yermakov et al. [20] is better understood as a complementary benchmark/generalization study rather than a survey and is therefore discussed as related work but excluded from Table 2. Third, entire research dimensions receive limited attention across existing surveys. Code availability and reproducibility (RQ5), terminology evolution (RQ9), and research-community demographics (RQ11) are addressed by at most one or two surveys, and not with the combined per-paper scope used here. The generator-detector arms race (RQ3) and evaluation protocol consistency (RQ6) likewise receive only partial coverage in several reviews.

This systematic review addresses these gaps by providing, to our knowledge, one of the broadest per-paper, full-text quantitative analyses focused specifically on deepfake detection research to date. With 1,410 papers analyzed across an eight-year span (January 2018–January 2026), it is among the largest reviews of synthetic-media forensics and the largest we identified that simultaneously covers methods, benchmarks, robustness, reproducibility, evaluation protocols, terminology, challenges, and research-community structure. The semi-automated analysis pipeline enables per-paper granularity across all 11 research questions, a scale that would be infeasible through fully manual coding alone. As Table 2 illustrates, this is the only review in our comparison that covers all eleven dimensions simultaneously while maintaining a systematic, quantitative, per-paper analysis pipeline.

1.3. Scope and Contributions

This paper presents a systematic literature review of deepfake detection research spanning from January 2018 to January 2026. We analyze 1,410 papers drawn primarily from arXiv and supplemented by Google Scholar, addressing 11 research questions that follow the evidence chain of the field: detector design and threat evolution, resource and evaluation validity, reproducibility, adversarial resilience, terminology, open challenges, and community structure. The key contributions of this work are as follows:

1.

Large-scale coverage across the full evidence chain. We provide one of the largest systematic analyses of deepfake detection to date, and the broadest in terms of RQ coverage that we identified, analyzing 1,410 included papers with structured metadata extraction. All included papers were processed through a full-text, LLM-assisted RQ-specific extraction pipeline with targeted human quality controls, identifying CNN-based methods as the largest single category (266 papers, 18.9%), transformer-based architectures (207 papers, 14.7%), and the rapid ascent of foundation model-based approaches (121 papers, 8.6%), along with generative method studies (104 papers), benchmark papers (101 papers), frequency-based methods (76 papers), hybrid approaches (72 papers), and multimodal techniques (60 papers).

2.

Resource-aware dataset landscape. We analyze RQ2 as a resource taxonomy rather than a flat dataset list, separating deepfake benchmark families from source/pretraining datasets and generator models or services. After alias normalization, evaluation remains concentrated around a small number of benchmark families—FaceForensics++ (330 normalized mentions), Celeb-DF (267), and DFDC (231)—while resources such as ImageNet, COCO, LAION, Stable Diffusion, Midjourney, Sora, StyleGAN, and ProGAN are classified outside the deepfake-benchmark totals.

3.

Identification of a systemic evaluation crisis. We document pervasive weaknesses in evaluation practices that undermine the reliability of reported results: 72.6% of papers (1,024) perform no robustness testing whatsoever, 73.1% of the 1,091 papers with evaluation metadata omit train-test split specifications, and 667 papers (47.3%) explicitly claim state-of-the-art performance, while heterogeneous protocols limit direct comparison. These findings reveal that the community’s ability to measure genuine progress is severely compromised.

4.

Multi-dimensional field analysis. Beyond technical methods and datasets, we examine code availability and reproducibility (51.1% raw code/model-link rate, 23.7% single-GitHub proxy, and 15-paper strict README/title-match lower bound), adversarial resilience (137 attack-oriented and 390 defense-oriented paper-label assignments, which may overlap), terminology evolution (“deepfake” appears in 546 papers alongside a broader dual vocabulary and multiple specialized labels), open challenges (generalization is the most frequently mentioned challenge with 436 mentions), and research community demographics spanning 56 geographic reporting entities. This holistic perspective exposes systemic issues that purely technical surveys overlook.

5.

Temporal trend analysis and RQ-level research roadmap. By covering an eight-year window that encompasses the full lifecycle of deepfake detection research—from its emergence in 2018 through the GAN era, the diffusion model revolution, and the foundation model paradigm shift—we identify inflection points, paradigm shifts, and emerging directions. We synthesize findings across all research questions into explicit RQ-level conclusions, showing how architectural evolution (RQ1), resource taxonomy (RQ2), generator adaptation (RQ3), robustness (RQ4), reproducibility (RQ5), evaluation protocols (RQ6), metrics (RQ7), adversarial resilience (RQ8), terminology (RQ9), open challenges (RQ10), and community structure (RQ11) jointly explain why standardized evaluation and cross-generator generalization must become the field’s dominant research agenda.

Figure 1 provides a visual map of all 11 research questions in the same order used throughout the Results section. The figure intentionally avoids assigning hard disciplinary labels to the RQs: several questions, especially reproducibility (RQ5), adversarial robustness (RQ8), and terminology (RQ9), cut across technical, methodological, and deployment concerns.

1.4. Paper Organization

The remainder of this paper is organized as follows. Section 2 describes the systematic review methodology, including search strategy, selection criteria, data extraction pipeline, and the 11 research questions. Section 3 presents the findings for each research question in turn: detection method evolution (RQ1), datasets and benchmarks (RQ2), the generator-detector arms race (RQ3), robustness analysis (RQ4), code availability and reproducibility (RQ5), evaluation protocols (RQ6), performance metrics and benchmarking (RQ7), adversarial attacks and defenses (RQ8), terminology evolution (RQ9), open challenges (RQ10), and research community analysis (RQ11). Section 4 synthesizes cross-cutting themes, discusses limitations of this review, and presents a research roadmap. Section 5 concludes with summary findings and recommendations.

2. Methodology

This systematic literature review uses PRISMA-informed reporting and a semi-automated evidence-mapping workflow. We employ a semi-automated pipeline that combines traditional search strategies with AI-assisted analysis to achieve large-scale, per-paper coverage while preserving auditable checkpoints for every research question. The automation is used for discovery, parsing, and structured extraction; the substantive claims in the paper are reported with explicit denominators, resource-type definitions, and targeted quality controls.

2.1. Search Strategy

We developed an automated research monitoring system for high-recall literature discovery. The system is a custom ingestion pipeline rather than a claim that automated monitoring is an established field standard. It executes query templates against arXiv and Google Scholar, stores candidate metadata and URLs, downloads available PDFs, computes duplicate keys from titles, identifiers, and version metadata, and queues the parsed full text for the RQ-specific extraction pipeline. This design was chosen because deepfake-detection research appears rapidly as preprints, benchmark releases, and generator-specific studies; a one-time database query would miss part of this moving target and would make the January 2026 cutoff harder to audit.

The primary search was conducted on arXiv, supplemented by Google Scholar, to capture both preprints and published venues. Our search covered the period from January 2018 to January 2026. Because the search cutoff occurs in January 2026, all 2026 counts should be interpreted as partial-year observations. For this reason, annual trend figures in the paper either stop at 2025 or explicitly omit the January 2026 slice; 2026 records remain included in aggregate corpus-level totals.

Search Keywords. We used a comprehensive set of keywords organized into three groups: (1) core terms: deepfake detection, deepfake, face forgery detection, facial manipulation detection; (2) method-specific terms: face swap detection, face reenactment detection, AI-generated face detection, synthetic face detection, GAN-generated face detection, diffusion-generated face detection; and (3) broader terms: media forensics, image forensics, video forensics, digital forensics, image manipulation detection. Each search query combined terms from multiple groups to maximize recall while maintaining relevance to the deepfake detection domain.

Source Coverage. arXiv served as the primary source due to its comprehensive coverage of preprints in computer science, particularly in computer vision (cs.CV) and cryptography and security (cs.CR) categories. The automated system performed daily searches to capture newly published papers. Google Scholar provided supplementary coverage of papers published at venues without strong preprint traditions.

2.2. Selection Criteria

We assessed papers for relevance through a two-stage filtering process. The initial search was deliberately broad: it included not only “deepfake detection” but also image forensics, video forensics, AI-generated media, generation methods, and benchmark terms. The resulting 9.4% inclusion rate should therefore be interpreted as the outcome of a high-recall candidate net, not as evidence that most deepfake-detection papers were rejected for low quality.

Stage 1: Automated Relevance Filtering. A large language model (LLM) evaluated each discovered paper against six domain-specific inclusion routes. A paper qualified as relevant if it addressed at least one of the following: (a) deepfake, face forgery, or AI-generated media detection methods, (b) datasets or benchmarks for deepfake or synthetic-media detection, (c) generative models used to create face, image, video, or multimodal manipulations, (d) adversarial attacks or defenses in the deepfake detection context, (e) evaluation methodologies for deepfake detectors, or (f) surveys or analyses of the deepfake detection landscape. Papers outside all six routes—for example, generic image classification, unrelated biometric recognition, generic diffusion-model training, or digital forensics papers without synthetic-media relevance—were removed at this stage.

Stage 2: Full-Text Quality Validation. We further validated papers passing Stage 1 by downloading and parsing the full PDF. Papers without accessible full text were excluded from the final review. Duplicate or superseded versions, short abstracts without sufficient technical content, and tangentially related records were also removed. The “tangentially related” category refers to targeted spot-check audit of borderline records, not to an end-to-end manual reading of every candidate. This process yielded 1,410 included papers from an initial pool of approximately 15,000 candidates.

2.3. PRISMA-Guided Selection Reporting

We report our study selection using a PRISMA-informed flow structure based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines [21]. Because this review uses automated monitoring, LLM-assisted relevance filtering, and arXiv-first coverage rather than a conventional database-only protocol, we describe the process as PRISMA-guided rather than claiming full PRISMA compliance. To avoid presenting the same attrition information twice, Figure 2 is the primary selection-flow summary; it reports the candidate counts, duplicate removal, six-criterion relevance screen, full-text validation, exclusion reasons, and final included corpus.

The six relevance criteria were intentionally broad to maximize recall, while Stage 2 removed false positives, inaccessible documents, duplicate or superseded versions, and records lacking sufficient technical content for full-paper analysis. This two-stage pipeline balances the comprehensiveness demanded by systematic reviews with the practical necessity of processing approximately 15,000 candidates, the vast majority of which fall outside the deepfake-detection domain once the six criteria are applied.

2.4. Data Extraction and Analysis

The 1,410 included papers were not manually coded end-to-end by human reviewers. Instead, each included full text was parsed and processed through a multi-pass LLM-assisted extraction pipeline, with human effort concentrated on protocol design, prompt specification, denominator-sensitive checks, high-impact field audits, and correction of known failure modes such as resource aliasing. This distinction is central to the methodology: the study is a large-scale semi-automated review with targeted human quality control, not a traditional fully manual coding exercise.

For each included paper, we extracted structured metadata through the following pipeline:

1.

Metadata Extraction: Title, authors, year, venue, abstract, and publication URLs were extracted from source APIs and PDF headers.

2.

Content Classification: Papers were categorized by content type (detection method, dataset, generation method, survey, etc.) and assigned paper-type tags (e.g., Method, Benchmark, Dataset).

3.

RQ-Specific Analysis: For each of the 11 research questions, a specialized prompt extracted relevant information from the full paper text. For example, RQ1 analysis extracted method category, backbone architecture, input type, supervision approach, and key innovation; RQ4 analysis extracted robustness testing protocols, manipulation types evaluated, and cross-dataset evaluation results. Each RQ analysis was independently checkpointed to enable verification and reproducibility.

4.

Aggregation: Per-paper classifications were aggregated into cross-paper statistics, trend analyses, and categorical distributions. Multi-label assignments (e.g., a paper addressing both generalization and adversarial challenges) were preserved to capture the full complexity of the literature.

This automated pipeline enabled analysis at a scale infeasible through fully manual coding alone, processing 1,410 papers across 11 research questions with per-paper granularity. The pipeline generated structured checkpoints for each RQ, facilitating both the quantitative analyses reported in §3 and the citation index used to link specific claims to supporting papers.

2.5. Quality Controls and Reporting Semantics

The scale of this review makes automated extraction necessary, but the strongest claims in the paper were not treated as unexamined LLM outputs. We applied targeted quality controls to the analyses most likely to affect the paper’s central conclusions. First, denominator-sensitive findings were checked for subset consistency: for example, the 73.1% split-reporting statistic is reported over the 1,091 papers with evaluation metadata rather than over the full 1,410-paper corpus. Second, RQ2 resource labels were manually consolidated for the most frequent families and separated by resource type, so that true benchmark families (e.g., FaceForensics++, Celeb-DF, DFDC), source or pretraining datasets (e.g., ImageNet, COCO, LAION), and generator models or services (e.g., Stable Diffusion, StyleGAN, Midjourney) are not collapsed into a single dataset count. Third, RQ5 code availability is reported as evidence tiers: a raw code/model-link rate, a single-GitHub proxy for likely author-released code, and a strict README/title-match lower bound; we avoid the stronger claim that repositories were verified to execute successfully. Fourth, RQ6 reporting statistics were checked against the evaluation-metadata subset so that train-test split, preprocessing, face-detector, and significance-testing claims are not generalized beyond the papers where those fields were extracted. Fifth, RQ11 percentages are reported over the resolved-affiliation subset and are explicitly non-additive because multi-affiliation papers can contribute to multiple geographic reporting entities.

Throughout the paper, counts such as “normalized mentions” should be read as corpus-level usage evidence rather than bibliographic citation counts. This distinction is especially important for RQ2, where a paper may mention a generator, a pretraining dataset, and a benchmark in the same experimental pipeline. We use aggressive field-level language when the evidence is structural and convergent across RQs, but keep denominator, resource-type, and proxy definitions explicit so that the conclusions remain auditable.

2.6. Research Questions

We investigate the following 11 research questions, covering the full evidence chain of deepfake-detection research:

RQ1 

How have detection methods evolved? — Taxonomy of detection approaches, backbone architectures, input modalities, and supervision types over time.

RQ2 

What datasets, benchmark families, and generation/source resources are used? — Resource-aware catalog of true deepfake benchmarks, source or pretraining datasets, generator models/services, modalities, and coverage gaps.

RQ3 

How does the generator-detector arms race manifest? — Co-evolution of generation and detection methods, adaptation strategies.

RQ4 

How robust are detectors across manipulation types? — Cross-type generalization, robustness to compression and social media processing.

RQ5 

What is the state of code availability and reproducibility? — Raw code-link rates, sharing platforms, and a conservative proxy for likely author-released implementations.

RQ6 

What evaluation protocols are used? — Train-test splits, preprocessing pipelines, face detectors, and evaluation granularity.

RQ7 

What performance metrics are reported? — Metric choices, state-of-the-art claims, cross-dataset evaluation practices.

RQ8 

What adversarial attacks and defenses exist? — Attack types, defense mechanisms, adversarial training practices.

RQ9 

How has terminology evolved? — Shifts in technical vocabulary reflecting conceptual changes in the field.

RQ10 

What are the open challenges and future directions? — Identified research gaps, urgency assessment, and proposed solutions.

RQ11 

What characterizes the research community? — Institutional distribution, geographic concentration, and collaboration patterns.

The RQs are reported in numeric order because that order mirrors the evidence chain used in the paper: detector methods and generative threats (RQ1–RQ3), evaluation validity (RQ4, RQ6, RQ7), reproducibility (RQ5), adversarial stress testing (RQ8), terminology and field discovery (RQ9), open problems (RQ10), and community structure (RQ11). Several questions cross category boundaries—for example, reproducibility is both technical and methodological, and adversarial robustness is both a detector property and an evaluation requirement. This is why the paper avoids treating the RQs as mutually exclusive “technical” versus “societal” bins. The cross-cutting synthesis in §4 focuses on relationships that emerge only when multiple RQs are interpreted together, such as the relationship between evaluation fragmentation (RQ6, RQ7) and the generalization challenge (RQ4, RQ10).

2.7. Reproducibility Materials

A fully reproducible release of this review requires the exact search strings and dates, retrieved identifiers, duplicate-resolution rules, inclusion/exclusion labels, RQ prompts and schemas, normalization registries, code-link audit rules, and figure-generation scripts. The public archival package accompanying the paper will expose these materials together with the per-paper structured records, subject to source-license constraints. Until that package is available, the quantitative results should be interpreted as an auditable evidence map rather than an independently reconstructed meta-analysis.

3. Results

This section presents findings for each of the 11 research questions. Each subsection synthesizes quantitative analysis with qualitative insights, supported by citations to representative works from the corpus, and ends with an explicit RQ-level answer that states the strongest implication supported by the evidence. The RQ-level verdict tables summarize the field-level claims before the detailed evidence is presented.

Table 3. Field-level synthesis of RQ1–RQ6. The table states the strongest claim supported by each RQ and its quantitative evidence.

RQ	Empirical finding	Field-level verdict
RQ1	CNNs (266), transformers (207), foundation models (121).	The architecture race has become a representation-and-evidence problem: the meaningful detector is the one that maintains reliable performance under unseen generators, modalities, and deployment conditions.
RQ2	True benchmark usage centers on FF++ (330), Celeb-DF (267), and DFDC (231).	Dataset abundance is partly illusory. The field has many resource labels but too few current, diverse, standardized benchmarks; benchmark design is now a scientific bottleneck.
RQ3	Diffusion-related work (∼330) overtakes GAN-focused work (∼297).	The field adapts, but reactively. Generator-specific detectors accumulate technical debt unless paired with continuously refreshed, generator-agnostic evaluation.
RQ4	72.6% of papers conduct zero robustness testing; only 7 test all four robustness dimensions.	Without cross-dataset, compression, social-media, and unseen-generator tests, a detector has not yet provided evidence of deployable forensic reliability.
RQ5	721 papers contain links; 334 meet the single-GitHub proxy; 15 pass strict README/title matching.	Link presence is not corresponding code. Reproducibility evidence must include runnable author code, checkpoints, splits, preprocessing scripts, and evaluation commands.
RQ6	798 of 1,091 evaluation-metadata papers omit split strategy; only 4.3% report significance.	Evaluation protocol is the hidden variable behind many leaderboard rankings; unreported splits and preprocessing make headline comparisons non-commensurable.

Table 3. Field-level synthesis of RQ1–RQ6. The table states the strongest claim supported by each RQ and its quantitative evidence.

RQ	Empirical finding	Field-level verdict
RQ1	CNNs (266), transformers (207), foundation models (121).	The architecture race has become a representation-and-evidence problem: the meaningful detector is the one that maintains reliable performance under unseen generators, modalities, and deployment conditions.
RQ2	True benchmark usage centers on FF++ (330), Celeb-DF (267), and DFDC (231).	Dataset abundance is partly illusory. The field has many resource labels but too few current, diverse, standardized benchmarks; benchmark design is now a scientific bottleneck.
RQ3	Diffusion-related work (∼330) overtakes GAN-focused work (∼297).	The field adapts, but reactively. Generator-specific detectors accumulate technical debt unless paired with continuously refreshed, generator-agnostic evaluation.
RQ4	72.6% of papers conduct zero robustness testing; only 7 test all four robustness dimensions.	Without cross-dataset, compression, social-media, and unseen-generator tests, a detector has not yet provided evidence of deployable forensic reliability.
RQ5	721 papers contain links; 334 meet the single-GitHub proxy; 15 pass strict README/title matching.	Link presence is not corresponding code. Reproducibility evidence must include runnable author code, checkpoints, splits, preprocessing scripts, and evaluation commands.
RQ6	798 of 1,091 evaluation-metadata papers omit split strategy; only 4.3% report significance.	Evaluation protocol is the hidden variable behind many leaderboard rankings; unreported splits and preprocessing make headline comparisons non-commensurable.

3.1. RQ1: Detection Method Evolution

Our analysis of 1,410 papers reveals a field undergoing a fundamental architectural transformation. Detection methods have evolved through three distinct phases—from CNN-dominated classifiers (2018–2020), through transformer diversification (2021–2023), to the current foundation model paradigm (2024–present). CNN-based approaches remain the most common single category at 266 papers (18.9%), but transformer-based methods (207 papers, 14.7%) have overtaken CNNs in annual output since 2024, and foundation model-based approaches account for 121 papers (8.6%), with the year-annotated series rising to 28 papers in 2024 and 50 in 2025. This section traces the evolutionary trajectory of detection architectures, backbone choices, input modalities, and supervision strategies that collectively define how the field detects synthetic media.

Table 4. Field-level synthesis of RQ7–RQ11.

RQ	Empirical finding	Field-level verdict
RQ7	Independent corpus-wide rates: 47.3% SOTA claims, 30.1% cross-dataset, 19.1% ablation.	The literature is score-rich but evidence-poor. SOTA should be an evidence contract: fixed protocols, cross-dataset tests, ablations, uncertainty, low-FPR, and efficiency.
RQ8	137 attack-oriented vs. 390 defense-oriented labels; adversarial training in 11 defense-oriented papers (2.8%).	Deepfake adversarial robustness is not solved; defense claims remain premature without adaptive, transfer, natural-processing, and cross-generator attacks.
RQ9	In 2025, “deepfake” appears in 38.5% of papers, “AI-generated” in 34.0%, and “face forgery” in 3.0%.	The field is converging on a dual vocabulary. Reviews and benchmarks should cross-index legacy deepfake labels and broader AI-generated-content labels with explicit generator and modality tags.
RQ10	Generalization is the most frequently mentioned open challenge (436 mentions).	The field diagnoses more than it repairs. The dominant open problems are coupled infrastructure failures, not isolated architecture gaps.
RQ11	The resolved-affiliation subset spans 56 geographic entities but is led by mainland China (377) and the USA (258).	Geography is scientifically consequential: benchmark provenance may encode demographic and media-distribution assumptions, motivating region-aware evaluation rather than inference from affiliation counts alone.

Table 4. Field-level synthesis of RQ7–RQ11.

RQ	Empirical finding	Field-level verdict
RQ7	Independent corpus-wide rates: 47.3% SOTA claims, 30.1% cross-dataset, 19.1% ablation.	The literature is score-rich but evidence-poor. SOTA should be an evidence contract: fixed protocols, cross-dataset tests, ablations, uncertainty, low-FPR, and efficiency.
RQ8	137 attack-oriented vs. 390 defense-oriented labels; adversarial training in 11 defense-oriented papers (2.8%).	Deepfake adversarial robustness is not solved; defense claims remain premature without adaptive, transfer, natural-processing, and cross-generator attacks.
RQ9	In 2025, “deepfake” appears in 38.5% of papers, “AI-generated” in 34.0%, and “face forgery” in 3.0%.	The field is converging on a dual vocabulary. Reviews and benchmarks should cross-index legacy deepfake labels and broader AI-generated-content labels with explicit generator and modality tags.
RQ10	Generalization is the most frequently mentioned open challenge (436 mentions).	The field diagnoses more than it repairs. The dominant open problems are coupled infrastructure failures, not isolated architecture gaps.
RQ11	The resolved-affiliation subset spans 56 geographic entities but is led by mainland China (377) and the USA (258).	Geography is scientifically consequential: benchmark provenance may encode demographic and media-distribution assumptions, motivating region-aware evaluation rather than inference from affiliation counts alone.

3.1.1. Method Taxonomy

Figure 3 and Table 5 present the distribution of consolidated paper-method and contribution categories across the corpus after normalization of 127 raw category labels. The chart is intentionally not a pure architecture taxonomy: it separates detector families (e.g., CNN, transformer, foundation-model, frequency, hybrid, multimodal) from adjacent contribution roles that shape detector evaluation (generative-method studies and benchmark studies). This separation prevents generation or benchmark papers from being misread as detector architectures while still preserving their role in the method-evolution narrative.

CNN-based approaches constitute the largest single category with 266 papers (18.9%), employing convolutional architectures—most commonly Xception, EfficientNet, and ResNet variants—as the primary feature extraction backbone [22,23,24,25,26]. While CNN-only approaches represented the dominant paradigm in early deepfake detection research, their relative share has declined over time, reflecting the emergence of competing architectural families that capture non-local spatial relationships more effectively.

Transformer-based methods represent the second largest category at 207 papers (14.7%), utilizing Vision Transformer architectures and their variants—Swin Transformer, DeiT, and Data-efficient Image Transformers—for spatial relationship modeling through self-attention mechanisms [27,28,29,30,31]. The self-attention mechanism captures global spatial dependencies that convolutional receptive fields miss, making transformers particularly effective for detecting artifacts that manifest as long-range inconsistencies across manipulated boundaries. Transformers surpassed CNNs in annual paper count in 2024, marking a decisive architectural transition.

Foundation model-based approaches have emerged as the most transformative development, totaling 121 papers (8.6%) and leveraging large pre-trained models such as CLIP, DINOv2, and multimodal large language models (MLLMs) for detection [32,33,34,35,36]. These methods exploit the rich, generalizable representations learned during large-scale pre-training, adapting them through parameter-efficient fine-tuning strategies including LoRA, prompt tuning, and adapter modules. The year-annotated series shows a sharp late-period expansion—28 foundation-model papers in 2024 and 50 in 2025—supporting a paradigm-shift interpretation without assigning an irreconcilable corpus-wide percentage.

Generative methods account for 104 papers (7.4%), encompassing work on deepfake generation techniques, diffusion-based synthesis, and GAN architectures [37,38,39,40,41]. This category captures the substantial body of research on understanding and characterizing the generation process itself, which directly informs detection strategies as documented in §3.3. Benchmark studies constitute 101 papers (7.2%), reflecting the field’s increasing maturity and emphasis on systematic, comparative evaluation [42,43,44,45]. Frequency-based analysis (76 papers, 5.4%) leverages spectral artifacts in Fourier or wavelet domains [46,47,48,49,50], exploiting the observation that generative models introduce characteristic frequency-domain signatures that differ from those of authentic images. Hybrid approaches (72 papers, 5.1%) combine multiple architectural paradigms—typically pairing CNNs with transformers, spatial with frequency-domain features, or image-level with patch-level analysis [51,52,53,54,55]. Multimodal methods (60 papers, 4.3%) integrate heterogeneous information streams such as RGB imagery, audio, textual metadata, and frequency representations [56,57,58,59,60].

Additional categories include temporal analysis (34 papers, 2.4%) [61,62,63,64], which models frame-to-frame consistency in video deepfakes; self-supervised methods (30 papers, 2.1%) [65,66,67,68], which learn manipulation-sensitive features through pretext tasks; GAN fingerprint analysis (20 papers, 1.4%) [69,70,71]; physical inconsistency detection (12 papers, 0.9%); and biological signal analysis (9 papers, 0.6%) [72,73,74].

It is important to note that the architecture-specific rows in this taxonomy primarily capture detection methods—binary classifiers that answer “is this content a deepfake?”—which constitute the overwhelming focus of the literature. The two role-oriented rows, generative-method studies and benchmark studies, are retained because they explain why detector research shifts: generator papers define the threat model, while benchmark papers define the evaluation target. A distinct sub-problem is localization, which produces spatial or temporal manipulation maps indicating where in an image or video the manipulation occurred. Detection and localization are technically distinct paradigms requiring different architectures (e.g., pixel-level segmentation networks vs. image-level classifiers) and evaluation protocols (e.g., pixel-level IoU and F1 vs. image-level AUC and accuracy) [18]. While some methods jointly optimize for both tasks, readers should be aware that performance comparisons across these paradigms are not directly commensurable.

3.1.2. Architectural Evolution Over Time

The annual extraction still reveals three distinct architectural phases, but the partial January 2026 slice is used only in aggregate counts and is not visualized as a standalone trend year. These phases also align with external shifts in generative AI: early detection work tracked GAN-era face manipulation tools; the 2022 diffusion wave created a new class of image artifacts and source distributions; and the 2024–2025 rise of vision-language and multimodal foundation models made representation transfer a central detector strategy [75,76,77]. This alignment is important because it shows that the method timeline is not merely an internal publication trend; it is a response to changing generator capabilities.

Phase 1: CNN dominance (2018–2020). CNN-based methods accounted for the overwhelming majority of detection approaches, with MesoNet and Xception-based pipelines serving as representative early detector architectures [78,79]. During this formative period, the field treated deepfake detection primarily as a binary image classification problem solvable with standard convolutional feature extractors. The small annual output (7–8 papers per year) reflects both the nascent state of the field and the limited diversity of generative threats, which were confined primarily to GAN-based face swaps.

Phase 2: Transformer diversification (2021–2023). The introduction of Vision Transformers for forgery detection in 2021 [80,81,82] catalyzed rapid architectural diversification. Transformer-based approaches grew from 5 papers in 2021 to 10 in 2023, while CNN-based methods maintained steady output at 14–23 papers per year. Hybrid and frequency-based approaches gained traction during this period—frequency-based methods grew from 5 papers in 2021 to 7 in 2023—as researchers recognized that spatial-only features were insufficient for detecting increasingly sophisticated forgeries. The multimodal fusion category also expanded, with researchers combining RGB, frequency, and temporal streams to capture complementary forensic signals.

Phase 3: Foundation model revolution (2024–present). Foundation model-based methods surged from 3 papers in 2023 to 28 in 2024 and 50 in 2025 [83,84,85,86,87]. This phase also saw transformer and CNN annual output become closely contested in the annotated annual extraction, with transformers slightly ahead in 2024–2025. The adoption of MLLMs—including GPT-4V, Qwen-VL, and LLaVA-based architectures—for explainable deepfake detection represents a particularly notable development, as these models can provide natural-language explanations of detected artifacts [34,86,88,89]. Vision-language and multimodal fusion methods expanded alongside CLIP-, BLIP-, and InstructBLIP-style representations, reinforcing the shift from single-stream classifiers toward detectors that combine visual, textual, temporal, and semantic evidence.

Diffusion forensics emerged as a distinct and rapidly growing subfield, with 57 papers addressing artifacts specific to diffusion-based generation. Of these, 89.5% were published in 2023–2025 [37,38,90,91,92], tracking precisely the proliferation of diffusion generators documented in §3.3. The growth from 1 paper in 2022 to 7 in 2023, 18 in 2024, and 26 in 2025 is consistent with a reactive detector-generator dynamic: in this annual series, corresponding detection work follows the public diffusion wave within roughly one to two publication years, although the counts do not establish causality.

3.1.3. Backbone Architecture Distribution

Beyond method categories, this analysis systematically tracks backbone architectures across the full corpus (Figure 4, Table 6). After normalizing 205 unique backbone labels, five architectures dominate the landscape.

CLIP leads with 72 papers, making it the single most frequent normalized detector-representation/ backbone label [32,33,54,93,94]. CLIP’s dual vision-language encoder provides rich semantic representations that transfer effectively across manipulation types, and its zero-shot capabilities enable detection without generator-specific training data. CLIP is the largest single normalized detector-representation label, surpassing any individual CNN label and indicating a shift toward representation-based detection; this does not imply that the aggregate foundation-model family exceeds the combined classic-CNN family. The coding records CLIP when it is used as a detector representation, backbone, or scoring model, and does not fully separate frozen encoders, fine-tuned encoders, and zero-shot use. CLIP’s prominence tracks the broader foundation-model trend: detector design is increasingly built around transferable representations rather than task-specific convolutional backbones.

Vision Transformer (ViT) follows with 65 papers [27,95,96,97], encompassing ViT-B/16, ViT-L/14, and related variants. The ViT family’s self-attention mechanism models long-range spatial relationships that convolutional architectures cannot capture within their fixed receptive fields, making it well suited for detecting manipulation artifacts at region boundaries.

ResNet appears in 57 papers [70,92,98,99], primarily ResNet-50 and ResNet-18 variants. Despite being the oldest major architecture in this ranking, ResNet remains prevalent due to its computational efficiency, well-understood behavior, extensive pre-trained checkpoint availability, and its role as a standard baseline in benchmark studies.

Xception is used in 52 papers [100,101,102,103,104]. Originally proposed for deepfake detection in the seminal FaceForensics++ work, Xception has become the de facto baseline that new methods are compared against. Its continued prevalence reflects both genuine utility and the field’s convention of including Xception comparisons for benchmarking purposes. EfficientNet rounds out the top five with 29 papers [26,105,106,107,108], valued for its favorable accuracy-to-computation tradeoff.

At the architecture family level, Vision Transformers (ViT, Swin, DeiT) collectively appear in approximately 110 papers (7.8%), while classic CNNs (ResNet, Xception, VGG) appear in approximately 165 papers (11.7%). Foundation models (CLIP, MLLMs, vision-language models) appear in approximately 100 papers (7.1%). The convergence of these three families in aggregate volume demonstrates that the backbone landscape has transitioned from CNN hegemony to a three-way contest, with foundation-model labels showing the strongest late-period growth but classic CNN families remaining larger in aggregate.

3.1.4. Supervision Paradigms

Figure 5 presents the distribution of training supervision strategies across the 1,013 papers with identifiable supervision types. Fully supervised learning remains the dominant paradigm, employed in 734 papers (72.5%), but alternative supervision strategies have gained substantial traction, particularly in 2024–2025.

Unsupervised approaches appear in 118 papers (11.6%), often leveraging reconstruction-based anomaly detection, distribution modeling, or one-class classification where only real images are required during training [47,59,66,109,110]. Zero-shot detection has emerged as a major research direction with 65 papers (6.4%), driven by foundation models whose pre-trained representations enable detection without any forgery-specific training [87,89,90,111,112]. The zero-shot trajectory is striking: from 3 papers in 2021 to 1 in 2022, 4 in 2023, then surging to 21 in 2024 and 35 in 2025. This growth closely tracks the adoption of CLIP and related foundation models, showing a close temporal association between zero-shot detection and the adoption of foundation-model representations.

Self-supervised methods account for 42 papers (4.1%), learning manipulation-sensitive features through pretext tasks such as contrastive learning, masked image modeling, and reconstruction error analysis [38,65,67,68,113]. Self-supervised methods grew from 1 paper in 2021 to 17 in 2025, driven by the recognition that hand-labeling forgery data cannot keep pace with the proliferation of new generators. Semi-supervised (23 papers, 2.3%) [33,50,114,115], weakly supervised (17 papers, 1.7%) [58,60,116,117], and few-shot learning (14 papers, 1.4%) [118,119,120,121] address the practical challenge of adapting to novel manipulation types with limited labeled examples.

The method taxonomy and supervision taxonomy are separate coding axes: 30 papers use self-supervised learning as their primary method-category label, whereas 42 receive a self-supervised supervision label. The growing diversity in supervision paradigms reflects a fundamental shift in research priorities: from maximizing accuracy on known forgery types to building generalizable detectors capable of identifying previously unseen manipulation techniques. The surge in zero-shot approaches temporally parallels the foundation-model trend documented above and is consistent with a shift toward learning what natural images look like—with deviations from this learned distribution flagged as potential forgeries—rather than cataloging the specific artifacts of each generator.

3.1.5. Synthesis

The detection method landscape has undergone two paradigm shifts in eight years. The first, around 2021, introduced transformer architectures and attention mechanisms that could capture global spatial dependencies missed by convolutional receptive fields. The second, beginning in 2023 and accelerating through 2025, introduced foundation models that leverage massive pre-training to provide generalizable representations without generator-specific training data.

Three structural trends emerge from this evolution. First, CLIP (72 papers) is the largest single normalized backbone label, ahead of ResNet (57) and Xception (52), reflecting a shift from task-specific architectures to general-purpose representations; classic CNN variants remain larger when aggregated as a family. Second, the supervision landscape is diversifying, with zero-shot (65 papers) and self-supervised (42 papers) methods rising sharply in the late-period annual counts, enabled by foundation models that encode sufficient visual knowledge to distinguish authentic from synthetic content without explicit forgery labels. Third, new subfields—diffusion forensics (57 papers, 89.5% from 2023–2025), multimodal detection (60 papers), and MLLM-based explainable detection—have expanded alongside advances in generative AI, consistent with the reactive dynamic discussed in §3.3.

These trends carry implications for both researchers and practitioners. The era of training Xception on FaceForensics++ and declaring state-of-the-art performance is ending. The field is increasingly exploring foundation-model-based detectors that operate across modalities, adapt with limited supervision, and can provide natural-language explanations of detected artifacts. This trajectory is driven equally by the increasing sophistication of generative models—which demand broader, more generalizable detection—and by the availability of powerful pre-trained representations that make such generalization architecturally feasible for the first time.

RQ1 answer and implication. Deepfake detection has crossed from an architecture-search problem into a representation-and-evidence problem. The CNN-to-transformer-to-foundation-model trajectory shows genuine technical progress, but the decisive question is no longer “which backbone wins on FF++?” It is whether a representation remains calibrated under unseen generators, modalities, compression regimes, and deployment distributions. Field-level verdict: a detector paper that does not test cross-generator transfer, robustness, and reproducible ablations is no longer a complete methodological contribution, regardless of its in-domain benchmark accuracy [20,76,77,122].

3.2. RQ2: Benchmark Datasets and Resource Taxonomy

The evaluation of deepfake detection methods depends critically on the resources against which detectors are trained and tested. Our RQ2 audit shows that the literature does not contain a single homogeneous “dataset” landscape. Instead, papers refer to at least three distinct resource types: (i) deepfake benchmark families used for evaluation, (ii) source, pretraining, or general-purpose datasets used to construct training data or initialize models, and (iii) generator models or commercial generation services used to synthesize fake samples. We therefore answer RQ2 as a resource taxonomy rather than a flat benchmark ranking.

The empirical base consists of 1,108 papers (78.6% of the corpus) that report explicit dataset or resource usage. The initial RQ2 extraction produced 926 distinct surface resource labels before final consolidation. These labels included alias variants and resource-type contamination. We therefore report paper-family mentions: repeated mentions of the same canonical family within one paper are collapsed to one use, while a paper may contribute to multiple resource families. These counts are not bibliographic citations to dataset papers. This distinction matters: “FF++,” “FaceForensics++,” and “FaceForensics++ (FF++)” should count toward the same benchmark family, while ImageNet and COCO should not be counted as deepfake benchmarks, and Stable Diffusion, Midjourney, Sora, StyleGAN, and ProGAN should be treated as generator resources rather than datasets.

3.2.1. Deepfake Benchmark Families vs. Other Resources

Figure 6 and Table 7 present the cleaned resource taxonomy. Among true deepfake detection benchmark families, FaceForensics++ remains the dominant resource with 330 normalized mentions after merging aliases such as FF++, FF+ (FaceForensics++), and FaceForensics++ (FF++) [79]. Celeb-DF follows with 267 normalized mentions after merging variants such as CelebDF, Celeb-DF v2, and Celeb-DF-v2 [123]. DFDC ranks third with 231 normalized mentions [124]. The next tier includes DeeperForensics (60, merging DeeperForensics and DeeperForensics-1.0), WildDeepfake (41), DFD/Google DFD (27), GenImage (24), and UADFV (18).

The audit also shows why a flat “benchmark dataset” title is misleading. Several highly frequent resources are not deepfake detection benchmarks. ImageNet, COCO/MS-COCO, CelebA, LFW, LAION, FFHQ, MORPH, IMDB-WIKI, UTKFace, and CIFAR-10 are better understood as source datasets, pretraining datasets, or general-purpose visual/face datasets [125,126,127,128]. They may be important for model pretraining, real-image sourcing, face attribute supervision, or negative-class construction, but they should not be used to support claims about the popularity of deepfake detection benchmarks. Conversely, Stable Diffusion, Midjourney, Sora, StyleGAN, ProGAN, DALL-E, GLIDE, and related entries denote generator models, generator families, or generation services [75,129,130]. They are threat sources or synthesis mechanisms, not evaluation datasets.

This resource-aware taxonomy changes the interpretation of the benchmark landscape. The strongest claim supported by the cleaned audit is not that the field has approximately 900 clean benchmark datasets. Rather, the field has a large and noisy long tail of resource labels, while the largest paper-family mention counts cluster around a much smaller set of benchmark families. We do not report a concentration index because the full denominator of audited benchmark-family mentions has not yet been published. This distinction is essential for reproducibility: a paper that trains on ImageNet and evaluates on FaceForensics++ is not using two deepfake benchmarks; it is using a source/pretraining dataset and a deepfake benchmark. Similarly, a paper that evaluates fakes generated by Stable Diffusion is referencing a generator family unless it names a fixed dataset constructed from those generations.

3.2.2. Dataset Characteristics and Modality Coverage

The modality distribution across the 1,108 papers with dataset information reveals the continued primacy of visual evaluation (Figure 7). Because modality is multi-label, this distribution is shown as a bar chart rather than a pie chart. Image-based datasets dominate, appearing in 71.1% of papers. Video datasets appear in 33.0% of papers, reflecting the prevalence of face-swap and reenactment benchmarks. Multimodal datasets account for 11.2%, while text-associated resources appear in 10.4%—including text inputs, prompts, or text-evaluation resources linked to visual-generation studies; this category should not be read as a pure text-output benchmark share. Audio datasets appear in just 5.9% of papers, despite the growing threat of synchronized audio-visual manipulations. Papers may report multiple modalities, so percentages exceed 100%.

The manipulation types covered by popular benchmarks cluster heavily around face swapping and face reenactment—the categories present in FaceForensics++. Face-swap methods are represented in virtually all widely used video benchmarks, while face reenactment is also well covered through FaceForensics++ manipulation types. However, newer manipulation paradigms—diffusion-based face editing, text-guided manipulation, fully synthetic image generation, and full-body synthesis—are represented primarily in recently introduced resources such as GenImage [113], DF40 [43], digital-human benchmarks [131], and newer video-generation benchmarks. The benchmark ecosystem is only beginning to catch up with the latest generation of manipulations.

3.2.3. Dataset Growth and Novelty Claims

The number of dataset/resource references per year has grown rapidly, mirroring the field’s overall expansion. From 55 total references in 2018, the count rose steadily through 355 in 2021 and 432 in 2022, then accelerated to 477 in 2023, 879 in 2024, and 1,320 in 2025. We do not visualize the January 2026 partial-year slice as a separate annual trend point because it would compress the apparent trajectory and distort the visual interpretation. The growth through 2025 reflects both increasing specialization of evaluation needs and the recognition that older benchmarks inadequately cover emerging manipulation types.

The automatically extracted novelty field should be interpreted as an audit queue rather than as a final count of newly introduced datasets. Manual inspection shows that this field still contains marker tokens, placeholders, and descriptive fragments such as “[NEW],” “new_dataset,” “Custom,” and sentence-level descriptions containing sample counts. These records are useful for identifying candidate new resources, but they are not yet a publication-ready registry. Until each candidate entry is manually validated against the paper title, release artifact, modality, collection protocol, and availability status, the strongest defensible claim is that many papers introduce specialized resources, while the exact number of genuinely new datasets remains under curation.

3.2.4. Benchmark Staleness and Alias-Aware Interpretation

The most important substantive finding remains benchmark staleness, and the alias-normalized counts sharpen the point. The three most widely used deepfake benchmark families—FaceForensics++ (2019), Celeb-DF (introduced in 2019 through its preprint and published at CVPR 2020), and DFDC (2020)—were all created before the diffusion model revolution, before text-to-image generation became mainstream, and before multimodal deepfakes emerged as a credible threat. Celeb-DF versioning should be handled explicitly: the family label should be used for aggregate analysis, while papers relying specifically on the expanded Celeb-DF(v2) release should name v2 rather than treating all Celeb-DF mentions as identical.

Year-by-year popularity data confirms the inertia. In 2025, FaceForensics++ remained the single most common benchmark family, followed by Celeb-DF and DFDC. GenImage, the highest-ranked diffusion-era benchmark in the cleaned benchmark-family table, remains far below the leading GAN-era benchmarks with 24 normalized mentions. The marginal distributions reveal a clear evaluation risk: diffusion-focused detector research is expanding while GAN-era, face-centric benchmarks still dominate overall usage. Establishing how often the same paper pairs a diffusion-focused method with a GAN-era benchmark requires a method-family-by-benchmark cross-tab and is not inferred from the marginal counts alone.

This staleness has measurable consequences for the reliability of reported results. As discussed in §3.4, methods achieving near-perfect accuracy on FaceForensics++ and Celeb-DF frequently exhibit substantial degradation when evaluated on cross-dataset or cross-generator protocols. Benchmark concentration creates a risk of inflated apparent performance and can obscure the generalization failures that matter most for real-world deployment.

3.2.5. Coverage Gaps and Limitations

Despite the large number of resource labels, several critical gaps persist in the benchmarking landscape:

Diffusion-generated content. GAN-based manipulations still dominate the most widely used benchmarks, while diffusion-based methods now produce a substantial share of newly created synthetic media. Dedicated diffusion forensics benchmarks are emerging, but GenImage’s 24 normalized mentions remain far below FaceForensics++’s 330.

Audio-visual deepfakes. Audio modality appears in only 5.9% of papers with dataset information, despite the growing threat of synchronized audio-visual manipulations in political disinformation and financial fraud. FakeAVCeleb and DeepSpeak represent notable efforts, but multimodal benchmarking infrastructure remains underdeveloped.

Social media degradation. Most benchmarks contain high-quality or lightly compressed content, yet real-world deepfakes are shared through platforms that introduce substantial compression, resolution reduction, and re-encoding artifacts. Only a small fraction of studies explicitly evaluate robustness to platform-specific degradations (see §3.4).

Demographic diversity. Many popular benchmarks draw from celebrity-focused or regionally concentrated data sources. Few benchmarks systematically evaluate detection performance across demographic groups such as age, ethnicity, and skin tone, raising fairness concerns for deployed systems.

Full-body and scene-level manipulation. Face-centric benchmarks dominate, yet generative AI increasingly enables full-body synthesis, scene composition, and object-level manipulation. Digital-human and newer manipulation benchmarks target full-body forgery and more general image editing, but comprehensive full-scene benchmarking remains an open challenge.

In summary, the deepfake detection community has built a large but structurally imbalanced resource ecosystem. A flat dataset list obscures the distinction between deepfake benchmarks, source/pretraining datasets, and generator resources. After alias-aware cleanup, the leading mention counts are dominated by a few aging, GAN-era, face-centric benchmark families, while diffusion-era, multimodal, full-body, and real-world degradation benchmarks remain underrepresented. Until the field’s evaluation infrastructure catches up with its generative frontier, reported detection accuracies remain at risk of overestimating real-world performance.

RQ2 answer and implication. The field is resource-rich but benchmark-poor. A flat count of dataset names exaggerates diversity because it conflates benchmarks, source/pretraining datasets, and generators; after alias-aware cleanup, evaluation still depends heavily on a few aging face-centric benchmark families. Field-level verdict: benchmark staleness is the silent confounder of deepfake detection. The next high-impact contribution is as likely to be a living, diffusion-era, multimodal benchmark suite as another detector architecture, because benchmark design now determines what the field can validly claim to know [79,122,123,124].

3.3. RQ3: Generator-Detector Arms Race

The adversarial co-evolution of generative and detection technologies fundamentally shapes the deepfake detection landscape. As generators produce increasingly realistic synthetic media, detectors must continuously adapt, creating a dynamic arms race that defines the trajectory of both fields. Our analysis reveals the structural dimensions of this interplay: how the research community allocates effort across generation, detection, and analysis roles; which generation methods receive the most defensive attention; and what strategies have emerged to bridge the widening gap between known and unknown manipulation techniques.

3.3.1. Role Distribution

Detection research continues to dominate the field, though with a more nuanced distribution than simple detector-vs-generator framing suggests. Of the 1,410 papers analyzed, 647 (45.9%) propose or evaluate detection methods [37,38,56,61,132], while 211 (15.0%) address both generation and detection simultaneously [118,119,133,134,135]. Analytical and survey contributions account for 158 papers (11.2%) [32,55,136,137,138], generation-focused papers total 144 (10.2%) [139,140,141,142,143], 16 survey papers (1.1%) provide comprehensive field overviews, and 234 papers (16.6%) fall into other specialized categories including benchmark creation, dataset construction, and application-specific studies.

The detection-only to generation-only ratio is approximately 4.5:1 (647/144); it excludes the 211 dual-role papers and therefore describes role labels rather than the total amount of detection activity. This ratio reflects the defensive posture of the research community: the field is primarily reactive, developing countermeasures in response to generative advances rather than proactively anticipating new threats. However, the 211 dual-role papers are particularly significant and represent one of the most notable trends in the data. These papers often reveal architectural insights where understanding the generator informs better detection—for example, using generative model internals to identify reconstruction artifacts, or training detectors by simultaneously learning to generate and distinguish forgeries [118,135,144,145,146].

Annual counts reveal evolving research priorities. In 2021, the ratio of detector to generator papers was 6.0:1 (66 detectors vs. 11 generators). By 2025, this ratio narrowed to 5.3:1 (235 detectors vs. 44 generators), coinciding with a larger absolute generation-focused literature and a broader commercial generator ecosystem. The most dramatic temporal trend is the rise of dual-role papers: from 13 in 2021 to 6 in 2022, then to 26 in 2023, 72 in 2024, and 84 in 2025. The series documents a six-fold increase from 2021 to 2025, but the annual counts alone do not explain the 2022 dip. Analysis papers similarly expanded from 13 in 2021 to 82 in 2025, reflecting the field’s increasing maturity and the growing need for systematic evaluation of the detection landscape.

The annual share of generation-focused research is highest in the printed series in 2022 at 28 papers (19.6% of that year’s output), whereas the absolute count reaches its maximum at 44 papers in 2025. This pattern temporally aligns with technology-release cycles: the high 2022 share coincides with the public releases of Stable Diffusion, DALL-E 2, and Midjourney. The descriptive counts do not establish that those releases caused the annual composition change.

3.3.2. Generation Methods Under Study

Understanding which generation methods are most commonly addressed by detection research reveals the threat landscape as perceived by the community. After normalizing over 300 distinct generation method labels extracted from the corpus, we identify several major categories with significant variation in research attention (Table 9).

Diffusion models have overtaken GANs as the most frequently addressed generation paradigm. After normalization, diffusion-related methods appear in approximately 330 papers [56,90,91,132,147], encompassing unconditional diffusion models (DDPMs), text-to-image pipelines such as Stable Diffusion [148,149,150], Midjourney [33,118,151], and DALL-E [103,152]. The temporal trajectory is dramatic: diffusion-targeted papers grew from near zero before 2022 to 5 in 2022, 32 in 2023, 88 in 2024, and 149 in 2025 (counting only the “Diffusion” label; the normalized total including “diffusion models,” “stable diffusion,” and related labels is substantially higher). This surge illustrates the arms-race dynamic in the publication record: detection-oriented attention expanded rapidly after diffusion models became a major generative paradigm.

Generative Adversarial Networks (GANs) remain the second most frequently targeted paradigm at approximately 297 papers after normalization [22,54,66,153,154]. GAN-based detection spans specific architectures including StyleGAN (40 papers) [70,71,155], ProGAN (21 papers) [156,157], StarGAN (12 papers), and BigGAN (12 papers), as well as general GAN detection frameworks [70,158,159]. The persistence of GAN-focused research, despite diffusion models receiving more attention since 2024, reflects both the maturity of GAN-based deepfake toolkits and the extensive benchmark infrastructure—particularly FaceForensics++, Celeb-DF, and DFDC—built around GAN-generated content.

The diffusion-GAN transition is the defining structural shift of the arms race. In 2021, GAN-related papers outnumbered diffusion papers 40 to 1. In 2023, diffusion first exceeded GAN in the raw annual labels (32 vs. 27); the lead became decisive in 2024 (88 vs. 67) and widened further in 2025 (149 vs. 98). This descriptive alignment shows a rapid pivot after the public diffusion wave, but annual counts alone do not establish a fixed causal lag.

Table 8 quantifies this lag for major generator families and tools, mapping each public release or public announcement year to the first detection paper targeting it in our corpus. Counts in this table come from the RQ3 generator-method field and therefore need not match the RQ2 resource-taxonomy counts, which measure dataset/resource mentions. The pattern is consistent: GAN-based generators (ProGAN, StyleGAN, StarGAN) were targeted by detection papers within 1–3 years of release, while diffusion-based and commercial text-to-image systems (Stable Diffusion, Midjourney) attracted detection papers within roughly one publication year in this corpus, consistent with faster observed responsiveness. Sora, publicly announced by OpenAI in early 2024 as a text-to-video model [160], already appears in detection papers within the following calendar year, showing that Sora-targeted work appeared quickly in the observed publication record; the annual resolution does not establish a precise lag mechanism. The paper counts further reveal where research attention concentrates: Stable Diffusion (47 papers) and StyleGAN (40 papers) receive the largest coded totals within their respective paradigms, while tools such as SimSwap (2 papers) and Sora (4 papers) receive substantially fewer mentions in this extraction.

Face swapping is addressed in approximately 147 papers after normalization (combining Face_Swap, FaceSwap, and related labels), making it the most studied face-specific manipulation [22,35,86,109,161]. Detection methods target classic encoder-decoder architectures such as FaceSwap and DeepFaceLab [162,163,164] as well as more recent approaches including FaceShifter [94,165] and SimSwap. The sustained attention to face swapping reflects its prominence in real-world malicious applications, from non-consensual media to identity fraud.

Face reenactment, studied in approximately 54 papers after normalization, addresses methods that transfer expressions, head poses, or full facial motion from a source to a target [22,56,109,166,167]. Classic methods such as Face2Face (43 papers after normalization) [165,168,169,170] and NeuralTextures (32 papers) [74,165,169] remain benchmark staples via the FaceForensics++ dataset, while newer audio-driven reenactment approaches [38,171,172] present emerging challenges for detection.

Text-to-image generation is addressed in 36 papers [22,69,83,109,173], with growth concentrated in 2024–2025 (13 and 24 papers respectively). This category reflects the growing threat of fully synthetic images generated from text prompts and is likely to remain important as commercial text-to-image and text-to-video services become more accessible. Related emerging paradigms include autoregressive models (18 papers), text-to-video generation (including Sora, Pika, and CogVideo), and video diffusion models.

VAE and autoencoder-based methods appear in 22 papers [171,173,174,175], representing variational autoencoders and related architectures. Additional categories include voice cloning and audio deepfakes (24 papers) [22,109,173,176], lip-sync manipulation, and emerging paradigms such as 3D Gaussian splatting and neural radiance fields. The long tail of generation methods—over 300 distinct labels identified—underscores the combinatorial challenge facing detection: any new generation technique potentially renders existing detectors ineffective.

Figure 8. Normalized generator families addressed by detection papers. Diffusion models (∼330 papers after alias normalization) have overtaken GAN-family methods (∼297) as the most frequently targeted paradigm, completing a transition that began in 2022.

Figure 8. Normalized generator families addressed by detection papers. Diffusion models (∼330 papers after alias normalization) have overtaken GAN-family methods (∼297) as the most frequently targeted paradigm, completing a transition that began in 2022.

Table 9. Generation method families addressed by detection papers after alias normalization. Counts are normalized paper-level mentions of generator families or manipulation families; categories are multi-label and therefore should not be summed as a partition of the corpus.

Normalized family	Papers
Diffusion family	∼330
GAN family	∼297
Face-swap family	∼147
Face-reenactment family	∼54
Text-to-image	36
Voice/audio cloning	24
VAE/autoencoder	22
Autoregressive	18

Table 9. Generation method families addressed by detection papers after alias normalization. Counts are normalized paper-level mentions of generator families or manipulation families; categories are multi-label and therefore should not be summed as a partition of the corpus.

Normalized family	Papers
Diffusion family	∼330
GAN family	∼297
Face-swap family	∼147
Face-reenactment family	∼54
Text-to-image	36
Voice/audio cloning	24
VAE/autoencoder	22
Autoregressive	18

To contextualize the generation method categories above, Table 10 catalogs the specific tools and models that have driven the arms race, tracing the progression from early graphics-based face replacement (FaceSwap, 2018) through GAN-era deepfake toolkits (DeepFaceLab, StyleGAN, SimSwap) to the current diffusion paradigm (Stable Diffusion, DALL-E, Midjourney) and emerging text-to-video platforms (SORA, Kling). This chronological view makes explicit how each successive generation of tools has expanded the threat surface—from face-specific manipulations to arbitrary image synthesis to fully generated video—and why detector generalization across these paradigms remains the field’s central challenge.

3.3.3. Detection Adaptation and Specialization

The detector side of the arms race reveals diverse problem formulations beyond simple binary classification (Table 11). Binary real-vs-fake classification remains the most common formulation at 499 papers (69.6% of the 717-paper subset with explicit detection-formulation labels) [32,37,38,56,132]. However, more nuanced approaches are gaining traction:

Forgery localization (64 papers, 8.9%) seeks to identify which regions of an image have been manipulated [34,47,58,95,136]. This formulation provides substantially more actionable information than binary classification, enabling forensic analysts to understand the extent and nature of manipulation.

Source attribution (37 papers, 5.2%) aims to identify which generator produced a given fake [31,33,64,89,114]. Attribution is critical for accountability and legal proceedings, where establishing the provenance of synthetic content can determine liability.

Multi-class classification (17 papers, 2.4%) distinguishes between manipulation types [35,50,118,177], enabling automated triage systems that categorize detected forgeries by technique. Additionally, active detection (4 papers) and passive detection (6 papers) represent contrasting paradigms: active methods embed proactive signals (watermarks, fingerprints) during content creation [178,179,180], while passive methods analyze content without prior modification [110,181,182].

The evolution from binary classification toward localization, attribution, and multi-class detection represents a maturation of the detection problem. As generators diversify, simply detecting “fake” is insufficient—practitioners need to know what was faked, where, and by which tool. This shift parallels trends in malware analysis, where attribution has become as important as detection, and in content moderation, where platform policies require different responses to different types of synthetic content. Table 11 summarizes these detector problem formulations without plotting a partial-year trend.

3.3.4. Generalization Strategies

Perhaps the most critical dimension of the arms race is the question of generalization: can a detector trained on known generators effectively detect content from unseen generators? Our analysis reveals seven distinct generalization strategies, with cross-dataset evaluation the most common and foundation-model adaptation a prominent late-period strategy (Figure 9).

Cross-dataset evaluation represents the most common generalization strategy, employed in 167 papers (combining “cross_dataset” and “cross-dataset” labels) [29,38,56,86,174]. This approach tests whether models trained on one collection of manipulations can detect manipulations from entirely different sources, serving as a proxy for real-world deployment where training and test distributions inevitably differ. The prevalence of cross-dataset evaluation reflects the community’s recognition that in-distribution accuracy, while necessary, is insufficient for practical deployment.

Cross-generator training appears in 64 papers that explicitly train on multiple generator types to improve robustness [183,184,185,186,187]. By exposing detectors to diverse generation methods during training, these approaches aim to learn generator-agnostic forgery signatures—such as blending artifacts, frequency-domain anomalies, or inconsistencies in image statistics—that transfer to unseen architectures.

Foundation-model adaptation is employed in 49 papers and is concentrated in the late-period literature [27,32,38,54,56]. These approaches leverage large-scale pretrained representations—most commonly CLIP, DINOv2, and various Vision Transformers—as feature backbones that encode rich semantic and perceptual information generalizable across generator types. The underlying hypothesis is that foundation models, trained on diverse natural image distributions, encode a sufficiently general notion of “realness” that deviations—regardless of which generator produced them—become detectable. This strategy connects directly to the foundation model paradigm documented in §3.1, where CLIP has become the most frequently used backbone overall.

Domain adaptation (34 papers) explicitly addresses the distribution shift between training and deployment conditions [28,33,55,90,132]. Methods include adversarial domain alignment, feature disentanglement to separate forgery-specific from domain-specific features [167,188], and test-time adaptation that adjusts model parameters at inference [189,190,191].

Cross-method evaluation (23 papers) tests detectors against manipulation types not seen during training, assessing fundamental robustness to novel forgery techniques. Few-shot learning (20 papers) tackles the challenge of adapting to new manipulation types with minimal labeled examples [49,63,192,193,194], a strategy particularly relevant as new generators emerge faster than large-scale labeled datasets can be collected. Continual learning (12 papers) addresses the temporal dimension of the arms race, developing methods that incrementally adapt to new generator types without catastrophic forgetting of older ones [195,196,197].

The diversity of generalization strategies—with cross-dataset evaluation, cross-generator training, and foundation-model adaptation yielding 280 strategy assignments in total, with possible paper overlap—shows sustained attention to cross-generator generalization as a pivotal technical challenge. The shift toward foundation model-based approaches represents a paradigmatic change from learning “what fakes look like” (generator-specific signatures) to learning “what reality looks like” (deviations from learned natural image distributions), with the latter offering inherently greater robustness to unseen generators [198,199].

3.3.5. The Arms Race Dynamic

Taken together, the role distribution, generation method trends, and generalization strategies reveal several structural features of the arms race with direct implications for the field’s trajectory.

Diffusion has become the leading generator family in the coded research attention. The rise of diffusion models coincided with the largest observed reallocation of detector research in this corpus. Prior to 2022, nearly all detection research targeted GAN-based content. By 2024–2025, diffusion-targeted papers overtook GAN-focused work in annual output, and the normalized totals (diffusion ∼330 vs. GAN ∼297) confirm that the transition has crossed the tipping point. Critically, early studies demonstrated that GAN-trained detectors generalize poorly to diffusion-generated content [191,200,201], validating the arms race concern and providing the empirical motivation for generator-agnostic detection strategies. The crossover unfolded across roughly three publication years—from the first diffusion-detection papers in 2022 to diffusion-leading annual counts by 2025—providing a descriptive measure of how quickly research attention shifted in this corpus.

Generalization has become the central challenge. The arms race analysis in this section converges with the robustness findings in §3.4 and the open challenges documented in §3.10: generalization is simultaneously the most studied strategy (280+ papers on cross-dataset, cross-generator, and foundation model approaches), the most frequently mentioned challenge (the top concern in §3.10), and the area where the widest gap between research attention and practical performance persists. The RQ3 coding records widespread acknowledgement of the arms-race and adaptivity problem, but the robustness evidence in §3.4 shows that claimed adaptability often exceeds demonstrated robustness.

The long tail of generators continues to expand. Our identification of over 300 distinct generation method labels highlights a combinatorial challenge: as generation becomes democratized through open-source tools (Stable Diffusion, FLUX), commercial APIs (Midjourney, DALL-E, Sora, Pika), and closed-source platforms (GPT-4o image generation), the number of distinct generator fingerprints that detectors must handle continues to grow as tools and model families proliferate. This long-tailed distribution structurally favors detection approaches that learn generalizable forensic features rather than generator-specific signatures. The emergence of text-to-video generators (Sora, Pika, CogVideo, Kling) in 2024–2025 adds a temporal dimension to this challenge, as video generation introduces new artifact types that image-only detectors cannot address.

Integrated research approaches are ascendant. The dramatic growth of dual-role papers—from 13 in 2021 to 84 in 2025—represents a maturing understanding that effective detection requires deep knowledge of generation processes. These papers often yield the most transferable insights: by studying how generators produce artifacts, researchers identify forensic signals that generalize across generator families. The 15.0% dual-role proportion suggests a shift from viewing generation and detection as separate problems to recognizing them as inherently coupled aspects of a single challenge.

A second-order arms race has emerged. Beyond the primary arms race between generators and detectors, a subset of papers explicitly studies adversarial attacks against detectors [48,202,203] and defenses against adversarial evasion [24,137,204]. This second-order dynamic—where generators are optimized not merely for visual quality but for explicit detector evasion—elevates the arms race from passive co-evolution to active strategic competition. The adversarial dimension is analyzed in detail in §3.8.

The video frontier. While the majority of arms race research has focused on still images, the rapid emergence of high-quality video generation models (Sora, Pika, Runway, Kling, CogVideo) in 2024–2025 signals that the next phase of the arms race will center on video content. Our data shows video as the target media for 269 papers, but most video-targeted detection work addresses face-swapped or reenacted video rather than fully AI-generated video. The gap between video generation capability and video-specific detection represents an emerging vulnerability window that the field has yet to address at scale.

The escalation narrative requires qualification. However, recent empirical evidence suggests the arms race dynamics may be more nuanced than the simple escalation model implies. Yermakov et al. [20] performed the broadest cross-dataset evaluation to date, spanning 14 benchmarks with 5-seed averaging, and found that detection difficulty has not strictly increased over time. Models trained on FaceForensics++ (2019 data containing four manipulation methods) achieve competitive or even superior cross-dataset generalization compared to models trained on newer 2024–2025 datasets. This counterintuitive result suggests that the diversity of manipulation methods in training data may matter more than their temporal recency: FF++ exposes detectors to four distinct forgery pipelines (Deepfakes, Face2Face, FaceSwap, NeuralTextures), providing a richer training signal than newer but less methodologically diverse benchmarks.

This finding reframes the central challenge of the arms race. Rather than an inexorable escalation where each generation of deepfakes is strictly harder to detect than the last, the primary failure mode is cross-paradigm generalization—detectors trained within one generative family (e.g., GANs) fail when confronted with a fundamentally different synthesis approach (e.g., diffusion models), regardless of which is “newer.” The practical implication is that training data diversity across generation paradigms may be a more reliable route to robust detection than continually chasing the latest generator, a conclusion that reinforces the shift toward foundation model-based approaches documented above.

In summary, the generator-detector arms race is characterized by a corpus-level GAN-to-diffusion crossover, a 4.5:1 ratio between detection-only and generation-only role labels that reflects the field’s defensive posture, an increasingly integrated research culture where 15.0% of papers address both sides simultaneously, and a convergence on generalization—including foundation-model-based approaches—as a leading candidate for robustness across a rapidly expanding set of generators. The field’s ability to pivot from GAN-focused to diffusion-focused detection in three years demonstrates adaptive capacity, but the accelerating pace of generative innovation (text-to-video, 3D generation, multimodal synthesis) will test whether this reactive paradigm can scale.

RQ3 answer and implication. The generator-detector arms race is accelerating, but it remains fundamentally reactive. The community can pivot—as shown by the rapid shift from GAN-centered to diffusion-centered detection—yet each pivot leaves a window in which detectors are evaluated on yesterday’s threat model. Field-level verdict: generator-specific detectors are technical debt. The durable research agenda is generator-agnostic representation learning, continuous benchmark refresh, and evaluation protocols that expose cross-paradigm failure before a new generator family becomes widely deployed [20,38,75,191].

3.4. RQ4: Robustness Analysis

A deepfake detector’s ability to generalize beyond its training distribution is a central determinant of real-world utility. Our analysis of all 1,410 papers reveals an evaluation crisis: 1,024 papers (72.6%) contain zero robustness testing beyond in-distribution evaluation on their training dataset. Cross-type evaluation appears in only 143 papers (10.1%), cross-dataset evaluation in 273 (19.4%), compression robustness in 106 (7.5%), and social media robustness in 58 (4.1%). Only 7 papers (0.5%) perform all four robustness tests. The field produces increasingly sophisticated detection methods but fails to evaluate them under the conditions they will encounter in deployment.

3.4.1. The Scale of the Problem

The headline finding demands emphasis: nearly three out of every four deepfake detection papers evaluate their method exclusively on held-out splits of the same dataset used for training. These 1,024 papers train on one benchmark—most commonly FaceForensics++ at quality level c23—test on that benchmark’s test partition, report high accuracy or AUC, and provide no evidence that their method generalizes to different manipulation types, datasets, compression levels, or social media conditions [37,58,61,205,206]. The practical consequence is that published performance metrics for the majority of the literature are optimistically biased and cannot be taken as indicators of deployment readiness.

Among the 386 papers (27.4%) that perform any form of robustness testing, a clear hierarchy of rigor emerges (Figure 10). Cross-dataset evaluation is most common (273 papers, 19.4%), followed by cross-type evaluation (143 papers, 10.1%), compression robustness (106 papers, 7.5%), and social media robustness (58 papers, 4.1%). Combining multiple tests is rare: only 96 papers (6.8%) conduct both cross-type and cross-dataset evaluation, 37 papers (2.6%) add compression testing to those two, and a mere 7 papers (0.5%) perform all four tests [12,191,207,208,209,210,211]. These 7 papers are the only identified works that meet this review’s four-dimension robustness criterion; the criterion is stringent but does not exhaust every possible form of generalization evidence.

3.4.2. Cross-Type Generalization

Cross-type evaluation—training on one set of manipulation methods and testing on held-out methods—provides a direct test of whether a detector has learned manipulation-agnostic forensic features rather than method-specific artifacts. Only 143 papers (10.1%) perform this evaluation [38,51,56,132,136]. The remaining 89.9% of the literature provides no evidence of cross-type generalization, yet many of these papers implicitly or explicitly claim broad detection capability.

Among papers that do test cross-type generalization, evaluation remains centered on the four canonical FaceForensics++ methods—Deepfakes, Face2Face, FaceSwap, and NeuralTextures. After normalizing the 273 unique manipulation type labels reported across the corpus—which include naming variants such as “face swap,” “faceswap,” “face_swap,” and “face-swap” for a single category—we identify approximately 20–30 distinct manipulation categories (Figure 11). The deepfake group (all naming variants combined) appears in 134 papers, face swap in 88, reenactment in 62, full synthesis in 54, and AI-generated images in 41 [104,165,212,213,214]. The most frequent manipulation type co-occurrences—Face2Face with FaceSwap (12 papers), Face2Face with NeuralTextures (11), FaceSwap with NeuralTextures (11)—confirm the FaceForensics++-centric evaluation paradigm. The most common protocol trains on one FF++ method and tests on the remaining three [212,215,216,217,218]. While this provides a standardized comparison point, it tests generalization only within the narrow set of methods included in a single 2019-era benchmark.

Emerging manipulation types remain critically underrepresented. Diffusion-based manipulations—the current frontier of generative media—appear in only 19 papers under various labels [219,220,221,222]. Lip-sync manipulations, increasingly prevalent in real-world deepfakes targeting political figures, appear in fewer than 10 papers [172,223,224,225,226]. Audio-visual manipulations are similarly sparse [227,228,229]. The field evaluates robustness primarily against manipulation types that were state of the art in 2019, not those defining the generative threat frontier at the review cutoff. The BioDeepAV benchmark [16] provides hard empirical evidence of this gap: when detectors trained on standard benchmarks encounter deepfakes generated by diffusion models, NeRFs, or Gaussian Splatting—methods absent from their training distribution—AUC drops exceed 30 percentage points. This out-of-distribution collapse corroborates our finding that 72.6% of papers perform no robustness testing: the methods that are tested under distribution shift reveal catastrophic degradation, suggesting that untested methods may harbor similar or worse vulnerabilities.

3.4.3. Cross-Dataset Evaluation

Cross-dataset evaluation tests whether a detector trained on one dataset generalizes to a different dataset with distinct manipulation methods, identities, capture conditions, and compression pipelines. At 273 papers (19.4%), it is the most commonly adopted robustness test, yet over 80% of the literature still omits it [23,27,38,64,114]. Cross-dataset evaluation is approximately 1.9 times as common as cross-type evaluation, suggesting that researchers more readily recognize dataset-specific biases as confounding factors [13,230,231] than they do manipulation-type overfitting.

FaceForensics++ serves as the dominant training source (appearing in 20 papers as a robustness evaluation anchor), with Celeb-DF (18 papers) and DFDC (16 papers) as the most frequent cross-dataset test targets [22,165,212,232,233]. WildDeepfake [74,234] and DeeperForensics [214,225] provide additional targets with in-the-wild collection, broader capture conditions, perturbations, poses, lighting, and compression profiles than the canonical controlled benchmarks. However, this narrow set of cross-dataset pairs remains concentrated in face-centric collections and may share demographic and processing biases. Truly stringent cross-dataset evaluation should include datasets with diverse demographics, social media processing artifacts, and manipulation techniques produced by generators entirely unseen during training.

3.4.4. Real-World Degradation Robustness

Robustness to post-processing degradations—compression, resolution reduction, and social media transformations—is the most deployment-critical yet least-studied dimension. Only 106 papers (7.5%) test compression robustness [22,38,113,235,236,237], despite the fact that virtually all media shared online undergoes lossy compression. Common protocols involve testing at different JPEG quality factors or comparing performance between FaceForensics++ compression levels c23 and c40 [215,216,238,239,240]. Studies that evaluate compression robustness frequently report substantial performance drops at aggressive compression levels [241,242,243,244,245], confirming that many artifact-based detectors rely on fragile signals destroyed by standard media processing.

Social media platform processing introduces degradations beyond simple compression: recompression, resizing, format conversion, filtering, and re-encoding. Only 58 papers (4.1%) evaluate this dimension [203,208,209,244,246]. Given that social media platforms are a major distribution channel for malicious deepfakes, the fact that over 95% of the literature ignores these conditions represents a fundamental disconnect between research evaluation and deployment reality.

Papers that test compression robustness tend to be more rigorous overall: 56.6% also perform cross-dataset evaluation and 41.5% include cross-type evaluation. By contrast, social media testing is frequently isolated—only 43.1% of social-media-testing papers also evaluate cross-dataset generalization, and only 17.2% combine it with cross-type evaluation. This pattern suggests that social media robustness is treated as a standalone concern rather than being integrated into comprehensive evaluation protocols.

3.4.5. Temporal Trends: Declining Rigor

The year-by-year trajectory of robustness testing reveals a pattern of initial improvement followed by regression. Compression robustness testing peaked at 16.3% of papers in 2021 (21 of 129) and declined to 5.8% in 2025 (31 of 538). Social media robustness testing peaked at 12.6% in 2022 (18 of 143) and dropped to 3.7% in 2025 (20 of 538). Cross-type evaluation emerged at 12.4% in 2021, briefly dipped to 3.5% in 2022, and has stagnated at approximately 10–12% since 2023. Cross-dataset evaluation shows the most variation, ranging from 14.5% in 2024 to 26.4% in 2021, with a partial recovery to 20.6% in 2025.

The field briefly prioritized real-world robustness during 2021–2022, coinciding with growing public awareness of deepfake threats and early deployment efforts. The subsequent decline correlates with the shift toward diffusion-model detection and novel architectures (transformers, foundation models) as described in §3.1. The publication pattern suggests an incentive structure that rewards architectural novelty more consistently than evaluation rigor: a new detector with in-distribution results on a single benchmark is easier to position as a contribution than a thorough robustness analysis of existing methods. The result is a growing body of literature whose practical relevance is increasingly uncertain.

3.4.6. Gold-Standard Papers

Only 7 papers in the entire corpus perform all four robustness evaluations—cross-type, cross-dataset, compression, and social media—spanning from 2021 to 2025: ForgeryNet [207], GLFF [208], the IEEE VIP Cup analysis [191], a comprehensive benchmark survey [12], CLUE [209], a study on deepfake detectors against face enhancement [210], and SocialDF [211]. Three of the seven are benchmark or survey papers rather than method papers, indicating that comprehensive robustness evaluation is more common in works designed to assess the field than in works designed to advance it. This asymmetry underscores the awareness-action gap: the community recognizes that robustness testing matters, but the publication incentive structure does not enforce it.

Figure 12. Modality distribution across papers performing robustness evaluations. Image-only studies dominate (42.0%), followed by multimodal (32.7%) and video (24.3%), with audio deepfake robustness nearly absent (0.9%).

Figure 12. Modality distribution across papers performing robustness evaluations. Image-only studies dominate (42.0%), followed by multimodal (32.7%) and video (24.3%), with audio deepfake robustness nearly absent (0.9%).

3.4.7. An Evaluation Crisis

The data reveal an unambiguous evaluation crisis. When 72.6% of papers present results with zero cross-domain evaluation, the published performance metrics for the majority of the literature are unreliable indicators of deployment readiness. Methods reporting 95%+ accuracy on FaceForensics++ c23 may—and frequently do—experience catastrophic performance degradation on unseen generators, social media-compressed content, or even the same benchmark at lower quality levels. The declining rates of compression and social media testing (from 16.3% to 5.8% and 12.6% to 3.7%, respectively) mean the field is moving further from deployment-ready evaluation even as practical stakes increase.

Moreover, the crisis extends beyond missing tests to the absence of a coherent multi-dimensional evaluation framework. Nguyen-Le et al. [17] propose four evaluation dimensions for passive detection systems: Generalization (cross-dataset and cross-generator transfer), Robustness (resilience to natural and adversarial perturbations), Attribution (identifying which specific generator produced the content), and Real-world Resilience (handling social media compression, varying input sizes, and temporal delays). Critically, these dimensions are not interchangeable—optimizing for one may harm another. Methods optimized for broad generalization may sacrifice attribution accuracy because they learn modality-agnostic artifacts rather than generator-specific fingerprints. This tension has direct implications for system design: a content moderation platform may prioritize generalization and resilience over attribution, while a forensic laboratory may require the opposite trade-off. Yet the corpus rarely documents evaluation across all four dimensions, making it difficult to assess how a method performs across the full space of deployment requirements.

Four structural reforms are needed. First, cross-dataset evaluation on at least three benchmarks (e.g., FF++, Celeb-DF, DFDC) should be a minimum publication requirement for any paper claiming generalization. Second, compression robustness testing at multiple quality levels should be expected for methods targeting deployment. Third, the community should adopt a standardized manipulation type taxonomy to replace the current naming chaos—273 raw labels for approximately 20–30 distinct categories. Fourth, top venues should incentivize comprehensive robustness evaluation—through dedicated tracks or mandatory supplementary evaluation—to close the gap between the field’s diagnostic awareness and its evaluative practice. Without these reforms, the deepfake detection literature will continue to accumulate impressive-sounding performance numbers that bear little relation to real-world detection capability, perpetuating a crisis of confidence in the field’s reported advances.

RQ4 answer and implication. Robustness is the clearest gap between laboratory success and deployable detection. A method that is not evaluated across datasets, compression levels, social-media transformations, and unseen generator families has demonstrated dataset fit, not forensic reliability. Field-level verdict: robustness testing should become a publication-level requirement for any detector that claims generalization, deployment relevance, or state-of-the-art status; without it, the result is best interpreted as a closed-world benchmark classifier [17,20,122].

3.5. RQ5: Code Availability and Reproducibility

Reproducibility is a cornerstone of scientific progress, yet a persistent gap separates claimed results from independently verifiable implementations across machine learning research. We assess code availability across the 1,410-paper corpus by extracting repository and model-hosting links from full texts and metadata, categorizing hosting platforms, and auditing whether link evidence plausibly corresponds to the paper’s own contribution. The central finding is sharper than a simple code-sharing rate: deepfake detection has a code-link abundance problem, not necessarily a corresponding-code abundance. Raw links are common; independently reusable author implementations are much harder to establish.

3.5.1. From Raw Links to Corresponding-Code Evidence

Of the 1,410 papers in our corpus, 721 (51.1%) include at least one link to a code repository, model hub, or dataset-hosting platform, yielding 1,801 extracted links. The remaining 689 papers (48.9%) provide no detected code or model artifact link. This raw 51.1% rate is useful as a broad openness signal, but it is not a reproducibility rate. The extracted links include authors’ own repositories, third-party baselines, detector libraries, generation toolkits, preprocessing code, dataset pages, and model hubs; all are useful resources, but only the first category directly enables reproduction of a paper’s own method.

Table 12 therefore reports RQ5 as a set of evidence tiers rather than a single binary statistic. Among the 721 link-bearing papers, 355 contain exactly one link. A stricter version of this proxy—exactly one GitHub link and no other platform link—identifies 334 papers, or 23.7% of the full corpus. We retain this 23.7% value as a conservative proxy for likely self-released implementation code, not as a verified count of runnable repositories.

We also inspected the README-based verification output generated by the corpus pipeline. That audit fetched GitHub/HuggingFace README content and counted title-word overlap between each paper title and the linked page. The 70% title-word threshold is not a field-standard cutoff; it was chosen as an intentionally strict, precision-oriented heuristic. Under this threshold, only 18 links from 15 papers were directly title-matched. This number should be interpreted as a high-precision lower bound, not as the true number of papers with corresponding code: repositories may use project acronyms, omit the full paper title, host code without a README, or release under a different repository name; conversely, some README/title matches may still point to model weights or datasets rather than complete training code. The important conclusion is not that exactly 15 papers have code. It is that raw link extraction alone cannot answer the corresponding-code question. The field should stop treating “has a GitHub/HuggingFace URL” as equivalent to “has reproducible author code.”

3.5.2. Platform Distribution

GitHub is overwhelmingly the dominant platform for code-related links. At the detected-link level, 1,545 of 1,801 links (85.8%) point to GitHub repositories, while 256 (14.2%) point to HuggingFace [33,60,247]. We report these as link-level facts rather than standalone reproducibility evidence because platform proportions are not paper-level code-release rates.

At the paper level, 687 of 721 papers with any code/model link (95.3%) include at least one GitHub URL, while 131 (18.2%) include at least one HuggingFace URL. The platform exclusivity breakdown is 590 GitHub-only papers (81.8% of link-bearing papers), 34 HuggingFace-only papers (4.7%), and 97 papers linking to both (13.5%) [109,136,248]. HuggingFace’s growing presence—particularly for pre-trained weights, dataset cards, and model checkpoints—is positive infrastructure, but it also reinforces the distinction between code release, model release, and dataset release.

3.5.3. Temporal Trends

Figure 13 and Table 13 present the evolution of raw code/model-link sharing from 2018 to 2025. The January 2026 partial-year slice is omitted from the trend plot and should not be used to infer annual behavior. Link availability improved substantially in the field’s early years, rising from 18.8% in 2018 (3 of 16 papers) to a peak of 58.2% in 2023 (106 of 182 papers). This trajectory tracks three phases of community maturation.

Phase 1: Low baseline (2018–2019). Fewer than one in four papers contained detected code/model links, reflecting the nascent state of the field and limited reproducibility expectations in early workshop and preprint venues.

Phase 2: Rapid adoption (2020–2023). The link rate jumped from 21.7% in 2019 to 47.8% in 2020 and continued climbing through 51.9% in 2021, 53.8% in 2022, and 58.2% in 2023 [231,249,250]. This inflection coincided with stronger conference reproducibility norms, the growing habit of including GitHub links in submissions, and the maturation of model-hosting infrastructure.

Phase 3: Plateau and decline (2024–2025). The upward trend reversed, with rates falling to 54.0% in 2024 and 49.8% in 2025. This decline likely reflects rapid field expansion into application domains with different artifact-sharing norms, as well as the growing volume of preprints whose code may not yet have been finalized. The practical interpretation is that voluntary norms appear to saturate around half of papers; stronger artifact expectations are needed to move beyond this ceiling.

3.5.4. The GitHub Ecosystem

Analysis of the 1,545 GitHub links reveals 793 unique GitHub users and organizations. The top linked organizations by link count include deepfakes (114 links), marekkowalski (74), nvlabs (38), shaoanlu (24), and iperov (23). These top contributors are predominantly generation tools, face-swapping libraries, or infrastructure repositories rather than detection-paper implementations. Major labs and infrastructure groups are also prominent: facebookresearch (18 links), openai (16), compvis (10), and huggingface (10). This pattern confirms that the code-link ecosystem captured by automatic extraction extends far beyond author implementations.

3.5.5. Implications for the Field

The reproducibility landscape presents three structural problems. First, the 27.4 percentage-point gap between raw link presence (51.1%) and the single-GitHub corresponding-code proxy (23.7%) means that for roughly three-quarters of papers, our conservative proxy does not identify a likely author implementation. Second, the strict README-title audit confirms how weak raw link evidence can be: only 15 papers meet the high-precision title-match criterion, even though 721 papers contain links. Third, link availability is only the first step. Functional reproducibility requires code, checkpoints, dataset versions, split files, preprocessing scripts, dependency specifications, and evaluation commands; a repository without these artifacts cannot support independent verification of SOTA claims.

This finding has direct implications for the evaluation crisis documented in §3.6 and §3.7. When 73.1% of the 1,091 papers with evaluation metadata omit train-test split details and 667 papers (47.3%) explicitly claim SOTA while protocols remain heterogeneous, missing code artifacts become a technical barrier to cumulative science rather than an open-science inconvenience. The field lacks two of the three pillars needed for reliable progress—standardized evaluation and reproducible implementations—leaving method novelty as the dominant basis for comparison.

Notably, the reproducibility crisis extends beyond visual deepfake detection. In the speech deepfake detection domain, Li et al. [15] report that only approximately 10% of published papers provide source code, which is even lower than our 23.7% conservative proxy for likely self-released code availability. This cross-modal corroboration suggests that the reproducibility deficit is a systemic property of the broader deepfake detection ecosystem, not a quirk of visual media.

We recommend that the community adopt stronger reproducibility standards: mandatory code and model release when ethically and legally possible, standardized evaluation benchmarks with fixed preprocessing pipelines, containerized environments for dependency management, and periodic reproducibility challenges that verify claimed results against released implementations. Several exemplary works in our corpus demonstrate that comprehensive artifact release—including training pipelines, pre-trained checkpoints, evaluation scripts, and dependency specifications—is achievable [38,49,62,251,252]. The challenge is making such practices the norm rather than the exception.

RQ5 answer and implication. The reproducibility question cannot be answered by asking whether a paper contains a URL. The defensible answer is tiered: 721 papers contain some code/model link, 334 papers satisfy a conservative single-GitHub proxy for likely self-released implementation code, and only 15 papers pass the strict README/title-match audit available in our pipeline. Field-level verdict: code-link presence is weak evidence; corresponding, runnable, documented author code is the reproducibility unit that matters. A SOTA claim without code, checkpoints, split files, preprocessing scripts, and evaluation commands should be treated as provisional until independently replicated [15,38,122,252].

3.6. RQ6: Evaluation Protocols

Meaningful comparison across deepfake detection methods requires robust and reproducible evaluation protocols. Our analysis of 1,091 papers with detailed evaluation metadata (from the 1,410-paper corpus) reveals a fragmented landscape: 73.1% of the 1,091 papers with evaluation metadata do not specify their data splitting strategy, preprocessing pipelines vary widely with no convergence toward a standard, and statistical significance testing remains vanishingly rare. These deficiencies collectively undermine the comparability and reliability of published results.

3.6.1. Data Splitting Strategies

The central finding is the pervasive absence of documented train-test split protocols. Of the 1,091 papers with evaluation protocol data, 798 (73.1%) do not specify their data splitting strategy [46,51,132,136,205]. Table 14 therefore conditions on the 293 papers that report a split rather than ranking missingness as if it were a protocol. Within this reported subset, custom splits account for 149 papers (50.9%), followed by cross-dataset split protocols at 109 papers (37.2%). Here, “cross-dataset” denotes the declared split/protocol category among papers with an explicit split; it is narrower than RQ7’s 424-paper count of any cross-dataset experiment. Standard benchmark splits account for only 13 papers (4.4% of specified protocols), while k-fold and leave-one-out protocols remain rare [23,33,56,58,65,93,138,153,177,189,212,242,247,253,254,255,256,257,258,259,260,261].

The consequences of this fragmentation are significant. When studies use different subsets of the same dataset for training and testing, reported performance metrics become incomparable. Custom splits—where researchers define their own partitions without adhering to established conventions—create a situation in which even results on the same benchmark may not be directly comparable. Among the few papers that report split ratios, common choices include 80/20, 70/30, and 720/140/140 (the FaceForensics++ convention), but these ratios are explicitly documented in fewer than 30 papers combined.

3.6.2. Preprocessing Pipelines

Preprocessing decisions—particularly face detection, alignment, and resizing—profoundly affect detection performance, yet these choices remain poorly standardized (Figure 14). Reporting completeness and operation choice are separated in the revised presentation: 194 papers (17.8% of the evaluation-metadata subset) do not specify any preprocessing steps [37,57,61,205,206], while Figure 14 ranks only named operations. The most frequently reported operations are frequency-domain transforms (45 papers), face detection (40), augmentation-as-preprocessing (28), resizing (20), normalization (15), and face alignment (10) [58,107,262,263,264]. The long tail of 600+ distinct preprocessing labels—many described only once—reveals extreme fragmentation with no convergence toward a reusable pipeline specification.

Face detection serves as the critical first step for face-centric deepfake detection, yet detector reporting remains weak (Figure 15). In the evaluation-metadata checkpoint, 925 papers are marked N/A for a dedicated face detector, a composite category that may indicate non-face-centric evaluation, no detector stage, or insufficient evidence that a detector was used. It is therefore not interpreted as detector non-reporting. Among the remaining 166 papers for which the face-detector field was applicable, 120 (72.3%) do not name the detector; this conditional rate should not be generalized to the full corpus. After alias normalization, MTCNN is the most commonly named detector (16 papers) [212,214,262,264,265], followed by dlib (12 papers) [56,263,266,267,268] and RetinaFace (5 papers) [131,269,270,271]. The remaining 13 papers name isolated detectors or toolchains such as S3FD [272], DSFD [169], YOLOv8 [273], MediaPipe [274], BlazeFace [258], and Mask R-CNN [275]. This heterogeneity is not cosmetic: face detector quality directly affects downstream detection through bounding-box precision, landmark alignment, failure rates on extreme poses, and sensitivity to occlusion.

Resizing conventions also vary widely, with target resolutions of 224×224 [165,269,276,277], 256×256 [278], 299×299, and 384×384 all appearing across studies. These resolution differences can meaningfully affect detection accuracy, particularly for methods that rely on fine-grained spatial or frequency artifacts that are resolution-dependent.

3.6.3. Evaluation Granularity

The granularity at which predictions are made and evaluated varies considerably across the literature (Figure 16). Image-level evaluation is the most prevalent, employed by 565 papers (51.8%), encompassing both full-image and face-crop predictions for still-image deepfake detection [32,37,56,132,247]. Video-level evaluation appears in 179 papers (16.4%), where frame-level predictions are aggregated—typically via majority voting, averaging, or percentile-based thresholding—to produce a single per-video decision [38,57,62,64,65]. Frame-level evaluation is used in 84 papers (7.7%), assessing each extracted frame independently without temporal aggregation [73,133,173,266,279]. Clip-level [72,280], audio-segment [228,281], and document-level evaluations appear less frequently.

A total of 235 papers (21.5%) do not clearly specify the evaluation level [22,60,61,63,136]. The distinction between frame-level and video-level evaluation is particularly consequential for video deepfake detection: frame-level AUC scores are typically inflated relative to video-level metrics because many frames within a manipulated video are trivially classifiable, while the few challenging boundary frames that determine video-level accuracy are outnumbered. The absence of consistent evaluation granularity makes cross-study comparison of video deepfake detection performance unreliable.

3.6.4. Data Augmentation Practices

Only 229 of 1,410 papers (16.2%) employ data augmentation, with the remaining majority either not using augmentation or not reporting its use [23,52,105,282,283]. The adoption rate fluctuates without a clear upward trend: 21.7% in 2021, 13.3% in 2022, 14.1% in 2024, and 18.2% in 2025, showing no clear upward trend despite growing evidence of its benefits for generalization. Among augmentation strategies, the most frequent types include compression artifacts (11 papers), rotation (10), random cropping (8), JPEG compression (8), horizontal flipping (7), resizing (7), and color jitter (7) [212,284,285,286]. More sophisticated techniques—CutMix, MixUp, frequency-domain augmentation, and adversarial training—appear less frequently.

Augmentation plays a particularly important role in cross-dataset generalization, where training-time augmentations simulating compression, blur, and resolution changes can substantially improve robustness to real-world perturbations [30,152,201,287,288]. Self-blended image augmentation [231] and forgery-aware augmentation strategies have demonstrated significant generalization improvements, yet adoption remains low. The 16.2% adoption rate is consistent with under-use of augmentation as a robustness tool; the present coding does not identify why papers omit or under-report it.

3.6.5. Reporting Standards and Scientific Rigor

The quality of experimental reporting presents the most concerning dimension (Figure 17). Only 47 papers (4.3% of the evaluation corpus) report any form of statistical significance testing [55,60,83,154,289]—confidence intervals, paired t-tests, or Wilcoxon signed-rank tests. The vast majority present performance numbers without any measure of statistical uncertainty, making it impossible to determine whether observed differences between methods are statistically meaningful or artifacts of random variation in data splitting, initialization, or training dynamics.

Reproducibility information—encompassing hyperparameter documentation and detailed experimental setup—is reported by 278 papers (25.5%). Hardware configuration is documented by only 81 papers (7.4%), despite the well-documented sensitivity of deep learning performance to computational resources [60,262,278,279,290]. Although recent papers show modest improvements in descriptive reporting, the aggregate rates remain too low to support statistically reliable meta-analysis: fewer than one in twenty evaluation-metadata papers reports significance testing, and fewer than one in twelve reports hardware.

3.6.6. Implications for the Field

The fragmentation of evaluation protocols creates a systemic barrier to scientific progress in deepfake detection. When 73.1% of the 1,091 papers with evaluation metadata omit their data splitting strategy, 120 of 166 papers with face-detector metadata do not specify the detector, 21.5% do not report evaluation granularity, and statistical significance testing occurs in only 4.3% of papers, the field operates on informal performance comparisons rather than rigorous empirical evidence. Several benchmark efforts—notably FaceForensics++ [291], DeepfakeBench [292], and challenge-based evaluations—have attempted to impose standardization, but our analysis demonstrates that these initiatives have not achieved widespread adoption beyond their immediate user communities.

Crucially, this fragmentation is not merely a theoretical concern—it produces measurably different results for identical methods on identical datasets. The DeepfakeBench standardized benchmark [122] directly compared self-reported results from original papers against results obtained under uniform conditions for the same methods. Nguyen-Le et al. [17] document discrepancies of up to 3 percentage points in AUC between self-reported and standardized results across methods tested on FF++, DFDC, and Celeb-DF. These standardized re-evaluations show that preprocessing, data splits, and implementation choices can materially change reported performance, undermining direct comparison across heterogeneous protocols.

A further under-discussed source of evaluation variability is whether training data is paired (each fake video has its real counterpart from the same source) or unpaired (real and fake videos come from different sources). Yermakov et al. [20] demonstrate that training on paired data yields a 4.7 percentage point improvement in mean AUROC across 14 cross-dataset benchmarks, because paired training forces the model to learn manipulation artifacts rather than superficial differences between unrelated videos. Yet this fundamental design choice is almost never reported or analyzed in detection papers, representing a hidden variable that contributes to the irreproducibility of reported results.

Compounding these issues, the widely used FaceForensics++ validation set is too similar to the FF++ training set to provide meaningful signal for cross-dataset generalization [20]. Consequently, the standard model selection procedure optimizes for in-distribution performance rather than the cross-dataset generalization that deployment requires. Yermakov et al. address this by constructing a custom validation set from held-out portions of diverse datasets, achieving substantially better generalization. This methodological insight is particularly relevant given that 73.1% of the 1,091 papers with evaluation metadata do not specify their split strategy, while the reported subset is itself dominated by custom rather than standardized splits.

The interaction between evaluation protocol fragmentation and the SOTA claims documented in §3.7 is particularly problematic. When 667 papers (47.3%) explicitly claim state-of-the-art results while the corpus exhibits heterogeneous splits, preprocessing, evaluation granularity, and little statistical testing, many headline rankings become operationally non-comparable. The field needs four structural reforms: (1) mandatory reporting of train-test split protocols and dataset versions; (2) standardized preprocessing pipelines with specified face detectors and resolution settings; (3) consistent evaluation granularity with explicit frame-to-video aggregation strategies; and (4) statistical significance testing and confidence interval reporting as a baseline publication requirement. Without such standardization, the deepfake detection literature will continue to grow in volume while its cumulative evidential value remains compromised.

RQ6 answer and implication. Evaluation protocol choice is a hidden variable large enough to invalidate many headline comparisons. Split strategy, face detector, preprocessing resolution, frame-to-video aggregation, and paired versus unpaired training can change results without changing the model. Field-level verdict: unreported protocols make rankings non-commensurable. The minimum credible reporting standard should include dataset version, split file, preprocessing pipeline, face detector, evaluation granularity, and statistical uncertainty for every headline metric [17,20,21,122].

3.7. RQ7: Performance Metrics and Benchmarking

Our analysis of 1,410 papers reveals significant fragmentation in how deepfake detection methods are evaluated, with widespread inconsistencies in metric selection, benchmarking practices, and reporting standards. The three independent corpus-wide rates are revealing: 47.3% of papers claim state-of-the-art performance, 30.1% conduct cross-dataset evaluation, and 19.1% include ablation studies. Because the fields were not conditioned on one another, these marginals do not establish the validation rate within the SOTA-claiming subset; they nevertheless motivate a direct subset audit and raise concerns about comparability across the literature.

3.7.1. Performance Metric Usage

The distribution of performance metrics across the literature reveals both convergence on a small set of core metrics and problematic fragmentation in nomenclature (Figure 18). Accuracy is the most frequently reported metric, appearing in 553 papers (39.2%), followed by AP (Average Precision) in 491 papers (34.8%), AUC (Area Under the ROC Curve) in 216 papers (15.3%), precision in 101 papers (7.2%), F1-score in 95 papers (6.7%), recall in 78 papers (5.5%), and EER (Equal Error Rate) in 47 papers (3.3%) [37,46,52,58,95]. Additional metrics including mAP (32 papers), IoU (30 papers), MAE (23 papers), FPR (12), and TPR (10) serve specialized evaluation needs [28,293,294,295,296].

When examining primary metrics—the main metric used to headline a paper’s best results—accuracy dominates with 388 papers (27.5%), followed by AP at 174 papers (12.3%) and AUC at 137 papers (9.7%) [49,57,185,297,298]. A primary metric could be identified for 771 of the 1,410 papers (54.7%); the remaining papers did not expose a single unambiguous headline metric under our coding. Within the identified subset, this concentration around three metrics provides a degree of consistency, but the underlying nomenclature remains fragmented: accuracy alone appears under at least seven distinct labels (accuracy, ACC, detection accuracy, classification accuracy, average accuracy, overall accuracy, mean accuracy), and AUC appears under five variants. This naming inconsistency complicates automated meta-analyses and systematic comparisons.

Table 15. Performance metric usage across 1,410 deepfake detection papers. “Reported” counts any appearance; “Primary” counts usage as the main headline metric. Metrics are normalized across naming variants (e.g., accuracy subsumes ACC, detection accuracy, classification accuracy).

Metric	Reported		Primary
	Papers	%	Papers	%
Accuracy	553	39.2	388	27.5
AP	491	34.8	174	12.3
AUC	216	15.3	137	9.7
Precision	101	7.2	11	0.8
F1	95	6.7	26	1.8
Recall	78	5.5	1	0.1
EER	47	3.3	13	0.9
mAP	32	2.3	5	0.4
IoU	30	2.1	5	0.4
MAE	23	1.6	10	0.7
FPR	12	0.9	–	–
TPR	10	0.7	1	0.1
Other	37	2.6	–	–

Table 15. Performance metric usage across 1,410 deepfake detection papers. “Reported” counts any appearance; “Primary” counts usage as the main headline metric. Metrics are normalized across naming variants (e.g., accuracy subsumes ACC, detection accuracy, classification accuracy).

Metric	Reported		Primary
	Papers	%	Papers	%
Accuracy	553	39.2	388	27.5
AP	491	34.8	174	12.3
AUC	216	15.3	137	9.7
Precision	101	7.2	11	0.8
F1	95	6.7	26	1.8
Recall	78	5.5	1	0.1
EER	47	3.3	13	0.9
mAP	32	2.3	5	0.4
IoU	30	2.1	5	0.4
MAE	23	1.6	10	0.7
FPR	12	0.9	–	–
TPR	10	0.7	1	0.1
Other	37	2.6	–	–

Accuracy is reported by 39.2% of papers and is the leading primary metric in 27.5%. This reliance is problematic for real-world deployment scenarios, where the ratio of authentic to manipulated content may be heavily skewed. Under highly imbalanced real-world prevalence, a detector biased toward the majority authentic class can achieve deceptively high accuracy while missing many manipulated samples. AUC provides a more threshold-independent assessment but conveys limited information at operationally relevant false positive rates. Only a small fraction of studies report deployment-appropriate metrics, such as TPR at fixed low FPR thresholds [47,109,299] or partial AUC [33]. The field’s reliance on accuracy as a primary yardstick can inflate apparent effectiveness and mask the difficult tradeoffs inherent in operational deployment.

Beyond classification metrics, image quality metrics such as PSNR and SSIM appear in studies evaluating adversarial perturbation methods or generative model quality [300,301], while localization-specific metrics including IoU and pixel-level F1 serve papers addressing manipulation localization [30,302,303]. This diversity of evaluation dimensions—detection, localization, attribution, robustness, and efficiency—further complicates cross-study comparisons and underscores the need for multi-dimensional evaluation standards.

3.7.2. State-of-the-Art Claims

A total of 667 of 1,410 papers (47.3%) explicitly claim state-of-the-art performance [37,38,46,56,57] (Figure 19). When nearly half of all papers claim to be the best, the concept of “state of the art” loses operational meaning. Several structural factors inflate these claims. First, the lack of standardized evaluation protocols (§3.6) allows papers to select favorable benchmark subsets, manipulation types, or compression settings that advantage their method. Second, many studies evaluate on only one or two datasets with a single train-test split, which is insufficient to capture generalization performance. Third, the proliferation of new benchmark datasets means that claiming SOTA on a newly introduced benchmark carries fundamentally different weight than demonstrating improvement on established benchmarks such as FaceForensics++, Celeb-DF, or DFDC [262,266,279,304,305].

The annual SOTA-claim rate is non-monotonic rather than stable. The small 2018–2020 cohorts (N=16, 23, and 23 papers) show approximately 69%, 74%, and 34.8%, respectively. In the larger 2021–2025 cohorts (N=129, 143, 182, 311, and 538), the rates are 52.7%, 53.8%, 37.4%, 43.7%, and 49.4%. The trajectory therefore shows a drop, rebound, and continued fluctuation; annual claim frequency alone cannot establish whether the substantive evidentiary bar for SOTA rose or fell.

3.7.3. Cross-Dataset Generalization

Cross-dataset evaluation—training on one dataset and testing on a different, independently collected dataset—is a central test of a detector’s practical utility. Our analysis finds that 424 of 1,410 papers (30.1%) conduct some form of cross-dataset evaluation [27,38,55,57,205]. The remaining 69.9% do not report cross-dataset evaluation under our coding definition; they should not all be assumed to use in-distribution settings because some papers provide insufficient detail for that classification.

The available year-coded points are incomplete and do not support a causal explanation for short-term dips. The aggregate adoption rate of 30.1% is therefore the primary result; annual values are treated as descriptive rather than as evidence that any specific technology wave changed evaluation practice.

A related concern is the narrow scope of cross-dataset evaluation when performed. The dominant protocol trains on FaceForensics++ and tests on Celeb-DF or DFDC, but these datasets share similar demographic distributions and processing pipelines. Truly challenging cross-dataset evaluation should include datasets with diverse demographics, varying compression levels, social media processing artifacts, and generators unseen during training [20,306,307,308,309]. The connection to §3.4 is direct: only 19.4% of papers perform cross-dataset evaluation in the robustness-specific analysis, with the difference reflecting broader definitions of “cross-dataset” in the evaluation practices context.

3.7.4. Ablation Studies

Beyond cross-dataset evaluation, ablation studies—systematic experiments isolating the contribution of individual components—provide critical insight into why a method works and which design choices matter most. Only 270 of 1,410 papers (19.1%) include ablation studies [33,46,93,205,278]. This rate is concerning given the architectural complexity of modern detectors, which typically combine multiple loss functions, augmentation strategies, feature extraction modules, and fusion mechanisms.

Ablation studies have increased in absolute number as the corpus has grown, but the available year-coded series is incomplete and does not justify a causal account of short-term changes. Among papers with ablations, common experiments remove or replace architectural components, vary loss weights, or compare backbones [34,50,105,114,132]. Ablations of hyperparameter sensitivity, training-data volume, and preprocessing robustness remain comparatively rare despite their importance for reproducibility and deployment.

3.7.5. Evaluation Practices Overview

Figure 20 summarizes the adoption of key evaluation practices across the corpus. The independent corpus-wide rates—47.3% SOTA claims, 30.1% cross-dataset evaluation, 19.1% ablations, and 11.6% real-world evaluation—show that strong progress language is more common than several forms of validation. Because these fields were not conditioned on one another, this comparison motivates a subset analysis rather than proving that every SOTA-claiming paper omits validation. Real-world evaluation—encompassing tests on social media-harvested content, diverse compression conditions, and adversarial scenarios—appears in only 163 papers (11.6%) [29,33,66,278,290]. Computational cost is reported by only 121 papers (8.6%) [27,53,60,139,305], despite its critical importance for deployment. The underreporting of computational efficiency is increasingly consequential as the field shifts toward foundation model-based detectors, which are often substantially heavier than lightweight CNN baselines and may require nontrivial memory, batching, and latency engineering. Real-time content moderation platforms processing large upload volumes cannot adopt methods whose computational costs are undisclosed.

The gap between laboratory and real-world performance is well-documented in the subset of studies that conduct deployment-relevant evaluations. Methods achieving near-perfect AUC on standard benchmarks often experience dramatic degradation when confronted with social media compression, diverse demographics, or unseen manipulation techniques [40,119,171,194,236,310,311,312,313,314]. This performance gap highlights a fundamental tension: optimizing for benchmark performance does not translate to the practical effectiveness that real-world applications demand.

As the field increasingly adopts foundation model-based approaches—which can require substantially more computation than lightweight CNN classifiers (§3.1)—the need for standardized efficiency reporting becomes pressing [20,216,272,315,316,317,318]. High-volume content-moderation platforms cannot responsibly assess deployment feasibility when computational costs are undisclosed, making inference latency and throughput critical reporting dimensions.

3.7.6. Reported Performance Trends Over Time

Extracting self-reported numerical performance values from the 1,131 papers with structured metric data reveals an apparent ceiling pattern in the reported benchmark scores (Figure 21). Median AUC has remained above 95% since 2022, with 90th-percentile values approaching 99%+ in 2024–2025. Median accuracy follows a similar trajectory, exceeding 90% from 2022 onward. This apparent ceiling in heterogeneous, self-reported in-distribution scores stands in stark contrast to the robustness failures documented in §3.4, where some cross-domain studies report degradation above 30 percentage points on unseen generators. The gap between near-ceiling reported performance and poor cross-domain generalization is itself a key finding: optimizing for benchmark AUC has become a poor proxy for real-world detection capability.

Two caveats apply. First, values are self-reported; as documented in §3.6, 73.1% of the 1,091 papers with evaluation metadata omit split specifications, meaning different papers may report headline numbers under incomparably favorable conditions. Second, the composition of the literature changes over time—including a growing foundation-model share—so the aggregate medians mix method progress with shifts in the evaluated paper population.

3.7.7. Implications for Benchmarking Standards

The cumulative picture is one of a field with rapidly advancing methods but insufficiently mature evaluation infrastructure. The fragmentation of metric nomenclature—with over 100 distinct metric names for what reduces to approximately 15 core measurements—complicates meta-analyses and systematic comparisons. Accuracy is the most frequently reported metric (39.2%) and the leading primary metric (27.5%); this reliance is misaligned with deployment reality, where class-imbalanced settings demand threshold-independent or operating-point-specific metrics. The independent rates—47.3% SOTA claims, 30.1% cross-dataset evaluation, and 19.1% ablation studies—show that progress language is more prevalent than several validation practices, but a conditional audit is required to determine how often SOTA-claiming papers include them.

The risk of metric-dependent conclusions is not hypothetical. Yermakov et al. [20] evaluated their GenD detector across 14 benchmarks and reported that GenD (DINO) achieved 91.6% mean AUROC but 49.5% TPR at FPR=1%—an operationally critical threshold where false positives must be minimized, and the relative ranking of competing methods shifts substantially. This metric sensitivity implies that many of the 47.3% of papers claiming state-of-the-art performance may be reporting rankings that hold only under their chosen metric. The community’s predominant reliance on a single headline number—typically AUC or accuracy—provides an incomplete and potentially misleading picture of detector capability, reinforcing the need for multi-metric evaluation as a reporting standard.

We identify four concrete steps toward more rigorous benchmarking. First, the community should adopt standardized reporting templates requiring both threshold-dependent (accuracy, F1) and threshold-independent (AUC, AP) metrics, along with TPR at operationally relevant FPR thresholds and computational cost. Second, cross-dataset evaluation on at least three benchmarks should be considered a minimum requirement, ideally including one dataset with social media processing artifacts and one with generators unseen during training. Third, ablation studies should be expected for any paper introducing architectural novelty, to distinguish genuine contributions from training recipe effects. Fourth, real-world evaluation pipelines—including social media simulation, adversarial robustness testing, and demographic fairness analysis—should be developed as community-standard toolkits, lowering the barrier to deployment-relevant assessment. The interaction between the evaluation protocol gaps documented in §3.6 and the metric fragmentation documented here creates a compounding problem: not only do papers use different metrics, but they compute those metrics under incomparable experimental conditions. Resolving both issues simultaneously is essential for the field to establish a meaningful and reliable measure of progress.

RQ7 answer and implication. The literature is over-supplied with headline scores but under-supplied with comparable evidence. Accuracy and AUC remain useful, but they are insufficient when operating points, false-positive costs, uncertainty, and cross-dataset performance determine deployment value. Field-level verdict: SOTA should be treated as an evidence contract rather than a phrase: fixed protocols, cross-dataset tests, ablations, uncertainty intervals, low-FPR operating points, and efficiency reporting should accompany every serious leaderboard claim [13,20,122].

3.8. RQ8: Adversarial Attacks and Defenses

Adversarial vulnerability of deepfake detectors represents a critical dimension of the detection problem with direct implications for deployment reliability. Our analysis assigns 137 attack-oriented and 390 defense-oriented paper labels across the full corpus. These labels are multi-label and may overlap, so their sum is not a unique-paper total. The 2.84:1 ratio describes the relative volume of defense and attack assignments and reflects growing attention to detector fragility. This section catalogs the attack landscape, taxonomizes defense mechanisms, traces the temporal evolution of the adversarial arms race, and identifies critical gaps in the community’s approach to robustness.

3.8.1. Temporal Evolution of Adversarial Research

Adversarial research in deepfake detection has grown from a niche concern to a central research theme. Figs. Figure 22 and Figure 23 visualize the 2021–2025 annual trajectory; the January 2026 partial-year slice is omitted from trend plots but retained in the aggregate 137 attack-oriented and 390 defense-oriented assignment totals.

Exploratory phase (2018–2020). Only six pre-2021 role assignments were recorded—three attack-oriented and three defense-oriented—reflecting limited adversarial coverage in the early corpus.

Growth phase (2021–2022). The coding contains 102 attack/defense role assignments across 2021–2022 (47 in 2021 and 55 in 2022), with defense-oriented assignments forming the majority. Attack-oriented assignments in 2021 numbered 9 against 38 defense-oriented assignments; by 2022, the corresponding counts were 17 and 38.

Highest attack-focused share (2023). Of 47 attack/defense role assignments in 2023, 24 (51.1%) were attack-oriented and 23 defense-oriented. This is the only year in the plotted series in which attack-oriented assignments slightly exceed defense-oriented assignments; the counts do not establish that the full vulnerability surface had been identified.

Defense-oriented growth (2024–2025). The publication mix shifts sharply toward defense-oriented assignments after 2023. In 2024, 70 defense-oriented and 33 attack-oriented assignments were recorded (32.0% attack-focused). In 2025, the corresponding counts were 203 defense-oriented and 51 attack-oriented assignments (20.1% attack-focused). The 254 role assignments recorded in 2025 are substantially above the annual assignment volume in earlier years, documenting rapid growth in adversarially coded work. The difference between the plotted 2021–2025 series and the full-corpus defense total is the omitted January 2026 partial-year slice.

3.8.2. Taxonomy of Adversarial Attacks

We categorize the 137 attack-oriented papers along two axes: the explicit attacker setting named by the paper and the perturbation strategy defining how adversarial modifications are constructed. The setting labels in Figure 24 should not be interpreted as a mutually exclusive taxonomy: transfer-based evaluation is often used as a practical black-box scenario, whereas adaptive attacks describe whether the attacker is defense-aware.

Attacker-Setting Classification. The most frequently named setting is transfer-based evasion (19 papers), where adversarial examples crafted against a surrogate model are applied to an unknown target detector [140,319,320,321,322]. Transfer attacks reflect a realistic deployment scenario: adversaries rarely have direct access to deployed detection systems but can train local surrogates. Recent work demonstrates that adversarial perturbations transfer effectively not only across CNN-based detectors but also from CNNs to Vision Transformers and CLIP-based architectures [323,324,325], suggesting that current detectors share fundamental vulnerabilities in their learned representations.

Adaptive or defense-aware attacks (17 papers) consider adversaries who dynamically adjust their strategy based on the detector’s behavior [109,142,193,326,327]. These are particularly dangerous because they can circumvent defenses that rely on fixed preprocessing or known attack patterns. The presence of adaptive attack evaluation is a marker of rigorous robustness assessment, yet only 17 of 137 attack-oriented papers (12.4%) include this explicit setting.

White-box access, which assumes full knowledge of the detector’s architecture and parameters, is studied in 14 papers [235,244,328,329,330]. Common white-box methods include Projected Gradient Descent (PGD) [329,331,332], the Fast Gradient Sign Method (FGSM) [205,321,331], and the Carlini & Wagner (C&W) attack [140,333]. While white-box access is the most favorable scenario for attackers and thus yields an upper bound on detector vulnerability, it serves as the standard stress test for evaluating architectural resilience.

Black-box or query access, where the adversary can only query the detector and observe outputs, is investigated in 13 papers [334,335,336,337,338]. Several works propose query-efficient strategies, including NES-based gradient estimation [339,340], genetic algorithm-based optimization [341], and decision-based frequency perturbations [48].

Attack Domain Classification. Orthogonal to the attacker-setting axis, which describes the adversary’s access, knowledge, or adaptation assumption, attacks can be organized by the domain in which perturbations are applied. Khan et al. [18] provide the most granular such taxonomy in the survey literature, identifying five attack domains: (i) pixel-space attacks, which directly perturb pixel values via gradient-based methods such as FGSM, PGD, and C&W—the dominant paradigm in our corpus (59.4% of attack instances); (ii) frequency-domain attacks, which inject perturbations in spectral representations to specifically evade frequency-based detectors [48,323]; (iii) latent-space attacks, which optimize perturbations within the latent representations of generative models, yielding higher transferability across architectures [142,328]; (iv) backdoor attacks, which poison training data with hidden triggers that cause targeted misclassification at inference [254,342]; and (v) watermarking-based attacks, in which adversarial watermarks are embedded during media generation to preemptively evade downstream detection. This domain-based view complements the attacker-setting labels: a single attack method (e.g., PGD) operates in pixel space under several possible access assumptions, whereas backdoor attacks require training-time access regardless of whether the deployed detector is evaluated as white-box or black-box. The distinction is practically important because defenses effective in one domain—such as spectral filtering against pixel-space perturbations—may offer no protection against attacks mounted in a different domain.

Perturbation Strategy Classification. The attack landscape extends well beyond classical ${\ell }_p$-norm bounded perturbations. Our categorization of 180 total attack instances reveals that adversarial perturbations constitute the dominant attack family at 59.4% of all attacks (107 instances), encompassing gradient-based methods (PGD, FGSM, MIM, AutoAttack) and their variants.

Gradient-based perturbations form the core of the adversarial perturbation category. PGD, FGSM, and their momentum-based variants (MI-FGSM, APGD) are the most frequently evaluated attacks [205,321,328,331,333,343]. AutoAttack, which combines multiple strategies, is also represented [333].

Compression and noise attacks account for 10.0% of instances (compression: 5.6%, noise: 4.4%), including JPEG recompression, Gaussian blur, and noise corruption. Although not adversarial in the classical sense, these transformations are frequently encountered in deployment and can substantially degrade detector accuracy [58,344,345].

Frequency-domain attacks represent an emerging class targeting the spectral artifacts upon which many detectors rely. The Frequency-based Post-train Bayesian Attack (FPBA) [205,323] perturbs both spatial and frequency representations, while the Frequency-based Black-box Decision-based Attack (FBA²D) [48] operates entirely in the frequency domain without requiring gradient access.

GAN- and diffusion-based attacks leverage generative models as attack tools. Super-resolution-based attacks apply SR processing to remove forensic traces [156,214], while adversarial camouflage approaches use GANs to induce blending inconsistencies [143]. Latent-space adversarial optimization in diffusion models produces outputs adversarially refined against forensic classifiers [142,328].

Backdoor and poisoning attacks (1.1%) target the training phase rather than inference. Liang et al. [254] evaluate multiple backdoor attack variants (BadNet, Blended, ISSBA, SIG, FTrojan) against deepfake detectors, while other work examines data poisoning [342] and trojan attacks [322] that compromise detector integrity at training time. These attacks are particularly insidious because they can be mounted without any interaction with the deployed system.

Geometric attacks (3.3%) include resizing, rotation, scaling, and cropping, while a small number of social media attacks (0.6%) target platform-specific transcoding artifacts. The low representation of social media attacks is notable given their practical relevance (see also §3.4).

3.8.3. Defense Mechanisms

The 390 defense-oriented papers propose a diverse range of robustness-enhancing mechanisms. Figure 25 visualizes the named defense mechanisms discussed below.

Robust Feature Extraction. The most prevalent defense strategy focuses on designing architectures that learn features inherently resilient to perturbations. Across the 173 named defense-mechanism instances in Figure 25, robust feature extraction accounts for 64 instances (37.0%). Dual-stream architectures that fuse semantic content with forensic residuals [58] provide complementary feature representations harder to simultaneously fool. Frequency-domain defenses leverage DCT or wavelet-based features that capture forensic artifacts in spectral bands less susceptible to pixel-space perturbations [68,235,334]. Graph-based approaches, such as LR-GCN with spectral band-pass filtering [62] and the GRACE framework with graph Laplacian regularization [346], provide robustness through relational modeling that is structurally resistant to localized perturbations.

Detection-Centric Defenses. The largest coded category comprises 66 of 173 mechanism instances (38.2%). These records propose specialized detection architectures designed with adversarial robustness as an explicit objective rather than a post-hoc addition. These include multi-modal fusion detectors that combine pixel-level, structural, and semantic similarity metrics [109]; foundation model-based detectors that leverage pre-trained representations with parameter-efficient fine-tuning [136,174]; and reasoning-based detectors that use multimodal large language models to identify inconsistencies through multi-step analysis [347].

Ensemble Methods and Data Augmentation. Ensemble approaches (11 of 173 mechanism instances, 6.4%) combine multiple detection models to increase the difficulty of constructing transferable adversarial examples [348,349]. Data augmentation strategies (6 of 173 mechanism instances, 3.5%) enhance robustness by exposing detectors to diverse perturbations during training, including forensic-oriented augmentation that simulates realistic post-processing and compression artifacts. Reinforcement learning-based adaptive augmentation [193] represents a recent innovation that dynamically selects the most challenging augmentation strategies during training.

Input Preprocessing and Purification. Preprocessing defenses (10 of 173 mechanism instances, 5.8%) apply transformations to detector inputs to neutralize adversarial perturbations before classification. Diffusion-based purification adds controlled noise and then denoises the image, recovering a cleaned version more likely to be correctly classified [209,333]. The Mask-guided Detection and Reconstruction (MagDR) framework uses adaptive masks to capture local facial region changes and applies hierarchical reconstruction to remove adversarial perturbations [327]. Test-time defenses based on randomized jitters—random resizing, cropping, gamma variation, and JPEG phase shifts—introduce stochasticity that disrupts adversarial patterns [58,68]. A key limitation of preprocessing defenses is their vulnerability to adaptive adversaries who anticipate the defense mechanism and optimize against it.

Proactive and Preemptive Defenses. Rather than defending at detection time, proactive defenses aim to disrupt the deepfake generation process itself. Huang et al. [350] add low-frequency perceptual perturbations to source face images that sabotage face-swapping pipelines. Yam et al. [351] apply latent-space adversarial perturbations to disrupt identity transfer in diffusion-based face-swapping. Neekhara et al. [244] embed semi-fragile neural watermarks robust to benign transformations but fragile to malicious manipulations. The FacePoison framework [352] adds perturbations that sabotage face detection as a preprocessing disruption. Zheng et al. [353] introduce dual-function adversarial perturbations combined with data-source poisoning to disrupt attacker retraining pipelines. These proactive approaches are promising but assume the defender has access to source images before manipulation—an assumption that does not hold for already-circulated content.

Content Provenance and Authentication. A distinct defense category sidesteps adversarial perturbations entirely by establishing verifiable provenance for media content. Digital watermarking [320,354], blockchain-based verification [354,355], and trusted execution environments [356] can provide verifiable provenance under stated trust and deployment assumptions. While these approaches cannot detect deepfakes in already-circulated media, they offer a principled long-term complement to reactive detection. Watermarking/provenance accounts for 5 of 173 mechanism instances (2.9%), reflecting growing but still modest community investment.

3.8.4. The Adversarial Training Paradox

Adversarial training—augmenting training data with adversarially perturbed examples to harden detectors—is widely regarded as the most principled defense paradigm in the adversarial ML literature, yet it appears in only 11 defense-oriented papers (2.8%); these 11 records also represent 6.4% of the 173 coded mechanism instances [193,205,244,331,344]. This underutilization constitutes one of the most significant gaps between the deepfake detection community and the broader adversarial robustness literature.

Variants of adversarial training employed in our corpus include standard PGD-AT and TRADES-based training [333], on-manifold adversarial training (OMAT) that constrains perturbations to the data manifold for improved generalization [40], and red-team training with worst-of-K counter-forensics per batch [58]. Domain-adversarial training, which learns domain-invariant features for cross-dataset generalization, is used by several additional works [355,357,358].

Several plausible hypotheses may contribute to the low adoption rate, but the corpus coding does not test them causally. Adversarial training can substantially increase computational cost for high-resolution images and video, may trade clean accuracy for worst-case robustness, and requires choosing among heterogeneous threat models spanning gradient-based, frequency-domain, generative, and real-world perturbations. These explanations should be tested directly rather than inferred from publication counts.

The few works that do employ adversarial training report a consistent pattern: robustness against trained-upon attacks improves substantially, but generalization to novel attack types remains limited [331,346]. Serrano et al. [205] systematically benchmark 11 adversarial training strategies and find that no single strategy dominates across all attack types and datasets. Zhou et al. [40] propose on-manifold adversarial training (OMAT) that partially mitigates the clean accuracy trade-off by constraining perturbations to realistic data manifolds. These results suggest that adversarial training for deepfake detection requires domain-specific adaptations rather than direct application of techniques from the image classification literature.

The defense-oriented literature instead predominantly pursues architectural modifications (robust features: 64 instances), detection-centric designs (66 instances), or ensemble methods (11 instances)—strategies that improve generalization without explicitly training against adversarial inputs. This pattern is consistent with a community orientation toward clean accuracy preservation over worst-case robustness: most deepfake detectors are developed and evaluated under the assumption that inputs are not adversarially manipulated, which is precisely the assumption an adversary would exploit.

3.8.5. The Attack-Defense Ecosystem

Beyond individual attacks and defenses, our analysis reveals a dynamic adversarial ecosystem with several structural characteristics that differentiate deepfake detection from other adversarial ML domains.

Broad and expanding attack surface. The 137 attack-oriented papers document successful evasion of state-of-the-art detectors using strategies spanning pixel-space perturbations [321,328], frequency-domain manipulation [48,323], generative model refinement [142,214], and backdoor injection [254,342]. Transfer attacks demonstrate that adversarial examples crafted against one detector frequently fool others [325,359,360], indicating that current detectors share common vulnerabilities rather than exhibiting independent failure modes.

Defense breadth without depth. Across the 390 defense-oriented papers, no single defense has demonstrated comprehensive resilience across all attack types. The defense literature is wide but shallow: many different robustness strategies are proposed, each evaluated against a narrow subset of attacks. This fragmentation makes it difficult to identify which defenses provide genuine security improvements versus which merely perform well against the specific attacks the authors chose to evaluate.

Incomplete adversarial evaluation. Many papers report only one or two standard attacks, typically FGSM and PGD, without covering the full threat spectrum. Our current coding establishes that only 17 of 137 attack-oriented papers (12.4%) explicitly use an adaptive or defense-aware attacker setting [109,361]; it does not provide the denominator needed to estimate what fraction of the 390 defense-oriented papers were adaptively evaluated. The low frequency of explicit adaptive settings nonetheless supports treating untested defenses as provisional rather than secure.

The 2023 inflection point. The shift from balanced attack-defense research in 2023 (51.1% attack-focused) to defense-dominated research in 2024–2025 (273 defense vs. 84 attack assignments, 76.5% defense-focused) documents a marked increase in defense-oriented publication activity; publication counts alone do not establish coordination or causality. Whether this defense surge translates to measurably more robust deployed systems remains an open question—one that can only be answered through standardized adversarial benchmarking, which the field currently lacks.

3.8.6. Open Challenges in Adversarial Robustness

Several fundamental challenges remain unaddressed by the current adversarial literature:

Standardized adversarial benchmarks. No universally accepted benchmark suite evaluates detectors against a comprehensive attack portfolio. The field needs an adversarial analogue to FaceForensics++ (§3.2)—a standardized evaluation protocol that tests white-box, black-box, transfer-based, and adaptive attacks against every proposed defense, with fixed perturbation budgets and evaluation metrics. Without such infrastructure, robustness claims across papers remain incomparable.

Certified defenses. Under our coding rule—an explicit, provable robustness guarantee within a stated perturbation set—we did not identify a certified deepfake defense in the corpus. This is an absence under the stated extraction rule, not proof that no related work exists outside the search scope. Randomized smoothing and related certification techniques remain a largely unexplored direction for deepfake detection.

Adaptive attack mandates. Following established best practices in adversarial ML, every proposed defense should be evaluated against an adaptive adversary that has knowledge of the defense mechanism. Testing only against standard attacks such as FGSM and PGD can provide a false sense of security.

Real-world adversarial threats. The gap between laboratory adversarial attacks (carefully optimized ${\ell }_p$-bounded perturbations) and real-world evasion strategies (social media compression, resolution reduction, format conversion) remains poorly characterized. The 0.6% representation of social media attacks in our attack catalog, compared to 59.4% for adversarial perturbations, suggests that the community’s threat model may be misaligned with practical deployment conditions (see also §3.4).

Cross-attack generalization. The observation that adversarial training improves robustness against trained-upon attacks but not novel ones points to a fundamental limitation: the space of possible attacks is too large and heterogeneous for any fixed training distribution to cover. Addressing this challenge may require moving beyond empirical defenses toward approaches grounded in the theoretical properties of detection functions—an area where the deepfake detection community has barely begun to engage.

In summary, the adversarial landscape in deepfake detection has evolved from a niche concern (6 role assignments before 2021) to a central research theme (137 attack-oriented and 390 defense-oriented assignments, with possible overlap), with a pronounced shift from attack discovery to defense development after the 2023 inflection point. The 2.84:1 defense-to-attack assignment ratio documents a strong publication emphasis on defenses, but the low adoption of adversarial training (2.8%), the absence of standardized adversarial benchmarks, and incomplete evaluation practices suggest that the field’s adversarial robustness is more fragile than headline results indicate.

RQ8 answer and implication. The adversarial literature has grown quickly, but the field has not yet produced evidence strong enough to support broad robust-security claims. A defense should not be considered robust unless it survives adaptive white-box and black-box attacks, transfer attacks, natural post-processing, and cross-generator shifts under a shared benchmark. Field-level verdict: current adversarial robustness claims should be read as stress-test results, not security guarantees, until they are validated under adaptive and deployment-realistic threat models [17,18,122].

3.9. RQ9: Terminology Evolution

Terminology is part of the field’s measurement infrastructure: it defines what is retrieved, compared, and eventually treated as the same research problem. We therefore analyzed title/abstract usage across all 1,410 papers using 20 post-retrieval analysis terms. These terms are distinct from the broader query families used during literature discovery in §2. Counts denote paper-level term presence rather than raw token frequency, and terms are non-exclusive because one paper may use several labels. The pipeline applies case-insensitive regular expressions with word-boundary anchors to titles and abstracts, normalizes the declared hyphen/space variants (e.g., “AI-generated” and “AI generated”), and counts each paper at most once per term per year; it does not apply open-ended stemming beyond the listed variants. The empirical pattern is not a simple replacement of “deepfake” by a new label; it is the emergence of a dual vocabulary in which face-centric and general AI-generated-content framings coexist.

3.9.1. Dominant and Emerging Umbrella Terms

“Deepfake” remains the dominant corpus-wide label, appearing in the title or abstract of 546 papers (38.7% of the full corpus). Its raw annual count rises from 45 papers in 2021 to 207 in 2025, but normalization by annual publication volume changes the interpretation: the term appears in 34.9% of 2021 papers, peaks at 54.5% in 2022, and settles at 38.5% in 2025. It is therefore better described as a durable umbrella term than as a continuously expanding share of the literature.

“AI-generated” is the clearest growth signal. It appears in 299 papers overall (21.2%) and rises from 2 papers in 2021 to 183 in 2025. The corresponding annual prevalence increases from 1.6% to 34.0%—a 32.4 percentage-point shift that cannot be explained by corpus growth alone. By 2025, “AI-generated” is only 4.5 percentage points behind “deepfake” (34.0% vs. 38.5%). The defensible conclusion is convergence, not a forecast of replacement: the field is increasingly describing deepfake detection as one instance of the broader problem of AI-generated-content detection.

“Generative AI” appears in 128 papers, with 120 of those term occurrences concentrated in 2023–2025 (17, 36, and 67). “AIGC” appears in 64 papers, rising from 7 in 2023 to 36 in 2025, while “GenAI” appears in 26 papers. These three labels contribute 218 term occurrences, not 218 unique papers, because labels can co-occur. In contrast, “face forgery” appears in 98 papers overall but falls from 12.4% of the 2021 literature to 3.0% in 2025. The shift is therefore visible both in rising general-purpose terminology and in the declining annual share of a canonical face-specific label.

Figure 26. Annual prevalence of representative terminology, normalized by yearly corpus size ($N=129,143,182,311,538$ for 2021–2025). Terms are non-exclusive. Normalization separates vocabulary change from publication-volume growth: by 2025, “AI-generated” appears in 34.0% of papers, approaching “deepfake” at 38.5%, while “face forgery” falls to 3.0%. The January 2026 partial-year slice is omitted.

Figure 26. Annual prevalence of representative terminology, normalized by yearly corpus size ($N=129,143,182,311,538$ for 2021–2025). Terms are non-exclusive. Normalization separates vocabulary change from publication-volume growth: by 2025, “AI-generated” appears in 34.0% of papers, approaching “deepfake” at 38.5%, while “face forgery” falls to 3.0%. The January 2026 partial-year slice is omitted.

3.9.2. Technology-Specific Terminology

Technology-specific labels show a smaller but directionally consistent pivot. “GAN-generated” appears in 17 papers, with annual counts of 4, 3, 3, 4, and 3 from 2021 through 2025. Because the corpus grew sharply, its annual prevalence falls from 3.1% to 0.6%. “Diffusion-generated” appears in 18 papers and rises from no occurrences in 2021 to 9 papers in 2025 (1.7% of that year’s corpus), overtaking “GAN-generated” in annual count in 2024. This timing aligns with the broad adoption of latent-diffusion and unCLIP-based text-to-image systems after 2022 [75]; however, annual term frequencies alone do not identify a causal release-to-vocabulary lag.

Technology-agnostic output labels occupy a middle ground. “Synthetic image” appears in 45 papers and grows from 1 paper in 2021 to 20 in 2025, while “synthetic media” appears in 40 papers, including 31 in 2025. These labels describe the content rather than the generator and are therefore compatible with a benchmark ecosystem that spans GANs, diffusion models, autoregressive generators, and future model families.

3.9.3. Conceptual Interpretation

The evidence supports three descriptive phases. First, the early field was organized around face-specific forensic tasks such as face forgery, face swap, and reenactment. Second, “deepfake” consolidated those tasks under a recognizable umbrella label. Third, from 2023 onward, “AI-generated,” “generative AI,” “AIGC,” and related terms expanded the framing beyond faces and beyond any single generation family. This third phase does not make “deepfake” obsolete; rather, it creates a two-level vocabulary in which “deepfake” remains the established forensic label while “AI-generated content” specifies a broader deployment scope.

Figure 27. Corpus-wide terminology frequency in titles and abstracts. The word cloud is descriptive rather than inferential: size represents paper-level term occurrence, and terms may co-occur within a paper.

Figure 27. Corpus-wide terminology frequency in titles and abstracts. The word cloud is descriptive rather than inferential: size represents paper-level term occurrence, and terms may co-occur within a paper.

This distinction matters for method claims. A paper framed as face-forgery detection can reasonably target a narrow manipulation family; a paper framed as AI-generated-content detection implicitly claims relevance across generators, content types, and often modalities. The vocabulary therefore changes the evidentiary burden attached to a detector, even when the underlying architecture is similar. Broader labels should be accompanied by broader benchmarks and cross-generator evaluation rather than treated as stylistic renaming.

3.9.4. Implications for Retrieval and Benchmarking

The coexistence of “deepfake” (546 papers), “AI-generated” (299), “generative AI” (128), “face forgery” (98), “AIGC” (64), and “synthetic media” (40) makes narrow retrieval strategies fragile. Our analysis does not directly estimate search recall or citation-flow effects, so those consequences should be treated as testable implications rather than measured outcomes. What the data do establish is vocabulary fragmentation: multiple non-equivalent labels describe overlapping detection problems, and their relative prevalence changes rapidly.

For future systematic reviews, the practical response is query sensitivity analysis: report which term families are searched, quantify the marginal papers contributed by each family, and preserve a cross-term alias registry. For benchmark maintainers, the analogous response is cross-indexing: datasets and leaderboards should expose face-specific, generator-specific, modality-specific, and general AI-generated-content tags simultaneously. This makes the field easier to retrieve without forcing premature agreement on a single canonical term.

RQ9 answer and implication. The field is converging on a dual vocabulary rather than replacing “deepfake” outright. In 2025, “deepfake” remains present in 38.5% of papers while “AI-generated” reaches 34.0%, and face-specific “face forgery” falls to 3.0%. Field-level verdict: terminology is a methodological variable because it changes the scope implied by a detector claim and the literature visible to a search strategy. High-quality reviews and benchmarks should therefore index both legacy deepfake terminology and broader AI-generated-content terminology, with explicit generator and modality tags [14,15,17].

3.10. RQ10: Open Challenges and Future Directions

The preceding research questions documented the rapid growth, methodological diversification, and persistent evaluation gaps in deepfake detection research. This section turns to what the community itself identifies as the most pressing open challenges and promising future directions. Through systematic extraction and coding of challenge statements, limitations, and proposed future work across 401 papers, we construct a taxonomy of open problems that serves as a research roadmap for the field. Because papers frequently identify challenges across multiple categories, we employ multi-label coding: the 401 papers produce 1,531 challenge mentions across eight primary categories. The community has achieved broad consensus on the nature of core challenges, yet proposed solutions remain fragmented and largely untested at scale.

3.10.1. Challenge Taxonomy

We categorize the challenge mentions extracted from 401 papers into eight primary categories, as shown in Figure 28 and Table 16. Generalization is the most frequently mentioned challenge by a wide margin, accounting for 436 mentions [34,59,134,153,248]—a rate of 1.09 mentions per paper, indicating that many papers identify multiple distinct generalization sub-challenges. This dominance is unsurprising given the findings of §3.4, which documented that 72.6% of papers conduct zero robustness testing of any kind. The generalization challenge encompasses multiple dimensions: cross-generator transfer (detectors trained on GAN-generated content failing on diffusion outputs), cross-dataset transfer (performance degradation when moving between benchmarks with different identities, compression levels, and capture conditions), and cross-manipulation transfer (inability to detect unseen manipulation types).

Scalability ranks second with 245 mentions [34,48,49,90,134], encompassing the difficulty of scaling detection systems from controlled research settings to real-world deployment at platform scale, the computational costs of processing high-throughput media streams, and the challenge of maintaining detection accuracy as the diversity of generative models grows exponentially. Data scarcity and dataset quality follows with 205 mentions [24,34,248,279,362], encompassing the lack of diverse, representative training datasets, insufficient coverage of emerging manipulation types, annotation burden, and the “reality gap” between controlled benchmark conditions and in-the-wild content.

Adversarial challenges are identified in 174 mentions [48,134,137,203,204], highlighting the vulnerability of current detectors to adversarial perturbations, evasion attacks, and the structural asymmetry between attackers and defenders. Explainability and interpretability accounts for 166 mentions [34,59,137,363,364], reflecting the tension between the black-box nature of deep learning detectors and the need for actionable forensic evidence. Papers note that binary real/fake classification without explanations is insufficient for legal proceedings, content moderation decisions, and building user trust. The challenge is compounded by the finding that current large language model-based explanation methods suffer from hallucination and inconsistent reasoning [42,365,366].

Ethical and societal concerns account for 127 mentions [137,158,248,289,367], spanning the dual-use nature of detection technology, privacy implications of forensic analysis, demographic bias in detector performance, the societal harms caused by non-consensual deepfake imagery, and regulatory gaps across jurisdictions [354,368,369]. Multimodal detection challenges are noted in 118 mentions [34,61,134,202,364], reflecting the difficulty of detecting coordinated manipulations across visual, auditory, and textual modalities—a problem that intensifies as generative models increasingly produce coherent audio-visual content. Real-time processing is identified in 60 mentions [55,61,139,305,370], underscoring the computational gap between state-of-the-art detection accuracy and the latency requirements of live content moderation.

3.10.2. Temporal Evolution of Challenges

The relative emphasis among challenge categories has shifted over the review period, tracking the evolution of generative technology. In 2021, generalization accounted for 29 mentions and data scarcity for 16, with adversarial challenges at 10. By 2025, these numbers had grown to 178, 80, and 60, respectively. Because annual corpus size also grew, relative emphasis is better described by paper-normalized rates. Explainability mentions rise from 4/129 papers in 2021 (3.1%) to 79/538 in 2025 (14.7%); multimodal mentions rise from 3.1% to 10.6% (4 to 57); scalability rises from 9.3% to 19.3% (12 to 104); and generalization rises from 22.5% to 33.1% (29 to 178). These multi-label rates show broadening attention beyond face-specific forgery without conflating publication-volume growth with changing emphasis.

The emergence of new challenge sub-categories is also revealing. Prior to 2023, adversarial robustness discussions focused primarily on additive perturbation attacks against GAN-based face forgery detectors. By 2025, the discourse had expanded to encompass decision-based attacks against API-deployed detectors [48], adversarial transferability across detector families [205], and the fragility of proactive defenses such as watermarking against LoRA-based bypass attacks [139]. Similarly, generalization challenges evolved from concerns about cross-dataset transfer within face forgery to fundamental questions about whether detectors designed for one generative paradigm (GANs) can detect outputs from an entirely different one (diffusion models) [49,90].

3.10.3. Critical Challenge Analysis

The generalization-robustness nexus. The most frequently mentioned challenge—generalization to unseen generators, datasets, and manipulation types—is deeply intertwined with robustness and scalability concerns. The combined 436 generalization and 245 scalability mentions reflect a field grappling with the fundamental tension between specialization (high accuracy on known manipulations) and universality (reliable detection across the full generative landscape). Multiple papers observe that detectors achieve impressive in-distribution accuracy (often exceeding 99%) but suffer catastrophic performance drops when evaluated on content from unseen generators [153,202,203]. This brittleness stems from detectors learning generator-specific artifacts rather than fundamental properties of synthetic content. Several recent studies have shown that even within the same generative model family, minor changes to the generation pipeline (e.g., different sampling schedules in diffusion models) can substantially degrade detector performance [49,70]. The problem is compounded by the rapid proliferation of new generative models: papers note that the time between a new generator’s release and the community’s ability to train effective detectors creates a persistent vulnerability window [24,34,161].

The arms race dynamic. A recurring theme across the 174 adversarial challenge mentions is the detection-evasion arms race. Detectors that achieve state-of-the-art performance on standard benchmarks can be defeated by adversarial attacks requiring minimal computational resources [48,204]. This asymmetry—where attackers need only find one weakness while defenders must protect against all possible attacks—is identified as a structural disadvantage for the detection community. Proactive defenses, including adversarial training and input preprocessing, provide partial mitigation but are shown to be fragile under adaptive attacks [139,315]. The binary framing of detection (real vs. fake) exacerbates this vulnerability; several papers advocate for calibrated confidence scores and uncertainty quantification as more robust alternatives to hard classification [109,134].

The evaluation crisis. Underpinning many of the above challenges is what we term an “evaluation crisis”: the absence of standardized, comprehensive benchmarks and evaluation protocols that reflect real-world conditions. Papers identify multiple dimensions of this crisis: inconsistent experimental settings across studies [363,371], evaluation limited to a small number of legacy datasets (particularly FaceForensics++) [55,372], lack of evaluation under real-world degradations such as social media recompression [203], and the difficulty of measuring fixed-threshold robustness when traditional protocols allow threshold retuning per condition [134]. The convergence of 73.1% of the 1,091 papers with evaluation metadata omitting split strategy specifications (§3.6) and 47.3% claiming SOTA results (§3.7) with only 19.4% conducting robustness-specific cross-dataset testing (§3.4) paints a sobering picture: the field cannot reliably determine which methods represent genuine progress.

The explainability imperative. The 166 explainability mentions—making it the fifth most frequently mentioned challenge—signal a maturation of the field beyond pure detection accuracy toward forensic accountability. Papers note that detection without explanation is insufficient for legal proceedings, platform trust and safety operations, and public credibility [59,364,365]. The challenge is two-fold: developing methods that produce faithful explanations of their detection rationale, and evaluating explanation quality against forensic expert judgment. Current multimodal LLM-based approaches show promise for generating natural-language forensic reasoning, but suffer from hallucination—producing plausible but factually incorrect explanations of why content is manipulated [42,366].

3.10.4. Proposed Future Directions

Of the 401 papers that discuss open challenges, 43.5% also propose concrete future directions. From these papers, we identify seven major research trajectories that the community collectively advocates.

1. Universal and adaptive detection. The most prominent future direction involves developing detectors that generalize across generator families, manipulation types, and data distributions without requiring retraining. Proposed approaches include learning generator-agnostic features based on fundamental properties of synthesis processes [24,153], continual learning frameworks that adapt to new generators while retaining knowledge of previous ones [34,202], and foundation model-based detection that leverages pre-trained representations for zero-shot or few-shot transfer [49,90]. Several papers specifically call for self-evolving detection frameworks that can autonomously update as new generation methods appear [161,364]. The year-annotated series records 28 foundation-model papers in 2024 and 50 in 2025, making this one of the field’s most prominent recent responses to the challenge.

2. Explainable and trustworthy forensics. A substantial cluster of future directions targets the interpretability deficit. Researchers propose moving beyond binary classification toward evidence-based forensic analysis that provides spatially localized artifact maps, textual explanations of detection rationale, and chain-of-thought reasoning traces [42,91,365]. The integration of multimodal large language models for forensic explanation is identified as a promising but immature direction, with current approaches suffering from hallucination in forensic reasoning [59,364]. Papers also advocate for the development of human-aligned evaluation frameworks that measure explanation quality against expert forensic judgments [42,366].

3. Comprehensive benchmarking and standardized evaluation. Many papers call for next-generation benchmarks that address the limitations of existing datasets. Desired properties include broad generator coverage spanning both GAN and diffusion families [363,371], diverse content categories beyond faces [90,202], real-world degradation conditions including social media processing [134,203], standardized evaluation protocols with fixed decision thresholds [109], and demographic diversity to enable fairness analysis [248,279]. The community increasingly recognizes that benchmark design is as important as method development, yet coordinated efforts to create such benchmarks remain limited.

4. Multimodal and cross-modal detection. As generative models produce increasingly coherent audio-visual content, researchers advocate for detection methods that exploit cross-modal consistency as a forensic signal [61,202,373]. Future directions include spatiotemporal fusion architectures that jointly model visual and auditory artifacts [55,173], cross-modal verification systems that check whether visual speech matches audio content [134,276], and multimodal reasoning frameworks that can detect subtle inconsistencies invisible to single-modality analysis [34,364]. The 118 multimodal challenge mentions confirm that this direction is not merely aspirational but reflects pressing operational needs.

5. Efficient and deployable detection. Bridging the gap between detection accuracy and deployment feasibility is a recurring theme, reflected in the 245 scalability and 60 real-time challenge mentions. Proposed directions include model compression and knowledge distillation to reduce computational requirements [49,305], edge-device deployment with real-time inference capabilities [18,155], and architecture-aware optimization that balances accuracy with latency constraints [204,370]. Several papers note that Green AI principles—minimizing the carbon footprint of detection systems—should be incorporated into system design [49,372].

6. Proactive and prevention-oriented approaches. Beyond reactive detection, some papers advocate for proactive strategies including content provenance and authentication systems, digital watermarking of generated content at the source, and platform-level policies that require provenance metadata [141,158]. However, this direction is tempered by findings that watermarking and other proactive defenses remain fragile against determined adversaries [139,315].

7. Ethical frameworks and governance. Papers increasingly call for the development of ethical guidelines governing deepfake detection deployment, including protocols for responsible disclosure of detector vulnerabilities, fairness auditing across demographic groups, regulatory frameworks that balance detection capability with privacy rights, and international coordination on AI-generated content governance [367,368,369,374]. The 127 ethical challenge mentions indicate that the community is acutely aware of the societal dimensions of this technology, even as actionable governance frameworks remain scarce.

3.10.5. Urgency Assessment

Of the 401 papers analyzed, 26 (6.5%) identify challenges classified as high urgency—problems requiring immediate attention due to their potential for near-term societal harm or fundamental impact on detection viability [34,59,109,153,205]. These high-urgency challenges cluster around three themes: (1) the rapid obsolescence of existing detectors as new generators emerge faster than detectors can be retrained [161,202,315]; (2) the vulnerability of deployed detection systems to adversarial evasion, including decision-based attacks that require only query access [48,203,204]; and (3) the insufficiency of current benchmarks to capture real-world deployment conditions, leading to overconfident deployment of fragile detectors [363,371,372]. The remaining 375 papers (93.5%) were not coded as high urgency; this residual category should not be interpreted as a validated medium-urgency tier.

Notably, only 43.5% of papers that identify challenges also propose concrete solutions [59,137,153,205,248]. This solution gap—where 56.5% of papers acknowledge challenges without proposing mitigation strategies—indicates that the community’s diagnostic capacity substantially exceeds its prescriptive capacity. Within the paired audit, the highest proportional gap is data scarcity (81.8%), while real-time processing has the lowest gap (64.0%); all categories nevertheless show more challenge records than solution records. The gap between identifying 436 generalization challenge mentions and deploying methods that actually solve cross-generator transfer underscores that awareness does not automatically translate into progress.

Table 17 reports a separate paired-field audit. Its gap-set counts are field-level records for which both a challenge field and a future-direction/solution field were available; they are not unique-paper counts and are not the same unit as the 1,531 broad challenge mentions in Table 16. Consequently, a gap-set count may be higher or lower than the corresponding broad mention count, and only within-row solution rates and gaps should be interpreted. The corpus-level 43.5% statistic is paper-level (a paper proposes any concrete future direction), whereas the per-category rates are multi-label record-level quantities and should not be averaged to reproduce 43.5%. Within this paired audit, generalization has the largest gap-set (273 records) and a 74.0% gap; data scarcity has the highest proportional gap (81.8%); and real-time processing has the lowest, though still majority, gap (64.0%). The pattern is consistent with a field that identifies systemic problems more often than it records category-matched remedies; publication incentives may contribute, but this review does not test that causal explanation.

3.10.6. Synthesis: A Research Roadmap

The challenge landscape reveals a field at a critical inflection point. The convergence of three trends—rapidly improving generative models, the broadening scope from face-specific to general AI-generated content, and the transition from research prototypes to deployed systems—has created a set of compounding challenges that resist piecemeal solutions. The most pressing needs, synthesized from our analysis of 401 papers and 1,531 challenge mentions, are:

Short-term priorities (1–2 years): Establishing standardized evaluation protocols and comprehensive benchmarks that reflect real-world conditions; developing robust uncertainty quantification methods that produce calibrated confidence scores rather than brittle binary decisions; and creating large-scale, demographically diverse datasets spanning current generator families. The 73.1% rate of unspecified split strategies among the 1,091 papers with evaluation metadata (§3.6) and 47.3% SOTA claim rate (§3.7) make evaluation reform the most urgent and actionable priority.

Medium-term priorities (2–5 years): Achieving reliable cross-generator generalization through generator-agnostic feature learning or continual adaptation; developing interpretable forensic systems that provide evidence-grade explanations acceptable in legal and regulatory contexts; and building efficient detection architectures deployable at platform scale with real-time latency. The 245 scalability mentions signal that deployment-readiness has become a first-class concern alongside accuracy.

Long-term priorities (5+ years): Resolving the fundamental detection-evasion arms race through paradigm shifts beyond reactive detection, potentially via content provenance infrastructure or generative model accountability mechanisms; establishing international governance frameworks for AI-generated content; and developing detection systems that remain robust across the full spectrum of future generative technologies.

This challenge taxonomy represents not a static inventory of problems but a dynamic map of a rapidly evolving research frontier. As generative AI continues to advance, the specific instantiations of these challenges will shift, but the underlying structural tensions—between generalization and specialization, accuracy and efficiency, detection and evasion, capability and governance—will persist as defining features of the field for years to come.

RQ10 answer and implication. The open challenges are not independent; they form a coupled systems problem. Generalization fails because benchmarks are stale, evaluation protocols are inconsistent, reproducibility is weak, and generators change faster than static datasets can be curated. Field-level verdict: the field diagnoses more than it repairs. The roadmap is not simply “better detectors,” but a coordinated infrastructure agenda: living benchmarks, standardized protocols, reproducible code, provenance-aware systems, and deployment-oriented robustness testing [13,20,21,122].

3.11. RQ11: Research Community Analysis

Understanding who conducts deepfake detection research—and where—is essential for assessing the diversity, sustainability, and potential blind spots of the field. We resolved at least one affiliation for 1,262 of the 1,410 papers (89.5%). The geographic-entity and institution-type analyses below use this resolved-affiliation subset. Institution-name normalization failed a cross-level consistency audit—for example, an earlier institution ranking assigned more papers to a single institution than to its geographic entity—so we do not publish institution-level league tables from that field. This section therefore reports the defensible geographic-entity, institution-type, and broad collaboration findings while treating specific institution rankings as requiring re-curation.

3.11.1. Geographic Distribution

The research landscape is geographically concentrated. Under the extraction taxonomy, mainland China appears in 377 papers (29.9%) and the United States in 258 (20.4%). Papers may include multiple affiliations, so percentages are non-additive. Table 18 uses the label geographic reporting entity: mainland China, Hong Kong, Taiwan, and Macao are coded separately because that is how the source affiliations were normalized, and the table should not be interpreted as a geopolitical classification [22,50,56,59,66,132].

A second tier includes the United Kingdom (61 papers), Hong Kong (54), Singapore (51), Italy (51), Australia (49), India (42), Germany (39), and South Korea (31). The long tail remains sparse in Africa and Latin America and uneven across South and Southeast Asia. These distributions motivate broader geographic evaluation, but affiliation counts alone do not establish benchmark bias or detector-performance disparities; those require demographic and region-by-benchmark cross-tabs.

3.11.2. Institution Type Distribution

Universities appear in 1,082 papers (85.7%) of the resolved-affiliation subset (Figure 29). Big-tech affiliations appear in 405 papers (32.1%), entries classified as other in 315 (25.0%), research institutes in 190 (15.1%), startups in 85 (6.7%), and direct government-laboratory affiliations in 2 ($<0.2\%$). Because a paper can contain several affiliation types, these are paper-level multi-label counts rather than a partition.

The annual affiliation counts show expansion across all major categories. Universities grow from 14 coded affiliations in 2018 to 416 in 2025, big tech from 4 to 143, and startups from 3 to 29. Startups therefore grew substantially in absolute and relative terms, but not faster than every other category under the printed counts. The January 2026 slice is retained in aggregate totals but is not interpreted as a standalone trend year.

3.11.3. Collaboration and Audit Boundaries

A broad collaboration indicator identifies 374 of 1,262 papers (29.6%) as having either multiple institutions or an academia–industry combination. This is not an academia–industry collaboration rate: multi-university and other cross-institutional papers also satisfy the indicator. A separate affiliation-type cross-tab is required before estimating direct academia–industry collaboration. Likewise, specific bilateral collaboration axes and top-institution rankings are withheld until the institution-name registry is re-normalized and reconciled with the geographic-entity field.

3.11.4. Implications

Three conclusions remain supported. First, research participation is geographically uneven, creating a need for benchmarks and evaluation protocols that explicitly test demographic and regional transfer. Second, publication activity is predominantly university-linked, while industry and startup participation is substantial but measured imperfectly by public affiliations. Third, the institution-level audit demonstrates a general methodological lesson: community-demographic claims require the same normalization discipline as dataset and method taxonomies, and internally inconsistent rankings should be withheld rather than polished into a league table.

RQ11 answer and implication. The community is international but unevenly distributed across geographic entities and institution types. The defensible evidence supports concentration and representation concerns, not a precise ranking of individual institutions. Field-level verdict: geography is a potential measurement-bias source, but its effect on detector performance must be tested with region-aware benchmark and error analyses rather than inferred from affiliation counts alone [13,17,375].

4. Discussion

The preceding analysis of 1,410 papers across 11 research questions reveals a field of remarkable energy and rapid evolution, yet one beset by structural tensions that limit the translation of research advances into reliable, deployable systems. The most fundamental of these tensions is the chasm between methodological sophistication and evaluative rigor: the community produces increasingly powerful detectors while 72.6% of papers conduct zero robustness testing. In this section, we synthesize cross-cutting themes that emerge from the interplay of our findings, discuss implications for both practitioners and researchers, and acknowledge the limitations of this review.

4.1. Cross-cutting Themes

Five overarching patterns emerge when the findings from all 11 research questions are considered jointly. Each reflects a structural tension in the field that cannot be resolved by addressing any single research question in isolation.

1. Method sophistication versus evaluation fragmentation. As documented in §3.1, the detection method landscape has diversified dramatically through three architectural phases, with CLIP becoming the largest single normalized backbone label while classic CNNs remain larger as an aggregate family and zero-shot detection emerging as a fast-growing supervision paradigm. Yet this methodological sophistication is not matched by evaluation rigor. The 73.1% rate of unspecified train-test splits among the 1,091 papers with evaluation metadata (§3.6), the 4.3% rate of statistical significance reporting, and the 21.5% rate of unreported evaluation granularity collectively undermine comparability. The corresponding independent corpus-wide rates are 47.3% for explicit SOTA claims, 30.1% for cross-dataset evaluation, and 19.1% for ablation studies (§3.7); these fields were not conditioned on one another. The result is a paradox: the community produces increasingly sophisticated methods but lacks the evaluative infrastructure to determine which genuinely represent progress. When nearly half of all papers claim SOTA performance while protocols remain heterogeneous, the concept of “state of the art” loses much of its meaning.

2. The generator-detector arms race and the generalization imperative. The arms race documented in §3.3—with a detector-to-generator ratio of 4.5:1 and over 300 distinct generation method labels—creates relentless pressure for cross-generator generalization. The completed diffusion transition (diffusion ∼330 papers overtaking GANs at ∼297 in just three years) demonstrates the field’s reactive capacity but also its vulnerability to each new generative paradigm.

This pressure converges with the robustness crisis of §3.4. Generalization is simultaneously the most studied strategy (§3.3), the most frequently mentioned challenge (436 mentions in §3.10), and the area where the widest gap between research attention and practical performance persists. With 72.6% of papers conducting zero robustness testing and only 7 papers (0.5%) performing all four robustness evaluations, the majority of published methods have never been tested under conditions that deviate from their training distribution. Foundation-model-based approaches—a prominent late-period generalization strategy—offer a plausible response through a shift from learning “what fakes look like” to learning transferable representations of authentic and synthetic content, but durable robustness against future generators remains unproven.

3. Low reproducibility undermining fair comparison. The reproducibility analysis (§3.5) reveals a reproducibility illusion: while 51.1% of papers include at least one code/model link, only 23.7% satisfy the conservative single-GitHub proxy for likely self-released implementation code, and only 15 papers pass the strict README/title-match audit available in our pipeline. The gap arises because raw link counts conflate authors’ own implementations with references to third-party codebases, baseline repositories, generation toolkits, dataset pages, and model hubs. Raw code/model-link sharing peaked at 58.2% in 2023 and has since plateaued near 50%, indicating a structural ceiling under voluntary norms.

For roughly three-quarters of papers, our conservative proxy does not identify likely self-released code; the stricter README/title audit makes clear that corresponding-code availability cannot be inferred from URL presence alone. Combined with the evaluation fragmentation documented above, this creates a compounding problem: not only are evaluation protocols inconsistent, but corresponding implementations are difficult to establish from available links. The field lacks two of the three pillars needed for cumulative scientific progress—standardized evaluation and reproducible implementations—leaving only method novelty as a basis for comparison.

4. Benchmark staleness amid rapid method evolution. A temporal mismatch persists between the pace of method innovation and the evolution of evaluation infrastructure. Despite the rapid architectural evolution documented in §3.1 and the corpus-level GAN-to-diffusion crossover in §3.3, FaceForensics++ (2019) remains the dominant benchmark family after alias normalization with 330 normalized mentions (§3.2). The three leading benchmark families—FaceForensics++, Celeb-DF, and DFDC—all originate from the 2019–2020 GAN-era benchmark cycle and predate diffusion models, text-to-image generation, and multimodal deepfakes. The cleaned RQ2 audit shows that the corpus contains a large long tail of resource labels, but many of those labels are source/pretraining datasets or generator models rather than deepfake benchmarks. GenImage, the highest-ranked diffusion-era benchmark in the cleaned table, has only 24 normalized mentions. The consequences are measurable: methods achieving near-perfect accuracy on FF++ frequently degrade substantially on unseen generators (§3.4), meaning benchmark concentration can inflate apparent performance while obscuring the generalization failures that matter for deployment.

5. Geographic concentration and terminology fragmentation. In the 1,262-paper resolved-affiliation subset, mainland China (377 papers) and the United States (258 papers) are the two largest geographic contributors across 56 coded entities (§3.11); geographic-entity percentages are non-additive because papers may include multiple affiliations. Universities appear in 85.7% of the resolved-affiliation subset, while 29.6% meet a broad cross-institutional or academia–industry collaboration indicator. This concentration raises concerns about demographic and media-distribution representativeness in benchmarks: systems optimized for the faces, platforms, and production pipelines most common in the dominant datasets may underperform in contexts that are underrepresented or undocumented.

Compounding geographic concentration, §3.9 documents substantial terminology fragmentation: “deepfake” (546 papers), “AI-generated” (299), “generative AI” (128), “face forgery” (98), and “AIGC” (64) coexist. After normalization by annual corpus size, “AI-generated” rises from 1.6% of papers in 2021 to 34.0% in 2025, approaching “deepfake” at 38.5%, while “face forgery” falls to 3.0%. This dual vocabulary complicates retrieval and benchmark indexing even without assuming a particular regional distribution of terms. The implications of geographic concentration for detection fairness are discussed in §4.3.

4.2. Cross-Modal Convergence and the Broader FAIGC Landscape

The challenges identified in this review are not unique to the visual modality—they are shared across the entire deepfake detection ecosystem. In the speech deepfake detection domain, Li et al. [15] report approximately 10% source-code availability and identify cross-dataset generalization and self-supervised pretrained representations as central themes. Their coding protocol is not identical to our link-evidence tiers, so the percentages are not directly comparable; the parallel nevertheless suggests that reproducibility and transfer problems extend beyond visual media. This convergence suggests that the fundamental challenges of deepfake detection—generalization, reproducibility, adversarial vulnerability—are properties of the detection problem itself, not artifacts of any particular modality or generation technology.

Furthermore, deepfake detection is increasingly understood as one sub-problem within a broader landscape of Fake AI-Generated Content (FAIGC) detection [14] that spans text (LLM-generated disinformation), images (diffusion-generated scenes), audio (synthetic speech), and multimodal content. The terminological expansion documented in §3.9—from “face forgery” to “AI-generated content”—reflects this conceptual broadening. As detection methods evolve from face-specific forensics toward general-purpose AI content authentication, the lessons from this review—particularly the evaluation crisis (§3.4, §3.6, §3.7), the reproducibility deficit (§3.5), and the concentration of research effort (§3.11)—apply increasingly to this broader domain.

4.3. Demographic Equity and Representational Fairness

The geographic and demographic patterns documented in §3.11 and §3.2 converge on a shared concern: deepfake detection research lacks sufficient evidence that its benchmark ecosystem represents the populations and media conditions most vulnerable to synthetic media harms. The resolved-affiliation subset is led by mainland China (377 papers) and the United States (258 papers), and the three dominant benchmarks—FaceForensics++, Celeb-DF, and DFDC—come from the 2019–2020 benchmark cycle and rely heavily on celebrity or consented-actor media. Many popular benchmarks do not publish a systematic diversity characterization of their subjects, so demographic representativeness is often unknown rather than demonstrated.

This matters for deployment fairness in two ways. First, detectors trained on demographically skewed benchmarks may exhibit differential error rates across populations—a pattern well-documented in face recognition and increasingly suspected but poorly measured in deepfake detection. Second, non-consensual intimate imagery (NCII) is a major harmful use case yet is severely underrepresented in publicly released benchmarks, precisely because curating such content raises profound ethical concerns. This creates a gap between research evaluation (which focuses on celebrity face-swaps) and the harm profile of real-world misuse.

Participation remains especially sparse in Africa and Latin America and uneven across South and Southeast Asia, despite substantial contributions from several countries in the latter region; this unevenness is itself a fairness concern. Detection systems optimized in well-represented research regions may perform poorly when deployed in regions with different media infrastructure, facial demographics, and manipulation tool prevalence. Expanding research participation through international funding programs, shared dataset initiatives with broad demographic representation, and community efforts to develop culturally specific forensic benchmarks is essential for ensuring that deepfake detection serves all populations rather than replicating existing inequities in AI systems.

4.4. Implications for Practitioners

Organizations seeking to deploy deepfake detection systems should be aware of several findings from this review:

• No universal detector has been demonstrated. Our analysis of method categories (§3.1) and robustness evaluations (§3.4) finds no corpus-wide evidence that one architecture generalizes across all manipulation types. The 72.6% zero-robustness rate means that the vast majority of published methods have never been tested outside their training distribution. Deployment strategies should validate complementary detector ensembles or foundation-model approaches under the target distribution rather than assume universality from benchmark scores.
• Real-world conditions remain poorly addressed. Only 7.5% of papers test compression resilience and 4.1% evaluate social media degradation (§3.4), yet these are precisely the conditions under which deployed detectors must operate. Compression robustness testing has actually declined, from 16.3% in 2021 to 5.8% in 2025. Practitioners should conduct their own robustness evaluations under platform-specific conditions rather than relying on benchmark performance figures.
• Reported performance metrics may not be comparable. The metric fragmentation documented in §3.7—with accuracy reported in 39.2% of papers and serving as the leading primary metric in 27.5%, despite its limitations in imbalanced settings, over 100 distinct metric names, and only 4.3% of papers reporting statistical significance (§3.6)—means that published comparisons should be interpreted with extreme caution. Practitioners should prioritize threshold-independent metrics (AUC), deployment-relevant metrics (TPR at low FPR), and computational efficiency reporting.
• Code availability does not guarantee reproducibility. While approximately half of papers provide some code link (§3.5), our conservative proxy identifies likely self-released implementation code for only roughly one in four papers, and strict README/title matching confirms far fewer. Practitioners should allocate significant engineering effort for adapting research prototypes to production environments and should favor methods with complete training pipelines, pre-trained checkpoints, and containerized environments.
• The threat landscape evolves faster than detectors. The arms race analysis in §3.3 shows that diffusion-related coding crossed GAN-related coding over roughly three publication years. The emergence of text-to-video generators (Sora, Pika, CogVideo, Kling) signals the next vulnerability window. Practitioners should plan for continuous model updates and consider proactive approaches—content provenance standards and digital watermarking—as complements to reactive detection. The 137 attack-oriented paper-label assignments documented in §3.8 show substantial research attention to adversarial evasion; transfer attacks reported in the corpus cross detector families.

4.5. A Minimum Evidence Contract for Future Detector Papers

Across RQ4–RQ7, the same conclusion appears from different angles: deepfake detection does not lack proposed models; it lacks enforceable evidence standards. We therefore propose a minimum evidence contract for future detector papers. A paper that claims state-of-the-art, robustness, or deployment relevance should report at minimum: (1) exact dataset versions and split files; (2) preprocessing, face detection, frame sampling, and aggregation rules; (3) in-domain and cross-dataset results; (4) at least one unseen-generator or cross-manipulation test; (5) robustness to compression or social-media processing; (6) threshold-independent metrics, low-FPR operating points, uncertainty and computational cost; (7) ablations isolating the claimed technical novelty; and (8) code, checkpoints, and evaluation scripts sufficient for independent reproduction.

This contract is intentionally demanding because weak evidence is not neutral. In a forensic domain, a detector with unreported uncertainty, unverifiable code, and unspecified splits can create misplaced confidence, not just an inflated leaderboard score. The publication norm should shift from “new detector plus higher accuracy” to “new detector plus reproducible evidence that the improvement survives distribution shift.” If adopted, this standard would convert many of the field’s current weaknesses—benchmark staleness, non-standardized metrics, low robustness testing, and poor reproducibility—into explicit review criteria rather than post-hoc limitations.

4.6. Implications for Researchers

Our synthesis of 1,410 papers identifies several priorities for advancing the field:

• Standardized evaluation protocols are urgently needed. The evaluation crisis documented across §3.4, §3.6, and §3.7 is perhaps the single most actionable finding of this review. When 72.6% of papers perform zero robustness testing, 73.1% of the 1,091 papers with evaluation metadata omit train-test split details, and 47.3% claim SOTA while protocols remain heterogeneous, the field cannot reliably identify its own progress. Community-wide adoption of standardized benchmarking suites—with fixed splits, specified preprocessing, mandatory cross-dataset evaluation, and statistical significance testing—would substantially improve the evidential value of published results.
• Cross-dataset evaluation should become mandatory. At present, only 30.1% of papers include cross-dataset experiments (§3.7), and only 19.4% in the robustness-specific analysis (§3.4). Cross-dataset evaluation should be a minimum requirement for any paper claiming generalization, ideally including at least three benchmarks spanning different generator families, one dataset with social media degradation, and one with unseen manipulation types.
• Robustness to real-world conditions requires dedicated attention. The hierarchy of evaluation rigor revealed in §3.4—from cross-dataset evaluation (19.4%) to cross-type (10.1%) to compression robustness (7.5%) to social media robustness (4.1%)—shows that the most deployment-relevant conditions are the least studied. The declining rates of compression and social media testing mean the field is moving further from deployment-ready evaluation even as practical stakes increase. The 7 gold-standard papers that conduct all four robustness tests should serve as the model, not the exception.
• The adversarial dimension demands stronger engagement. The adversarial landscape (§3.8) contains 137 attack-oriented and 390 defense-oriented paper-label assignments, which may overlap; adversarial training appears in only 11 defense-oriented papers (2.8%), and only 17 of the 137 attack-oriented papers (12.4%) explicitly evaluate adaptive adversaries. The field needs standardized adversarial benchmarks, mandatory adaptive attack evaluation, and greater investment in domain-adapted adversarial training.
• Geographic diversity and collaboration should be expanded. With mainland China (377 papers) and the United States (258 papers) as the two largest geographic contributors in the resolved-affiliation subset across 56 coded entities (§3.11), the field would benefit from broader international participation. The terminology fragmentation documented in §3.9 adds a separate discovery barrier: benchmark and literature indexes should cross-reference “deepfake,” “AI-generated,” “AIGC,” and modality-specific labels rather than assume one universal vocabulary.
• Reproducibility practices must improve. The gap between raw code/model-link rate (51.1%), the single-GitHub corresponding-code proxy (23.7%), and the strict README/title-match lower bound (15 papers) in §3.5 signals that current norms are insufficient. The plateau near 50% suggests that voluntary norms alone may not drive substantially higher adoption. The community should move toward stronger artifact expectations: code and model release when possible, standardized evaluation scripts, executable environments, and periodic reproducibility challenges.

4.7. Threats to Validity

We acknowledge several limitations that may affect the generalizability of our findings.

Search scope. Our primary search targeted arXiv, supplemented by Google Scholar (§2.1). While arXiv provides comprehensive coverage of preprints in computer vision and machine learning, this strategy may underrepresent papers published exclusively at specialized venues (e.g., ACM Multimedia, IEEE WIFS, ICASSP) or in journals without preprint traditions. Papers from audio forensics and natural language processing communities that address deepfake-adjacent problems may also be underrepresented, a concern underscored by the finding that audio datasets appear in only 5.9% of papers (§3.2).

LLM-assisted analysis. Our semi-automated pipeline uses large language models for relevance filtering, metadata extraction, and content categorization (§2). While this approach enables analysis at a scale infeasible through fully manual coding, it introduces the potential for classification errors and inconsistent handling of ambiguous cases. The 9.4% inclusion rate—1,410 papers retained from a larger candidate pool—means that some relevant papers may have been incorrectly excluded. We report conservative estimates where possible (e.g., the tiered code-evidence estimates in §3.5) and acknowledge that exact figures should be interpreted as informative approximations rather than precise measurements.

Statistical precision of key estimates. The LLM-assisted pipeline processes each paper independently without a full human-labeled validation set. We therefore report a sensitivity scenario rather than an empirically estimated error interval: if 5% of papers currently coded as having no robustness test were false negatives, the 72.6% estimate would shift to approximately 69%, preserving the qualitative conclusion. The 51.1% code-link rate is an upper bound on link presence, 23.7% is a conservative proxy rather than a verified lower bound on author code, and the 15-paper README/title tier is the high-precision lower-bound audit available in the pipeline. The 47.3% SOTA-claim rate is based on explicit superlative language. Until a human-labeled validation set is released, all percentages should be read as informative estimates rather than formal uncertainty intervals.

Temporal bias. The distribution of papers is heavily weighted toward recent years, with 2024 (311 papers) and 2025 (538 papers) collectively accounting for over 60% of the dataset. This reflects genuine field growth but means that early-year trends are based on smaller samples. Papers from 2025–January 2026 may not yet have finalized code releases or received peer review, potentially inflating certain statistics relative to their eventual steady state.

English language bias. We conducted our search in English, which may underrepresent research published in Chinese, Korean, or other languages—particularly relevant given that mainland China is the leading geographic contributor with 377 papers (§3.11). The coexistence of “deepfake,” “AI-generated,” and “AIGC” labels (§3.9) further motivates multilingual and cross-term retrieval audits, but this review does not estimate region-by-term usage directly.

Normalization challenges. Several analyses required normalization of heterogeneous labels—127 raw method categories (§3.1), over 300 generation method labels (§3.3), 273 manipulation type labels (§3.4), and over 100 metric names (§3.7). The “approximately” qualifier for normalized totals (e.g., diffusion ∼330, GAN ∼297) reflects inherent uncertainty in this process.

Snapshot nature. This review represents a snapshot of a rapidly evolving field as of early 2026. The emergence of text-to-video generation, 3D synthesis, and multimodal generation in 2024–2025 has already begun to reshape the field in ways not yet fully captured by the corpus.

Despite these limitations, the scale of the review and the convergence of several independently coded RQs provide a broad evidence map of the deepfake-detection landscape. The principal patterns—low robustness testing, protocol fragmentation, weak corresponding-code evidence, benchmark staleness, and geographic concentration—are qualitatively consistent across multiple analyses, but their exact magnitudes should be re-estimated when a human-labeled validation set and the archival review package are available.

5. Conclusions

This paper presented one of the broadest per-paper, full-text systematic quantitative reviews focused on deepfake detection research, analyzing 1,410 papers published from January 2018 to January 2026 across 11 research questions spanning the full evidence chain of the field. Through a semi-automated extraction pipeline with targeted quality controls at a scale infeasible for fully manual coding, we constructed a quantitative portrait of a research domain that has undergone extraordinary growth and transformation in under a decade. The resulting picture is not simply that the field is large; it is that the field is producing methods faster than it is producing reliable evidence for those methods.

Our analysis reveals several defining characteristics of the field’s current state. Detection methodologies have evolved through three distinct phases—from CNN dominance (266 papers, 18.9%) through transformer diversification (207 papers, 14.7%) to the foundation-model shift (121 papers, 8.6%; 28 year-annotated papers in 2024 and 50 in 2025)—with CLIP as the largest single normalized backbone label, while classic CNNs remain larger in aggregate (RQ1). Despite this diversification, evaluation remains concentrated around aging benchmark families—most notably FaceForensics++ (330 normalized mentions), Celeb-DF (267), and DFDC (231), all from the 2019–2020 GAN-era benchmark cycle—while the cleaned RQ2 audit shows that source/pretraining datasets and generator models must be separated from true deepfake benchmark totals (RQ2). The generator-detector arms race has undergone a defining transition, with diffusion-related papers (∼330) overtaking GAN-focused work (∼297) in just three years, while dual-role papers grew from 13 in 2021 to 84 in 2025 (RQ3).

The robustness crisis stands as the review’s most alarming finding: 72.6% of papers (1,024) contain zero robustness testing of any kind, only 7 papers (0.5%) conduct all four robustness evaluations, and compression testing declined from 16.3% in 2021 to 5.8% in 2025, while social-media testing declined from 12.6% in 2022 to 3.7% in 2025 (RQ4). The field faces a reproducibility illusion: while 51.1% of papers include code/model links, only 23.7% satisfy the conservative single-GitHub proxy for likely self-released implementation code, only 15 papers pass the strict README/title-match audit, and raw link sharing has plateaued near 50% (RQ5). Evaluation protocols are highly fragmented, with 73.1% of the 1,091 papers with evaluation metadata omitting split strategies and only 4.3% reporting statistical significance (RQ6). Performance reporting suffers from compounding inconsistencies: the independent corpus-wide rates are 47.3% for explicit SOTA claims, 30.1% for cross-dataset evaluation, and 19.1% for ablation studies (RQ7). The adversarial landscape has grown from 6 pre-2021 role assignments to 137 attack-oriented and 390 defense-oriented assignments, which may overlap; adversarial training appears in 11 defense papers (2.8%) (RQ8). Terminology is in active transition: by 2025, “deepfake” appears in 38.5% of papers and “AI-generated” in 34.0%, while “face forgery” falls to 3.0%, revealing convergence on a dual vocabulary rather than simple term replacement (RQ9). Generalization is the most frequently mentioned open challenge (436 mentions), yet only 43.5% of papers identifying challenges propose solutions (RQ10). In the resolved-affiliation subset, mainland China (377 papers) and the United States (258 papers) are the two largest geographic contributors, and geographic-entity percentages are not additive because papers may include multiple affiliations (RQ11).

Synthesizing across all research questions, we identify five priorities essential for the field’s continued progress:

1.

Standardized evaluation benchmarks and protocols. The absence of agreed-upon benchmarks, fixed evaluation splits, and reporting standards is the single largest obstacle to measuring genuine progress. When 72.6% of papers perform zero robustness testing and 73.1% of the 1,091 papers with evaluation metadata omit split details, the field cannot determine which methods represent real advances. The community must converge on comprehensive evaluation suites spanning current generator families, real-world degradations, and routine statistical significance testing.

2.

Robust cross-domain generalization. Detection methods must move beyond generator-specific artifact memorization toward learning fundamental properties of synthetic content. The foundation-model paradigm—with CLIP as the largest single backbone label—offers a prominent trajectory, shifting from learning “what fakes look like” to “what reality looks like.” Continued advances in generator-agnostic representations, continual learning, and few-shot adaptation are essential.

3.

Real-world deployment readiness. Bridging the gap between laboratory and field performance demands evaluation under conditions encountered in practice—social media recompression, resolution reduction, adversarial perturbations, and emerging generators. The declining rates of compression and social media robustness testing indicate the field is moving further from this goal. Detection systems must produce calibrated confidence scores within the latency constraints of real-world platforms.

4.

Reproducibility and open science. The reproducibility illusion—51.1% raw code/model-link availability, 23.7% under the single-GitHub corresponding-code proxy, and only 15 papers passing strict README/title matching—is insufficient for a field with direct societal impact. Artifact and code release when ethically and legally permitted, standardized experimental configurations, containerized environments, and periodic reproducibility challenges would accelerate cumulative progress and reduce redundant effort.

5.

Proactive defenses for emerging generative technologies. As generative AI advances from diffusion models to video generation, multimodal synthesis, and beyond, the detection community must develop anticipatory methods that address new paradigms without full retraining. The observed diffusion crossover unfolded across roughly three publication years, but accelerating generative innovation may outstrip this reactive capacity. Content provenance infrastructure and collaboration with generative model developers are essential complements.

The deepfake detection landscape reflects a field of considerable vitality and growing maturity, yet one facing structural challenges—a 72.6% zero-robustness rate, fragmented evaluation, a reproducibility illusion, benchmark staleness, and an asymmetric arms race—that incremental methodological improvements alone cannot resolve. The strongest conclusion of this review is deliberately blunt: many current deepfake-detector comparisons are not yet reliable evidence of real-world progress. As generative AI systems become more capable, more accessible, and more deeply integrated into content creation workflows, the stakes of reliable detection will only increase. The ability to distinguish authentic from synthetic media is no longer merely a computer vision problem; it is a foundational requirement for maintaining trust in digital communication, evidence, and public discourse. We hope that this review, by providing a comprehensive empirical foundation across 1,410 papers and identifying the most critical gaps with quantitative evidence, will help the research community direct its efforts where they are most urgently needed.