Talking Face Generation in Socio-Technical Systems: A Systematic Review of Deep Learning Architectures, Deployment Contexts, and Governance Frameworks

1. Introduction

1.1. Background and Motivation

Modern deep learning can now synthesize a photorealistic speaking face from a single portrait photograph and an audio clip. Talking face generation (TFG), illustrated in Figure 1, is the automated creation of temporally coherent video in which a face speaks in synchrony with a given audio stream. The problem sits at the intersection of computer vision, audio signal processing, and generative modelling, and in one decade it has grown from a narrow scientific question into a technology with commercial weight and social consequences.

The technical lineage is short but dense. Bregler et al. [1] showed in the 1990s that vocal-tract articulation is tightly coupled with visible facial motion, making data-driven synthesis plausible, and the statistical models of the 2000s formalized the mapping without approaching perceptual realism. Deep learning changed the trajectory. The Synthesizing Obama system of Suwajanakorn et al. [2] demonstrated that broadcast-quality lip synchronization could be learned directly from audio-visual corpora, without hand-crafted features. Generative Adversarial Networks (GANs) [3] then became the generative backbone of choice, and Wav2Lip [4] achieved lip-sync fidelity good enough for in-the-wild use. Large audio-visual corpora such as VoxCeleb [5], VoxCeleb2, and HDTF [6] supplied the training signal to push generalization beyond single-speaker settings.

Three further transitions followed from around 2021. Transformer architectures such as FaceFormer [7] brought long-range audio-visual context modelling; Neural Radiance Fields (NeRFs) [8] were adapted to dynamic portraits in AD-NeRF [9], giving 3D-consistent talking heads; and Diffusion Models [10] then displaced GANs, with DiffTalk [11] and DreamTalk [12] delivering probabilistic, high-fidelity synthesis with better training stability and diversity. By 2024, Diffusion Transformers (DiT) and Gaussian Splatting had begun to remove the long-standing latency barrier: VASA-1 [13] produces 512×512 video at 45 fps in real time, and OmniHuman-1 [14] extends the DiT paradigm to full-body digital human animation.

1.2 Applications and Open Problems

TFG has become socio-technical in the sense of Trist and Bamforth [15] and later socio-technical systems (STS) theory [16]: its deployment consequences are now inseparable from the human and institutional contexts it operates in. In healthcare, TFG-based conversational agents are one response to a projected global shortage of 11.1 million health workers by 2030 [17]; AI physician avatars can deliver patient education with perceived empathy comparable to human clinicians [18], and real-time systems such as MuseTalk [19] and VASA-1 [13] make live telehealth interaction feasible, with emotionally adaptive models such as EmotiveTalk [20] pointing toward mental-health applications. In education, embodied agents with realistic faces improve engagement and retention over disembodied voices or text [21], support virtual reality (VR) anatomy and clinical-communication training [22], and, with dyadic systems such as INFP [23], open up conversational skills training. In media and personalized communication, TFG enables multilingual dubbing, lip-readable synthetic media for hearing-impaired audiences, and consumer applications of latent-space face synthesis [13,24,25]; public institutions can use the same capability for low-cost multilingual civic communication.

1.3. The Regulatory Turn

The potential harms of TFG have triggered a wave of regulation. The EU AI Act (Regulation 2024/1689), adopted in 2024 and fully enforceable from August 2026, classifies deepfakes and synthetic facial media as a transparency risk under Article 50: generated content must be labelled, users must know when they are interacting with an AI-generated face, and Article 50(2) with Recital 134 mandates watermarking, metadata identifiers, and provenance tooling so synthetic media stays machine-detectable [26]. In the United States, the TAKE IT DOWN Act (Public Law 119-12, May 2025) criminalizes distribution of non-consensual AI-generated intimate imagery, with prison terms of up to two years (three where the subject is a minor) and a 48-hour platform takedown obligation [27]. C2PA v2.0 provides cryptographically signed provenance manifests already adopted by major platforms, and China’s synthetic media labelling rules took effect on 1 September 2025 [28].

The timing matters as much as the content. VASA-1 [13] and OmniHuman-1 [14] were published and deployed before any of these instruments entered force. That gap between capability and governance is a familiar pattern in socio-technical systems [16], and closing it requires the kind of systems-level account of TFG that this review sets out to provide.

1.4. Gap, Objectives, and Research Questions

Surveys of TFG and adjacent fields have appeared steadily since 2022 [29,30,31,32,33,34], and Section 2 analyses them in detail: none addresses the regulatory environment, dataset equity, or the socio-technical systems TFG is deployed into, and no existing review is simultaneously rigorous, current to 2024-2025, and comprehensive across technical, ethical, and socio-technical dimensions.

This review addresses that gap through six research questions:

RQ1: What deep learning architectures have dominated TFG from 2016 to 2025, and what trajectory characterizes their evolution?

RQ2: How do biases in the datasets used to train and evaluate TFG models affect generalization, and what are the equity implications for deployment?

RQ3: What metrics are currently used to evaluate TFG systems, and how adequately do they capture the requirements of socio-technical deployment?

RQ4: How have Diffusion Transformers, Gaussian Splatting, and related modern architectures advanced the quality, efficiency, and controllability of TFG relative to GANs and classical NeRFs?

RQ5: What ethical safeguards (detection, consent, watermarking, provenance) are integrated into state-of-the-art TFG systems, and how does this align with emerging regulatory requirements?

RQ6: How does TFG embed within, and generate risks and opportunities for, the socio-technical systems in which it is deployed, including healthcare, education, identity management, and democratic governance?

1.5. Contributions and Structure

This review contributes: (i) an architectural taxonomy extended to Gaussian Splatting and DiT, mapping the TFG design space from 2016 to 2025; (ii) the first systematic assessment of demographic representation across the most widely used TFG benchmark datasets; (iii) a critical analysis of the evaluation metric ecosystem and three proposed deployment-fitness metrics, the Perceived Trustworthiness Score (PTS), Cross-Cultural Authenticity Index (CCAI), and Deepfake Detectability Rate (DDR); (iv) documentation of the safeguard gap, showing that no reviewed system natively integrates watermarking, consent, or provenance, mapped against the EU AI Act, the TAKE IT DOWN Act, and C2PA v2.0; (v) a socio-technical analysis of four deployment domains; (vi) the Responsible Talking Face Generation in Socio-Technical Systems (RTFG-STS) Framework, grounded in Floridi et al.’s AI4People principles [35] and operationalized through five mechanisms aligned to EU AI Act Article 14; and (vii) a formalized three-stage process pipeline for TFG.

Section 2, Section 3, Section 4, Section 5 and Section 6 present, in turn, prior surveys, methodology, the synthesis by research question, the RTFG-STS Framework, and conclusions.

2. Related Surveys and Reviews

Secondary literature on TFG and its neighbors falls into three groups: deepfake generation and detection reviews, TFG-specific surveys, and reviews of adjacent domains. We assess each against four dimensions: methodological rigor, temporal coverage, thematic scope, and socio-technical perspective. Figure 2 summarizes the comparison and shows that no prior review satisfies all four.

2.1. Deepfake Generation and Detection

Rana et al. [29] surveyed deepfake detection (2018-2020) with a declared methodology adapted from Kitchenham and Charters [36]; generation, datasets, and ethics are excluded by design, and the window predates the transformer, diffusion, and Gaussian Splatting transitions. Heidari et al. [33] extended this line to 2021 with an artefact-based detection taxonomy, closing before diffusion models arrived, whose outputs defeat detectors calibrated to GAN artefacts. Romero Moreno [37] analyses deepfakes through human rights law and evaluates the EU AI Act; it is a legal analysis rather than a technical survey, but its treatment of consent and provenance as rights-enabling mechanisms directly informs this review’s RQ5 and the framework in Section 5.

2.2. Talking Face and Talking Head Surveys

Zhen et al. [30] surveyed talking-head generation (2017-2022) with a 2D/3D, pipeline/end-to-end classification but declare no methodology: no search string, databases, criteria, or quality assessment, and the survey ends before DiffTalk [11], VASA-1 [13], and OmniHuman-1 [14] existed. Toshpulatov et al. [31] add technical depth and a Convolutional Neural Network (CNN)/GAN/NeRF taxonomy, likewise without a protocol, and a frontier placed at NeRFs is now a full architectural generation out of date. Bigioi and Corcoran [32] frame the field through multilingual dubbing, also without a declared method. Rakesh et al. [34] is the only prior survey to reach diffusion and Gaussian Splatting, but it too follows no systematic method and is entirely technical: nothing on ethics, governance, or deployment. Nisar et al. [38] take an application-oriented view, among the first to treat TFG as socially consequential, though the coverage is illustrative rather than systematic.

2.3. Adjacent Domains

Three adjacent literatures matter here. Audio-visual speech learning supplied both the field’s dominant training corpus (VoxCeleb [5], used in over 25 of the 92 included studies) and its dominant synchrony metric, SyncNet [39], built for correspondence detection in broadcast television rather than naturalness assessment, a design intent central to the RQ3 analysis. 3D face reconstruction supplied the parametric priors that intermediate-representation systems such as SadTalker [40] and GeneFace [41] depend on; such priors average over their training demographics and can encode bias geometrically, which is relevant to RQ2. The embodied conversational agent (ECA) literature supplies the socio-technical evidence base: Provoost et al. [21] found ECA-administered clinical interviews can match human-administered assessment in specific contexts, and Bickmore and Cassell [42] showed that trust in agents depends on human-like appearance and conversational naturalness. These findings motivate RQ6 and the proposed RTFG-STS Framework.

2.4. Comparative Analysis and Gaps

Four gaps follow from Figure 2. First, methodological rigor: only the two detection reviews [29,33] declare a systematic method, and every survey of TFG generation itself [30,31,32,34] is non-reproducible, which biases coverage toward the most-cited work. Second, temporal obsolescence: no methodologically rigorous review reaches the diffusion or Gaussian Splatting eras at all. Third, a governance blind-spot: no TFG survey addresses the EU AI Act [26], the TAKE IT DOWN Act [27], C2PA v2.0, or China’s labelling rules [28], despite these creating concrete design obligations within the review period. Fourth, no socio-technical analysis: every existing review treats TFG as a benchmark optimization problem. The literature explains how to build a talking face system; it says almost nothing about what happens when one enters a clinical workflow, a curriculum, a KYC (Know Your Customer) pipeline, or a political information environment. This review is designed to close all four gaps at once.

3. Methodology

3.1. Review Framework and Rationale

The review follows the SLR methodology of Kitchenham and Charters [36] in three phases (planning, conducting, reporting), with study identification and reporting per PRISMA 2020 [43]. A scoping review would be inappropriate because RQ3 and RQ5 need quality-filtered synthesis to support defensible conclusions. A meta-analysis is impossible because TFG studies report incompatible metric subsets of FID, SyncNet, PSNR, SSIM, LPIPS, and FVD on incompatible test sets; making statistical pooling meaningless in this context. The Kitchenham protocol accommodates qualitative synthesis of heterogeneous studies while keeping selection transparent and auditable. Figure 3 presents the PRISMA 2020 flow diagram.

3.2. Scope

The review covers deep learning approaches to TFG across the full pipeline from feature extraction to rendering: audio-driven single-image animation, audio-plus-video reenactment, 3D-aware synthesis (NeRF and Gaussian Splatting), diffusion and DiT architectures, and detection or safeguard work directly tied to TFG assessment. Coverage runs from January 2016, when deep recurrent architectures first reached talking face synthesis, to December 2025, fixed at the fourth search iteration in March 2026. Pre-2016 methods (HMMs, DBNs, AAMs) fall outside the deep learning pipeline the research questions examine. Because RQ5 and RQ6 require it, the disciplinary scope extends past computer vision into applied ethics, AI governance, and STS analysis.

3.3. Search Strategy

Keywords were identified in three stages: a scoping search that yielded 47 highly cited papers and 31 candidate terms in three semantic clusters; forward snowballing through the 20 most-cited candidates, adding five terms; and a 2025 revision adding frontier terminology. The final Boolean string, applied to titles, abstracts, and keywords, was:

(“talking face” OR “talking head” OR “audio-driven face” OR “speech-driven face” OR “audio-visual synthesis” OR “portrait animation” OR “lip sync” OR “lip synchronization” OR “video reenactment” OR “deepfake” OR “facial animation deep learning” OR “neural talking” OR “video dubbing” OR “face reenactment” OR “Gaussian splatting avatar” OR “diffusion portrait” OR “facial diffusion transformer” OR “audio-driven digital human”)

The string was executed in tiers: a core tier (“talking face,” “talking head,” “lip sync”) in every iteration; an extended tier (deepfake, portrait animation, reenactment, dubbing) from the second iteration; and a frontier tier only in the fourth, so that terminology which did not exist before 2025 could not distort coverage of older work. Six repositories were searched for disciplinary relevance and complementary coverage: IEEE Xplore, ScienceDirect, the ACM Digital Library, SpringerLink, Google Scholar with Web of Science, and arXiv, where landmark systems such as VASA-1 appeared months before formal publication. Four iterations ran between March 2023 and March 2026; a single frozen search would have missed the diffusion and Gaussian Splatting literature entirely. Table 1 reports retrieval counts.

3.4 Selection Criteria and Procedure

Table 2 states the inclusion and exclusion criteria, which operationalize the scope above. All of I1-I6 must hold for inclusion and any one of E1-E6 excludes.

Selection ran in three stages. Deduplication (DOI match, then title-plus-first-author matching at a 0.95 Levenshtein threshold) reduced 18,343 records to 8,741, and coarse title screening against I3-I5 left 448 for abstract review. Two reviewers screened all 448 abstracts independently (Cohen’s κ = 0.84), excluding 118 (off-topic DL application n = 64; not primary research n = 31; temporal n = 14; language n = 9); eight full texts proved unretrievable, and the remaining 322 were assessed against all criteria and the quality instrument below. Of these, 230 were excluded (opaque methodology n = 78; topic mismatch n = 91; tutorial or chapter n = 31; other n = 30), leaving 92 studies in the synthesis corpus.

3.5. Quality Assessment

Each of the 322 full texts was scored on five criteria adapted from Kitchenham and Charters [36]: problem clarity (QA1), methodological transparency (QA2), evaluation rigor (QA3), limitation acknowledgement (QA4), and contribution significance (QA5). Each criterion scores 1.0, 0.5, or 0.0; the inclusion threshold of 3.0 of 5.0 excludes fundamental deficiencies while admitting honestly reported early-stage work. Scores of the 322 full texts ranged from 0.5 to 5.0; 92 studies (28.6%) met the threshold, with a mean of 4.27 (SD = 0.39) among those included. QA1 was the criterion most often fully satisfied (93.5%), QA4 the most often only partial (41.3% scored 0.5), and QA2 was the most discriminating: studies scoring 0.0 on methodological transparency never exceeded 2.0 overall, making opacity the main driver of exclusion.

3.6. Extraction and Synthesis

A structured extraction form, piloted on 10 studies and refined once, captured bibliographic metadata, technical characteristics, datasets and metrics with quantitative results, five ethical and governance sub-fields (detection, watermarking/provenance, consent, risk discussion, regulatory reference), and deployment characteristics. The lead author extracted all 92 studies and a co-author independently verified a stratified 25% subsample (23 studies), with disagreements resolved against the source text and, in four cases, by email to the original authors. Given the heterogeneity of study types and outcomes, we adopted qualitative narrative synthesis following Cochrane guidance [44]: thematic synthesis within each RQ, followed by a cross-RQ analysis of patterns no single question exposes; quantitative results are tabulated where comparable but never statistically pooled. Where fewer than three independent studies support a finding, this is flagged in Section 4. The full bibliographic record of all 92 included studies, with extracted characteristics and quality scores, is provided as supplementary material (Appendix A).

3.7. Corpus Characteristics

The corpus comprises 56 conference papers (60.8%), 24 journal articles (26.1%), and 12 preprints (13.0%), consistent with a field whose flagship venues are conferences: CVPR leads with 22, followed by ECCV (9), ACM Multimedia (8), ICCV (7), and others. Annual output grows from 2 included studies in 2016-2017 to a roughly 20 per year across 2023-2025.

3.8. Threats to Validity

Four threats deserve note. The English-language restriction and database selection likely underrepresent Chinese-language venues, a real limitation given how much TFG research originates from Chinese institutions. Publication bias favors positive results; including arXiv preprints softens but does not remove it. The frontier described here is accurate to March 2026 and may be partially superseded at reading time; the multi-iteration search was designed to limit this decay. Finally, quality scoring involves judgement, particularly on QA3 and QA5; the κ = 0.84 agreement and explicit scoring descriptors constrain but do not eliminate subjectivity. RQ6 rests on a thinner evidence base than RQ1-RQ4, since primary studies rarely report deployment outcomes; the analysis there leans on the ECA literature and regulatory documents.

4. Data Synthesis and Analysis

4.1. Corpus Overview

The 92 studies span 2016 to 2025 and nine architecture families, drawing on 19 distinct named training and evaluation corpora (plus custom single-identity datasets) and 17 distinct evaluation metrics. GAN-based work is the largest single family (22 studies, 23.9%), a legacy of its 2018-2022 dominance, but the three post-GAN families together, Diffusion Models (16, 17.4%), Diffusion Transformers (7, 7.6%), and Gaussian Splatting (8, 8.7%), account for 33.7% of the corpus and overtake it. Two numbers foreshadow the RQ5 findings: only 8 studies (8.7%) contain any ethical content, and only 1 (1.1%) embeds a safeguard mechanism. Fewer than half (44, 47.8%) state a deployment domain at all.

4.2. The TFG Process Pipeline

Across all nine families, virtually every reviewed system decomposes into three sequential modules: feature extraction, a mapping network, and a transformation network (Figure 4). This shared scaffold is what makes cross-era comparison possible in RQ1, RQ3, and RQ4.

Feature extraction encodes the source image $x^I$ and audio sequence $x^A$ into latent representations:

$$F^I=E^I\left(x^I\right),$$

(1)

$$F^A=E^A\left(x^A\right),$$

(2)

where $E^I$ is an image encoder (a face-recognition backbone, or a VAE encoder in diffusion systems) and $E^A$ an audio encoder (a mel-spectrogram CNN or a pre-trained speech encoder). The mapping network learns the cross-modal correspondence between audio features and facial motion:

$$M_t=f_{map}\left(F_t^A,F^I\right),$$

(3)

This is the most architecturally variable component in the corpus, realised as recurrent lip-displacement regression (GAN era), audio-visual cross-attention (transformers), audio-conditioned deformation fields (NeRF), or conditioning vectors injected into the denoising network (diffusion); $M_t$ may cover lips, 3DMM expression coefficients, pose, gaze, and blinks. The transformation network then synthesises each frame from the source image and motion parameters:

$${\widehat{x}}_t=G\left(x^I,M_t\right),$$

(4)

The form of $G$ is what distinguishes the architectural eras: an adversarially trained generator with combined adversarial, lip-synchrony (SyncNet [39]), identity, and perceptual losses in GAN systems; iterative denoising conditioned on identity and motion in diffusion systems [10]; and, in Gaussian Splatting systems, differentiable rasterisation of learned 3D Gaussians whose position, opacity, covariance, and colour parameters are jointly optimised from the audio-conditioned motion prior.

4.3. RQ1: Architecture Evolution, 2016-2025

Figure 5 shows the distribution of included studies by primary architecture family and year. Five phases are identifiable, each anchored by a specific breakthrough.

Phase I, recurrent foundations (2016-2018). LSTM and bidirectional RNN models mapped mel-spectrogram frames to lip displacements: Fan et al. [45] set the deep recurrent baseline, and Suwajanakorn et al. [2] reached broadcast quality with 17 hours of single-speaker footage, at the cost of strict speaker dependence.

Phase II, GAN ascendancy (2019-2022). Adversarial training lifted the quality ceiling: Zhou et al. [46] disentangled audio and visual representations for speaker-agnostic synthesis, Wav2Lip [4] achieved the first in-the-wild lip sync by training against a pre-trained SyncNet discriminator, and MakeItTalk [47] and PC-AVS [48] added speaker-aware animation and pose control.

Phase III, transformers and NeRFs (2021-2023). FaceFormer [7] replaced recurrence with self-attention for long-range temporal coherence, while AD-NeRF [9] conditioned a radiance field on audio, giving 3D-consistent talking portraits at the price of hours of per-identity optimization and inference far below real time.

Phase IV, diffusion (2023-2024). DiffTalk [11] and DreamTalk [12] showed that audio- and identity-conditioned diffusion beats GANs on diversity and artefacts without mode collapse; SadTalker [40] added pose-controllable synthesis without per-identity tuning; and VASA-1 [13] closed the phase with a disentangled face latent space and flow matching, at 512×512 and 45 fps in real time.

Phase V, DiT and Gaussian Splatting (2024-2025). Two paradigms now co-define the frontier: DiTs replace the U-Net with a transformer over latent tokens and scale quality with compute [49], with OmniHuman-1 [14] dissolving the boundary between TFG and full-body digital humans, while Gaussian Splatting systems (GSTalker [50]; GaussianEmoTalker [51]; VASA-3D [52]) replace volume rendering with fast rasterization and remove the latency problem outright. Table 3 summarizes the full taxonomy.

The trajectory is a series of paradigm shifts, not a smooth curve, and each shift has an identifiable cause. GANs gave way to diffusion because mode collapse produced systematic artefacts in high-motion regions (teeth and hair most visibly). NeRFs gave way to Gaussian Splatting because volume rendering could not meet real-time requirements. U-Net diffusion is giving way to DiT because transformers scale better with parameter count. Phase V is distinguished by simultaneity: DiT-scale models produce the highest quality but demand server-class hardware, Gaussian Splatting runs in real time at quality sufficient for most deployments, and a hybrid, DiT-generated motion rendered through Gaussian Splatting, is the most promising route to closing that gap. Of the 41 corpus studies dated 2024 or later, 15 (36.6%) are DiT or Gaussian Splatting systems, while only one is GAN-based (2.4%).

4.4. RQ2: Dataset Biases and Generalization

Figure 6 shows dataset usage across the corpus: VoxCeleb2 dominates (60 studies, 65.2%), followed by HDTF (36, 39.1%) and MEAD (18, 19.6%); because most systems train and evaluate on several corpora, counts sum to more than the corpus size. A further 12 studies (13.0%) rely wholly or partly on custom single-identity corpora, and the 3D vertex-animation corpora VOCASET and BIWI (3 studies each) serve the FaceFormer line of work. Figure 6(b) characterizes the most used corpora.

The most consistent finding is under-documentation. Of the 19 named corpora identified, only CREMA-D reports ethnic diversity statistics in its own paper, and CREMA-D appears in exactly one included study. VoxCeleb2, which underpins nearly two-thirds of the corpus, reports subject counts and languages but nothing systematic about ethnicity, geography, or age. Studies running cross-demographic evaluations on it ([S16], [S21], [S31]) consistently note that South Asian, Middle-Eastern, African, and Latin American faces are severely underrepresented, several estimating these groups at under 8% of subjects combined. HDTF is built from 362 news and political broadcast subjects, predominantly white and East Asian male speakers of American or British English.

The skew is measurable in performance. Multiple studies report degraded lip-sync confidence when VoxCeleb2-trained models are evaluated on South Asian, African, and Middle-Eastern subjects; exact figures vary, but the direction is consistent and matches the face-recognition fairness literature [64]. The practical meaning is stark: a telehealth avatar trained on the dominant benchmark will articulate less accurately for the faces and languages of Sub-Saharan Africa, South Asia, and the Middle East, precisely where clinician shortages make AI-mediated care most valuable.

Two further gaps compound this. Affective coverage is thin: only MEAD (18 studies) and CREMA-D (1 study) carry emotion labels, and neither exceeds 91 subjects, so the emotion disentanglement reported by systems such as EmotiveTalk [20] and EAT [65] rests on very small affective corpora. Linguistic coverage is thinner still: 14 of the 19 named corpora are English-only, the VoxCeleb pair and CelebV-HQ are nominally multilingual but Western-dominant in practice, and ViCo (2 studies) is the sole substantively non-English corpus. Models trained on English phonemes produce implausible lip shapes for tonal languages (Mandarin, Thai, Yoruba) and for complex consonant clusters (Arabic, Polish).

The problem is not dataset size but diversity and documentation. Fixing it requires demographic audits as a publication prerequisite, open release of phonemically and ethnically diverse corpora, and demographic-stratified evaluation of the kind standard in face-recognition fairness work [64]; EU AI Act Article 10, requiring training data representative of the served population, now supplies regulatory pressure in the same direction.

4.5. RQ3: Evaluation Metrics and Deployment Fitness

Figure 7 shows metric frequency across the corpus. Distribution- and synchrony-level measures lead: FID appears in 60 studies (65.2%) and SyncNet-derived synchrony scores in 51 (55.4%), with the signal-fidelity pair SSIM (43, 46.7%) and PSNR (34, 37.0%) close behind and human MOS evaluation in a third of studies (33, 35.9%). Figure 7(b) assesses each metric family against three deployment-fitness criteria.

Three problems stand out. First, the dominance of PSNR and SSIM is hard to defend. Both measure pixel correspondence to a specific reference, which conflates quality with reference proximity: a slightly blurred but naturally animated face can score below a sharp but temporally incoherent one. The community has known PSNR is a poor perceptual proxy since at least 2012 [66]; its persistence in 2024-2025 papers is evaluation inertia, and it matters: a video that scores well on PSNR while looking uncanny to a patient is not fit for use as a virtual clinician.

Second, SyncNet [39] was trained on English broadcast correspondences, and no included study validates it on non-English phonemic inventories. It also measures correspondence, not naturalness: minimal lip movement scores well on neutral phonemes while missing the jaw openings, bilabial closures, and fricative patterns that make speech look real. Models optimized hard against SyncNet drift toward stiff, over constrained lip motion that is metrically accurate and perceptually unconvincing.

Third, the metrics that matter most for deployment remain rare or shallow. CSIM, the only standardized identity-preservation measure, appears in 15.2% of studies, and FVD, which captures the temporal coherence that per-frame metrics miss, in 13.0%. Human MOS evaluation is more widespread (35.9%) but typically shallow: panels of fewer than 30 raters are the norm, below accepted sample sizes for reliable MOS work, and no study recruits raters from its intended deployment population. The one metric whose adoption is clearly rising, FPS (15.2%), tracks the field’s real-time turn rather than any perceptual or governance property. Human perception remains the fitness measure that deployment actually turns on.

We therefore propose three metric dimensions as a research agenda. The Perceived Trustworthiness Score (PTS) is a human evaluation protocol in which raters drawn from the target deployment population (patients for healthcare avatars, students for educational agents) score credibility, warmth, and professional appropriateness on a multi-item Likert scale; it supplements rather than replaces MOS. The Cross-Cultural Authenticity Index (CCAI) extends SyncNet-style evaluation across at least five typologically distinct language families (for example Indo-European, Sino-Tibetan, Afro-Asiatic, Niger-Congo, Dravidian), exposing the cross-lingual limits of English-trained models and creating an incentive for diverse training data. The Deepfake Detectability Rate (DDR) is the proportion of generated frames correctly classified as synthetic by an ensemble of state-of-the-art detectors; it bears directly on EU AI Act Article 50, and systems below 80% DDR should carry mandatory C2PA watermarking as a compensating control. All three are proposed with provisional parameters and need validation through large, demographically diverse human studies before serving as standards.

4.6. RQ4: DiT and Gaussian Splatting Versus GANs and NeRFs

Figure 8 compares four architecture eras on visual quality, lip synchrony, and real-time capability; Table 4 reports benchmark data for representative systems.

On quality, the FID trajectory is unambiguous: GAN and early hybrid systems report FIDs in the 41-48 range (2021-2023), early diffusion at 35-47 in 2023, VASA-1 at 22.1, OmniHuman-1 at 18.4, a 55% relative reduction from the GAN baseline in four years, a pattern matching image generation more broadly, where DiT architectures keep improving predictably with scale.

On latency, 2024-2025 resolved the long-standing trade-off in three separate ways: flow matching (VASA-1 [13]) cut inference-time function evaluations without losing diversity; Gaussian rasterization (GSTalker [50]; VASA-3D [52]) replaced volume rendering for an order-of-magnitude speedup; and autoregressive streaming (Teller [60]) predicts motion tokens at sub-frame latency for continuous live output. Three architecturally unrelated solutions arriving at once suggest the barrier is genuinely down, not narrowly circumvented.

Controllability advanced in parallel and is invisible to FID and SyncNet. Wav2Lip [4] controlled lips only; PC-AVS [48] added pose; SadTalker [40] added stylistic motion via 3DMM coefficients; EmotiveTalk [20] decouples audio into content and emotion for independent affective control; and VASA-1 [13] disentangles pose, expression, gaze, blink, and speaking style into orthogonal latent dimensions with fine-grained post-hoc editing. For deployment this is not a luxury: a tutor that cannot adapt its emotional register teaches worse, and a clinician avatar with one fixed expression reads as uncanny.

4.7. RQ5: Ethical Safeguards and Regulatory Alignment

Figure 9 presents the starkest finding in the corpus. Of 92 included studies, only 8 (8.7%) contain any ethical content at all, and exactly 1 (1.1%) integrates a technical safeguard: a frequency-domain watermark added as a post-processing step, non-C2PA-compliant and untested against compression. No study implements a consent protocol. No study cites the EU AI Act, the TAKE IT DOWN Act, C2PA, GDPR Article 9, or any other governance instrument as a design constraint. Table 5 details the inventory.

Regulatory instruments are now in force and the gap is comprehensive. EU AI Act Article 50(2) requires machine-detectable marking by August 2026 (no reviewed system embeds a compliant manifest); Recital 134 requires marks robust to compression and conversion (no study tests robustness); Article 10 requires representative training data (no study audits its dataset demographically); and Article 14 requires meaningful human oversight (no system includes an oversight or revocation mechanism). The TAKE IT DOWN Act’s consent conditions go unaddressed, and C2PA is not referenced in a single included paper.

We do not read this as individual negligence: publication incentives reward benchmark quality, ethics review does not yet require safeguard implementation, and liability-creating regulation arrived only at the end of the review period. The result is a compliance cliff: a large body of deployable state-of-the-art systems facing EU AI Act obligations from August 2026 while satisfying none of them. Section 5 offers an operational path across.

4.8. RQ6: TFG in Socio-Technical Systems

Figure 10 shows the coded deployment domains. Most studies (48, 52.2%) specify no domain, treating TFG as a context-free technical problem; among the rest, entertainment leads (18, 19.6%), then education (9), healthcare (7), identity management (5), governance (3), and accessibility (2). The analysis below covers the four domains most consequential for this review, drawing on the 24 domain-specific studies plus the adjacent literature from Section 2.

Healthcare. DeTore et al. [67] found TFG-powered avatars could administer validated clinical instruments (PHQ-9, GAD-7) with response quality comparable to human-administered assessment; Haider et al. [18] found avatar-delivered patient education matched human instruction on comprehension while cutting clinician time by 34%. The risk profile has three parts: patients must know they are talking to an AI (an Article 50 obligation no healthcare-focused study reports implementing); the RQ2 performance gaps translate into reduced comprehension of clinical information, with patient safety implications; and liability for incorrect AI-delivered medical advice is unassigned in every jurisdiction we could identify.

Education. Chheang et al. [22] showed higher knowledge retention with TFG-powered VR anatomy assistants than with text; EmotiveTalk [20] sketches emotionally adaptive tutoring at scale, though the cross-lingual limits from RQ2 constrain language learning uses. The risks are distinctive: interactive tutors that read student facial affect are collecting biometric data governed by GDPR Article 9 and FERPA, which no included study addresses; the authority effect of face-to-face communication [42] may lead students to under-scrutinize AI-delivered content; and convergence on a few commercial avatar products could homogenize pedagogy.

Identity management. This is where the threat is already quantifiable. Venkatesh et al. [68] show that presentation attack detection (PAD) systems generalize poorly to attack types unseen in training. PAD calibrated on GAN artefacts (spectral inconsistencies, flicker, blink anomalies) can fail outright against the smooth, physiologically coherent output of flow-matching systems such as VASA-1 [13] or Gaussian Splatting systems such as GSTalker [50], which produce none of those signatures. The iProov threat intelligence report [69] documents sharply rising real-world PAD bypasses through 2023-2024, coinciding with diffusion TFG availability, and neither VASA-1 nor GSTalker has been formally evaluated against current liveness detection in any peer-reviewed study we identified. The institutional consequences are already on record: the USD 25 million Arup fraud [70], and Pindrop’s finding that over a third of a 300-profile applicant sample was wholly fabricated. A convincing deepfake video interview now costs a consumer GPU and a public model checkpoint. In systems terms, this erodes the trust infrastructure of digital identity itself [16]; no single countermeasure suffices, which is why Section 5 argues for coordinated governance across standards, law, platforms, and institutional practice.

Governance and democratic information. Political deepfakes of real politicians were documented in at least seven national election contexts between 2022 and 2024, and photorealistic talking faces raise the credibility and virality of false information, with lower-income and lower-digital-literacy populations disproportionately exposed. Detection is caught in an arms race: as generation improves (RQ4), detectors trained on older outputs decay. Mandatory provenance (EU AI Act Article 50; C2PA v2.0) is a structural exit from that race, binding content cryptographically to its generating model at creation so verification no longer depends on out-detecting generation quality. Table 6 summarizes the four domains.

4.9. Cross-RQ Patterns

Four patterns emerge only at the intersection of the research questions.

Optimization-deployment misalignment. The field optimizes PSNR, SSIM, and SyncNet; the deployment contexts that matter require identity preservation, cross-cultural authenticity, trustworthiness, and detectability, which the field does not measure. This reflects incentive structure, not oversight, and it will persist until venues and funders require PTS, CCAI, and DDR alongside FID.

Democratization and concentration at once. Gaussian Splatting and lightweight diffusion push high-fidelity TFG onto consumer hardware, while DiT-scale models (OmniHuman-1 [14]; EMO2 [71]) concentrate frontier capability in organizations with large compute budgets. The result is a two-tier field in which the hardest-to-detect systems are least accessible for beneficial use, and the most accessible systems are both the most useful and the easiest to misuse.

The absent safeguard as structural failure. That 91.3% of studies contain no ethical content and 98.9% no safeguard reflects the field’s incentive design. Ethics review at NeurIPS (2021) and CVPR (2023) has not measurably shifted the pattern, suggesting domain-specific requirements (watermarking, consent, DDR reporting) need to become submission conditions.

Governance lag as predictable risk. Each capability jump (2019 GAN photorealism, 2023 diffusion fidelity, 2024 real-time Gaussian Splatting) was followed by roughly 18-24 months of misuse before any governance response. The EU AI Act’s technology-neutral design may weather the next transition better than capability-specific rules; whether enforcement is agile enough is open.

5. The RTFG-STS Framework

5.1. Theoretical Foundations

The framework’s ethical core is the AI4People report of Floridi et al. [35], which joined the four bioethical principles of Beauchamp and Childress [72] (beneficence, non-maleficence, autonomy, justice) with a fifth, AI-specific principle, explicability. AI4People underpins the European Commission’s AI ethics guidelines and the philosophy of the EU AI Act [26], and it specifies what an ethical system must do, not only what it must avoid; on its own, though, it addresses the AI system in isolation. Socio-technical systems theory [15,16] supplies the missing half: a virtual clinician must be judged by its behavior inside a clinical workflow, its effect on patient trust, and its interaction with consent regulation, not by FID alone. Bijker’s concept of socio-technical stabilization [16] explains why the governance lag in Section 4.9 is costly: governance built after a technology hardens into institutional practice faces one that is already difficult to reshape. The RTFG-STS Framework therefore applies AI4People’s principles at the level of the deployed socio-technical system and adds a sixth principle, systemic embeddedness, which AI4People lacks and the STS analysis demands.

5.2. Six Ethical Principles of the RTFG-STS Framework

Table 7 states the six principles, their interpretation for TFG, and the operational mechanisms (Section 5.3) that implement each.

5.3. Five Operational Mechanisms

M1: C2PA-compliant watermarking and provenance (P2, P5). Every deployed TFG system must embed a cryptographically signed provenance manifest per C2PA v2.0, satisfying EU AI Act Article 50(2) and Recital 134:

$$C2PA\left(I_t\right)=Sign\left(Manifest\left(I_t,metadata,mode{l}_{hash}\right),K_{issuer}\right),$$

(5)

The manifest encodes the provenance claim, the generating model’s identity hash, a timestamp, hashes of the inputs, and the issuer’s certificate, signed with a key on the C2PA Trust List. Recital 134 requires robustness to H.264/H.265 compression, rescaling, and format conversion, a requirement the corpus’s single watermarking study does not meet; DCT-domain embedding with error-correction coding is current best practice pending a C2PA-TFG robustness standard, and systems should report the proportion of frames whose manifest survives compression, treating results below 80% as a barrier to high-risk deployment.

M2: Risk-tiered consent (P2, P3). Consent obligations scale with deployment risk, in three tiers aligned to the EU AI Act’s classification (Table 8). At Tier 3, consent is stored as a Verifiable Credential:

$$VC\left(I\right)=Sign\left(DI{D}_{subject}\parallel DI{D}_{issuer}\parallel consentScope\parallel timestamp\parallel revocationEndpoint,\hspace{0.25em}{K}_{subject}\right),$$

(6)

where the DIDs identify the consent subject and system operator, consentScope bounds permitted uses, and the revocation endpoint lets the subject withdraw consent, obliging removal of synthesized content within 48 hours, the TAKE IT DOWN Act timeframe [27].

M3: Socio-Technical Impact Assessment (P1, P4, P6). Before deployment beyond controlled research, the system undergoes a STIA, modelled on the EU AI Act’s Fundamental Rights Impact Assessment (Article 27) but extended to the full deployment context, and maintained as a living document across major revisions. Its six modules assess the target socio-technical system and the TFG system’s role in it; demographic performance equity (stratified evaluation plus CCAI); effects on the behavior, trust, and wellbeing of interacting humans (via PTS); the institutional processes and authority structures the system will intersect; the feedback loops likely to emerge over time; and regulatory compliance against the EU AI Act, the TAKE IT DOWN Act, GDPR, C2PA, and jurisdiction-specific rules, with a remediation timeline for gaps.

M4: Deepfake detectability audit (P2, P4, P5). DDR is defined as

$$DDR={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}1\left(D_{ensemble}\left(f_i\right)=\text{"}\mathrm{AI}-\mathrm{generated}\text{"}\right),$$

(7)

where $f_i$ is the i-th generated frame and $D_{ensemble}$ an ensemble of at least three state-of-the-art detectors, computed after H.264 compression at 2,000 kbps to approximate platform distribution. Three thresholds apply: at DDR ≥ 80%, content is reliably machine-detectable and a C2PA manifest suffices; between 60% and 80%, C2PA plus interaction-point disclosure is mandatory and Tier 3 deployment requires a regulatory waiver; below 60%, Tier 2-3 deployment is prohibited pending improvement, with research use permitted under manifest. The 80% floor follows NIST SP 800-76-2 guidance for detection rates in government identity contexts [73]; the 60% boundary matches the empirical detection floors reported for GAN-era detectors facing novel synthesis modalities [33,68]. Both are provisional pending a standardized TFG-specific DDR benchmark suite.

M5: Human oversight layer (P3, P5, P6). For Tier 3 deployments, a human must stand between generated output and consequential action, per EU AI Act Article 14. Concretely: a qualified clinician reviews clinical content an avatar delivers, with the AI nature disclosed to the patient; a trained reviewer verifies identity through a non-video channel before any decision rests on a video interview, with C2PA verification mandatory; a certified forensic reviewer attests provenance for TFG content in legal proceedings; and TFG video of public figures carries Article 50-compliant disclosure under the editorial responsibility of a publisher who accepts liability.

5.4. Application Across Deployment Domains

Table 9 translates the framework into concrete obligations for the four domains analysed in RQ6.

5.5. Framework Limitations

The framework is a normative proposal derived from the synthesis findings, AI4People, and STS theory; it has not been validated in live deployments, and pilot studies in healthcare and education are needed to test whether the STIA and consent tiers are operationally feasible. It is calibrated to the regulatory picture of December 2025 and should be re-checked against regulation annually. DDR depends on the currency of its detection ensemble: an ensemble calibrated to GAN artefacts says nothing valid about Gaussian Splatting output, so it must be refreshed annually, ideally against a public benchmark maintained by standards bodies. Finally, the framework specifies what responsible deployment requires but cannot enforce it; enforcement will come from venue ethics policies, funder conditions, and supervisory authorities, and advocacy toward venue-level adoption at CVPR, NeurIPS, and ACM MM is the most direct route to uptake.

6. Conclusions

6.1. Principal Findings

Across 92 primary studies from 2016 to 2025, the picture is of a field whose technical progress has consistently outrun its governance and evaluation. Architecturally (RQ1), TFG crossed five phases in nine years; the 2024-2025 frontier is contested between DiT and Gaussian Splatting, which together account for roughly 36% of the most recent studies and have jointly resolved the historic latency-quality trade-off. The training data beneath this achievement (RQ2) skews toward East Asian and Western European faces, with no dataset paper in the corpus reporting a demographic audit, so geographic and linguistic bias is written into the model weights serving the very populations the data underrepresents. Evaluation (RQ3) still runs on PSNR, SSIM, and SyncNet, none of which measures identity preservation, cross-cultural naturalness, trustworthiness, or detectability. Modern architectures (RQ4) delivered a 55% FID reduction over the GAN baseline and three independent routes to real time, yet this capability exists without safeguards (RQ5): zero systems with compliant watermarking, zero consent protocols, zero references to any regulatory instrument as a design constraint. The consequences (RQ6) are no longer hypothetical: USD 25 million lost to a single deepfake video call, systematic liveness bypass, and measurable erosion of trust in digital identity.

Read through STS theory, these findings share one cause: the field’s technological frame, its goals, metrics, and rewards, is built entirely around benchmark optimization, with no structural place for governance, equity, or deployment fitness. Widening that frame is beyond any individual researcher; it needs coordinated action from venues, funders, regulators, and the institutions now deploying TFG.

6.2. Implications

For researchers, the most actionable change is ethics-by-design reporting: DDR alongside FID and SyncNet, disclosed demographic audits of training data, and a stated deployment context with a fitness assessment against it. The cost is modest next to building a frontier system. For practitioners, the message is blunter: no currently available state-of-the-art system meets the obligations EU AI Act Article 50 imposes from August 2026, so deploying VASA-1, MuseTalk, or OmniHuman-1 in a patient-facing, student-facing, or KYC context today without C2PA watermarking, Tier 3 consent, and human oversight is a compliance liability with a dated deadline. For policymakers, three interventions exceed any current instrument: public funding for an open, demographically diverse, multilingual TFG dataset program; extension of ISO/IEC SC37 presentation attack standards to TFG synthesis attacks; and platform liability for hosting synthetic media without C2PA verification, which does not depend on winning the detection arms race.

6.3. Limitations

Four limitations apply. The English-language restriction underrepresents Chinese-language venues despite Chinese institutions being among the field’s most active contributors. The regulatory analysis is dated to December 2025 and may be sharpened by the EU AI Office’s forthcoming Code of Practice. The RQ6 analysis rests on 24 domain-specific studies plus adjacent literature, so its claims about political communication and education are propositions to test rather than settled findings.

6.4. Closing Statement

TFG can now synthesize photorealistic, real-time, emotionally controllable speech video from one image and an audio clip. Whether we can build it is settled; whether we can govern it is not. The EU AI Act, the TAKE IT DOWN Act, and C2PA v2.0 are the regulatory answers, and the RTFG-STS Framework is ours: embed consent, provenance, equity, and deployment fitness as design prerequisites rather than afterthoughts. Whether the next nine years produces systems that serve the rural clinic and the multilingual classroom without also arming the fraudster and the misinformed is not a technical question. It is a socio-technical one, and it deserves a socio-technical answer.