IterVocoder: Fast High-Fidelity Speech Synthesis via GAN-Guided Iterative Refinement

1. Introduction

Neural vocoders have emerged as a cornerstone in modern speech synthesis systems, serving as the critical component that transforms intermediate acoustic features into time-domain waveforms [1,2,3,4]. These models play a vital role in a wide range of speech-related tasks, including text-to-speech (TTS) [5,6,7,8,9,10], voice conversion [11,12], speech-to-speech translation [13,14,15], and enhancement [16,17,18,19]. Furthermore, neural vocoders have shown effectiveness in challenging scenarios such as restoration [20,21] and low-bitrate coding [22,23,24,25]. Among the early vocoder architectures, autoregressive (AR) models [1,26,27,28] set a new benchmark in audio quality, albeit with a major limitation: their sequential nature severely impedes real-time synthesis due to inherent dependencies across time steps.

To overcome this bottleneck, researchers have turned to non-autoregressive (non-AR) models, which enable parallel waveform generation and thus significantly improve inference efficiency. Notable approaches include flow-based vocoders [3,4,29], which exploit invertible neural transformations to map white noise to audio. More recently, GAN-based vocoders [31,32,33,34,35,36,37,38,39,40,41] have achieved impressive results by leveraging discriminators to refine generator outputs towards perceptual realism. These adversarial frameworks allow training on waveform-level losses, thereby producing sharper and more natural outputs compared to models trained solely on mean squared error or spectrogram losses.

In parallel, diffusion-based methods have introduced an alternative paradigm by reversing a noise corruption process in multiple steps to generate speech waveforms [42,43,44,45,46,47,48,49]. These denoising diffusion models (DDPMs) can match or exceed AR models in fidelity, but often require hundreds of iterative steps to reach optimal quality, leading to high computational costs. A fundamental trade-off exists between the number of refinement iterations and the quality of generated speech [42]. Various improvements have been proposed to address this, including better noise schedules [44], adaptive priors [45,46], and improved network structures [47,48]. Nevertheless, high-fidelity speech generation within a small number of iterations remains a significant challenge in practical settings.

Interestingly, recent work has revealed that GANs and diffusion models are not mutually exclusive and can be effectively combined [50,51]. For instance, Denoising Diffusion GANs predict clean signals from noisy inputs while adversarially regularizing the intermediate outputs. This dual mechanism enables more sample-efficient and perceptually aligned synthesis, and has already been applied in TTS settings [51] with promising outcomes. These hybrid frameworks open new possibilities for fast yet accurate synthesis architectures.

Motivated by this line of research, we propose a novel vocoder architecture called IterVocoder, which synergizes iterative denoising with adversarial optimization. Drawing inspiration from fixed-point iteration theory [52], our model repeatedly applies a shared neural mapping that progressively removes noise while minimizing a multi-resolution GAN loss at every iteration stage. Unlike traditional GANs which enforce realism at the final output only, our formulation encourages all intermediate steps to contribute constructively to the final waveform quality. This enforces convergence behavior while leveraging adversarial gradients to guide the denoising trajectory.

To further enhance spectral fidelity and temporal coherence, our loss combines GAN-based metrics with STFT-domain objectives [35], effectively balancing phase robustness and amplitude accuracy. Comprehensive subjective evaluations indicate that IterVocoder generates speech with quality comparable to natural recordings when using as few as five iterations. In addition, it achieves inference latency over 240× faster than conventional autoregressive models like WaveRNN [27], making it an attractive solution for real-time deployment.

In summary, our work demonstrates that integrating fixed-point refinement with adversarial supervision enables fast, stable, and high-quality neural vocoding. The proposed IterVocoder bridges the gap between fidelity and speed and offers a compelling alternative to both GAN-only and diffusion-only models for waveform synthesis.

2. Related Work

2.1. Advancements in Neural Vocoders

Neural vocoding has undergone a significant transformation in recent years, revolutionizing the synthesis of speech waveforms from abstract acoustic representations. Early works in this field were predominantly based on autoregressive (AR) models such as WaveNet [26], SampleRNN [1], WaveRNN [27], and LPCNet [28]. These models demonstrated outstanding performance in generating natural-sounding speech by modeling the conditional probability of waveform samples in a sequential manner. However, the sequential dependency inherent in AR models impedes parallelization, making real-time synthesis challenging.

To alleviate the latency issues of AR models, non-autoregressive (non-AR) neural vocoders were introduced. Normalizing flow-based approaches such as Parallel WaveNet [29], WaveGlow [3], and WaveFlow [4] employ invertible transformations to map noise vectors to waveform space. These models enable efficient parallel synthesis while maintaining decent audio quality. Despite these improvements, flow-based models often require carefully designed architectures and training tricks to ensure stability and fidelity.

2.2. GAN-Based Non-Autoregressive Vocoders

Generative adversarial networks (GANs) [31] have emerged as a powerful framework for non-AR waveform generation. In this context, a generator learns to synthesize waveforms from acoustic features, while a discriminator attempts to distinguish between real and synthesized signals. Pioneering efforts such as MelGAN [34], Parallel WaveGAN [35], and HiFi-GAN [33] showcased that GAN-based training objectives can significantly enhance the perceptual quality of generated speech. Variants such as UnivNet [38], BigVGAN [41], and iSTFTNet [39] further improved frequency resolution and generalization by incorporating multi-resolution spectrogram losses and discriminator designs tailored for fine-grained audio characteristics.

Nevertheless, GANs are known to suffer from stability issues during training and may produce artifacts such as pitch jitter or harmonic collapse. Recent work attempts to mitigate these issues via improved loss designs, better normalization strategies, and discriminator ensembles. However, GANs still generally require significant architectural and objective engineering to yield consistently high-quality audio.

2.3. Diffusion-Based Vocoders

Denoising diffusion probabilistic models (DDPMs) [42,43] have introduced a fundamentally different approach to speech synthesis. These models start from Gaussian noise and apply a series of denoising steps to gradually recover the waveform. Inspired by nonequilibrium thermodynamics, DDPMs offer a principled probabilistic framework and have demonstrated exceptional audio quality that matches or exceeds that of AR models. Subsequent improvements such as BDDM [44], PriorGrad [45], SpecGrad [46], and InferGrad [49] have focused on accelerating inference via better noise schedules, architectural modifications, and training procedures.

Despite their merits, a major drawback of diffusion-based vocoders is the high number of iterations required during inference, often ranging from 50 to 200, which increases latency and limits practical deployment. Strategies like progressive denoising and early stopping are commonly used but often at the cost of degraded output fidelity.

2.4. Combining GANs and Diffusion: Emerging Synergies

Recent studies suggest that GANs and diffusion processes are not mutually exclusive and can be integrated to harness the strengths of both [50,51]. Denoising Diffusion GANs [50] combine adversarial supervision with diffusion-style denoising to achieve improved convergence and sample quality. This hybridization allows the model to produce high-fidelity outputs in fewer iterations while benefiting from the perceptual realism that GANs offer. Similar concepts have been applied to TTS pipelines [51], where the spectrogram generation phase is guided through both adversarial and denoising objectives.

While these hybrid methods are promising, they are often complex and require careful calibration between iterative refinement and adversarial constraints. Furthermore, most existing works focus on mel-spectrogram generation rather than direct waveform synthesis.

2.5. Fixed-Point Iteration: A Theoretical Foundation for Iterative Refinement

The concept of fixed-point iteration [52] offers a rigorous theoretical foundation for iterative refinement methods. In this framework, a function $f\left(x\right)$ is repeatedly applied to an input until it converges to a point $x^{*}$ such that $f\left(x^{*}\right)=x^{*}$. This concept has found applications in numerical solvers, denoising algorithms, and more recently in deep learning architectures that benefit from iterative feedback loops. Applying fixed-point iteration to waveform denoising implies that a denoising network can be repeatedly applied until its output stabilizes—conceptually aligning with the structure of DDPMs and suggesting convergence properties beneficial for vocoding.

2.6. Positioning Our Work: The Contribution of IterVocoder

Our work builds upon these foundations by proposing a non-autoregressive neural vocoder— IterVocoder—that fuses fixed-point iteration with adversarial learning. Unlike prior GAN-based vocoders that generate the waveform in a single forward pass, IterVocoder applies a denoising transformation iteratively, each time refining the waveform closer to the natural target. Unlike conventional DDPMs, the refinement process is accelerated and guided by adversarial feedback, applied not only at the final output but across all intermediate iterations.

Additionally, our model introduces a composite loss that includes adversarial, STFT-domain, and multi-scale time-domain components, promoting spectral fidelity and perceptual realism. This design enables IterVocoder to synthesize natural and intelligible speech in as few as five iterations, significantly outperforming conventional DDPM-based systems in both speed and quality.

In summary, IterVocoder stands at the intersection of several research threads—non-autoregressive vocoding, adversarial learning, diffusion processes, and fixed-point theory—combining their strengths into a coherent and efficient architecture for real-time, high-quality waveform synthesis.

Figure 1. Overview of the overall framework.

Figure 1. Overview of the overall framework.

3. Methodology

This section presents IterVocoder, a novel non-autoregressive neural vocoder that synergistically combines the strengths of denoising diffusion probabilistic models (DDPMs), generative adversarial networks (GANs), and fixed-point iteration theory. Our aim is to generate natural-sounding waveforms with high fidelity and fast inference through a learnable iterative denoising framework. By unifying these three modeling paradigms, IterVocoder achieves the dual objective of generation quality and inference efficiency—two crucial properties in real-world speech synthesis systems.

We begin by outlining the neural vocoding task formulation, followed by a detailed review of diffusion-based and GAN-based approaches. We then reinterpret denoising processes from the perspective of fixed-point iterations and finally present our proposed iterative refinement architecture, its learning objectives, and the joint training mechanism. The overall pipeline reflects a principled blend of iterative modeling and adversarial training, grounded in signal processing theory and deep generative modeling.

3.1. Neural Vocoding Task Definition

Neural vocoding refers to the process of reconstructing high-fidelity speech waveforms from compact acoustic representations such as mel-spectrograms or cepstral features. Let $\mathit{c}=({\mathit{c}}_1,...,{\mathit{c}}_K)\in {\mathbb{R}}^{FK}$ denote the conditioning sequence, where each ${\mathit{c}}_k\in {\mathbb{R}}^F$ represents the spectral feature at frame k, and F is the feature dimensionality. The goal is to learn a function ${\mathcal{F}}_{\theta }$ that generates a waveform ${\mathit{y}}_0\in {\mathbb{R}}^D$ such that ${\mathit{y}}_0\approx {\mathit{x}}_0$, where ${\mathit{x}}_0$ is the reference natural speech.

This conditional generative task poses multiple challenges: (1) modeling temporal dependencies without introducing excessive latency, (2) producing perceptually natural audio, and (3) ensuring stable training and inference. IterVocoder is specifically designed to tackle all three by employing a principled iterative update mechanism inspired by fixed-point theory, while incorporating perceptual loss functions from GAN-based models.

3.2. Preliminary: DDPM-Based Neural Vocoders

DDPMs formulate waveform generation as a reverse diffusion process, progressively denoising Gaussian noise into coherent audio signals. The generative process follows a Markov chain, whereby each step stochastically refines a noisy sample conditioned on both the prior step and the input features $\mathit{c}$.

The forward diffusion process adds noise to the target waveform over T timesteps, defined as:

$$\begin{array}{c}\hfill p\left({\mathit{x}}_t\right|{\mathit{x}}_{t-1})=\mathcal{N}(\sqrt{1-{\beta }_t}{\mathit{x}}_{t-1},{\beta }_t\mathit{I}),\end{array}$$

(1)

where ${\beta }_t$ controls the noise schedule. The cumulative formulation enables direct sampling from intermediate states:

$$\begin{array}{c}\hfill {\mathit{x}}_t=\sqrt{{\overline{\alpha }}_t}{\mathit{x}}_0+\sqrt{1-{\overline{\alpha }}_t}\mathit{ϵ},\mathit{ϵ}\sim \mathcal{N}(0,\mathit{I}).\end{array}$$

(2)

To recover ${\mathit{x}}_0$, a neural network is trained to estimate the noise component $\mathit{ϵ}$, optimized via the loss:

$$\begin{array}{c}\hfill {\mathcal{L}}^{WG}={∥\mathit{ϵ}-{\mathcal{F}}_{\theta }({\mathit{x}}_t,\mathit{c},{\beta }_t)∥}_2^2.\end{array}$$

(3)

While DDPMs offer remarkable synthesis quality, they require hundreds of denoising iterations for convergence, making them impractical in latency-sensitive applications. Consequently, various refinements have been proposed to reduce the number of steps without compromising quality.

SpecGrad Loss

SpecGrad modifies the diffusion prior to incorporate structured noise shaped by spectral energy distributions. This results in a more effective supervision signal, particularly in frequency-sensitive synthesis tasks:

$$\begin{array}{c}\hfill {\mathcal{L}}^{\mathrm{SG}}={∥{\mathit{L}}^{-1}(\mathit{ϵ}-{\mathcal{F}}_{\theta }({\mathit{x}}_t,\mathit{c},{\beta }_t))∥}_2^2.\end{array}$$

(4)

Here, $\mathit{L}={\mathit{G}}^{+}\mathit{M}\mathit{G}$ acts as a spectrally informed transform matrix, emphasizing perceptually critical regions in the loss computation.

InferGrad Strategy

To address inconsistencies between intermediate outputs and the final waveform, InferGrad adds a reconstruction loss on the ultimate prediction:

$$\begin{array}{c}\hfill {\mathcal{L}}^{IG}={\mathcal{L}}^{WG}+{\lambda }_{\mathrm{IF}}·{\mathcal{L}}^{\mathrm{IF}}({\mathit{y}}_0,{\mathit{x}}_0),\end{array}$$

(5)

where ${\mathcal{L}}^{\mathrm{IF}}$ captures perceptual dissimilarities between generated and real signals, facilitating more stable inference-time behavior.

3.3. GAN-Based Vocoders

GAN-based vocoders directly synthesize waveforms from conditioning features using a generator-discriminator paradigm. The generator ${\mathcal{F}}_{\theta }$ learns to produce signals that fool a set of discriminators ${\left\{{\mathcal{D}}_r\right\}}_{r=1}^R$, each operating at different resolutions or domains.

The adversarial objective is formulated as follows:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{Gen}}^{\mathrm{GAN}}& ={\displaystyle \frac{1}{R}}\sum _{r=1}^R-{\mathcal{D}}_r\left({\mathit{y}}_0\right)+{\lambda }_{\mathrm{FM}}{\mathcal{L}}_r^{\mathrm{FM}}({\mathit{x}}_0,{\mathit{y}}_0),\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{Dis}}^{\mathrm{GAN}}& ={\displaystyle \frac{1}{R}}\sum _{r=1}^{R}max(0,1-{\mathcal{D}}_r\left({\mathit{x}}_0\right))+max(0,1+{\mathcal{D}}_r\left({\mathit{y}}_0\right)).\hfill \end{array}$$

(7)

Here, ${\mathcal{L}}^{\mathrm{FM}}$ denotes the feature matching loss, which stabilizes training by aligning internal activations between real and generated samples.

Multi-Resolution STFT Loss

In addition to adversarial losses, STFT-based criteria provide fine-grained frequency-domain supervision:

$$\begin{array}{c}\hfill {\mathcal{L}}^{\mathrm{MR}-\mathrm{STFT}}={\displaystyle \frac{1}{R}}\sum _{r=1}^R\left[{\displaystyle \frac{\parallel {\mathit{X}}_r-{\mathit{Y}}_r{\parallel }_2}{\parallel {\mathit{X}}_r{\parallel }_2}}+{\displaystyle \frac{1}{N_r{K}_r}}{\parallel log{\mathit{X}}_r-log{\mathit{Y}}_r\parallel }_1\right],\end{array}$$

(8)

where ${\mathit{X}}_r$, ${\mathit{Y}}_r$ are the STFT spectrograms of real and generated signals at resolution r. This loss emphasizes both spectral amplitude and structure.

3.4. Fixed-Point Iteration in Neural Vocoding

The concept of fixed-point iteration originates from numerical analysis, where repeated applications of a contraction mapping $\mathcal{T}$ drive convergence to a stable solution. In vocoding, we reinterpret the denoising operation as a fixed-point process:

$$\begin{array}{c}\hfill {\parallel \mathcal{T}\left(\mathit{\xi }\right)-\mathit{\varphi }\parallel }_2\le {\parallel \mathit{\xi }-\mathit{\varphi }\parallel }_2,\end{array}$$

(9)

ensuring convergence under contraction. We define:

$$\begin{array}{c}\hfill \mathcal{T}\left({\mathit{y}}_t\right)=\mathcal{G}({\mathit{y}}_t-{\mathcal{F}}_{\theta }({\mathit{y}}_t,\mathit{c},t),\mathit{c}),\end{array}$$

(10)

as our iterative refinement operator, where $\mathcal{G}$ adjusts the residual to align with perceptual energy constraints.

3.5. Proposed IterVocoder Framework

The core of IterVocoder lies in its learned iterative refinement steps:

$$\begin{array}{cc}\hfill {\mathit{z}}_t& ={\mathit{y}}_t-{\mathcal{F}}_{\theta }({\mathit{y}}_t,\mathit{c},t),\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathit{y}}_{t-1}& =\mathcal{G}({\mathit{z}}_t,\mathit{c}),\hfill \end{array}$$

(12)

where ${\mathit{z}}_t$ is the residual error, and $\mathcal{G}$ enforces spectral energy consistency via a normalization:

$$\begin{array}{c}\hfill P_c=\sum _k\sum _f{c}_{k,f}^2,P_z=\sum _d{z}_{t,d}^2.\end{array}$$

(13)

The output ${\mathit{y}}_{t-1}$ is scaled such that $P_z\approx P_c$, matching the energy profile of the input features.

Final Objective

We jointly supervise all intermediate outputs to encourage early convergence:

$$\begin{array}{c}\hfill {\mathcal{L}}_{\mathrm{total}}={\displaystyle \frac{1}{T}}\sum _{t=0}^{T-1}\left[{\mathcal{L}}_{\mathrm{Gen}}^{\mathrm{GAN}}({\mathit{x}}_0,{\mathit{y}}_t)+{\lambda }_{\mathrm{STFT}}·{\mathcal{L}}^{\mathrm{STFT}}({\mathit{x}}_0,{\mathit{y}}_t)\right].\end{array}$$

(14)

This formulation ensures that each refinement step contributes to the fidelity and realism of the final waveform, allowing IterVocoder to achieve high-quality synthesis in only a few iterations.

4. Experiments

In this section, we present a comprehensive evaluation of our proposed vocoder framework, which we call IterVocoder. We conduct both subjective and objective experiments to validate the audio quality, inference efficiency, and robustness of IterVocoder in comparison to various representative baselines, including autoregressive (AR), GAN-based, and DDPM-based neural vocoders. Our experiments are designed to answer the following key questions: (1) How does IterVocoder perform in terms of speech naturalness and generation speed? (2) How does it compare to prior diffusion-based and GAN-based systems? (3) What are the effects of key components such as fixed-point iteration, multi-resolution losses, and adversarial training?

4.1. Model Architecture and Implementation Details

• Architecture Overview: The IterVocoder framework is built on top of the WaveGrad Base model [42], consisting of 13.8M trainable parameters. For the denoising network $\mathcal{F}$, we adopt a U-Net-like architecture equipped with residual connections, layer normalization, and dilated convolutions for long-range temporal modeling. For adversarial supervision, we employ three GAN discriminators ${\left\{{\mathcal{D}}_r\right\}}_{r=1}^{R_{\mathrm{GAN}}}$ at different temporal resolutions (original, 2x, and 4x down-sampled) using the MelGAN backbone [34]. Each discriminator processes audio segments and outputs multi-frame logits, which are then aggregated through averaging.
• Noise Initialization and Conditioning: The initial noisy input ${\mathit{y}}_T\sim \mathcal{N}(\mathbf{0},\Sigma )$ is sampled using the SpecGrad algorithm [46]. For conditional input $\mathit{c}$, we compute 128-dimensional log-mel spectrograms using a 24 kHz sampling rate with a Hann window of 50 ms, 12.5 ms frame shift, and a 2048-point FFT.
• Loss Functions: Our generator is jointly supervised by the following loss functions:

  -

  Adversarial Losses: Generator and discriminator losses are defined via Equations Equation ?? and Equation ??, respectively.

  -

  Multi-resolution STFT Loss: Using three STFT settings ([360, 900, 1800], [80, 150, 300], and [512, 1024, 2048]), we apply ${\mathcal{L}}^{\mathrm{MR}-\mathrm{STFT}}$ to capture spectral fidelity at different temporal scales.

  -

  Mel-spectrogram Amplitude Loss: MAE is computed between the 128-dimensional mel features of generated and reference audio.

4.2. Training Setup and Baseline Models

• Dataset and Training: For training, we use a proprietary 184-hour US English dataset and the LibriTTS dataset [67]. Training is done using 128 Google TPU v3 cores with a global batch size of 512. We crop 1.5-second segments for training and follow optimizer hyperparameters from WaveGrad.
• Loss Weighting: On the proprietary dataset, we set ${\lambda }_{\mathrm{FM}}=100$ and ${\lambda }_{\mathrm{STFT}}=1$ following SEANet [64]; for LibriTTS, we use ${\lambda }_{\mathrm{FM}}=10$ and ${\lambda }_{\mathrm{STFT}}=2.5$, and omit the mel amplitude loss.
• Baselines: We include:

  -

  AR Baseline: WaveRNN [27], trained for 1M steps.

  -

  DDPM Baselines: SpecGrad [46] and InferGrad [49] with varying noise schedules and fine-tuning steps.

  -

  GAN Baselines: HiFi-GAN [33] and MB-MelGAN [36], using checkpoints provided in [66].

4.3. Objective Evaluation of Iterative Denoising

We analyze how well the intermediate outputs ${\mathit{y}}_t$ approach the clean target ${\mathit{x}}_0$ over iterations. We evaluate log-magnitude absolute error ${\mathcal{L}}^{\mathrm{Mag}}$ and spectral convergence ${\mathcal{L}}^{\mathrm{Sc}}$ across three STFT settings with different configurations than used in training. Results show that IterVocoder consistently improves both metrics per iteration and converges faster than SpecGrad or InferGrad. Unlike DDPM-based models which suffer from artifacts due to noise injection, IterVocoder maintains smoother transitions across iterations.

4.4. Main Results: MOS and Inference Speed

We report mean opinion scores (MOS) and real-time factors (RTFs) for all competing vocoder models across 1,000 test utterances drawn from both proprietary and public evaluation datasets, as shown in Table 1. These two metrics serve complementary purposes: MOS provides a human-centric perceptual quality rating, while RTF reflects computational efficiency, a critical aspect in real-time deployment scenarios.

IterVocoder achieves the best MOS among all DDPM-based vocoders and simultaneously delivers significant inference speed improvements. Specifically, with just two iterations, IterVocoder-2 attains a MOS of 4.13, surpassing both InferGrad-2 and SpecGrad-3 while being computationally more efficient. As the number of iterations increases, the perceptual quality steadily improves, culminating in IterVocoder-5 reaching a MOS of 4.44. This score is statistically on par with WaveRNN (MOS 4.41), a strong autoregressive baseline, but with a staggering reduction in inference time: the RTF of IterVocoder is below 0.07, compared to over 17.0 for WaveRNN, indicating more than 240x speedup.

This result highlights the advantage of combining fixed-point iteration and adversarial training: the model converges to high-fidelity audio in only a few steps. Moreover, it demonstrates that IterVocoder’s iterative mechanism captures the denoising trajectory more effectively than conventional DDPMs. Notably, this balance between efficiency and quality is achieved without sacrificing stability, making IterVocoder a practical alternative for latency-sensitive applications such as TTS or live voice conversion.

4.5. Side-by-Side Preference Results

Side-by-side (SxS) preference testing offers a more fine-grained perceptual comparison than MOS by forcing listeners to choose between paired samples. Table 2 summarizes these preference outcomes. We observe a strong listener bias toward IterVocoder-3 over InferGrad-3 (SxS score: 0.375 ± 0.073, $p<0.001$), confirming its superiority at similar iteration depths.

Interestingly, when comparing IterVocoder-3 to WaveRNN, the preference difference becomes statistically smaller and even reverses slightly in favor of WaveRNN (SxS: -0.051). However, when moving to IterVocoder-5, the preference margins narrow further, with differences against WaveRNN and Ground-truth being statistically insignificant. For instance, the SxS score of IterVocoder-5 vs WaveRNN is only -0.018 ($p=0.29$), and vs Ground-truth is -0.027 ($p=0.056$), indicating perceptual parity in practical terms.

These findings corroborate the MOS trends and illustrate that IterVocoder can deliver naturalness indistinguishable from AR and real human recordings, provided enough iterations are applied. Additionally, it demonstrates that preference saturation occurs beyond three iterations, suggesting diminishing perceptual returns beyond that point.

4.6. Evaluation on GAN Baselines and Robustness

To further benchmark IterVocoder against high-performing GAN-based vocoders, we evaluate performance on the LibriTTS test set. As shown in Table 3, IterVocoder-5 obtains a MOS of 3.98, marginally lower than HiFi-GAN V1 (4.03) but significantly higher than MB-MelGAN (3.37). In side-by-side comparisons, listeners strongly prefer IterVocoder-5 over MB-MelGAN ($p<0.001$), while its results are statistically tied with HiFi-GAN V1.

Despite strong perceptual results, we did observe rare but noticeable synthesis artifacts during qualitative inspection. Specifically, IterVocoder sometimes introduces pulsive distortions when exposed to noisy or reverberant acoustic conditions during training. These artifacts are hypothesized to stem from mismatches between training and inference noise schedules and the generator’s overreliance on mel-spectral consistency.

To improve robustness, future work could incorporate multi-condition training, stochastic conditioning paths, or adversarial augmentation methods that better simulate real-world acoustics. Additionally, architectural advances such as incorporating dual-domain constraints or perceptual regularizers could enhance performance under noisy or mismatched input conditions.

In summary, our proposed IterVocoder framework consistently demonstrates high-quality audio synthesis capabilities while achieving superior inference speed. It significantly outperforms traditional DDPM-based systems in both perceptual quality and efficiency and competes with autoregressive and GAN-based models in listener evaluations. Moreover, its convergence speed and iteration efficiency suggest strong potential for real-time applications. Future efforts can focus on enhancing noise robustness, exploring adaptive iteration strategies, and extending to multilingual or zero-shot vocoding scenarios.

5. Conclusion and Future Directions

This paper introduced IterVocoder, a novel neural vocoder architecture that fuses the strengths of fixed-point iteration and adversarial training. Inspired by the convergence behavior of denoising diffusion models (DDPMs), yet free from the stochastic degradation steps, IterVocoder performs deterministic iterative refinement toward clean speech, using a generator optimized under adversarial and spectral consistency losses. This design allows our model to benefit from the stable reconstruction trajectories of DDPMs while accelerating convergence by integrating GAN-based learning signals. Distinct from conventional diffusion models, which rely on pre-defined noise schedules and often require hundreds of sampling steps, IterVocoder operates under a deterministic iterative framework that dramatically reduces the number of denoising iterations without compromising output quality. By interpreting waveform refinement as a fixed-point optimization problem, our model adapts its generator architecture to efficiently traverse the speech manifold toward a clean target signal. This perspective not only provides theoretical grounding but also motivates a unified training objective that balances fidelity and efficiency.

Through extensive subjective listening experiments, we demonstrated that IterVocoder generates speech waveforms of high perceptual quality. Specifically, our model surpassed state-of-the-art DDPM-based vocoders across multiple test conditions and listener evaluations. When configured with only five iterations, IterVocoder achieved a MOS score statistically comparable to that of WaveRNN and natural human speech, while reducing the inference latency by over 240 times. These findings underscore the practicality of IterVocoder in real-time speech synthesis scenarios where both quality and speed are essential. Moreover, preference-based evaluations revealed that IterVocoder outperforms recent GAN-based models like MB-MelGAN and achieves competitive performance with HiFi-GAN V1. This suggests that our framework is capable of capturing fine-grained audio features crucial for human perception. However, we also identified edge cases where synthesis artifacts emerged under acoustically mismatched or noisy training conditions, pointing to opportunities for robustness improvements.

Looking forward, several promising research directions remain. First, future work could investigate incorporating stochastic variation during inference to increase diversity and expressiveness. Second, further gains in generalization and robustness may be achieved through domain-aware data augmentation strategies, multi-resolution supervision, or hybrid training regimes that combine frame- and waveform-level objectives. Third, extending the IterVocoder architecture to accommodate multilingual or code-switched speech corpora would enable broader applicability. Lastly, combining our framework with attention-based contextual modules may enable expressive TTS synthesis with emotion, speaker identity, or prosody control. In conclusion, IterVocoder represents a significant step toward efficient and perceptually faithful neural vocoding. By bridging deterministic iteration with adversarial learning, it sets a new foundation for future work in high-speed, high-fidelity speech synthesis systems.