Submitted:
11 November 2026
Posted:
13 November 2026
You are already at the latest version
Abstract
Continuous-space diffusion and flow language models generate text by updating real-valued vectors (such as token embeddings, compressed codes, or sentence vectors) and decoding them into discrete words at readout. We organize this literature by three coupled choices: the continuous state being refined, the transition operator that updates it, and the text interface that converts states into tokens. Because language is discrete while the state space is continuous, adapting image diffusion to text encounters rounding ambiguity and embedding collapse under squared-error regression, uninformative noise schedules, distribution shift between clean-text codecs and learned priors, and mode averaging in few-step sampling. Across published comparisons with autoregressive and discrete-diffusion baselines, continuous models show no general advantage over same-scale autoregressive models on open-ended text; the closest prompt-continuation tie uses different training sequence lengths and separately tuned samplers. RePlaid matches or improves upon masked diffusion in likelihood bounds past compute-optimal budgets, and large-scale blockwise adaptation brings continuous models close to masked baselines on mathematical reasoning and code; however, continuous models still trail autoregressive baselines in held-out likelihood bounds, and scaling fits indicate that matching exact autoregressive negative log-likelihood requires substantially more training compute. Their clearest gains over autoregressive decoding appear in gradient-guided control, plan-level diversity, and task-specific reasoning after targeted fine-tuning. Under few-step budgets, trajectory-consistent distillation and stochastic bridges prevent entropy collapse, yet no standalone continuous generator has shown an end-to-end latency win over optimized autoregressive decoding at matched quality; the reported matched-quality latency gains rely on autoregressive verification of continuous drafts. We close with open problems in likelihood scaling, readout fidelity, few-step serving, representation reuse, and post-training.
Keywords:
diffusion language models
; flow matching
; continuous-space text generation
; latent diffusion
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.