DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models

1. Introduction

Diffusion large language models (D-LLMs) have recently gained increasing attention as an emerging paradigm for efficient and high-quality text generation [1,2,3]. Rather than decoding tokens strictly from left to right [4,5,6], D-LLMs generate a sequence through iterative denoising. Despite their advantages over autoregressive LLMs, D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually incorrect or unsupported content [7,8]. Detecting such errors from output-level evidence (i.e., predictive uncertainty of output tokens) is often insufficient [9,10,11], because hallucination-related cues may appear during the denoising process in the form of uncertainty rebound or inconsistent refinement among token [12,13]. This motivates D-LLM-specific hallucination detection methods that explicitly model denoising trajectories, rather than solely relying on static features of the generated output [14,15].

Recent studies have begun to exploit predictive uncertainty throughout the denoising trajectory, typically quantified by token-level entropy, for hallucination detection in D-LLMs [16,17]. One line of work focuses on step-level evidence, which compresses the denoising trajectory along the diffusion-step dimension by selecting or reweighting the steps whose uncertainty patterns are most informative for hallucination prediction [18,19]. Another line models token-level uncertainty dynamics, which compresses the trajectory along the token dimension by identifying hallucination-indicative tokens and using their denoising dynamics as indicators of factual reliability [20]. Although these methods demonstrate the usefulness of uncertainty at intermediate denoising states, both lines of works compress the original trajectory along one dimension before detection. As a result, the token-step structure of D-LLM denoising trajectories, which jointly captures how token-level uncertainty evolves across diffusion steps, is not fully preserved, potentially leading to information loss for hallucination detection.

To investigate whether compressing denoising trajectories may overlook critical cues for hallucination detection, we examine the evolution of token-level uncertainty under different Q&A scenarios, and then identify two representative patterns that require explicit modeling of token-step uncertainty evidence. Pattern ❶: Inconsistent Convergence in Hallucination Answers (Fig. Figure 1a). In hallucinated responses, token uncertainties may converge to markedly different levels during the late denoising stage: some tokens become highly confident, while others remain relatively uncertain. Such inconsistent convergence across tokens reveals abnormal inter-token relations and provides hallucination evidence beyond any individual token. Pattern ❷: Cross-Token Fault Propagation in Hallucination Answers (Fig. Figure 1b). During the denoising process of some hallucination answers, an unstable or incorrect token state can affect the decoding of other contextually connected tokens, causing their entropy trajectories to deviate together from a coherent factual path. Capturing this propagative pattern for hallucination detection requires explicit modeling of dynamic dependencies among token trajectories, further highlighting the necessity of preserving the full token-step uncertainty structure. Recognizing the importance of two-dimensional evidence modeling, we pose a central research question:

A promising solution to this question is to formulate the token-step uncertainty trajectory as a multivariate time series, which naturally preserves its two-dimensional structure: denoising steps define the temporal axis, while token-associated uncertainty signals serve as multiple interacting variables. Although this formulation is natural, directly applying existing approaches for multivariate time series to D-LLM hallucination detection is non-trivial, since they typically assume that each variable has a stable identity over time and among different variables [21,22]. This assumption does not hold for D-LLMs, where the semantic role of a potential variable (i.e. token position) may vary across samples and change during denoising process, thus giving rise to two key challenges. Challenge 1: Stable Variable Construction. Raw token positions cannot directly serve as variables, as they lack consistent semantic identities. They must be reorganized into semantically stable variables while preserving fine-grained denoising dynamics relevant to hallucination detection. Challenge 2: Relational Temporal Modeling. The temporal dynamics of the constructed variables are coupled with evolving dependencies among them. In this case, the hallucination detector must jointly capture both aspects, as hallucination cues can manifest as inconsistent convergence or propagate across related trajectories.

To address these challenges, we propose DeMTS, a hallucination detection framework that formulates D-LLM Denoising trajectories as Multivariate Time Series over learnable latent variables. To handle Challenge 1, DeMTS introduces a trajectory-preserving Token-to-Variable assignment (T2V) module, which reorganizes token uncertainty trajectories into latent variables with more consistent identities, considering uncertainty states, contextual semantics, and positional structure. A trajectory-preservation constraint is further imposed to retain local denoising variations during token-to-variable assignment. To address Challenge 2, DeMTS introduces Dynamic Multivariate Temporal Modeling (DMTM), which jointly captures relational and temporal patterns over the constructed latent variables. It performs step-adaptive inter-variable interaction to suppress unreliable early-step dependencies and strengthen interactions among stabilized variables, followed by variable-wise temporal modeling to encode the evolution of each latent variable. In this way, DeMTS captures both hallucination-related temporal dynamics and cross-token dependencies. In summary, our contributions are as follows:

• We formulate D-LLMs hallucination detection as a multivariate time series learning problem, and identify two key challenges in adapting existing time series approaches to denoising trajectories.
• We propose DeMTS, a trajectory-based detector which transforms semantically unstable token signals into stable latent variables and jointly captures dynamic inter-variable dependencies and variable-wise temporal evolution.
• Experiments on two D-LLM backbones and three benchmarks show that DeMTS consistently outperforms baselines, while remaining robust, transferable, and efficient.

2. Related Works

Hallucination Detection in D-LLMs. The iterative denoising process of D-LLMs [23] provides rich trajectory-level uncertainty evidence for hallucination detection [16]. One line of existing methods focuses on step-oriented evidence selection [18,24,25], which identifies or reweights informative denoising stages and uses the selected trajectory segments for hallucination prediction. The second line focuses on dynamics-oriented trajectory modeling [20], which detects hallucinations by modeling abnormal token evolution during denoising. They show that hallucination evidence can emerge from discriminative denoising steps or dynamic trajectory patterns. Differently, DeMTS models D-LLMs two-dimensional denoising trajectories as multivariate time series for hallucination detection.

Multivariate Time Series Learning. Deep learning for multivariate time series has been widely studied to capture evolving patterns across multiple variables [26,27,28]. Existing methods are commonly categorized by their channel modeling strategies. Channel-independent methods model each variable separately to preserve variable-specific temporal patterns [29,30,31], while channel-dependent methods jointly model all variables to capture global inter-variable correlations [22,32,33]. More recent channel-partial methods seek a balance between these two views by allowing each variable to interact only with relevant variables [34,35]. These strategies assume that variables have stable identities across samples and time [36,37]. However, in D-LLM denoising trajectories, token positions may correspond to changing semantic roles during generation. Therefore, DeMTS converts changing token signals into stable latent variables for temporal and inter-variable modeling. Detailed literature review please refer to Appendix A.

3. Preliminary

Diffusion Large Language Models. Given an input query $\mathbf{q}$, the diffusion large language models (D-LLMs) generates a fixed-length response through iterative denoising. Let ${\mathbf{r}}^{\left(t\right)}=(r_1^{\left(t\right)},r_2^{\left(t\right)},\dots ,r_N^{\left(t\right)})\in {\mathcal{V}}^N$ denote the intermediate response at denoising step $t\in \{T,\dots ,0\}$, where N is the sequence length and $\mathcal{V}$ is the vocabulary. Starting from a highly masked sequence ${\mathbf{r}}^{\left(T\right)}$, the model progressively reconstructs the response according to ${\mathbf{r}}^{(t-1)}\sim p_{\theta }\left({\mathbf{r}}^{(t-1)}\mid {\mathbf{r}}^{\left(t\right)},\mathbf{q}\right),$ until obtaining the final response ${\mathbf{r}}^{\left(0\right)}$.

At each denoising step t, the D-LLMs predicts a categorical distribution over the vocabulary for each token position i: ${\pi }_{i,t}=p_{\theta }\left(r_i^{\left(0\right)}\mid {\mathbf{r}}^{\left(t\right)},\mathbf{q}\right)\in {\mathbb{R}}^{\left|\mathcal{V}\right|}.$ The corresponding predictive uncertainty is measured by token entropy: $H_{i,t}=-{\sum }_{v\in \mathcal{V}}{\pi }_{i,t}\left(v\right)log{\pi }_{i,t}\left(v\right).$ Collecting token uncertainties across all denoising steps and positions yields the uncertainty trajectory $\mathbf{H}=\left[H_{i,t}\right]\in {\mathbb{R}}^{N\times (T+1)},$ where the temporal axis corresponds to denoising steps and the token axis corresponds to sequence positions. We further denote by ${\mathbf{e}}_{i,t}\in {\mathbb{R}}^{d_h}$ the contextual embedding of token position i at step t, and collect them as $\mathbf{E}\in {\mathbb{R}}^{N\times (T+1)\times d_h}$. The uncertainty trajectory $\mathbf{H}$ provides the primary detection signal, while $\mathbf{E}$ supplies contextual semantics for constructing the latent-variable trajectory.

Hallucination Detection in D-LLMs. Given the uncertainty trajectory $\mathbf{H}$, contextual trajectory $\mathbf{E}$, and the generated response, hallucination detection aims to determine whether the response contains factually incorrect or unsupported content. Let $\mathcal{D}={\left\{({\mathbf{H}}_n,{\mathbf{E}}_n,y_n)\right\}}_{n=1}^M$ denote the training set, where M is the number of samples and $y_n\in \{0,1\}$ is the hallucination label, with $y_n=1$ indicating a hallucinated response and $y_n=0$ indicating a factual response. The objective is to learn a detector $f_{\varphi }$ that maps the denoising trajectories to a hallucination probability: ${\widehat{y}}_n=f_{\varphi }\left({\mathbf{H}}_n,{\mathbf{E}}_n\right).$ The task is formulated as trajectory-based binary classification:

$$\underset{\varphi }{min}{\displaystyle \frac{1}{M}}{\displaystyle \sum _{n=1}^M}{\mathcal{L}}_{\mathrm{cls}}\left(y_n,f_{\varphi }({\mathbf{H}}_n,{\mathbf{E}}_n)\right),$$

(1)

where ${\mathcal{L}}_{\mathrm{cls}}$ denotes the binary cross-entropy loss.

4. Methodology

In this section, we provide an overview of DeMTS, a hallucination detection framework that models D-LLMs denoising trajectories as multivariate time series. DeMTS takes a token-step uncertainty trajectory as input which jointly preserves temporal evolution and cross-token relationships. It first performs Trajectory-Preserving Token-to-Variable (T2V) Assignment to map schedule-biased and semantically unstable token signals into latent variables with more consistent identities. Based on the resulting latent variable trajectories, DeMTS then applies Dynamic Multivariate Temporal Modeling to jointly capture variable-wise temporal evolution and dynamically evolving interactions among latent variables throughout the denoising process. The learned trajectory representation is finally used for hallucination detection. The overall framework of DeMTS is illustrated in Fig. Figure 2.

4.1. Trajectory-Preserving T2V Assignment

The raw denoising trajectory of a D-LLM is organized by token positions, where each trajectory corresponds to the uncertainty evolution of a fixed sequence position. However, the same position may correspond to different semantic roles and evolve into different token contents during denoising, making it unsuitable as a variable for multivariate temporal modeling. To construct a more reliable variable axis, we propose Trajectory-Preserving Token-to-Variable Assignment (T2V), which converts the position-indexed token trajectory into a latent variable-indexed trajectory, providing a more stable basis for multivariate time series learning.

The goal of T2V is to learn how each token-level signal should contribute to K latent variables with stable semantics while preserving its denoising dynamics. To this end, a schedule-aware uncertainty normalization scheme is introduced to reduce the global uncertainty bias induced by the denoising process. Then, a latent variable axis is constructed by assigning token-level signals to latent variables according to their uncertainty, semantic content, and positional structure. Moreover, T2V imposes a trajectory-preservation constraint on the assignment, preventing local hallucination cues from being smoothed out during token-to-variable aggregation.

Schedule-Aware Uncertainty Normalization. Raw token uncertainty can be affected by the denoising schedule, where early steps are generally more uncertain and later steps become more stable. To reduce this step-wise bias, we normalize token uncertainty within each denoising step by

$$\begin{array}{c}\hfill \begin{array}{c}{\mu }_t={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{H}_{i,t},{\sigma }_t=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\left(H_{i,t}-{\mu }_t\right)}^2},\\ {\widehat{H}}_{i,t}={\displaystyle \frac{H_{i,t}-{\mu }_t}{{\sigma }_t+ϵ}},\end{array}\end{array}$$

(2)

where $H_{i,t}$ denotes the raw uncertainty of token position i at denoising step t, ${\widehat{H}}_{i,t}$ denotes the normalized uncertainty, and $ϵ$ is a small constant. After normalization, ${\widehat{H}}_{i,t}$ can capture the relative uncertainty level of token i within the uncertainty distribution at each denoising step, providing a schedule-calibrated signal for token-to-variable assignment.

Variable Axis Construction. After normalization, the denoising trajectory remains indexed by raw token positions, preventing their direct use as stable variables for multivariate temporal modeling. Since a token position lacks a stable semantic identity in most scenarios, we construct a latent variable axis by learning a soft assignment from each token-level signal to K latent variables, grouping semantically related token signals into latent variables with more consistent identities. In this way, the position-indexed trajectory is reorganized into a stable variable-indexed trajectory, providing a more reliable basis for subsequent procedures.

To construct a latent-variable axis with stable semantics, the assignment should depend on both uncertainty dynamics and token semantics. We therefore combine the normalized uncertainty with the contextual token embedding extracted from the D-LLMs via:

$$\begin{array}{c}\hfill {\mathbf{u}}_{i,t}={\mathrm{Linear}}_h\left({\widehat{H}}_{i,t}\right)+\gamma {\mathrm{Linear}}_e\left({\mathbf{e}}_{i,t}\right),\end{array}$$

(3)

where ${\mathbf{u}}_{i,t}\in {\mathbb{R}}^{N\times (T+1)\times d_u}$ denotes the joint representation of token (i) at denoising step (t), encoding both its uncertainty state and contextual semantics for subsequent latent-variable assignment. Specifically, ${\mathbf{e}}_{i,t}$ denotes the contextual embedding of token i at denoising step t, ${\mathrm{Linear}}_h(·)$ and ${\mathrm{Linear}}_e(·)$ project the uncertainty and semantic signals into the same space, respectively, and $\gamma$ controls the contribution of contextual information.

Based on ${\mathbf{u}}_{i,t}$, we further compute the content-conditioned assignment logits over K latent variables as:

$$\begin{array}{c}\hfill {\mathbf{s}}_{i,t}^{\mathrm{con}}={\mathcal{F}}_{\mathrm{con}}\left({\mathbf{u}}_{i,t}\right),\end{array}$$

(4)

where ${\mathcal{F}}_{\mathrm{con}}(·)$ is a learnable mapping function, and ${\mathbf{s}}_{i,t}^{\mathrm{con}}$ represents the assignment preference of token signal $(i,t)$ over the K latent variables.

Although token positions cannot directly serve as stable variables due to their unstable semantic identities, positional information still provides useful structural cues for latent variable construction. Specifically, relative positions preserve the sequential order and local neighborhood relationships among tokens, offering structural guidance for grouping semantically related token-level signals into latent variables. To capture structural cues at different spatial scales, we adopt a multi-frequency Fourier representation to encode each position:

$${\varphi }_i={\left[sin\left({\displaystyle \frac{2\pi fi}{N}}\right),cos\left({\displaystyle \frac{2\pi fi}{N}}\right)\right]}_{f=1}^F,$$

(5)

where F denotes the number of Fourier frequencies. Low-frequency components capture coarse global ordering, whereas high-frequency components preserve fine-grained local neighborhood relations, providing complementary structural information for latent-variable construction. We then map the Fourier representation into structural assignment logits:

$${\mathbf{s}}_i^{\mathrm{str}}={\mathcal{F}}_{\mathrm{str}}\left({\varphi }_i\right),$$

(6)

where ${\mathcal{F}}_{\mathrm{str}}(·)$ is a learnable mapping function and ${\mathbf{s}}_i^{\mathrm{str}}$ represents the structural assignment preference of position i over the K latent variables. Since positional structure should be adaptively injected according to the current token state, we introduce a content-adaptive gate

$${\mathbf{g}}_{i,t}=\sigma \left({\mathcal{F}}_{\mathrm{gate}}\left({\mathbf{u}}_{i,t}\right)\right)\in {\mathbb{R}}^K,$$

(7)

to regulate their contribution. The final token-to-variable assignment weights are obtained by combining content-conditioned and gated structural assignment logits:

$$\begin{array}{c}\hfill {\mathbf{a}}_{i,t}=Softmax\left({\mathbf{s}}_{i,t}^{\mathrm{con}}+{\mathbf{g}}_{i,t}\odot {\mathbf{s}}_i^{\mathrm{str}}\right)\in {\mathbb{R}}^K,\end{array}$$

(8)

where ${\mathbf{a}}_{i,t}$ denotes the soft assignment distribution of token signal $(i,t)$ over the K latent variables. In this way, token semantics and positional structure are jointly considered for token-to-variable assignment.

Preservation-Constrained Variable Construction. Given the token-to-variable assignment, we further construct the variable-indexed trajectory through assignment-weighted aggregation, transforming token-level representations into latent-variable representations. We first define the aggregation operator for latent variable k at denoising step t:

$${\mathcal{A}}_{k,t}\left({\mathbf{x}}_t\right)={\displaystyle \frac{{\sum }_{i=1}^N{a}_{i,t,k}{\mathbf{x}}_{i,t}}{{\sum }_{i=1}^N{a}_{i,t,k}+ϵ}},$$

(9)

where ${\mathbf{x}}_t={\left\{{\mathbf{x}}_{i,t}\right\}}_{i=1}^N$ denotes a generic token-indexed input to the aggregation operator instantiated as the joint token representation ${\mathbf{u}}_t$. Moreover, ${\mathbf{a}}_{i,t}=[a_{i,t,1},a_{i,t,2},\dots ,a_{i,t,K}]\in {\mathbb{R}}^K$ is the soft assignment distribution of token i over all latent variables at denoising step t, and $a_{i,t,k}$ denotes its k-th element. It measures the contribution of token-level input ${\mathbf{x}}_{i,t}$ to the k-th latent variable, while $ϵ$ is a small constant

for numerical stability. Using this operator, the latent variable k is written as ${\mathbf{z}}_{k,t}={\mathcal{A}}_{k,t}\left({\mathbf{u}}_t\right)\in {\mathbb{R}}^{d_u},$ and the complete variable-indexed trajectory is $\mathbf{Z}={\left[{\mathbf{z}}_{k,t}\right]}_{\begin{array}{c}k=1,\dots ,K\\ t=0,\dots ,T\end{array}}.$

A potential concern with assignment-weighted aggregation is that it may smooth out local denoising dynamics that are informative for hallucination detection. We therefore introduce a trajectory-preservation constraint to retain these dynamics. Specifically, we project the uncertainty change of token i into the representation space: $\Delta {\widehat{\mathbf{H}}}_{i,t}={\mathrm{Linear}}_h({\widehat{H}}_{i,t}-{\widehat{H}}_{i,t-1}).$ Collecting all token-level variations gives $\Delta {\widehat{\mathbf{H}}}_t={[\Delta {\widehat{\mathbf{H}}}_{1,t},\dots ,\Delta {\widehat{\mathbf{H}}}_{N,t}]}^{\top }.$ We then aggregate the token-level variations into the latent-variable space: $\Delta {\mathbf{Z}}_{k,t}={\mathcal{A}}_{k,t}(\Delta {\widehat{\mathbf{H}}}_t).$ The latent-variable variations are subsequently projected back to the token level: $\Delta {\tilde{\mathbf{H}}}_{i,t}={\sum }_{k=1}^K{a}_{i,t,k}\Delta {\mathbf{Z}}_{k,t}.$ This aggregation and reconstruction process constrains the T2V assignment to preserve token-level denoising dynamics during variable construction.

To preserve both the magnitude and temporal direction of these dynamics, we decompose the preservation constraint into two complementary terms. The first term is the Amplitude Preservation Loss, which addresses amplitude smoothing, where token-to-variable aggregation weakens the magnitude of local uncertainty changes:

$${\mathcal{L}}_{amp}={\mathbb{E}}_{i,t}\left[{\left(|\Delta {\widehat{\mathbf{H}}}_{i,t}|-|\Delta {\tilde{\mathbf{H}}}_{i,t}|\right)}^2\right].$$

(10)

By matching the magnitude of the reconstructed variation to that of the original token-level variation, ${\mathcal{L}}_{amp}$ encourages variable construction to retain the strength of local denoising dynamics. The second term is the Direction Consistency Loss, which addresses direction distortion, where the reconstructed variation may evolve in the opposite temporal direction:

$${\mathcal{L}}_{dir}={\mathbb{E}}_{i,t}\left[max\left(0,-\Delta {\widehat{\mathbf{H}}}_{i,t}\Delta {\tilde{\mathbf{H}}}_{i,t}\right)\right].$$

(11)

This penalty is activated when the original and reconstructed variations have opposite signs, thereby discouraging variable construction from reversing uncertainty trends. The final preservation constraint (with balance hyperparameters $\lambda$ ) is defined as

$${\mathcal{L}}_{pres}={\lambda }_{amp}{\mathcal{L}}_{amp}+{\lambda }_{dir}{\mathcal{L}}_{dir}.$$

(12)

4.2. Dynamic Multivariate Temporal Modeling

With the latent-variable trajectory constructed by T2V, the remaining challenge is to model the temporal evolution with the evolving inter-variable dependency for hallucination detection. To this end, we propose Dynamic Multivariate Temporal Modeling (DMTM), which progressively integrates inter-variable dependency modeling with variable-wise temporal encoding for hallucination prediction.

Stage-Adaptive Inter-Variable Interaction. In D-LLM denoising trajectories, the dependencies among latent variables evolve throughout denoising and provide important relational evidence for hallucination detection. To model the dependencies, we aggregate cross-variable relational evidence through attention-based interaction. Specifically, we construct contextualized variable representation via:

$$\mathbf{C}=Softmax\left({\displaystyle \frac{\mathbf{Q}{\mathbf{K}}^{\top }}{\sqrt{d}}}\right)\mathbf{V},$$

(13)

where $\mathbf{Q}=\mathbf{Z}{\mathbf{W}}_Q$, $\mathbf{K}=\mathbf{Z}{\mathbf{W}}_K$, and $\mathbf{V}=\mathbf{Z}{\mathbf{W}}_V$ denote the query, key, and value projections, respectively. The row-wise softmax normalizes the dependency scores for each latent variable across all variables, producing relation weights that specify how much information it should receive from the others at step t. Using these weights, each latent variable aggregates information from its related variables, adaptively capturing step-specific cross-variable dependencies during denoising.

While the above interaction captures step-specific dependencies, their reliability varies with the noise level at each denoising step. In the D-LLMs denoising process, larger t corresponds to earlier and noisier stages, where variable are less reliable and the estimated dependencies are more susceptible to spurious correlations. As t decreases, the variables gradually stabilize, making their relational patterns increasingly representative of the underlying semantic dependencies [19]. To calibrate cross-variable information fusion according to this reliability transition, we introduce a trajectory-reliability gate:

$$\mathbf{U}=\mathbf{Z}+\rho \mathbf{C},\rho ={\displaystyle \frac{T-t}{T}},$$

(14)

where ${\rho }_t$ increases as the trajectory approaches the final step, assigning limited fusion strength to noisy early-step relations while progressively strengthening information exchange among stabilized variables. By limiting interaction at noisy early stages and strengthening it as the variables stabilize, this design reduces the influence of spurious dependencies and emphasizes more reliable semantic relations.

Variable-Wise Temporal Dynamics Modeling. After incorporating stage-adaptive relational evidence into each latent variable, we further model how its dependency-enhanced state evolves throughout the denoising trajectory. To capture temporal evolution patterns, we first incorporate trajectory-order information into the dependency-enhanced variable states: ${\overline{\mathbf{U}}}_k={\mathbf{U}}_k+{\mathbf{e}}^{\mathrm{time}},$ where ${\mathbf{e}}^{\mathrm{time}}$ denotes the temporal embedding of trajectory step t. For latent variable k, we then apply temporal self-attention to its complete trajectory:

$${\mathbf{R}}_k=\mathrm{TemporalAttn}\left({\overline{\mathbf{U}}}_k\right).$$

(15)

Applying it to all variables yields the multivariate temporal representation $\mathbf{R}$ for hallucination prediction. Finally, the hallucination probability is predicted through a lightweight classifier: $\widehat{y}=MLP\left(\mathbf{R}\right)$.

Training Objective. The detector is trained with a classification objective together with the preservation constraint introduced in T2V. Given the ground-truth hallucination label $y\in \{0,1\}$ and the predicted probability $\widehat{y}$, we define the classification loss as the binary cross-entropy:

$${\mathcal{L}}_{cls}=-\mathbb{E}\left[ylog\widehat{y}+(1-y)log(1-\widehat{y})\right].$$

(16)

The overall training objective is:

$$\mathcal{L}={\mathcal{L}}_{cls}+{\mathcal{L}}_{pres},$$

(17)

which optimizes the detector for hallucination prediction while encouraging the learned variable trajectory to retain fine-grained token-level denoising dynamics. Algorithm 1 summarizes the procedure of DeMTS in Appendix B, with complexity analysis in Appendix C.

5. Experiments

5.1. Experimental Setup

Datasets. Our evaluation covers three complementary question-answering scenarios. TriviaQA [38] focuses on open-domain knowledge recall, HotpotQA [39] requires the integration of multiple factual clues, and CommonsenseQA [40] examines reasoning grounded in everyday knowledge. We collect responses and their complete denoising trajectories from two representative D-LLMs, LLaDA-8B-Instruct [1] and Dream-7B-Instruct [3], using step generation lengths of 64 and 128.

Baselines and Evaluation. We compare DeMTS with three categories of hallucination detectors: ❶ Output-based methods, including Perplexity [41], Length-Normalized Entropy (LN-Entropy) [42], Semantic Entropy [43], and Lexical Similarity [44]; ❷ Latent-based methods, including EigenScore [45], Contrast-Consistent Search (CCS) [46], and Truthfulness Separator Vector (TSV) [47]; ❸ Trajectory-based methods, including TraceDet [18] and DynHD [20]. We use AUROC as the evaluation metric. Hallucination labels are determined by Qwen3-8B [48,49] based on the question, answer, and generated response. Detailed descriptions of the baselines and implementation settings are provided in Appendix D.1 and Appendix D.2.

5.2. Experimental Results and Analysis

Performance Comparison.Table 1 presents the main comparison across two D-LLM backbones, three QA benchmarks, and two generation-length settings. We further analyze the experimental results from the perspectives of these three model categories: ❶ To examine whether final-output statistics are sufficient for D-LLMs hallucination detection, we compare DeMTS with output-level methods. DeMTS consistently outperforms output-based baselines across different settings. These results show that output-level evidence cannot adequately characterize hallucinations formed throughout iterative denoising, highlighting the necessity of exploiting denoising-process information. ❷ To evaluate the benefit of explicitly modeling the generation process over using static internal representations, we compare DeMTS with latent-based detectors. Although latent-based methods use richer model-side signals than output-level statistics, they still underperform DeMTS in most cases. This suggests that static representation signals are insufficient to capture the evolving temporal dependencies involved in hallucination formation. ❸ To verify whether preserving the multivariate structure of denoising trajectories provides additional detection evidence, we compare DeMTS with TraceDet and DynHD. DeMTS achieves the best overall performance across different settings. The consistent improvement supports our central motivation: jointly modeling variable-wise evolution and inter-variable interactions better captures abnormal temporal relations for hallucination detection.

Ablation Study. As shown in Table 2, removing any component consistently degrades performance on backbones, indicating the effectiveness of multivariate dynamics modeling. ❶ Replacing schedule-normalized signals with raw uncertainty trajectories leads to clear performance drops, suggesting that the schedule-induced global uncertainty trend can mask sample-specific abnormal dynamics. ❷ Using token positions as variables causes the most pronounced degradation, confirming that semantically unstable token positions do not provide reliable variable identities for time-series modeling. ❸ Removing either ${\mathcal{L}}_{amp}$ or ${\mathcal{L}}_{dir}$ weakens detection performance, demonstrating the importance of preserving fine-grained denoising dynamics during T2V assignment. We provide a sensitivity analysis of the loss hyperparameters in the Appendix D.3. ❹ Removing either inter-variable interaction or temporal evolution modeling also substantially weakens detection performance, showing that trajectory-level hallucination evidence depends on both variable-wise evolution and cross-variable dependency modeling. Comprehensive ablation studies are provided in Appendix D.4.

Robustness to the Number of Latent Variables. As shown in Figure 3a, we vary the number of latent variables K to examine the effect of variable resolution. A small K over-compresses token-level denoising signals, while a large K leads to fragmented and redundant variables. The results show that a moderate K better balances information preservation and variable stability.

Visualization of Learned Variables. To examine whether the Token-to-variable assignment (T2V) module constructs variables with stable identities, we visualize the assignment patterns across multiple samples in Figure 3b. Although token contents and semantic roles vary substantially across samples, each latent variable consistently aggregates token-level denoising signals with similar semantic or dynamic characteristics. This cross-sample consistency allows the learned latent variables to capture transferable trajectory patterns, thereby improving their generalization to unseen questions and generation contexts.

Cross-Task Generalization. To assess whether DeMTS captures transferable hallucination dynamics rather than dataset-specific semantic patterns, we train the detector on one benchmark and test it on the other benchmarks without further adaptation. As shown in Table 3, DeMTS achieves the highest average AUROC indicating that the stable latent variables constructed through T2V assignment, along with the modeling of temporal evolution and inter-variable interactions, capture transferable hallucination patterns.

Efficiency Analysis. We compare the inference time and AUROC of different methods to evaluate their practical efficiency as shown in Figure 4. DeMTS achieves the highest AUROC with the shortest runtime. These results show that DeMTS provides the best AUROC-efficiency trade-off by effectively exploiting multivariate denoising dynamics.

6. Conclusion

In this work, we investigated hallucination detection in D-LLMs from a multivariate time-series perspective. Unlike existing approaches that compress denoising trajectories along the temporal or variable dimension, we preserve their two-dimensional token-step structure to capture how hallucination-related uncertainty evolves and propagates throughout generation. Extensive experiments across multiple D-LLMs, benchmarks, and generation settings demonstrate that DeMTS exhibits strong effectiveness, cross-task generalization, robustness, and computational efficiency.

Appendix A.1. Hallucination Detection in D-LLMs

D-LLMs generate text through iterative denoising, which exposes intermediate predictions, uncertainty estimates, and hidden states before the final response is obtained [1,3,23]. This process provides trajectory-level evidence for hallucination detection, since factual errors may emerge during refinement rather than only in the final output [16]. Existing studies mainly explore this evidence from two perspectives.

The first line focuses on step-oriented evidence selection. TraceDet [18] formulates the denoising process as a decoding trace and selects informative sub-traces for hallucination prediction, showing that different denoising stages contribute unequally to detection. HIVE [25] further exploits hidden evidence from intermediate denoising states and uses selected evidence for verification. These methods demonstrate the value of intermediate states, but they mainly rely on selecting or compressing denoising stages.

The second line focuses on dynamics-oriented trajectory modeling. DynHD [20] models the evolution of uncertainty evidence and detects hallucinations by measuring deviations from expected denoising dynamics. TDGNet [24] constructs temporal dynamic graphs to capture evolving token-level relations during denoising. These methods reveal that hallucination cues can lie in dynamic uncertainty patterns and token interactions. However, they still do not fully preserve the complete token-by-step trajectory structure. In contrast, DeMTS models D-LLM denoising trajectories as multivariate time series, aiming to jointly capture token-wise temporal evolution and evolving inter-token dependencies for hallucination detection.

Appendix A.2. Multivariate Time-Series Modeling

Multivariate time-series modeling aims to capture temporal patterns across multiple variables and their dependencies [26,28,50]. Existing methods are often discussed according to how they model variable channels. Channel-independent methods, such as PatchTST [30], model each variable separately to preserve variable-specific temporal patterns and reduce interference from noisy correlations. Channel-dependent methods, such as Crossformer [22] and iTransformer [32], jointly model variables to capture cross-variable dependencies. More recent channel-partial methods, such as DUET [34], CrossGNN [51], and TimeFilter [21], allow each variable to interact with only relevant variables, balancing variable-specific modeling and dependency modeling.

These methods provide useful inspiration for modeling D-LLMs denoising trajectories [52,53,54,55], where denoising steps form the temporal axis and token signals form the raw variable axis. However, conventional multivariate time-series methods usually assume that each variable has a stable identity and meaning across samples and time. This assumption does not hold for D-LLMs, because token positions may correspond to different semantic roles across samples and may change during denoising. Therefore, directly treating token positions as variables can lead to unstable temporal patterns and unreliable dependencies. To address this issue, DeMTS first converts changing token signals into stable latent variables, and then models both their temporal evolution and stage-adaptive inter-variable interactions.

Appendix D.1. Baselines

We compare DeMTS with three categories of hallucination detectors, covering output-level signals, internal representations, and denoising-trajectory evidence.

❶Output-based methods. These methods assess hallucination risk using information derived from the generated responses, including model likelihood, token-level uncertainty, and consistency among sampled outputs. They serve as widely used baselines for evaluating whether final-output signals are sufficient for hallucination detection.

⊳Perplexity [41] uses the likelihood of the generated response as a confidence signal, where responses with lower model likelihood are considered less reliable.

⊳Length-Normalized Entropy (LN-Entropy) [42] estimates predictive uncertainty from token-level entropy and normalizes it by sequence length, reducing the bias caused by different output lengths.

⊳Semantic Entropy [43] samples multiple responses, groups them according to semantic equivalence, and computes the entropy over semantic clusters to measure meaning-level uncertainty.

⊳Lexical Similarity [44] measures the surface-level agreement among multiple sampled responses, where lower similarity indicates higher generation instability.

Table A1. Ablation study in AUROC (%) on two D-LLMs backbones.

Ablation Variant	TriviaQA		HotpotQA		CSQA
	64	128	64	128	64	128
LLaDA-8B-Instruct
DeMTS	89.84	89.13	88.14	88.03	83.42	85.08
w/o SAUN	84.32	84.72	85.65	84.37	80.05	82.05
w/o T2V	81.78	87.40	78.60	75.65	81.15	74.04
w/o SAIVI	83.41	86.45	81.70	81.37	79.21	74.89
w/o VWTDM	83.58	86.54	87.97	84.20	78.57	76.18
w/o ${\mathcal{L}}_{amp}$	86.59	86.97	79.97	79.85	81.53	77.53
w/o ${\mathcal{L}}_{dir}$	82.17	85.43	80.82	77.61	80.07	76.92
Dream-7B-Instruct
DeMTS	87.74	88.89	88.16	84.63	86.95	87.86
w/o SAUN	81.07	82.90	79.50	79.04	83.07	86.67
w/o T2V	80.50	84.84	79.17	77.84	77.18	83.85
w/o SAIVI	77.95	86.11	79.87	72.77	81.38	83.01
w/o VWTDM	80.27	82.32	80.98	74.65	82.61	84.21
w/o ${\mathcal{L}}_{amp}$	79.61	82.17	81.21	78.47	78.57	83.41
w/o ${\mathcal{L}}_{dir}$	82.13	81.29	80.15	79.96	78.69	82.06

Table A1. Ablation study in AUROC (%) on two D-LLMs backbones.

Ablation Variant	TriviaQA		HotpotQA		CSQA
	64	128	64	128	64	128
LLaDA-8B-Instruct
DeMTS	89.84	89.13	88.14	88.03	83.42	85.08
w/o SAUN	84.32	84.72	85.65	84.37	80.05	82.05
w/o T2V	81.78	87.40	78.60	75.65	81.15	74.04
w/o SAIVI	83.41	86.45	81.70	81.37	79.21	74.89
w/o VWTDM	83.58	86.54	87.97	84.20	78.57	76.18
w/o ${\mathcal{L}}_{amp}$	86.59	86.97	79.97	79.85	81.53	77.53
w/o ${\mathcal{L}}_{dir}$	82.17	85.43	80.82	77.61	80.07	76.92
Dream-7B-Instruct
DeMTS	87.74	88.89	88.16	84.63	86.95	87.86
w/o SAUN	81.07	82.90	79.50	79.04	83.07	86.67
w/o T2V	80.50	84.84	79.17	77.84	77.18	83.85
w/o SAIVI	77.95	86.11	79.87	72.77	81.38	83.01
w/o VWTDM	80.27	82.32	80.98	74.65	82.61	84.21
w/o ${\mathcal{L}}_{amp}$	79.61	82.17	81.21	78.47	78.57	83.41
w/o ${\mathcal{L}}_{dir}$	82.13	81.29	80.15	79.96	78.69	82.06

❷Latent-based methods. These methods exploit internal representations of LLMs to identify factuality-related signals beyond surface outputs. They are included to examine whether static hidden-state features are sufficient for detecting hallucinations in D-LLMs without explicitly modeling the denoising trajectory.

⊳EigenScore [45] detects hallucinations by measuring the semantic consistency of internal representations, using the spectral structure of hidden-state covariance matrices.

⊳Contrast-Consistent Search (CCS) [46] discovers truth-related directions in the activation space without direct supervision by enforcing consistency between contrastive statements.

⊳Truthfulness Separator Vector (TSV) [47] learns a lightweight steering vector in the latent space to improve the separation between truthful and hallucinated responses.

❸Trajectory-based methods. These methods are specifically designed for D-LLMs and leverage intermediate information exposed during iterative denoising. They provide the most direct comparison to DeMTS, since they also use process-level evidence rather than relying only on final outputs or static representations.

⊳TraceDet [18] is designed for D-LLMs and formulates the denoising process as a decoding trace. It selects hallucination-relevant sub-traces from intermediate denoising steps for final prediction.

⊳DynHD [20] constructs semantic-aware uncertainty evidence and learns reference denoising dynamics for factual responses. Hallucinations are detected by measuring the deviation between observed evidence trajectories and the learned reference trajectories.

Appendix D.2. Implementation Details

We conduct experiments on TriviaQA, HotpotQA, and CommonsenseQA (show in Table A3) using LLaDA-8B-Instruct and Dream-7B-Instruct as D-LLM backbones. For each dataset, we sample 1,600 training examples and split an additional 400 examples into 200 validation and 200 test instances. The random seed is fixed to 42. The number of denoising steps is set to 64 or 128. At each denoising step, we record token-wise predictive entropy and token IDs.

Our detector uses a hidden dimension of 64 and constructs (K=32,48) latent variables from the denoising trajectory. It is trained with Cross Entropy and the trajectory-preservation loss. We use AdamW with a learning rate of $2\times {10}^{-4}$, weight decay of 0.1, and batch size of 8. Training lasts for at most 100 epochs and early stopping with a patience of 20. The checkpoint with the highest validation AUROC is selected for testing. AUROC is used as the primary evaluation metric. All experiments are implemented in PyTorch 2.6.0 and conducted on 4 NVIDIA L40 GPUs. All experimental hyperparameter settings are shown in Table A2.

Table A2. Hyperparameter settings of DeMTS.

Hyperparameter	Value
Number of latent variables K	32, 48
Hidden dimension d	64
Fourier frequencies F	16
Cross-variable attention heads	2
Temporal attention heads	8
Dropout rate	0.05
Optimizer	AdamW
Learning rate	$2\times {10}^{-4}$
Weight decay	0.1
Batch size	8
Maximum epochs	100
Early stopping patience	20
Trajectory preservation weight	0.1
Random seed	42

Table A2. Hyperparameter settings of DeMTS.

Hyperparameter	Value
Number of latent variables K	32, 48
Hidden dimension d	64
Fourier frequencies F	16
Cross-variable attention heads	2
Temporal attention heads	8
Dropout rate	0.05
Optimizer	AdamW
Learning rate	$2\times {10}^{-4}$
Weight decay	0.1
Batch size	8
Maximum epochs	100
Early stopping patience	20
Trajectory preservation weight	0.1
Random seed	42

Table A3. Examples of factual and hallucinated responses from HotpotQA, TriviaQA, and CommonsenseQA. Correct responses are highlighted in blue, while hallucinated responses are highlighted in red.

Dataset	Type	Question & Response
HotpotQA	Factual	Q: Professor A. Selvanathan is a professor at a university that is public or private?
		R: The university is public.
	Hallucination	Q: Which French aristocrat and military officer who fought in the American Revolutionary War visited the historic home located at Enfield, Halifax County, North Carolina, known as The Cellar?
		R:James Hogun visited The Cellar. The reference answer is Marquis de Lafayette.
TriviaQA	Factual	Q: What country has the largest Muslim population?
		R: The country with the largest Muslim population is Indonesia.
	Hallucination	Q: Which supposedly non-lethal weapon was named by the inventor after his childhood hero “Tom Swift”?
		R: The weapon was called the Tom Swifty Gun.
CommonsenseQA	Factual	Q: The townhouse was a hard sell for the realtor; it was right next to a high-rise what? Choices: (A) suburban development; (B) apartment building; (C) bus stop; (D) Michigan; (E) suburbs.
		R:B. apartment building.
	Hallucination	Q: There is a star at the center of what group of celestial bodies? Choices: (A) Hollywood; (B) skyline; (C) outer space; (D) constellation; (E) solar system.
		R:A. Hollywood. The correct answer is E. solar system.

Table A3. Examples of factual and hallucinated responses from HotpotQA, TriviaQA, and CommonsenseQA. Correct responses are highlighted in blue, while hallucinated responses are highlighted in red.

Dataset	Type	Question & Response
HotpotQA	Factual	Q: Professor A. Selvanathan is a professor at a university that is public or private?
		R: The university is public.
	Hallucination	Q: Which French aristocrat and military officer who fought in the American Revolutionary War visited the historic home located at Enfield, Halifax County, North Carolina, known as The Cellar?
		R:James Hogun visited The Cellar. The reference answer is Marquis de Lafayette.
TriviaQA	Factual	Q: What country has the largest Muslim population?
		R: The country with the largest Muslim population is Indonesia.
	Hallucination	Q: Which supposedly non-lethal weapon was named by the inventor after his childhood hero “Tom Swift”?
		R: The weapon was called the Tom Swifty Gun.
CommonsenseQA	Factual	Q: The townhouse was a hard sell for the realtor; it was right next to a high-rise what? Choices: (A) suburban development; (B) apartment building; (C) bus stop; (D) Michigan; (E) suburbs.
		R:B. apartment building.
	Hallucination	Q: There is a star at the center of what group of celestial bodies? Choices: (A) Hollywood; (B) skyline; (C) outer space; (D) constellation; (E) solar system.
		R:A. Hollywood. The correct answer is E. solar system.

Appendix D.3. Sensitivity Analysis of Trajectory Preservation Losses

We further analyze the sensitivity of the two trajectory-preservation losses, i.e., the amplitude consistency loss ${\mathcal{L}}_{amp}$ and the direction consistency loss ${\mathcal{L}}_{dir}$. As shown in Figure A1, the detector maintains stable performance across a broad range of loss weights, indicating that the proposed trajectory-preservation objective is not overly sensitive to specific hyperparameter choices. When either loss is removed or assigned a very small weight, the performance decreases, suggesting that preserving only the classification objective is insufficient to retain fine-grained denoising dynamics during token-to-variable assignment. Meanwhile, excessively large weights also lead to performance degradation, since overly strong trajectory constraints may suppress discriminative feature learning for hallucination detection. Overall, the heatmaps show that ${\mathcal{L}}_{amp}$ and ${\mathcal{L}}_{dir}$ play complementary roles: ${\mathcal{L}}_{amp}$ helps preserve the magnitude of local uncertainty variations, while ${\mathcal{L}}_{dir}$ maintains the temporal direction of denoising evolution. Their joint use provides a stable and effective regularization scheme for constructing reliable variable-level trajectories.

Figure A1. Sensitivity analysis of the amplitude consistency loss ${\mathcal{L}}_{amp}$ and direction consistency loss ${\mathcal{L}}_{dir}$.

Figure A1. Sensitivity analysis of the amplitude consistency loss ${\mathcal{L}}_{amp}$ and direction consistency loss ${\mathcal{L}}_{dir}$.

Appendix D.4. Complete Ablation Results

To provide a more comprehensive analysis of each component in DeMTS, we report the complete ablation results on both D-LLMs backbones in Table A1. The results exhibit consistent trends across different models, datasets, and generation-length settings, further verifying the robustness and generality of our design.

The complete ablation results further confirm the effectiveness of each component in DeMTS. Replacing schedule-normalized trajectories with raw uncertainty signals consistently reduces performance, showing the importance of suppressing schedule-induced global trends. Directly using token positions as variables also leads to clear degradation, verifying the necessity of constructing semantically stable latent variables before time-series modeling. Removing either ${\mathcal{L}}_{amp}$ or ${\mathcal{L}}_{dir}$ also weakens performance, indicating that both amplitude consistency and direction consistency contribute to preserving fine-grained denoising dynamics during token-to-variable assignment. Moreover, removing inter-variable interaction or temporal evolution modeling consistently degrades performance, demonstrating that DeMTS benefits from jointly preserving local denoising dynamics and modeling multivariate temporal dependencies.

Overall, the complete ablation results are consistent with the observations in the main paper. They demonstrate that the performance gain of DeMTS does not come from a single isolated component, but from the integration of schedule normalization, stable latent-variable construction, trajectory-preserving regularization, and dynamic multivariate temporal modeling.

Appendix E.1. D-LLM Response Generation

We design task-specific generation prompts according to the answer format and evidence setting of each benchmark. TriviaQA requires concise factual answers, HotpotQA requires responses grounded in the provided context, and CommonsenseQA requires selecting a single option without additional explanations. Across all datasets, the final answer is enclosed within <answer></answer> tags to enable consistent answer extraction and subsequent hallucination annotation. The complete prompt templates are provided below.

Appendix E.2. Automatic Hallucination Annotation

For TriviaQA and HotpotQA, we use Qwen3-8B to automatically determine whether the generated response is hallucinated. For TriviaQA and HotpotQA, the judgment yes is mapped to label 1, indicating a hallucinated response, whereas no is mapped to label 0, indicating a non-hallucinated response. CommonsenseQA does not use the Qwen3-8B judge; its labels are obtained through rule-based matching of the predicted option or answer text.