Narrative-Dynamical Systems (NDS): A Closed-Loop Architecture for Long-Horizon Autoregressive Decoding via Orthogonal Logit Projection and Dynamic Barriers

1. Introduction

The ability of neural language models to sustain coherent open-ended generation is not solely a function of model scale or training data; it remains critically sensitive to the decoding algorithm employed at inference time. While models are capable of high-quality outputs, standard truncation heuristics and mode-seeking objectives can produce text that is locally grammatical yet globally degenerate, exhibiting high redundancy and diminished diversity over long horizons [1,2]. This phenomenon highlights a fundamental probability–quality tension, explaining why favoring high-probability continuations often results in dull or repetitive sequences in unconstrained settings [3].

To mitigate these issues, recent inference-time strategies have proposed alternative sampling rules that regulate local information content [4] or employ diversity-oriented decoding objectives, such as diversity-augmented beam search [5]. A complementary line of work introduces auxiliary predictors or experts to steer generation without updating the base generator parameters [6,7,8]. Furthermore, representation-aware decoding approaches based on contrastive objectives provide evidence that degeneration can be mediated by interventions that explicitly couple token selection to representation geometry [9].

In this work, we adopt a systems engineering perspective: we treat long-horizon decoding as a feedback process where the model constantly conditions on its own prior outputs. Within this framework, the induced closed-loop dynamics can drift into unstable regimes characterized by low representation drift, high token redundancy, and collapsing entropy. NDS targets this mechanism by specifying an explicit architecture and a logit-level control law that (i) ensures the model remains within a local trust region around the base distribution and (ii) actively suppresses empirically detected attractor sets. By treating the generator as a fixed inference component that exposes logits prior to sampling, we implement this control action as a modular, plug-and-play logit processor.

2. Related Work

Degeneration under open-ended decoding was characterized by Holtzman et al. [1], who show that truncation-based sampling changes the shape of the sampled distribution yet does not eliminate repetitive failure modes. Unlikelihood training penalizes repeated tokens at training time and provides a widely used point of comparison when parameter updates are permitted [2]. Meister et al. analyze the probability–quality paradox [3] and propose locally typical sampling as an inference-time correction [4]. Diversity-promoting decoding has been studied in the context of structured search procedures such as Diverse Beam Search [5]. Controlled generation methods such as PPLM and FUDGE modify token probabilities using auxiliary predictors while keeping the base generator frozen [6,7]; decoding-time expert mixtures provide another parameter-free control mechanism [8]. Contrastive frameworks connect degeneration to representation anisotropy and propose decoding rules that trade off plausibility and diversity in representation space [9]. Distributional evaluation metrics such as MAUVE quantify the gap between generated and reference text distributions under decoding interventions [10]; benchmarking toolkits such as Texygen support standardized evaluation of diverse generators and metrics [11].

3. System Architecture

NDS is defined as the composition of a frozen autoregressive generator, a frozen encoder, online statistics maintained over a sliding window, and a pre-sampling logit processor that injects control.

3.1. Components and Interfaces

Let $G_{\theta }$ be a generator (decoder-only LM) that maps a prefix $h_t$ to logits $z_t$:

$$G_{\theta }:h_t\mapsto z_t=z_{\theta }\left(h_t\right)\in {\mathbb{R}}^{\left|\mathcal{V}\right|},p_t(·)=Softmax\left(z_t\right).$$

Let $E_{\psi }$ be an encoder that maps the same prefix to a compact state:

$$E_{\psi }:h_t\mapsto x_t=\varphi \left(h_t\right)\in {\mathbb{R}}^d.$$

The control module computes ${\delta }_t\in {\mathbb{R}}^{\left|\mathcal{V}\right|}$ from $\left(x_t\right)$, token and distribution statistics on a sliding window, and a one-step candidate set computed from $p_t$. The controlled distribution is

$$q_t(·\mid h_t)=Softmax(z_t+{\delta }_t),y_t\sim q_t(·\mid h_t).$$

The generator parameters $\theta$ and encoder parameters $\psi$ remain fixed throughout inference; all interventions are applied to logits prior to sampling.

Figure 1. NDS architecture. Control is injected as a logit processor prior to sampling; the encoder and generator remain frozen.

Figure 1. NDS architecture. Control is injected as a logit processor prior to sampling; the encoder and generator remain frozen.

4. Coupled Diagnostics for Long-Horizon Degeneration

NDS does not rely on a single proxy. Instead, it triggers control when evidence is consistent across (i) representation drift, (ii) token redundancy, and (iii) distributional concentration.

4.1. Representation Drift

Define the state and drift:

$$x_t=\varphi \left(h_t\right),v_t=∥x_t-x_{t-1}∥.$$

Let ${\overline{v}}_t$ be a windowed drift average:

$${\overline{v}}_t\triangleq {\displaystyle \frac{1}{W}}\sum _{i=t-W}^{t-1}{v}_i.$$

Low ${\overline{v}}_t$ indicates that the representation evolves slowly under the current decoding regime.

4.2. Token Redundancy

Let ${\mathcal{G}}_n\left(h_t\right)$ be the multiset of n-grams in the last W tokens of $h_t$. Define the repeated n-gram participation rate:

$$r_n\left(t\right)\triangleq {\displaystyle \frac{1}{W}}\sum _{j=t-W}^{t-1}\mathbf{1}\left[\exists g\in {\mathcal{G}}_n\left(h_t\right):g\mathrm{occurs}\mathrm{at}\mathrm{least}\mathrm{twice}\mathrm{and}{y}_j\in g\right].$$

This quantity is invariant to embedding choices and detects local redundancy in the generated token stream.

4.3. Distributional Concentration

Let $p_t=Softmax\left(z_t\right)$. Define entropy:

$$H\left(p_t\right)\triangleq -\sum _{i\in \mathcal{V}}{p}_t\left(i\right)log{p}_t\left(i\right).$$

Low entropy indicates concentration of probability mass on a small subset of tokens and is frequently associated with repetitive continuations under long-horizon decoding [1,3].

4.4. Trigger Predicate

Definition 

(Trigger predicate). For thresholds $({ϵ}_v,{ϵ}_r,{ϵ}_H)$,

$${\mathtt{trigger}}_t\triangleq \mathbf{1}\left[{\overline{v}}_t<{ϵ}_v\wedge r_n\left(t\right)>{ϵ}_r\wedge H\left(p_t\right)<{ϵ}_H\right].$$

Triggering is intentionally conservative: the controller activates only when all three channels indicate entry into a low-drift, high-redundancy, low-entropy regime.

5. Attractor Sets and Dynamic Barriers

NDS constructs an attractor set ${\mathcal{A}}_t\subset \mathcal{V}$ online from windowed statistics and imposes a sparse barrier in logit space.

5.1. Frequency and Repetition Criteria

Let $f_t\left(w\right)$ be the empirical frequency of token w in the last W steps:

$$f_t\left(w\right)\triangleq {\displaystyle \frac{1}{W}}\sum _{j=t-W}^{t-1}\mathbf{1}[y_j=w].$$

Fix thresholds $({\tau }_f,{\tau }_r)$. Define the attractor set:

$${\mathcal{A}}_t\triangleq \{w\in \mathcal{V}:f_t\left(w\right)\ge {\tau }_f\}\cup \{w\in \mathcal{V}:w\mathrm{participates}\mathrm{in}\mathrm{repeated}n-\mathrm{grams}\mathrm{at}\mathrm{rate}\ge {\tau }_r\}.$$

5.2. Barrier Definition

Let $\sigma (·)$ be a smooth step function (e.g., logistic). Define the recent attractor density

$${\rho }_t\triangleq {\displaystyle \frac{1}{W}}\sum _{j=t-W}^{t-1}\mathbf{1}[y_j\in {\mathcal{A}}_t].$$

The barrier is a sparse logit penalty:

$${\left({\delta }_t^{\mathrm{bar}}\right)}_i\triangleq -\beta \mathbf{1}[i\in {\mathcal{A}}_t]·\sigma ({\rho }_t-\theta ),\beta >0.$$

Proposition 

(Barrier suppresses attractor-set probability mass). Let $p=Softmax\left(z\right)$ and let $\mathcal{A}\subset \mathcal{V}$. Define $q=Softmax(z+\delta )$ where ${\delta }_i=-\beta$ for $i\in \mathcal{A}$ and 0 otherwise, with $\beta >0$. Then

$$q\left(\mathcal{A}\right)={\displaystyle \frac{e^{-\beta }p\left(\mathcal{A}\right)}{e^{-\beta }p\left(\mathcal{A}\right)+(1-p\left(\mathcal{A}\right))}}\le e^{-\beta }{\displaystyle \frac{p\left(\mathcal{A}\right)}{1-p\left(\mathcal{A}\right)}}.$$

Proof. 

Since $q\left(i\right)\propto p\left(i\right)e^{{\delta }_i}$, we have

$$q\left(\mathcal{A}\right)={\displaystyle \frac{{\sum }_{i\in \mathcal{A}}p\left(i\right)e^{-\beta }}{{\sum }_{i\in \mathcal{A}}p\left(i\right)e^{-\beta }+{\sum }_{i\notin \mathcal{A}}p\left(i\right)}}={\displaystyle \frac{e^{-\beta }p\left(\mathcal{A}\right)}{e^{-\beta }p\left(\mathcal{A}\right)+(1-p\left(\mathcal{A}\right))}}.$$

The inequality follows by bounding the denominator below by $(1-p(\mathcal{A}\left)\right)$.    □

6. Control Law: Orthogonal Projection under a KL Trust Region

When ${\mathtt{trigger}}_t=1$, NDS computes a projected step designed to increase a one-step diversification surrogate while remaining near the base distribution.

6.1. One-Step Surrogate and Score-Function Direction

Define the window centroid in state space:

$${\overline{x}}_t={\displaystyle \frac{1}{W}}\sum _{i=t-W}^{t-1}{x}_i,\mathcal{J}\left(x\right)={\displaystyle \frac{1}{2}}{∥x-{\overline{x}}_t∥}^2.$$

Construct a candidate set ${\mathcal{C}}_t$ from $p_t$ (top-k or nucleus). Define one-step scores

$$\widehat{\mathcal{J}}(y;h_t)=\mathcal{J}\left(\varphi (h_t\oplus y)\right).$$

Using ${\nabla }_{z}log{p}_t\left(y\right)={\mathbf{e}}_y-p_t$, define a finite-sum score-function direction:

$$g_t\triangleq \sum _{y\in {\mathcal{C}}_t}{w}_t\left(y\right)(\widehat{\mathcal{J}}(y;h_t)-b_t)({\mathbf{e}}_y-p_t),$$

where $w_t$ are normalized base probabilities restricted to ${\mathcal{C}}_t$ and $b_t$ is a baseline (e.g., the weighted mean of $\widehat{\mathcal{J}}$ over ${\mathcal{C}}_t$).

6.2. Second-Order KL Approximation

Let $p=Softmax\left(z\right)$ and $q=Softmax(z+\delta )$. For sufficiently small $\delta$, a second-order approximation holds.

Lemma 

(Second-order KL approximation in logit space). Let $p=Softmax\left(z\right)$ and $q=Softmax(z+\delta )$. Then

$$KL(q\parallel p)={\displaystyle \frac{1}{2}}{\delta }^{\top }H\delta +{O(\parallel \delta \parallel }^3),H=diag\left(p\right)-p{p}^{\top }.$$

Proof. 

A Taylor expansion of $KL(Softmax(z+\delta )\parallel Softmax(z\left)\right)$ around $\delta =0$ yields a zero first-order term and a Hessian equal to the Fisher information of the categorical family parameterized by logits, namely $diag\left(p\right)-p{p}^{\top }$.    □

6.3. Orthogonal Projection and Minimum-Norm Characterization

Let $c_t$ be a constraint direction. Define the orthogonal projection:

$$g_t^{\perp }\triangleq g_t-{\displaystyle \frac{〈g_t,c_t〉}{{∥c_t∥}^2+{ϵ}_c}}{c}_t.$$

Lemma 

(Minimum-norm projection). For $c\ne 0$, $g^{\perp }$ is the unique solution of

$$\underset{u}{min}{\parallel u-g\parallel }^2\mathrm{s}.\mathrm{t}.〈c,u〉=0.$$

Proof. 

Let $L(u,\lambda )={\parallel u-g\parallel }^2+2\lambda 〈c,u〉$. Stationarity gives $u=g-\lambda c$. Enforcing $〈c,u〉=0$ yields $\lambda =〈c,g〉/{\parallel c\parallel }^2$, and thus $u=g-(〈c,g〉/\parallel c{\parallel }^2)c$.    □

6.4. Trust-Region Scaling

A practical scaled step is obtained by using the quadratic KL surrogate:

$${\delta }_t^{\mathrm{proj}}\triangleq \sqrt{{\displaystyle \frac{2\kappa }{{\left(g_t^{\perp }\right)}^{\top }{H}^{-1}{g}_t^{\perp }}}}{H}^{-1}{g}_t^{\perp }\mathrm{when}{g}_t^{\perp }\ne 0.$$

6.5. Composite Control

The final control is

$${\delta }_t\triangleq {\mathtt{trigger}}_t·{\delta }_t^{\mathrm{proj}}+{\delta }_t^{\mathrm{bar}},q_t=Softmax(z_t+{\delta }_t).$$

7. Algorithm

Algorithm 1 NDS closed-loop decoding (single step)

Require: prefix $h_t$, logits $z_t$, window size W  1: $x_t\leftarrow E_{\psi }\left(h_t\right)$; update ${\overline{v}}_t$, $r_n\left(t\right)$, and $H\left(p_t\right)$  2: ${\mathtt{trigger}}_t\leftarrow \mathbf{1}[{\overline{v}}_t<{ϵ}_v\wedge r_n\left(t\right)>{ϵ}_r\wedge H\left(p_t\right)<{ϵ}_H]$  3: $p_t\leftarrow Softmax\left(z_t\right)$; build candidate set ${\mathcal{C}}_t$ from $p_t$ (top-k/nucleus)  4: Compute one-step scores $\widehat{\mathcal{J}}(y;h_t)$ for $y\in {\mathcal{C}}_t$  5: Compute $g_t$; project to $g_t^{\perp }$; scale to ${\delta }_t^{\mathrm{proj}}$ under the KL surrogate  6: Build ${\mathcal{A}}_t$ from window counts and repeated n-grams; compute sparse barrier ${\delta }_t^{\mathrm{bar}}$  7: ${\delta }_t\leftarrow {\mathtt{trigger}}_t·{\delta }_t^{\mathrm{proj}}+{\delta }_t^{\mathrm{bar}}$  8: Sample $y_t\sim Softmax(z_t+{\delta }_t)$; update $h_{t+1}\leftarrow h_t\oplus y_t$

8. Implementation Considerations

NDS requires two operational interfaces: (i) an encoder that maps the evolving prefix to a fixed-dimensional representation, and (ii) a generator runtime that exposes logits prior to sampling so that a logit processor can inject ${\delta }_t$. The fragments below illustrate a deterministic windowed-state implementation and sparse logit-bias injection using the llama.cpp library [13]; they are representative of a standard inference stack where pre-sampling hooks are available.

8.1. Windowed Statistics and Attractor-Set Construction

A ring buffer maintains the last W tokens and W embeddings, along with a hash map for token frequencies. Repeated n-gram participation is computed on the token ring buffer; in a production setting the repeated n-gram detector is implemented with rolling hashes to avoid quadratic scans in W.

Listing 1. C++: windowed statistics and frequency-based attractor set.

8.2. Sparse Logit Injection

Projection is computed on candidate-supported vectors; the barrier is applied only to the sparse attractor set ${\mathcal{A}}_t$. Both enter as additive bias terms prior to the sampling decision.

Listing 2. C++: sparse barrier insertion in a sampler chain.

9. Evaluation Methodology

This section states an experimental design aligned with prior decoding studies.

9.1. Baselines

Baselines are implemented as alternative pre-sampling interventions operating on the same generator logits. We include: (i) nucleus sampling [1], (ii) locally typical sampling [4], (iii) diversity-oriented search procedures [5], (iv) auxiliary-predictor steering [6,7], and (v) decoding-time expert mixtures [8]. Where applicable, representation-aware decoding rules are included for comparison [9]. All methods share identical tokenization, stopping criteria, and prompt suite.

9.2. Prompt Suite

A fixed prompt suite is used with stratified lengths and styles to probe long-horizon behavior. For each prompt, generation is run for a fixed token budget and evaluated under identical termination rules. The prompt suite is held constant across methods to isolate the effect of decoding-time interventions.

9.3. Metrics

We report complementary metrics capturing redundancy, diversity, and distributional shift: (i) Distinct-n (ratio of unique n-grams to total n-grams) [11]; (ii) repetition rate via repeated n-gram participation $r_n\left(t\right)$ aggregated over the generation; (iii) Self-BLEU as a corpus-level indicator of mode concentration [11]; and (iv) MAUVE as a distributional metric comparing generated and reference text [10]. Metrics are interpreted jointly due to known trade-offs between diversity and local coherence under decoding interventions.

10. Conclusion

NDS formalizes long-horizon decoding as a feedback process and proposes a closed-loop architecture in which a fixed generator is coupled to a fixed encoder through a pre-sampling logit processor. Control is activated only under conservative, multi-signal triggering and is implemented as the sum of an orthogonally projected trust-region step and a sparse dynamic barrier that exponentially suppresses probability mass on empirically identified attractor sets. The analysis provides closed-form derivations for the KL surrogate and the projection operator, together with an explicit attenuation bound for attractor-set mass, yielding a principled decoding-time intervention that does not require updating generator parameters.