An Exact Saturation Law for Pairwise-Agreement Symbolic Entropy and Why Entropy-Saturation Criteria Cannot Recover the Decorrelation Delay

1. Introduction

Phase space reconstruction is the standard route from a scalar measurement to the dynamics that produced it [1,2,3]. The method requires two parameters: a delay $\tau$ and an embedding dimension m. Both are chosen by the analyst, and both materially affect everything computed downstream: recurrence quantification, Lyapunov spectra, entropy measures, and the classifier features built from them [4]. Selecting them badly degrades all of these. Selecting them well is an old problem with a settled core: delay from the first minimum of the average mutual information (AMI) [5] or from the autocorrelation, dimension from the false nearest neighbours (FNN) criterion [6]. This general challenge — inferring unknown model parameters purely from measured time-series observations — also appears broadly across the mechanical-engineering community, for instance within Bayesian probabilistic model-updating frameworks that infer model posterior distributions from experimental response data [7,8]. However, for nonlinear dynamical time-series, phase-space reconstruction has developed its own set of dedicated heuristic criteria.

A more recent family of methods abandons these criteria and reads both parameters off a symbolic entropy curve $H(m,\tau )$ [10]. The curve is obtained by thresholding the signal at $\epsilon$, encoding the embedding vector by which of its components exceed $\epsilon$, and computing the Shannon entropy of the resulting symbol distribution as a function of the delay. The proposal is that H grows while the components are still dependent, stops growing once they decorrelate, and that the transition is a robust and noise-tolerant delay estimator [10]. The C-C method [11] is the well-known precursor of this idea; multiscale and fuzzy-entropy variants followed [12,13,14]. The most elaborate member of the family replaces the local-extremum rule by a signal-processing pipeline: the entropy curve is low-pass filtered with a Butterworth filter whose cutoff is chosen from the curve’s own spectrum, the delay is placed where the filtered growth rate falls below a threshold, and the dimension is inferred from the spacing of the filtered curve’s minima. On the systems and datasets reported in that literature, this Low-Pass Filtered Symbolic Entropy (LPF-SEM) method is claimed to reduce parameter estimation error by around 70% relative to five competing methods and to yield $O\left(N^{1.41}\right)$ cost.

This paper began as an attempt to reproduce and extend that result. It did not reproduce. Rather than stop at a discrepancy, we asked what the entropy curve actually measures, and found that the question has an exact answer, one that explains the failure and rules out the whole family rather than any particular implementation.

1.1. What We Prove

The pairwise-agreement symbolisation these methods use is invariant under complementation of the thresholded sign vector $\mathbf{s}\mapsto \mathbf{1}-\mathbf{s}$. Agreement indicators only record which components share a side of the threshold, not which side. The map from sign vector to symbol is therefore two-to-one, the effective alphabet has exactly $2^{m-1}$ rather than $2^m$ elements, and the entropy cannot exceed $m-1$ bits (Lemma 1, Theorem 1). Any mixing process drives the symbol distribution toward uniformity on those $2^{m-1}$ symbols, so $H(m,\tau )\to m-1$ as $\tau \to \infty$ (Proposition 1). Saturation is thus guaranteed a priori for every signal, and the ceiling $m-1$ is set by the symbolisation rather than by the data. Reading a delay off the saturation point means reading the shape of a curve’s approach to a fixed asymptote that the method itself imposed.

This has three consequences we develop and test. First, the saturation point is not the decorrelation delay: the AMI minimum is a second-order (linear) functional of the process and has no fixed relation to the rate at which a nonlinear symbol agreement probability equilibrates. Second, $\tau$ and m are not independently estimable within this family, because the ceiling that defines the saturation point depends on m; we measure the resulting drift. Third, the criterion needs a threshold parameter (LPF-SEM needs two, the filter cutoff and the curvature threshold $\gamma$) that is not constrained by the data and must be set per system if the reference delay is to be recovered at all. We show that a single threshold cannot work: the value of $\gamma$ that reproduces the AMI delay varies fivefold across five systems on clean data, drifts with noise, and does not exist for two of them.

We stress the scope of the claim, because it is easy to over-read. The theorem concerns the pairwise-agreement symbol, the construction that the saturation family, from the C-C method through LPF-SEM, actually computes. Its complement invariance is a property of that specific symbol and not of symbolic entropy in general: permutation entropy [9], whose alphabet is the set of ordinal patterns, has no such degeneracy, and nothing proved here shows that a saturation criterion built on a non-degenerate symbol would fail. What is closed off is the line of method development that reads the decorrelation delay off the saturation of a pairwise-agreement entropy, not the use of symbolic entropies as a diagnostic.

1.2. What We Measure

To test the family rather than a straw man we rebuilt the pipeline from scratch on genuine dynamics: five chaotic systems integrated with fourth-order Runge–Kutta (four ODEs and one delay differential equation), each verified chaotic through a positive largest Lyapunov exponent computed with renormalisation. All six estimators were run over five systems × six signal-to-noise ratios × twenty trials, with a component ablation, a filter-parameter sweep, and a threshold-transferability study added. On clean data LPF-SEM returns delays $1.17$–$4.69\times$ the reference, so the discrepancy is not a constant offset that could be calibrated away, and it is the largest bias of the six estimators at every noise level. The ablation is unambiguous: the low-pass filter, increases the mean delay bias from $28.8$ to $56.4$ samples, and adding the curvature criterion leaves it at $48.6$, worse than the unfiltered criterion it was meant to improve.

1.3. What This Means in Practice

A negative result is only useful if it bounds the damage, so we carried the study through to the downstream task: RQA-feature fault classification on the MFPT bearing set and a rotor set, with five-fold cross-validation grouped by record so that windows from one record cannot straddle the train/test split. The outcome, in Section 5, is not what we expected and is more interesting for it. On MFPT the accuracy ordering is the reverse of the delay ordering: LPF-SEM, the worst estimator by every measure above, classifies best (random forest $0.930$) while C-C, the best, classifies worst ($0.599$). Neither the delay’s accuracy nor its length explains that: a sweep of $\tau$ at fixed dimension is flat or non-monotone over the whole range, and a control that keeps only the calibrated pair $(\tau ,m)$ reproduces $88\%$ of LPF-SEM’s score, because every window of a record is embedded at that record’s own parameters. On the rotor set the effect vanishes. The family was therefore not adopted without evidence; it was adopted on evidence that does not support the explanation offered. The theorem closes off a line of method development, and the downstream study replaces a wrong justification with a correct but narrower one.

1.4. Structure

Section 2 proves the ceiling and saturation results and derives a noise-inflation bound. Section 3 defines the estimators and reference criteria. Section 4 reports the numerical study on chaotic systems. Section 5 applies the estimators to real machinery data. Section 6 discusses why the family fails and the limitations of our study.

2. Symbolisation, Ceiling, and Saturation

2.1. Pairwise-agreement Symbolisation

Let $x={\left\{x_t\right\}}_{t=0}^{N-1}$ be a scalar time series, fixed $m\ge 2$ and $\tau \in {\mathbb{N}}^{+}$, and take a threshold $\epsilon >0$. Write

$$\sigma \left(u\right)=\mathbf{1}[u>\epsilon ]\in \{0,1\},u\in \mathbb{R}.$$

(1)

The embedding vector at time t is ${\mathbf{x}}_t^{\left(m\right)}=(x_t,x_{t+\tau },\dots ,x_{t+(m-1)\tau })$, and its sign vector is

$${\mathbf{s}}_t={\left(\sigma \left(x_{t+j\tau }\right)\right)}_{j=0}^{m-1}\in {\{0,1\}}^m.$$

(2)

The symbolisation used by the methods studied here does not record ${\mathbf{s}}_t$ itself. It records, for every pair of components, whether the two agree:

$$A_t={\left([\sigma \left(x_{t+j\tau }\right)=\sigma \left(x_{t+k\tau }\right)]\right)}_{0\le j<k\le m-1},$$

(3)

so that $A_t\in {\{0,1\}}^{\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)}$. The symbolic entropy is the Shannon entropy of the empirical distribution of $A_t$ over the $T=N-(m-1)\tau$ admissible positions, with $\widehat{p}\left(A\right)=\frac{1}{T}\#\{t:A_t=A\}$:

$$H(m,\tau )=-\sum _A\widehat{p}\left(A\right){log}_2\widehat{p}\left(A\right).$$

(4)

This is the quantity plotted as the entropy curve and, in the saturation family, used to select $\tau$. The threshold is conventionally tied to the data through $\epsilon =\alpha \mathrm{sd}\left(x\right)$; the methods below fix $\alpha$ and treat it as a design constant rather than a parameter to be estimated.

2.2. Complement Invariance and the Effective Alphabet

Lemma 1 

(Complement invariance). Write $b_j=\sigma \left(x_{t+j\tau }\right)$ for the j-th component of ${\mathbf{s}}_t$. For every t and every $j<k$,

$$[b_j=b_k]=[(1-b_j)=(1-b_k)],$$

(5)

so $A_t$ is invariant under the complementation ${\mathbf{s}}_t\mapsto \mathbf{1}-{\mathbf{s}}_t$. Consequently the map $\mathbf{s}\mapsto A\left(\mathbf{s}\right)$ is constant on the two-element orbits of the complement action on ${\{0,1\}}^m$, and its image contains at most $2^{m-1}$ elements.

Proof. 

Both sides of the first display assert that two bits agree; replacing both by their complements preserves agreement, giving the identity. For the second part, the complement action $\mathbf{s}\mapsto \mathbf{1}-\mathbf{s}$ is a fixed-point-free involution on ${\{0,1\}}^m$ (since $m\ge 1$, no $\mathbf{s}$ equals its complement), hence partitions the $2^m$ sign vectors into $2^m/2=2^{m-1}$ orbits. Since A is constant on orbits, it is a function of the orbit index, and the image of A therefore has at most $2^{m-1}$ elements. □

Geometrically, replacing $\sigma \left(u\right)=\mathbf{1}[u>\epsilon ]$ by $\mathbf{1}[u<\epsilon ]$, equivalently $\epsilon \to -\epsilon$, flips every component bit without changing A: reflecting the signal about the threshold leaves the symbol sequence, and hence H, identically unchanged, so the symbolisation is blind to which side of the threshold a component falls on.

Theorem 1 

(Entropy ceiling). For all $m\ge 2$, $\tau \in {\mathbb{N}}^{+}$ and any x and $\epsilon >0$,

$$0\le H(m,\tau )\le {log}_2{2}^{m-1}=m-1.$$

(6)

Proof. 

$A_t$ is a function of ${\mathbf{s}}_t$, which takes at most $2^{m-1}$ distinct values on orbits by Lemma 1. A random variable supported on k points has entropy at most ${log}_{2}k$, attained only when the distribution is uniform on the support. Hence $H(m,\tau )\le {log}_2{2}^{m-1}=m-1$. Non-negativity is immediate. □

The bound is tight and, importantly, the effective alphabet is often smaller than $2^{m-1}$ for a given signal, because distinct orbits can produce the same agreement pattern (the map is injective on orbits only when the binary comparison structure separates them, which fails for small m). Numerical verification that the bound is attained is in Section 4.2.

Remark 1 

(Scope of the ceiling). Theorem 1 is a statement about the pairwise-agreement symbol $A_t$ of (3), and only about it. Its engine is the complement degeneracy of that symbol; any symbolic construction whose alphabet is not invariant under complementation is untouched by the argument, and the ceiling $m-1$ does not apply to it. Permutation or ordinal-pattern entropy [9] is the clearest example of such a construction, and the class is wider than that. What the theorem excludes is the use of the saturation of a pairwise-agreement entropy as a delay estimator, not symbolic entropy as a tool.

2.3. Saturation for Mixing Processes

Proposition 1 

(Saturation). Suppose that for some ${\tau }_0$ the m components ${\left(x_{t+j\tau }\right)}_{j=0}^{m-1}$ are mutually independent with $\mathbb{P}(x_{t+j\tau }>\epsilon )=\frac{1}{2}$ for every j. Then $A_t$ is uniform on the $2^{m-1}$ agreement patterns and $H(m,\tau )=m-1$ exactly. In particular, if the process is mixing and ε is the median of the marginal, then $H(m,\tau )\to m-1$ as $\tau \to \infty$.

Proof. 

If the m signs are independent Bernoulli$\left(\frac{1}{2}\right)$, the sign vector ${\mathbf{s}}_t$ is uniform on ${\{0,1\}}^m$, hence uniform on the $2^{m-1}$ complement orbits by Lemma 1. The orbits inherit equal probability, so the induced distribution of $A_t$ is uniform on its support whenever A separates orbits; the entropy is then ${log}_2{2}^{m-1}=m-1$. For a mixing process the components become asymptotically independent as $\tau \to \infty$, and H converges to the independent value. □

Proposition 1 is the crux of our argument. Every signal (chaotic, noisy, periodic, or i.i.d. noise) has an entropy curve that rises and then saturates at $m-1$. The family locates $\tau$ at the onset of that saturation and reads it as the decorrelation delay, but the asymptote being approached is a constant fixed by the symbolisation, and information about the dynamics is carried entirely in the rate of approach. The recovered delay is therefore fixed by the shape of H near its ceiling (a nonlinear functional of the joint threshold-crossing statistics), and not by the second-order quantity (AMI) it is taken to estimate. Two structural consequences follow.

Remark 2 

($\tau$–m coupling). The ceiling $m-1$ depends on m, so the curve whose saturation onset is being located changes with m. Because saturation is located through a threshold on the normalised growth rate $|dH/d\tau |/max|dH/d\tau |$, changing m both raises the ceiling and displaces the whole curve, and $\widehat{\tau }$ therefore depends on the trial value of m. The two parameters are not separately identifiable within this family. In the LPF-SEM pipeline the dependence is circular: the dimension is derived from $\widehat{\tau }$ through the pseudo-period test, so an m-dependent $\widehat{\tau }$ feeds an $\widehat{m}$ that justifies the m used to obtain $\widehat{\tau }$. We quantify the drift in Section 4.8.

Remark 3 

(Thresholds are not identifiable either). The criterion interprets "saturation" as $|dH/d\tau |\le \gamma max|dH/d\tau |$ after low-pass filtering at cutoff $f_c$. Neither constant is determined by the data. Since H approaches its ceiling monotonically and smoothly for every signal, the first crossing of any such threshold traces out a wide range of delays as the constants vary, with no reason for any particular choice to be the decorrelation delay. Section 4.6 and Section 4.7 measure this.

2.4. Noise Inflation and the Looseness of the Standard Bound

Symbolic entropies are often argued to be noise robust on the grounds that additive noise only perturbs which side of the threshold a sample falls on. The following bound makes this precise, and the numerical study below shows it is valid but too loose to certify anything at practical noise levels.

Proposition 2 

(Noise-inflation bound). Let x have symbolisation entropy $H_x$ and let $y=x+\eta$ with $\eta \sim \mathcal{N}(0,{\sigma }_{\eta }^2)$ independent of x. Write q for the probability that a single threshold comparison of x is flipped by η, and let $\beta =2q(1-q)$ be the probability that two independent comparisons both flip or both do not, so that their agreement is preserved. With $h_2\left(\beta \right)=-\beta {log}_2\beta -(1-\beta ){log}_2(1-\beta )$ the binary entropy function,

$$H_y-H_x\le \left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)h_2\left(\beta \right).$$

(7)

Proof. 

Each of the $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ agreement bits of $A_t$ is unchanged when an even number of its two comparisons flip. Noise therefore acts on each agreement bit as a binary symmetric channel, active with probability $\beta$. A binary symmetric channel cannot decrease conditional entropy, $H\left(\mathrm{output}\right)\le H\left(\mathrm{input}\right)+h_2\left(\beta \right)$ per bit, and entropy is subadditive across the $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ coordinates, giving the bound. □

Remark 4 

(Sharpened constant). The constant $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ in (7) over-counts the available randomness. By Theorem 1 the agreement vector is determined by the $2^{m-1}$ complement orbits of an m-bit sign vector, i.e. by $m-1$ independent binary degrees of freedom, not by $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ independent bits. Replacing $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ by $m-1$ preserves the proof and gives a strictly tighter bound, which we evaluate alongside (7) in Section 4.10. Even the tightened form remains far above the observed inflation, which we take as evidence that bounds of this type cannot support the noise-robustness claims made on their behalf.

3. Estimators Under Test

3.1. Reference Criteria

We take as ground truth the two criteria the reconstruction literature has relied on longest.

Delay. The first minimum of the average mutual information [5], estimated on 64 equiprobable bins with $x_t$ and $x_{t+\tau }$ cross-tabulated for lags up to 100. We use the first local minimum, and denote the reference ${\tau }^{*}$.

Dimension. The first embedding dimension at which the fraction of false nearest neighbours [6] drops below $1\%$, with tolerance parameters $r_{\mathrm{tol}}=15$ and $a_{\mathrm{tol}}=2$ on the normalised distances, and candidate dimensions up to 12. We denote the reference $m^{*}$.

Both criteria are algorithm-independent and use no free parameter tuned to the present study.

3.2. The Saturation Family

We implement six estimators, reproducing the LPF-SEM pipeline exactly as specified in the source literature (adaptive Butterworth filter, normalised curvature criterion, pseudo-period consistency test), including its own internal constants.

• SEM [10]. The delay is the first local maximum of the unfiltered entropy curve $H(m,\tau )$ evaluated at $m=6$, with $\epsilon =0.1\mathrm{sd}\left(x\right)$. The dimension is fixed at 6.
• LPF-SEM. The entropy curve is low-pass filtered by a fourth-order zero-phase Butterworth filter with cutoff $f_c$ selected adaptively as $f_c=\mathrm{clip}(2.5f_p,0.02,0.3)$, where $f_p$ is the dominant frequency of the curve’s own power spectrum. The normalised filtered curve $\tilde{H}$ is differentiated, and the delay is the first index beyond ${\tau }_0=3$ where $|\mathrm{d}\tilde{H}/\mathrm{d}\tau |<\gamma max|\mathrm{d}\tilde{H}/\mathrm{d}\tau |$ with $\gamma =0.05$; if no index qualifies, the maximiser of the normalised curvature $\kappa =|{\tilde{H}}^{\prime \prime }|/{(1+{\tilde{H}}^{\prime 2})}^{3/2}$ is used instead. The dimension comes from the pseudo-period consistency test: the mean spacing of the filtered curve’s minima is accepted as a pseudo-period $T_p$ only if its coefficient of variation is below $0.5$, and then $m=\mathrm{clip}(\mathrm{round}({\tau }_{max}/T_p)+1,2,30)$; otherwise m falls back to the design default 6.
• MSE [12,14]. Coarse-grained multiscale sample entropy over scales $1\dots 20$ with $m=2$, $r=0.15\mathrm{sd}$; the delay is the scale minimising the entropy.
• SFE [13]. Symbolic fuzzy entropy over the delay axis for lags $1\dots 20$; the delay is the first local minimum.
• MS-SFE. Multiscale fuzzy entropy over scales $1\dots 20$; the delay is the scale minimising the entropy.
• C-C [11]. The correlation-integral statistic $S\left(t\right)$ and $\Delta \overline{S}\left(t\right)$ over $t=1\dots 60$ with 400 random reference points [15]; the delay is the first zero of $\overline{S}\left(t\right)$, the window the minimum of $\Delta \overline{S}$, and the dimension is derived from the window.

Where the source method returns a dimension through a rule of its own, we use that rule rather than imposing a common one, so that each estimator is judged as it would be used. All entropy curves are computed on at most 800 delay points per curve, and the search range is $\tau \le 200$ throughout. The complete pipeline is implemented in Python with NumPy [16], SciPy [17] and scikit-learn [18].

3.3. Reference Values Used

Applied to the five systems, the reference criteria give ${\tau }^{*}=16,30,40,43,21$ and $m^{*}=3,2,3,3,3$ for Lorenz, Rössler, Duffing, Mackey–Glass and Chen respectively. These values are used unchanged throughout all sections that follow.

4. Numerical Study on Chaotic Systems

4.1. Systems and Their Verification

We use four autonomous ODEs and one delay differential equation, all integrated from scratch with fixed-step fourth-order Runge–Kutta; no closed-form or synthetic surrogate is used anywhere in this study.

Lorenz$\dot{x}=10(y-x)$, $\dot{y}=x(28-z)-y$, $\dot{z}=xy-\frac{8}{3}z$, at $\mathrm{d}t=0.01$ [19]; Rössler$\dot{x}=-y-z$, $\dot{y}=x+0.2y$, $\dot{z}=0.2+z(x-5.7)$, at $\mathrm{d}t=0.02$ [20]; Duffing$\ddot{x}=-0.15\dot{x}+x-x^3+0.3cos\left(1.2t\right)$, at $\mathrm{d}t=0.02$, a forced oscillator; Chen$\dot{x}=35(y-x)$, $\dot{y}=-7x-xz+28y$, $\dot{z}=xy-3z$, at $\mathrm{d}t=0.005$ [21]; and the Mackey–Glass delay equation $\dot{x}=0.2x(t-17)/(1+x{(t-17)}^{10})-0.1x\left(t\right)$ at $\mathrm{d}t=0.1$ [22,23]. The observed component is x in every case, after discarding a transient that is at least 5000 steps.

Each system is verified chaotic rather than asserted to be, using a Benettin two-trajectory estimator with renormalisation [24,25]. For the ODEs the perturbation is rescaled every step of a 40000-step run; for the delay equation a scalar perturbation never reaches the delayed term, so the trailing delay-length history is rescaled instead, every one time unit, over a long run. Table 1 reports the largest exponents per unit time. All five are positive, and the ODE estimates agree with the standard literature values ($0.0725$ for Rössler against $0.0714$, $1.97$ for Chen against $2.03$). For Mackey–Glass the long-time average is step-size converged: $0.0086$ at 200 time units of accumulation settling to $0.0052$ at 2000, identically at $\mathrm{d}t=0.10$ and $0.02$.

White Gaussian noise is added at the specified signal-to-noise ratio as $y=x+{\sigma }_n\xi$ with ${\sigma }_n^2=\mathbb{E}\left[x^2\right]/{10}^{\mathrm{SNR}/10}$, so that the reported ratio is the in-band SNR of the record. Each cell of the main comparison is averaged over twenty independent noise realisations and five systems, using fixed random seeds so that every estimator sees identical data.

4.2. Numerical Verification of the Ceiling

Theorem 1 and Proposition 1 predict that $H(m,\tau )$ saturates at $m-1$ and that the bound is attained for a process whose components are independent. Table 2 tests both. We evaluate $H(m,\tau )$ at a large delay ($\tau =97$) for $m=2,\dots ,8$ on (i) i.i.d. standard normal samples, which maximise the uniformity of the sign vector, and (ii) the Lorenz trajectory.

The i.i.d. case attains the predicted ceiling to three decimal places in every row ($1.0000$, $1.9998$, $2.9998$, $3.9996$, $4.9978$, $5.9989$, $6.9958$ against $m-1=1,\dots ,7$). The small residual below the ceiling is finite-sample bias, since $2^{m-1}$ grows to 128 symbols while the effective sample size at $\tau =97$ is fixed. The Lorenz curve approaches the same ceiling from below without reaching it, because a deterministic flow at $\tau =97$ is not yet fully independent at $m=8$; the gap widens with m exactly as the enlarged alphabet makes attainment harder.

Figure 1(a) shows the full curves. Every system’s $H(m,\tau )$ rises and flattens, and the flattening occurs at a system-dependent delay that has the visual signature the family reads as a decorrelation onset. Figure 1(b) shows what noise does to the same construction on Lorenz: the approach to the ceiling becomes faster and the plateau becomes noisy, but the curve has the same qualitative shape, which is precisely why the criterion is described as noise tolerant: the feature it detects is present in every curve, including one that is pure noise.

4.3. Delay Recovery on Clean Data

Table 3 is the first substantive result. It reports the mean delay returned by each estimator on noiseless data, together with the ratio to the AMI reference. Since this is clean data, a correct estimator should return the reference up to integer rounding; a consistent multiplicative bias would at least be correctable.

Neither holds. LPF-SEM’s ratio ranges from $1.17$ (Rössler) to $4.69$ (Lorenz), a fourfold spread with no common factor. The unfiltered SEM is worse where it matters most, returning 123 for Duffing and 70 for Lorenz against references of 40 and 16. MSE and MS-SFE collapse onto small delays for four of the five systems ($\widehat{\tau }=5$ for Rössler, Duffing, Mackey–Glass and Chen, ratios $0.12$–$0.24$), the signature of an $argmin$ rule landing on the smallest admissible scale, and SFE returns 2–14 samples throughout, effectively a constant floor. C-C is the most consistent of the six, near $2{\tau }^{*}$ for Lorenz and Rössler but collapsing to 1 sample for Duffing, an error larger than the reference itself.

No member of the family returns a value proportional to the decorrelation delay, and the failure mode differs from member to member. This is not a noise problem.

4.4. Behaviour under Noise

Figure 2 and Table 4 extend the comparison to six noise levels. Two quantities are separated: the bias $|\widehat{\tau }-{\tau }^{*}|$ averaged over systems, and the run-to-run dispersion over the twenty realisations.

LPF-SEM carries the largest mean bias at every SNR, and the bias is not degraded by noise so much as uniformly large: $48.6$ samples on clean data rising to $53.8$ at 5 dB. Its dispersion is also the worst and nearly constant at about 50 samples, indicating a systematic offset rather than noise-driven scatter. MSE has the smallest bias, falling from $23.2$ to $13.9$ at 0 dB: its floor-like behaviour is less wrong than the ceiling-based methods, for the wrong reason. C-C is competitive at high SNR but degrades monotonically ($18.3$ at 20 dB to $26.3$ at 0 dB), and SEM improves with noise from $28.8$ to $24.2$, an artefact of the noise flattening the spurious local maxima that trap it on clean data.

The dimension results (right-hand panels) are more revealing still. On clean data LPF-SEM returns $m=6$ for Lorenz, Duffing, Mackey–Glass and Chen in $100\%$ of trials, which is exactly the design default used when the pseudo-period test is rejected, giving a constant dimension error of $3.0$ against $m^{*}=3$ on those four systems. Rössler is the exception: its pseudo-period test always passes on clean data and returns $m=4$ (error $2.0$ against $m^{*}=2$), though under added noise the fallback $m=6$ reappears. The pseudo-period test does not in general produce a dimension; it returns the default. The reported dimension is inert, which explains the otherwise puzzling uniformity and fixes the $\tau$–m coupling of Remark 2 at $m\approx 6$: LPF-SEM’s delay output is in effect an unfiltered-curve criterion evaluated at a single dimension with two extra smoothing stages.

4.5. Component Ablation

The ablation isolates each ingredient by building the method up in stages, holding the data fixed. Results are in Table 5.

All three readings are unfavourable. Filtering doubles the clean-data bias ($28.8\to 56.4$) and the penalty grows with noise, reaching $70.7$ at 0 dB: the filter is not neutral but actively harmful. The curvature criterion recovers part of the loss ($56.4\to 48.6$) but never returns to the unfiltered level, so the two-stage NCC remains worse than the criterion it replaces. The pseudo-period test is the only component with a consistent benefit, and its gain is confined to the dimension ($3.2\to 2.8$), is small, and as Section 4.4 established it is usually the design default rather than a value derived from the signal.

4.6. Filter Parameter Sensitivity

Figure 3 sweeps the two filter constants over $f_c\in \{0.02,\dots ,0.25\}$ and order $\in \{2,4,6\}$ for each system and noise level, using the same crossing rule for the delay.

Over this grid the returned delay spans 5 to 197 samples on clean data, essentially the entire search window $\tau \le 200$, with a standard deviation of 42 across the grid, and the spread is larger under noise (52 at 10 dB). The reported delay is therefore dominated by the filter constants rather than by the signal: a plausible arbitrary choice changes the answer fortyfold, two orders of magnitude beyond the deviation the method is claimed to reduce. The adaptive cutoff rule fixes one point in this space but chooses it from the dominant peak of the entropy curve’s own spectrum, a property of the same saturating curve whose onset is being located, so it inherits the circularity rather than resolving it.

4.7. Non-transferability of the Threshold

If the criterion measured something physical, the threshold $\gamma$ that reproduces the reference delay would be roughly the same across systems. We sweep $\gamma$ over $[0.005,1]$ and record the implied delay, then ask which $\gamma$, if any, returns ${\tau }^{*}$ to within one sample.

The answer, in Table 6, is that no single value works. On clean data the required threshold is $0.15$ for Rössler, $0.25$ for Mackey–Glass and $0.75$ for both Duffing and Chen, a fivefold spread, and for Lorenz no value in the entire swept range reproduces the reference at all. The required threshold then drifts with noise: Rössler moves $0.15\to 0.25$, Mackey–Glass $0.25\to 0.60$, and Duffing passes from feasible at $0.75$ to infeasible entirely. At 0 dB, only four of the five systems admit any feasible threshold within an order of magnitude.

A criterion whose free parameter must be re-tuned per system, drifts as the data degrades, and sometimes has no valid setting cannot serve as the basis of a general estimator. This is the operational form of Remark 3.

4.8. Coupling between the Delay and the Dimension

Remark 2 predicts that $\widehat{\tau }$ depends on the trial embedding dimension, because the ceiling defining saturation is $m-1$. Sweeping $m=3,\dots ,8$ at fixed data and fixed reference, we find the drift is real and system dependent: over that range the LPF-SEM delay moves $73\to 70$ for Lorenz, $45\to 33$ for Rössler, $177\to 173$ for Duffing, $69\to 50$ for Mackey–Glass and, in the opposite direction, $31\to 56$ for Chen. The drift reaches $1.8\times$ (Chen) and $1.4\times$ (Rössler, Mackey–Glass) even though ${\tau }^{*}$ is a fixed property of the signal.

Because the pipeline then derives the dimension from the delay, the two outputs are not independent estimates of two parameters; they are two views of one arbitrarily-parameterised computation. Reporting them as separate calibrated quantities, as the source methodology does, misrepresents what has been computed.

4.9. Computational Cost

Figure 4(a) reports wall-clock time for every estimator on identical hardware, using the exact implementations compared everywhere else in this study, with median times over five repeats at each series length. The fitted exponents p in $T\sim N^p$ are $0.26$ for C-C, $0.79$ for LPF-SEM, $0.87$ for SEM, $0.95$ for MSE, $0.99$ for MS-SFE and $0.22$ for SFE.

LPF-SEM is close to linear in the series length and cheaper than C-C by an order of magnitude, but its ordering against the unfiltered SEM reverses between $N=5000$ and $N=10000$. SFE’s fit is poor ($R^2=0.48$ against $0.94$–$0.997$ for the others) because its timings are non-monotone in N, so its exponent is not a scaling law. Nothing above depends on cost: an estimator can be cheap and wrong, and LPF-SEM is at once the most biased and among the cheapest, which explains its adoption without justifying it.

4.10. The Noise-Inflation Bound

Proposition 2 bounds the inflation of the symbolic entropy by additive noise, and Remark 4 sharpens its constant from $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ to $m-1$. Figure 4(b) evaluates both against the measured inflation on Lorenz at $\tau \le 120$, $m=6$.

Both bounds hold at every SNR, as they must. Both are also loose by a wide margin. At 0 dB the measured inflation is $1.00$ bit while (7) predicts $14.2$ bits and the sharpened form predicts $4.7$; at 30 dB the measured inflation is $0.009$ bit against $1.73$ and $0.58$. The looseness is $14\times$–$183\times$ for the original bound and $5\times$–$61\times$ for the sharpened one.

The reason connects to the main theorem. The bound treats the $\left({\displaystyle \genfrac{}{}{0pt}{}{m}{2}}\right)$ agreement bits as independently degraded, but by Theorem 1 they originate in only $m-1$ independent binary degrees of freedom, and noise flipping one sign bit flips all agreement bits involving that component simultaneously, so the degradations are strongly positively correlated. Positive correlation lowers the entropy of the flip pattern, hence the information gained, which is why the union bound is loose; accounting for it requires the joint flip distribution of the m comparisons, which is signal dependent.

We report this as a secondary negative result: bounds of the form (7) are often offered as the warrant for calling symbolic methods noise robust, but a bound permitting fourteen times the observed inflation cannot certify robustness at 0 dB.

5. Downstream Task: Fault Diagnosis

The measurements so far score the delay against a reference. A practitioner cares about it only through the task the reconstruction feeds. We therefore carry the comparison through to the application that motivates the family, RQA feature [26,27] fault classification on real machinery data [28,29], because the outcome reframes the practical claim, and it is more useful reported against the numbers than described.

5.1. Data and Protocol

Two sets are used.

MFPT bearing [30]: three baseline (healthy) records, seven outer-race and seven inner-race records under variable load. The baseline records are stored at twice the sampling rate of the fault records, so they are decimated by two; this equalises bandwidth across classes rather than favouring one. Each record is then cut into windows of 1024 samples, at most forty per record, giving 680 windows over three classes, chance $1/3$.

Rotor [31]: 180 denoised segments of 2048 samples, forty-five per class over normal, rub-impact, unbalance and misalignment, giving 360 windows over four classes, chance $1/4$.

For each estimator the delay and dimension are calibrated on a held-out slice of each record (the first 30000 samples, or the whole record where shorter) and then held fixed while that record’s windows are embedded. Each window is mapped to a nine-dimensional feature vector by recurrence quantification at the calibrated $(\tau ,m)$ with a Theiler window [32] of $2\tau$: recurrence rate, determinism, mean and maximum diagonal line length, diagonal-line entropy, laminarity, trapping time, and the maximum and the entropy of the vertical line lengths. Four classifiers are applied (an RBF-kernel SVM, k-nearest neighbours with $k=5$, a 200-tree random forest, and linear discriminant analysis) under five-fold cross-validation grouped by record, so that windows from one record never straddle the train/test split and the reported scores are not inflated by within-record similarity.

5.2. Results

Table 7 and Figure 5 give the accuracies, with the mean calibrated delay alongside.

On MFPT the estimator matters a great deal, and the ranking is the reverse of the one the delay study predicts. LPF-SEM is best on every classifier ($0.930$ with the random forest, $0.820$ even with a linear discriminant, the weakest model in the set), for a mean of $0.860$. C-C is worst on every classifier, averaging $0.588$: its random forest scores $0.599$, barely above the $0.333$ chance line, for a feature set that supports $0.930$ when the delay is LPF-SEM’s. The spread between best and worst, $0.27$ of accuracy, exceeds any difference reported between the methods under test in the source literature.

The ordering does not follow delay accuracy. C-C carried the smallest mean delay deviation of the six on clean data ($22.8$ samples) and at 20 dB ($18.3$), and it is worst here. LPF-SEM carried the largest deviation at every noise level, with a ratio to the reference spanning $1.17$–$4.69$ and a delay error above 48 samples in every row of Table 4; it is best here.

On the rotor set the picture inverts again, and less informatively: every estimator lands between $0.886$ and $0.986$, with C-C ($0.986$ SVM) and LPF-SEM ($0.978$ SVM) tied at the top and the rest within a point. The four classes are nearly linearly separable in these features, so the delay has little to do. This insensitivity is real, but it is a property of the rotor set rather than of the procedure, and it does not generalise to MFPT.

5.3. What Drives the MFPT Result

The obvious reading is that the delay is simply too short for the losers: LPF-SEM embeds at $\overline{\tau }=41.4$ and wins, MS-SFE and MSE at $19.6$ and $12.9$ occupy the middle, and C-C and SFE at $3.2$ and $4.1$ occupy the bottom. A recurrence plot at $\tau =3$ on a 1024-sample window is close to diagonal (the embedding barely separates successive coordinates), so its features should be correspondingly impoverished.

That reading cannot be the whole story, because SEM returns a delay as small as C-C’s ($\overline{\tau }=3.1$) yet places second at a mean of $0.773$; what separates them is the dimension, $11.7$ against $4.5$. The task responds to the $(\tau ,m)$ pair, and neither estimate being accurate is what selects the pair that works. We test the magnitude reading, and a less flattering one, directly.

5.4. Does the Delay Magnitude Explain It?

We sweep $\tau$ over 1–100 with m held fixed and everything else (windows, features, folds, classifier) unchanged. The sweep is run at the two dimensions the main experiment actually used: $m=6$ (LPF-SEM’s calibrated mean of $6.5$) and $m=12$ (SEM’s $11.7$); Figure 6(a) gives the result.

At $m=6$ the accuracy is flat to within $0.04$ for every delay from $\tau =1$ to $\tau =35$, and falls only past $\tau =50$, well beyond the range any estimator returns. At $m=12$ the curve is likewise shallow but peaked: it rises to $0.835$ (RF) near $\tau =18$ and declines by $0.17$ to $\tau =70$. Neither curve is monotone, and neither approaches the $0.930$ that LPF-SEM reaches in Table 7. The delay’s magnitude is not what separates the estimators; if it were, the sweep would climb toward LPF-SEM’s $\overline{\tau }=41.4$ and it does the opposite.

5.5. Does the Calibration Explain It?

Since the delay itself does not, we remove the signal and keep the calibration. Each window’s nine-dimensional feature vector is replaced by the two numbers $(\tau ,m)$ used to compute it, and the same grouped cross-validation is re-run; the last column of Table 7 and Figure 6(b) give the part of the reported score attributable to the calibration alone.

For LPF-SEM that control returns $0.817$ under the random forest and $0.683$ under the SVM, $88\%$ of the feature-based figure, from two numbers per record; for the other five it returns $0.28$ to $0.62$, and their ordering is close to the full pipeline’s. The mechanism is not mysterious: every window of a record is embedded at that record’s own $(\tau ,m)$, and the classes are disjoint in records, so a parameter estimate that separates the classes is copied into every feature vector of that record, and the classifier can read the class partly off the calibration rather than off the signal. LPF-SEM’s delays range from 8 to 52 across the 17 records and separate them well; C-C’s range from 2 to 5 and do not, which is why the most accurate delay estimator scores $0.433$ here and finishes last.

Two qualifications keep this honest. The control does not exhaust the scores: the features add $0.11$ over the parameters for LPF-SEM and much more for SEM and MS-SFE, so the signal still does work. And the effect is not universal: on the rotor set the control is high for several methods ($0.844$ for C-C, $0.767$ for SEM) while every method’s feature accuracy lies within $0.10$ of every other’s, so where the features are saturated the calibration adds nothing either way. The MFPT ranking, however, is dominated by it.

5.6. What the Inversion Means

The family was adopted because it improved fault-classification scores, and on the testbed where it was adopted it does: the evidence was real, the mechanism inferred from it was not. The method is a useful heuristic with an unsound explanation. What the sweep and the control together show is that the downstream evidence sustaining it was never evidence about the delay: the delay’s size does not drive the score, its accuracy does not drive it, and most of LPF-SEM’s MFPT advantage is reproduced by its calibrated parameters alone. A negative result about delay accuracy measured against AMI therefore has no purchase on that literature, which is why the criterion’s inability to recover the decorrelation delay went unnoticed through several variants and a decade of use.

6. Discussion

6.1. Why the Saturation Criterion Cannot Work

The failure is not an implementation defect and not a matter of tuning; it follows from what the entropy curve is. Saturation-based criteria rest on the premise that the delay at which $H(m,\tau )$ stops growing is the delay at which the embedding components become independent. Theorem 1 and Proposition 1 show that this happens for every signal whatsoever, because the symbolisation guarantees a ceiling at $m-1$ that mixing processes inevitably reach. The delay at which it happens is set by the rate at which the joint distribution of the m thresholded signs equilibrates: a nonlinear functional of the threshold geometry and the local expansion rates near that level set. The AMI minimum, by contrast, is a second-order statistic of the linear correlation structure, invariant to sign changes and to any strictly monotone transformation. Nothing forces the two to coincide; the measured ratios of $1.17$ to $4.69$ are what one expects of two unrelated functionals.

The extreme case makes the point. Applied to i.i.d. noise, which has ${\tau }^{*}=1$ trivially, since consecutive samples are independent, the curve still rises, still saturates smoothly at $m-1$ (Table 2), and still shows a plausible saturation onset. The criterion assigns a delay to a signal with no dynamics at all: the definition of a detector responding to an artefact. Two problems compound this. The ceiling depends on m, so the delay and the dimension are not independent though they are reported as such; and the thresholds ($\gamma$, and $f_c$ in the filtered version) are unconstrained by the data, so any threshold picks out some delay from a smooth monotone curve. The returned number is better described as a function of the analyst’s constants than of the signal.

6.2. What a Usable Replacement Should Preserve

Nothing in our argument disturbs the standing criteria. First-minimum AMI and first-false-neighbour selection remain the sensible default, for the reasons they always had: AMI is well defined for any process, has no free parameter beyond the binning, and its first minimum is the delay at which the reconstruction coordinates carry the least redundant linear information; FNN has a threshold, but one calibrated against the noise floor and attractor size rather than against a shape heuristic.

If one wants a symbolic criterion, and there are good reasons to want cheap threshold-based indicators of dependence, it must compare H against its own ceiling rather than a fixed threshold. Define the normalised occupancy

$$g(m,\tau )={\displaystyle \frac{m-1-H(m,\tau )}{m-1}}\in [0,1],$$

(8)

which measures the distance from the uniform state of Proposition 1 and is free of the ceiling artefact by construction. Selecting the first $\tau$ with g below a level tied to the sampling noise of H would at least measure saturation relative to what saturation means for that m. We have not established that such a criterion estimates the decorrelation delay, and our results give no reason to expect it; we offer it as the minimal repair rather than a validated method.

6.3. Practice: a Decision Rule for the Two Goals

Section 5 answers the question reviewers will ask first: if the estimator is this wrong, how was it ever adopted? Because on the downstream task it is not wrong but better, for a reason that, as Section 5.4 and Section 5.5 show, has nothing to do with the delay. The practical advice therefore has to be split by the practitioner’s goal, and we state it as two branches.

If the goal is physically correct reconstruction parameters, such as a delay for a recurrence plot, a correlation or Lyapunov dimension, an attractor reconstruction, a Theiler window, or any analysis in which $\tau$ is itself the object of interest, then do not use a saturation-based entropy criterion. Take $\tau$ from the first minimum of the AMI and m from FNN, as the field has done. This is not a matter of tuning: Section 4.6 and Section 4.7 show that no setting of LPF-SEM’s filter constants or threshold recovers the reference consistently, and its $\tau$ is wrong by a factor reaching $4.69$, so the number carries no dynamical meaning even where it happens to be close.

If the goal is only classification accuracy on a task already solvable from RQA features, LPF-SEM may be used as a black-box heuristic feature-engineering step, and on MFPT it is the best of those compared here ($0.930$, random forest). Two caveats carry over. First, its $\tau$ must not be reported or interpreted as the decorrelation delay: the parameter is a per-record index whose separating power is what helps the classifier, not the fact that it is a delay (Section 5.5), and the two are different quantities that happen to live in the same pipeline. Second, the advantage is task-specific rather than a property of the method (it is not reproduced on the rotor set, where every estimator lands within a point), so the heuristic should be validated on the target data rather than assumed to transfer.

6.4. Limitations

Six limitations should be stated.

First, and most important for reading the paper, the object of criticism is the saturation criterion, not the reference criteria. Our target throughout is the specific claim that the saturation onset of a pairwise-agreement entropy estimates the decorrelation delay, and specifically the methods that present themselves as replacements for AMI and FNN. Theorem 1 and the empirical study bear on that claim. They do not show that AMI and FNN are optimal, and we do not claim it. The critique is of a substitution, not of the standard practice that substitution proposed to replace.

Second, AMI and FNN have defects of their own. The AMI minimum depends on the binning, and for some processes the mutual information has no interior minimum, so the first-minimum rule becomes ill-defined; FNN depends on tolerance parameters and is noise-sensitive, tending to over-estimate m at low SNR. We use them as the field’s conventions and as the criteria under test claim to replace, which makes them the appropriate reference for this comparison, and not as ground truth. The correct reading of our result is that the saturation family is worse than the prevailing convention, which is what the family claimed to improve on, not that the convention is beyond criticism.

Third, the symbolisation threshold was fixed at $\epsilon =\alpha \mathrm{sd}\left(x\right)$ with $\alpha =0.1$ throughout; other choices move all curves, though Theorem 1 holds for every $\epsilon >0$ and the ceiling is unaffected, so the structural argument is independent of this choice while the specific numbers are not.

Fourth, our systems are stationary, and we did not evaluate non-stationary or intermittently chaotic signals: records with drifting parameters, transient laminar phases, or intermittent bursting. The theorem is unconditional in x, so the ceiling and the saturation regime hold for such records too, and the structural argument is unaffected; but the measured delay ratios, the ablation, and the downstream inversion are established only for the stationary benchmarks examined here and should not be assumed to transfer to non-stationary data.

Fifth, the entropy curves were thinned to 800 delay points and the search restricted to $\tau \le 200$, so the sensitivity result is bounded by that window and the true parameter dependence may be larger rather than smaller.

Sixth, the five systems are standard benchmarks plus one delay equation; we do not claim the measured ratios transfer beyond them.

7. Conclusions

We set out to reproduce a reported improvement in phase-space parameter estimation and instead proved why the underlying family of methods cannot work as advertised. The result is a boundary statement as much as a theorem: it closes the pairwise-agreement branch of the saturation family and makes no claim about symbolic entropies of other design.

The theorem, and its scope. The pairwise-agreement symbolisation at the heart of saturation-based entropy criteria is invariant under complementation of the thresholded sign vector, so its effective alphabet has exactly $2^{m-1}$ symbols, so $H(m,\tau )\in [0,m-1]$ and $H\to m-1$ for any mixing process. Saturation is built into the symbolisation rather than discovered in the data, and the delay read off the saturation onset is the argument of the approach to a self-imposed asymptote. Verified to three decimal places on i.i.d. noise and approached from below by chaotic flows, the law rules out the premise on which the family rests. The complement degeneracy is a property of the pairwise-agreement symbol itself, so the argument covers the C-C method through LPF-SEM and any future variant of the same construction; it does not extend to permutation entropy or to any symbolisation whose alphabet is not complement-degenerate (Remark 1). Symbolic entropy as a tool is not impugned, only its use, via pairwise agreement, to estimate the delay.

The evidence, stated plainly. The empirical study measures the consequences rather than asserting them. The low-pass filter, the method’s headline ingredient, doubles the delay bias instead of reducing it ($28.8\to 56.4$ samples); the returned delay sweeps a fortyfold range across the filter’s own constant grid; the threshold that would reproduce the AMI reference varies fivefold across systems and does not exist for two of five; the dimension output is the method’s own fallback constant in $100\%$ of runs for four of five systems; and LPF-SEM carries the largest delay bias of the six estimators at every one of six noise levels, spanning $1.17$–$4.69$ times the reference with no common factor. A secondary negative result is that the standard noise-inflation bound for symbolic entropies is valid but loose by one to two orders of magnitude at practical SNRs, even after its constant is sharpened using the ceiling theorem, so bounds of that form cannot certify noise robustness.

The counterintuitive part. In the downstream fault-classification study on MFPT bearing data the ordering inverts: the most biased estimator classifies best ($0.930$) and the most accurate one worst ($0.599$). Neither the delay’s magnitude nor its accuracy explains it (a sweep of $\tau$ at fixed dimension is flat or non-monotone over the whole range), and a control that discards the signal and keeps only the calibrated pair reproduces $88\%$ of the winner’s score, because every window of a record is embedded at that record’s own parameters. A criterion can be useful and wrong at once, and the downstream evidence that sustained this family was never evidence about the delay.

The advice. For a physically meaningful reconstruction (recurrence plots, correlation or Lyapunov dimensions, attractor analysis, Theiler windows), the delay should come from the first minimum of the average mutual information and the dimension from false nearest neighbours; saturation-based pairwise-agreement entropy criteria should not be used as delay estimators, and no tuning of their constants repairs this. Where only classification accuracy on an RQA feature set is wanted, LPF-SEM may be used as a black-box heuristic (it wins on MFPT), but its output must be read as a tuned index of the pipeline, not as the decorrelation delay of the signal, and validated on the target data rather than assumed to transfer.