Deterministic chaos models-based Gaussian noise generator

1. Introduction

Chaos theory has radically revolutionized our understanding of the potential for complex, unpredictable behavior within seemingly simple deterministic systems. This phenomenon, known as chaos, arises in nonlinear systems and is characterized by extreme sensitivity to initial conditions, often leading to the iconic “butterfly effect”. Chaotic dynamics are remarkably pervasive, manifesting in diverse domains such as social, biological, and technical systems [1,2,3,4,5,6].

Security threads and challenges prompt scientists to research how chaotic signals are suitable to resolve these issues. Numerous applications have been proposed including secure and covert communication, pseudo-random number generator (PRNG) and cryptography [7,8,9,10,11,12,13,14]. Despite the diverse applications, varying advantages and drawbacks in each of these scopes, in all of them, chaotic systems are used as sources of entropy, a fundamental property crucial for security applications.

Chaos generator, present in both hardware and software implementation, produces time series that either mask information-bearing signals or undergo postprocessing for various cryptographic purposes. Chaotic trajectories form strange attractors in phase space, which, in contradiction to limit cycle that represents regular behavior, has fractional or fractal dimension [1]. There are several variants of fractal dimension, generally measuring the average closeness, density, and distribution of points on chaotic attractors in phase space. Chaos sources are categorized into two groups including continuous time and discrete maps systems. Due to differences in attractor properties, the trajectories of a chaotic system exhibit different statistics reflected in their probability density functions (PDFs). Chaos applications necessitate a high-quality entropy source, emphasizing the necessity to maximize the entropy of chaotic signals. Statistical analyses have been proposed by Yong Wang et al. to increase the quality of chaos for the same purpose as in this paper [15]. Concerning PDF, the most necessary is a number with uniform PDF and Gaussian distributed signals with bell-shaped PDF. The first requirement is common in cryptography, the second is useful in communication. Both of them maximize uncertainty of range-limited and variance-limited signals, respectively [16].

A two-dimensional discrete-time white Gaussian noise generator was proposed in [17]. To obtain an independent time series with a Gaussian probability density function, a special transformation was used on the tent map. A similar approach to the same tent map demonstrated the efficiency of the transformation method [18].

The Gaussian distribution is not the sole target widely sought in signal processing. Contributions in this direction also focus on achieving a uniform probability density. In paper [19] it is proved that the folding sums of chaotic trajectories, bounded within a given interval by modulus operation, tend to approach uniformity. It is worth noting, that certain discrete chaotic systems yield series with a uniform probability density function without requirement of additional transformations [20,21,22,23,24,25].

Significant achievements have been attained in the realm of optical chaos generators [26,27,28,29,30,31]. Wideband optical chaos exhibits a heightened propensity for Gaussian-distributed probability density functions. This characteristic has prompted suggestions for harnessing optical chaos generators as high-quality, ultra-fast entropy sources [28,29]. A recently reported experimental technique focuses on generating a Gaussian-invariant distribution [32]. Such sources hold potential for diverse applications spanning communication [33,34], random number generation [26,27] and cryptography [30,35].

Covert communication, originally designed to conceal and render the transmission undetectable, is now being harnessed for commercial and private applications to provide physical-level protection. In contrast to conventional communication, where Gaussian white noise diminishes the signal-to-noise ratio, covert communication employs it as a countermeasure against eavesdroppers. Thermal noise is utilized to randomize the transmitted signal in a Gaussian-distributed spread spectrum [36]. Additionally, extending this concept, the utilization of artificially generated Gaussian-distributed noise has been proposed across diverse scenarios [37,38,39,40].

Despite hundreds of chaotic systems have been designed and studied, little of them can produce Gaussian-distributed signals. Therefore, finding an approach for obtaining the most complex chaotic signal based on simple and well-studied nonlinear systems is important. In this paper, we study the statistical properties of some chaotic signals to find out how much of them are necessary to form a Gaussian-distributed waveform with the following experimental implementation and achievement verification that allow an implementation of ”deterministic noise” generators.

The manuscript is organized as follows. In section 2, we describe the quality metrics used to evaluate and compare chaotic signals. The properties of time series produced by mathematic models of Chua’s circuit and Lorenz system are studied in section 3. Experimental verification of the results is provided by the section 4. The discussion and conclusion are presented in section 5. The appendix presents the result of computing statistical properties of more than 40 continuous-time chaotic systems found in literature.

2. Materials and Methods

2.1. Central Limit Theorem

The central limit theorem (CLT) relates to probability theory and explains why Gaussian noise is widespread in nature. For identically distributed independent samples, the standardized sample mean tends towards the standard normal distribution even if the original variables themselves are not normally distributed. There are several variants of CLT. In its common Lindeberg-Lévy form the CLT is following [41,42]:

Theorem 1.

Suppose $\{X_1,\cdots ,X_n\}$ is a sequence of independent and identically distributed random variables with expectation $\mathbb{E}\left[X_i\right]=\mu$ and variance $Var\left[X_i\right]={\sigma }^2<\infty$. Then as n approaches infinity, the random variables $\sqrt{n}(\frac{1}{n}{\sum }_{i=1}^n{X}_i-\mu )$ converge in distribution to a normal $\mathcal{N}(0,{\sigma }^2)$.

The CLT shows a way to obtain a variable with Gaussian-like distribution from several others by summation ones. The key condition for particular components is their limited variance which chaotic signals satisfy due to existence of their attractors in bounded phase space. The question is to find how many independent chaotic signals are necessary and which criteria the resulting signal should meet to be considered Gaussian-like. Therefore, the following consideration of the basic distributions will give a possibility to develop an algorithm for testing of chaotic signals.

2.2. Basic Distribitions

As a reference signal, we chose white Gaussian noise whose statistical properties can be described by three basic distributions which are the PDF of the shape, envelope and phase of the signal.

The term ”Gaussian noise” or ”Gaussian signal” refers to a normal or bell-shaped probability density function of this signal. Such signals can be either random noise or information-bearing. A Gaussian variable has following PDF

$$p\left(x\right)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{(x-\mu )}^2}{2{\sigma }^2}},$$

(1)

where $\sigma$ – standard deviation; $\mu$ – mathematical expectation or mean.

The envelope of white noise has Rayleigh distribution

$$p(x,\sigma )=\frac{x}{{\sigma }^2}{e}^{-\frac{x^2}{2{\sigma }^2}},$$

(2)

where $\sigma$ – standard deviation of the gaussian distribution (1).

Uniform distribution describes the phase of signal

$$p\left(\varphi \right)=\frac{1}{2\pi },$$

(3)

where $0\le \varphi \le 2\pi$.

2.3. Measures of Similarity of Probability Density Functions

To evaluate similarity of any signal to WGN, the normality tests that includes computing of skewness and excess kurtosis are conventionally utilized [43].

Skewness is a measure of the asymmetry of PDF around its mean and is defined as the normalized third central moment of signal

$${\tilde{\mu }}_3=\mathbb{E}\left[{\left(\frac{x-\mu }{\sigma }\right)}^3\right],$$

(4)

where $\mathbb{E}$ – expectation operator.

Excess kurtosis is the metric of the height and sharpness of PDF, relative to that of a standard bell curve

$${\tilde{\mu }}_4=\mathbb{E}\left[{\left(\frac{x-\mu }{\sigma }\right)}^4\right]-3.$$

(5)

For WGN both skewness and excess kurtosis are equal to zero.

It is worth noting, that if a signal exhibits a symmetric distribution with zero excess kurtosis, it by no means guarantees that its probability density function is normal. There are non-Gaussian distributions with ${\tilde{\mu }}_3=0$ and ${\tilde{\mu }}_4=0$ [44,45]. Therefore, at least, a graphical analysis is necessary to confidently assert the normality of the distribution, or confirmation through more sophisticated tests.

The envelope and phase of a signal can be found by using Hilbert transform [46]. For a given function $x\left(t\right)$, Hilbert transform is given as

$$\widehat{x}\left(t\right)=\mathbb{H}\left(x\left(t\right)\right)=\frac{1}{\pi }{\int }_{-\infty }^{\infty }\frac{x\left(t\right)}{t-\tau }d\tau .$$

(6)

Then the envelope and phase of $x\left(t\right)$ are

$$a\left(t\right)=\sqrt{x^2\left(t\right)+{\widehat{x}}^2\left(t\right)},$$

(7)

$$\varphi \left(t\right)=arctan\frac{\widehat{x}\left(t\right)}{x\left(t\right)}.$$

(8)

If $x\left(t\right)$ is WGN, the envelope $a\left(t\right)$ has Rayleigh PDF (2), and phase $\varphi \left(t\right)$ has a uniform PDF (3).

The informational properties of signals can be assessed by their differential entropy. For a variable x with a probability density function $p\left(x\right)$, the differential entropy is given by

$$h=-{\int }_{-\infty }^{\infty }p\left(x\right)logp\left(x\right)dx.$$

(9)

The Gaussian distribution (1) attains the maximum differential entropy, which, for $\mu =0$ and variance ${\sigma }^2$ is equal

$$h_g=\frac{1}{2}ln2\pi e{\sigma }^2.$$

(10)

For any other signal with the same variance ${\sigma }^2$, the differential entropy $h\left(x\right)$ does not exceed the value given in (10), i.e., $h\left(x\right)\le h_g\left(x\right)$.

Entropy power, a concept pioneered by Shannon [47], allows us to compare the probability density functions of a signal and Gaussian noise:

$$k_1=\frac{e^{2h}}{2\pi e},0\le k_1\le 1,$$

(11)

where h – entropy of signal.

The entropy power of a distribution coincides with the variance of a Gaussian distribution possessing an identical entropy.

To compare two arbitrary PDFs we used entropy power ratio, mathematically defined as follows for distributions $p_1\left(x\right)$ and $p_2\left(x\right)$ bearing entropies $h_1$ and $h_2$, respectively:

$$\frac{k_1\left(h_2\right)}{k_1\left(h_1\right)}=e^{2\Delta h},$$

(12)

where $\Delta h=h_1-h_2$ – the difference between differential entropies of two distributions.

To restrict the value of ratio (12) into interval $0\le k_1\le 1$ the following expression can be applied:

$$k=e^{-2\left|\Delta h\right|}.$$

(13)

The entropy power (13) increases when the difference between entropies of distributions becomes smaller.

To evaluate the similarity of probability density functions of amplitude and phase with reference distributions, we utilize the entropy power relative to these baseline distributions. Thus, the entropy power of the envelope of the signal is determined as:

$$k_2=e^{-2|h_e-h_R|},$$

(14)

where $h_e$ – entropy of the envelope of signal; $h_R=1+ln\frac{\sigma }{\sqrt{2}}+\frac{\gamma }{2}$ – entropy of Rayleigh PDF (2) [44]; $\gamma \approx 0.577$ – Euler-Mascheroni constant.

The entropy power for evaluating the similarity of phase distribution of signal and uniform one would be determined as:

$$k_3=e^{-2|h_{ph}-h_u|},$$

(15)

where $h_{ph}$ – entropy of phase of signal; $h_u=ln2\pi$ – entropy of uniform PDF (3) [44].

3. Properties of Sums of Chaotic Signals

The approach for generating a Gaussian-distributed chaotic signal is schematically illustrated in Figure 1. Selected chaotic outputs undergo normalization, followed by summation, to achieve a transformation from their original probability density function to a Gaussian PDF. The normalization step may be omitted depending on the properties of the original chaotic signals and the desired variance of the resulting sum.

There are lots of different nonlinear systems exhibiting chaotic behavior. We investigated more than 40 mostly three-dimensional chaotic systems through computing PDFs, skewness, excess kurtosis, and entropy power of their output signals. Before estimation of the mentioned in the algorithm characteristic, each chaotic signal was centered around its zero mean and normalized to have unit variance. The results show that the majority of the studied chaotic systems exhibit characteristics that are far from being considered Gaussian-like signals.

The obtained results for the variety of chaotic systems appear in the Appendix, and a detailed description of the algorithm is analytically and experimentally demonstrated below with examples from two widely used and extensively studied systems: Chua and Lorenz. They were specifically chosen since their signals, among all examined, possess the most favorable parameters to serve as a platform for generating deterministic chaotic Gaussian signals.

3.1. Chua’s Circuit

The Chua’s circuit is one of the most well-known systems that generate deterministic chaos and can be easily implemented as an electrical circuit [48]. Moreover, various types of nonlinearities can be utilized in this circuit [49,50].

The mathematical model of Chua’s circuit in dimensionless form is a system of three differential equation

$$\begin{array}{c}\dot{x}=\alpha (y-x-f\left(x\right))\\ \dot{y}=x-y+z\\ \dot{z}=-\beta y\end{array}$$

(16)

where x, y, z – output variables; $\alpha$, $\beta$ – parameters of the model; $f\left(x\right)=m_{2}x+0.5((m_0-m_1)\left(\right|x+a_1|-|x-a_1\left|\right)+(m_1-m_2)\left(\right|x+a_2|-|x-a_2\left|\right))$ – nonlinear function, $a_1=1$, $a_2=6.88$, $m_0=-1.238$, $m_1=-0.6665$, $m_2=500$. The connection between the circuit’s and the model’s parameters can be found in [48,51].

We studied Chua’s circuit with the piecewise linear characteristic of the nonlinear part which provides various dynamical modes including a single or double-scroll chaotic attractor. As shown in Figure 2, the circuit encompasses inductance, two capacitors, and nonlinearity implemented on two operational amplifiers [51].

When $\alpha =10$, $\beta =14.6$, the circuit operates in double-scroll mode and all the outputs possess a symmetric PDF with skewness close to zero. The y output produces low excess kurtosis ${\tilde{\mu }}_4=0.1421$, which also has the highest entropy power $k_1=0.9039$, $k_2=0.9756$, and $k_3=0.998$. Nevertheless, there are distinct visual differences between PDFs of chaotic and basic distributions which can be seen on Figure 3a,e,i and numerically from Table 1.

The Central Limit Theorem states that the summation of two or more chaotic signals yields a PDF of the sum that exhibits a greater resemblance to a Gaussian PDF compared to that of a single signal. The evolution of PDFs when the number n of independent chaotic signals in sum increases is shown in Figure 3. To minimize the number of chaotic signal in the sum, it is efficient to use the variable y. When only three independent outputs of y are added, the PDF of the sum, along with its envelope and phase, closely resemble the basic ones, i.e., normal, Rayleigh, and uniform PDFs. The entropy powers of the sum $k_1,k_2,k_3$ exceed $0.99$.

The quality metrics of the sum of signals are contingent upon the number of signals incorporated, as illustrated in Figure 4. The excess kurtosis decreases to zero, and all entropy powers converge to 1 with an increasing value of n. This suggests that achieving chaos resembling Gaussian noise necessitates the addition of a minimum of three original chaotic signals from Chua’s circuit.

A change of circuit parameters with preservation of chaotic double-scroll mode causes significant changes in quality metrics, particularly, excess kurtosis and entropy power. For example, when $\alpha =9.273$, $\beta =16.8$, $a=1587$ Chua’s circuit has following metrics: $E=-0.9334$, $k_1=0.8739$, $k_2=0.5006$ and $k_3=0.999$ which are quite different from ones mentioned above. This result suggests that the PDFs of signals are sensitive to the parameters of chaotic system.

3.2. Lorenz System

While the previously considered Chua’s circuit is one of the simplest in both as an analytical model and experimental implementation, Lorenz system is the well-known and extensively investigated source of deterministic chaos such as the system was originally developed to model atmospheric convection in meteorology and allows describing of different atmospheric phenomena [52]. Lorenz system is usually applicable in its mathematical representation (a system of three differential equations with nonlinearity (17)) that found widespread use in diverse fields such as physics, engineering, biology, economics, and even art. However, its hardware implementation is inherently more complex than Chua’s circuit due to involvement of analog multipliers (Figure 5) [53].

$$\begin{array}{c}\dot{x}=\sigma (y-x)\hfill \\ \dot{y}=(r-z)x-y\hfill \\ \dot{z}=xy-bz\hfill \end{array}$$

(17)

where x, y, z – system variables; $\sigma =10$, $r=28$, $b=\frac{8}{3}$ – parameters.

We simulated the system (17) with two different values of the parameter r: $r=28$ and $r=60$ keeping other parameters constant. The statistic parameters of Lorenz system is shown in Table 2. Two of the three outputs in the Lorenz system, namely x and y, exhibit symmetric PDFs. Similarly to the Chua’s circuit, the results indicate a significant impact of model parameters on the appearance and characteristics of the PDFs of the output signals. For the investigated parameter values, the signal most resembling Gaussian noise is y with $\sigma =10$, $r=28$, and $b=\frac{8}{3}$, featuring coefficients ${\tilde{\mu }}_4=-0.1573$, $k_1=0.9013$, $k_2=0.9368$, and $k_3=0.8877$.

The original histograms of normalized output signals from the Lorenz system are shown in Figure 6. Despite the asymmetry in the output signal z, combining all three outputs can yield a normal PDF when summing only 3-4 signals. The relationship between the excess kurtosis coefficient and entropy powers for different n is presented in Figure 7 and confirms this conclusion.

The entropy power values ($k_1,k_2,k_3$) for the sum of three and four y variables exceeded 0.98 and 0.99, respectively. Furthermore, a comparison of PDFs in Figure when $n=4$ suggests no significant discrepancies between the PDFs of the sum and the basic PDFs.

Analyses of Chua’s circuit and Lorenz system reveal that the summation of chaotic non-Gaussian signals proves to be an effective method for obtaining deterministic chaotic Gaussian signals.

4. Experimental Verification

4.1. Experiment with Chua’s Circuit

The following results were obtained for the Chua circuit with a piecewise-linear nonlinear element implemented using operational amplifiers TL082CP, as depicted in Figure 2. Four circuits were assembled with slightly different parameters. The parameter difference was provided be the used components feature – the used capacitors $C1=100\mu F$ and $C2=10\mu F$, as well as inductor $L=18mH$ have tolerance $\pm 5\%$. The parameters of nonlinear element were as $R2,R3=220Ohm$, $R4=2.2kOhm$, $R5,R6=22kOhm$, $R7=3.3kOhm$.

Additionally, a potentiometer was utilized as the resistance $R1=1580Ohm$ whose value could be adjusted with increment $\pm 100Ohm$ to change the chaotic circuit behavior. The voltages across the capacitors $C_2$, corresponding to the variable y in the model of circuit, were recorded and processed for further analysis. The set of signal records (50 samples of voltages across the capacitors $C_2$) was performed where each record contained $2\ast {10}^6$ sweep points.

The experimental results have confirmed the theoretical assumptions and analytical outcomes. As depicted in Figure 8, increasing the number of chaotic systems, rapidly drives the distribution of their summed signals towards a normal distribution. This observation extends to the envelope and phase of the signal, where Rayleigh and uniform distributions were observed, respectively.

4.2. Experiment with Lorenz System

The experimental investigation was performed using a simplified electronic realization of the Lorenz oscillator, constructed in accordance with the specifications outlined in [53,54] and circuit in Figure 5. We used analog multiplier AD633 and operational amplifier TL084CN. The others circuit elements were following: $R1,R2=100kOhm$, $R3,R6=10kOhm$, $R7=1MOhm$ with tolerance 5% and $R4=370Ohm$ with tolerance 0.5%. The variable resistor $R5$ allows tuning of the parameter r and was set $R5=35.72kOhm$.

Four circuits were assembled on a breadboard, and the appropriate oscillograms were recorded. Subsequently, they were subjected to the suggested set of tests. The probability density functions of the normalized output x from the electronic circuit of Lorenz system are illustrated in Figure 9. A comparison between Figure 6 and Figure 9 reveals a good agreement between analytic and experimental results.

5. Discussion and Conclusions

The results underscore a promising approach for generating deterministic Gaussian-like signals using an optimal number of chaotic systems. The PDFs of the original chaotic signals exhibit essential deviations from a normal distribution, indicating their inherently lower information capacity compared to fully random Gaussian noise. This observation prompts the need to establish criteria for selecting chaotic systems to achieve Gaussian signal formation through summation chaotic signals. It requires an optimization process to minimize the number of independent simple chaotic signals. The performed analysis demonstrates that adherence to the following guidelines significantly enhances the likelihood of successful chaotic system and signal selection:

• Prioritize signals possessing symmetric probability density functions.
• Minimize the excess kurtosis of the selected signals, ideally aiming for ${\tilde{\mu }}_4<1$.
• Ensure that all three entropy powers ($k_1,k_2,k_3$) of the original chaotic signal surpass the value $0.85$.

While strict adherence to these guidelines does not ensure Gaussian signal achievement, however, it substantially increases the probability of generating a Gaussian chaotic signal with desired entropy coefficients $k_1,k_2,k_3>0.95$ by summing merely three or four independent chaotic signals.

It is crucial to underscore that chaotic signals are generated by deterministic systems, rendering their summation equally deterministic. This underscores the capability of simple summation to yield intricate, noise-mimicking chaotic signals that retain their deterministic nature. The most innovative aspect of this method emphasizes the possibility of attributing the output voltage and current of random chaotic circuits to a well known signal as the Gaussian noise, whose properties are well-known and exploitable in many applications. The possibility of obtaining a Gaussian like signal description allows the opportunity to design and model chaotic circuits’ output voltage, preserving all the general properties of the Gaussian distribution. Moreover, since many chaotic circuits do not show output’s signals linked to Gaussian noise, this method allows widening the field of investigation even to those circuits that apparently do not have these properties. It is also considerable that more than 40 chaotic systems have been evaluated that have led to the confirmation of this theory, confirming the validity of this method.

Output signals, obtained with this process, hold considerable value across a variety of applications, including generating deterministic Gaussian noise, electronic warfare systems, coherent and/or covert communication systems, radars, statistical modeling, machine learning, cryptography and more.

The possibility of obtaining a pseudo-random number generator through the simple sum of chaotic signals, predisposes this method to a wide usability, bypassing any possible complication of chaotic signal processing. All the possible applications mentioned have the support of both hardware and software implementation of chaotic circuits, with the addition in this case, of greater properties and guidelines that can be exploited during the implementation guaranteed by the traceability to Gaussian noise. Since the main aim is to give more known laws to circuits that in certain conditions are dominated by chaos, other aspects and elaborations of the output signals of the proposed chaotic circuits are also being investigated, which could lead back to the properties of a Gaussian pseudo noise. Future investigations will be aimed at trying to reconstruct more and more well-known properties of apparently chaotic signals, trying to make it easier to implement the chosen application.