Revealing Short–Term Memory Communication Channels Embedded in Alphabetical Texts: Theory and Experiments

1. Introducing an Equivalent Input–Output Model of the Short–Term Memory

Humans can communicate and extract meaning both from spoken and written language. Whereas the sensory processing pathways for listening and reading are distinct, listeners and readers appear to extract very similar information about the meaning of a narrative story – heard or read – because the brain assimilates a written text like the corresponding spoken/heard text [1]. In the following, therefore, we consider the processing of reading or writing a text – a writer is also a reader of his/her own text – due to the same brain activity. In other words, the human brain represents semantic information in a modal form, independently of input modality.

How the human brain analyzes the parts of a sentence (parsing) and describes their syntactic roles is still a major question in cognitive neuroscience. In References [2,3], we proposed that a sentence is elaborated by the short–term memory (STM) with two independent processing units in series (equivalent surface processors), with similar size. The clues for conjecturing this input–output model emerged by considering many novels belonging to the Italian and English literatures. In Reference [3], we showed that there are no significant mathematical/statistical differences between the two literary corpora, according to the so–called surface deep–language parameters, suitably defined.

The model conjectures that the mathematical structure of alphabetical languages – digital codes created by the human mind for communication – seems to be deeply rooted in humans, independently of the particular language used or historical epoch. The complex and inaccessible mental process lying beneath communication – still largely unknown – can be studied by looking at the input–output functioning revealed by the structure of alphabetical languages.

The first processor is linked to the number of words between two contiguous interpunctions, variable indicated by$I_p$ – termed word interval (Appendix A lists the mathematical symbols used in the present article) – approximately ranging in Miller’s $7\pm 2$ law range [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]. The second processor is linked to the number $M_F$of$I_p$’s contained in a sentence, referred to as the extended short–term memory (E–STM), ranging approximately from 1 to 6. These two units can process sentences containing approximately a number of words from $8.3$ to$61.2$, values that can be converted into time by assuming a reading speed. This conversion gives $2.6~19.5$ seconds for a fast–reading reader [14], and $5.3~30.1$ seconds for a reader of novels, values well supported by experiments [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30].

The E–STM must not be confused with the intermediate memory [31,32]. It is not modelled by studying neuronal activity, but by studying only the surface aspects of human communication due, of course, to neuronal activity, such as words and interpunctions, whose effects writers and readers experience since the invention of writing. In other words, the model proposed in References [2,3] describes the “input–output” characteristics of the STM. In Reference [33], we further developed the theory by including an equivalent first processor that memorizes syllables and characters to produce a word.

In conclusion, in References [2,3,33] we have proposed an input–output model of the STM, made of three equivalent linear processors in series, which independently process: (1) syllables and characters to make a word, (2) words and interpunctions to make a word interval; (3) word intervals to make a sentence. This is a simple, but a useful approach because the multiple brain process regarding speech/texts is not yet fully understood but characters, words and interpunctions – these latter are needed to distinguish word intervals and sentences – can be easily studied [34,35,36].

In other words, the model conjectures that the mathematical structure of alphabetical languages is deeply rooted in humans, independently of the particular language used or historical epoch. The complex and inaccessible mental process lying beneath communication – still largely unknown – is revealed by looking at the input–output functioning built–in in alphabetical languages of any historical epoch.

The literature on the STM and its various aspects is immense and multidisciplinary – we have recalled above only few references – but nobody – as far as we know – has considered the connections we found and discussed in References [2,3,33]. Our modelling of the STM processing by three units in series is new.

A sentence conveys meaning, of course, therefore the theory we have developed might be one of the necessary starting points to arrive at the Information Theory that will finally include meaning.

Today, many scholars are trying to arrive at a “semantic communication” theory or “semantic information” theory, but the results are still, in our opinion, in their infancy [37,38,39,40,41,42,43,44,45]. These theories, as those concerning the STM, have not considered the main “ingredients” of our theory, namely the number of characters per word $C_P$, $I_P$ and $M_F$, parameters that anybody understands and can calculate in any alphabetical language [34,35,36], as a starting point for including meaning, still a very open issue.

Aim of the present paper is twofold: (a) to further develop the theory proposed in Reference [2,3,33], and (b) apply it to the flow of linguistic variables making a sentence. This “signal” flow is built–in in the model proposed in Reference [33], namely, the transformation of: (a) characters into words; (b) words into word intervals; (c) word intervals into sentences, according to Figure 1. Since the connection between these linguistic variables is described by regression lines [34,35,36], in the present article we analyze experimental scatterplots between these variables

The article is ideally divided in two parts. In the first part – from Section 2 to Section 4 – we recall and further develop the theory of linear channels [2,3,33]; in the second part – from Section 5 to Section 8 – we apply it to a significant database of literary texts.

The database of literary texts considered is a large set of the New Testament (NT) books, namely the Gospels according to Matthew, Mark, Luke, John, the Book of Acts, the Epistle to the Romans, and the Apocalypse – 155 chapters in total, according to the traditional subdivision of these texts. We have considered the original Greek texts and their translation to Latin and to 35 modern languages, texts partially studied in Reference [35]. Notice that in this paper, “translation” is indistinguishable from “language” because we deal only with one translation per language.

We consider the NT books and their modern translations for two reasons: (a) they tell the same story, therefore it is meaningful to compare the translations in different languages; (b) they use common words – not the words of scientific/academic disciplines – therefore, they can give some clues on how most humans communicate.

After this introductory section, Section 2 presents the theory of linear regression lines and associated communication channels; Section 3 presents the connection of single linear channels; Section 4 proposes and discusses the theory of series connection of single channels affected by noise; Section 5 reports an exploratory data analysis of the NT texts; Section 6 reports findings concerning single channels, Section 7 concerning series connection of channels and Section 8 concerning cross channels; finally Section 9 summarizes the main findings and indicates future studies.

2. Theory of Linear Regression Lines and Associated Communication Channels

In this section, we recall and further expand the general theory stochastic variables linearly connected, originally developed for linguistic channels [35,36] but applicable to any other field/specialty in which a linear relationship holds between two variables.

Let $x$ (independent variable) and $y$ (dependent variable) linked by the line:

$$y=mx+b$$

(1)

Notice that Eq. (1) models a deterministic relationship through the slope $m$ and the intercept $b$. Since in most scatterplots between linguistic variables $b=0$, in the following we assume

$$b=0$$

(2)

However, notice that if $b\ne 0$, the theory can be fully applied by defining a new dependent variable $\check{y}=y-b$.

In general, the relationship between $x$ and $y$ is not deterministic, i.e., given by Eq. (1), but stochastic (random). Eq. (1) models, in fact, two variables perfectly correlated – correlation coefficient$r=1$ – characterized by a multiplicative “bias” $m.$ In general, however, these conditions do not hold, therefore Eq. (1) can be written as:

$$y=mx+n$$

(3)

In Eq. (3) $n$ is an additive Gaussian stochastic variable with zero mean value [34,35,36], therefore Eq. (3) models a noisy linear channel. Notice that $n$ must not be confused with an intercept $b$.

Figure 2 shows the flow chart describing Eq. (1) and Eq. (3) with a system/channel representation. The black box indicated with $m$ represents the deterministic channel, i.e. Eq. (1); the black box indicated with $r$ represents the parallel channel due to the scattering of $y$ around the regression line. The additive noise $n$ is a Gaussian stochastic variable with zero mean that makes the linear channel partially stochastic, namely “noisy”.

Now, let us consider:

a)

The variance of the difference between the values calculated with Eq. (1) $\left(m\ne 1\right)$and those calculated with $y=x\left(m=1\right)$ (45° line) at given $x-$ value, as the “regression noise” power $N_m$ [35]. This “noise” is due to the multiplicative bias between the two variables.

b)

The variance of the difference between the values not lying on Eq. (1) ($r\ne 1$), and those lying on it ($r=1$), as the “correlation noise” power $N_r$ [35]. This “noise” is due to the spread of $y$ around the line given by Eq. (1), modelled by $n$.

c)

Let $s_x^2$ and $s_y^2$ be the variances of $x$ and$y$.

In case (a), we get the difference $\left(m-1\right)x,$ therefore the variance (or power) of the values lying on the regression line, regression noise, is given by:

$$N_m={\left(m-1\right)}^2{s}_x^2$$

(4)

Now, we define the regression noise−to−signal power ratio (NSR), $R_m,$ as:

$$R_m={\displaystyle \frac{N_m}{s_x^2}}={\left(m-1\right)}^2$$

(5)

In case (b), the fraction of the variance $s_y^2$ due to the values of $y$ not lying on the regression line (correlation noise power,$N_r$) is given by [46]:

$$N_r=\left(1-r^2\right)s_y^2$$

(6)

The parameter ${\mathit{r}}^2$ is called the coefficient of determination and it is proportional to the variance of $\mathit{y}$ explained by the regression line [46]. However, this variance is correlated with the slope $\mathit{m}$ because the fraction of the variance ${\mathit{s}}_{\mathit{y}}^2$ due to the regression line, namely ${\mathit{r}}^2{\mathit{s}}_{\mathit{y}}^2$, is related to $\mathit{m}$ according to [46]:

$$r^2{s}_y^2=m^2{s}_x^2$$

(7)

Figure 3 shows the flow chart of variances.

Therefore, inserting Eq. (7) in Eq. (6), we get the correlation NSR, $R_r$:

$$R_r={\displaystyle \frac{N_r}{s_x^2}}={\displaystyle \frac{(1-r^2)}{r^2}}{m}^2$$

(8)

Now, since the two noise sources are disjoint, the total NSR $R$of the channel shown in Figure 2, Figure 3 is given by:

$$R=R_m+R_r$$

(9)

Therefore, $R$depends only on the two parameters $m$and $r$ of the regression line:

$$R={\left(m-1\right)}^2+{\displaystyle \frac{(1-r^2)}{r^2}}{m}^2$$

(10)

Finally, the signal−to−noise ratio (SNR) $\mathsf{\gamma }$ is given by:

$$\mathsf{\gamma }={\displaystyle \frac{1}{R}}={\displaystyle \frac{1}{{(m-1)}^2+\frac{1-r^2}{r^2}{m}^2}}$$

(11)

In decibel:

$$\mathsf{\Gamma }=10\times {\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\mathsf{\gamma }$$

(12)

Of course, we expect that no channel can yield $\left|r\right|=1$and $\left|m\right|=1$, therefore $\mathsf{\gamma }=\infty$. In empirical scatterplots, very likely, $\left|r\right|<1,\left|m\right|\ne 1$.

In conclusion, the slope $m$ measures the multiplicative “bias” of the dependent variable $y$ compared to the independent variable $x$ in the deterministic channel; the correlation coefficient $r$measures how “precise” the linear best fit is.

Finally, notice the more direct and insightful analysis that can be achieved by using the NSR instead of the more common SNR because, in Eq. (9), the single channel NSRs simply add together. This makes easy to study, for example, which addend determines $R$, and thus $\mathsf{\Gamma }$, while this is by far less easy with Eq. (11). Moreover, this choice leads also to a useful graphical representation of Eq. (10) that can guides analysis and design [11], as shown in Section 8.

In the next sections, we apply the theory of linear channel modeling to specific cases.

3. Connection of Single Linear Channels

We first study how the output variable $y_k$ of channel $k$ relates to the output variable $y_j$ of another similar channel $j$ for the same input $x$. This channel is termed “cross channel” and it is fundamental in studying language translation [35]. Secondly, we study how the output of a deterministic channel, modelled by Eq. (1), relates to the output of its stochastic version, Eq. (3).

3.1. Cross Channels

Let us consider a scatterplot $k$ and a scatterplot $j$ in which the independent variable $x$ and the dependent variable $y$ are linked by linear regression lines:

$${\mathit{y}}_{\mathit{k}}={\mathit{m}}_{\mathit{k}}{\mathit{x}}_{\mathit{k}}$$

(13)

$${\mathit{y}}_{\mathit{j}}={\mathit{m}}_{\mathit{j}}{\mathit{x}}_{\mathit{j}}$$

(14)

As discussed in Section 2, Eqs. (13)(14) do not give the full relationship between the two variables because they link only conditional average values, measured by the slopes ${\mathit{m}}_{\mathit{k}}$ and ${\mathit{m}}_{\mathit{j}}$ in the deterministic channels. According to Eq. (3), we can write more general linear relationships by considering the scattering of the data, always present in experiments, modelled by additive Gaussian zero–mean noise sources ${\mathit{n}}_{\mathit{k}}$ and ${\mathit{n}}_{\mathit{j}}$.

$${\mathit{y}}_{\mathit{k}}={\mathit{m}}_{\mathit{k}}{\mathit{x}}_{\mathit{k}}+{\mathit{n}}_{\mathit{k}}$$

(15)

$${\mathit{y}}_{\mathit{j}}={\mathit{m}}_{\mathit{j}}{\mathit{x}}_{\mathit{j}}+{\mathit{n}}_{\mathit{j}}$$

(16)

Now, we can develop a series of interesting investigations on these equations. By eliminating $\mathit{x}$ we can compare the dependent variable ${\mathit{y}}_{\mathit{j}}$ of Eq. (16) to the dependent variable ${\mathit{y}}_{\mathit{k}}$ of Eq. (15) for ${\mathit{x}}_{\mathit{k}}={\mathit{x}}_{\mathit{j}}=\mathit{x}$. In doing so, we can find the regression line and the correlation coefficient of the new scatterplot linking ${\mathit{y}}_{\mathit{j}}$ to ${\mathit{y}}_{\mathit{k}}$ without the availability of the scatterplot itself.

By eliminating $\mathit{x}$ between Eq.(15) and Eq. (16), we get:

$${\mathit{y}}_{\mathit{j}}={\displaystyle \frac{{\mathit{m}}_{\mathit{j}}}{{\mathit{m}}_{\mathit{k}}}}{\mathit{y}}_{\mathit{k}}-{\displaystyle \frac{{\mathit{m}}_{\mathit{j}}}{{\mathit{m}}_{\mathit{k}}}}{\mathit{n}}_{\mathit{k}}+{\mathit{n}}_{\mathit{j}}$$

(17)

Compared to the new independent variable ${\mathit{y}}_{\mathit{k}}$, the slope ${\mathit{m}}_{\mathit{k}\mathit{j}}$of the regression line is given by:

$${\mathit{m}}_{\mathit{k}\mathit{j}}={\mathit{m}}_{\mathit{j}}/{\mathit{m}}_{\mathit{k}}$$

(18)

Because the two Gaussian noise sources are independent and additive, the total noise is given by:

$${\mathit{n}}_{\mathit{k}\mathit{j}}=-{\displaystyle \frac{{\mathit{m}}_{\mathit{j}}}{{\mathit{m}}_{\mathit{k}}}}{\mathit{n}}_{\mathit{k}}+{\mathit{n}}_{\mathit{j}}={-\mathit{m}}_{\mathit{k}\mathit{j}}{\mathit{n}}_{\mathit{k}}+{\mathit{n}}_{\mathit{j}}$$

(19)

Figure 4 shows the flow chart describing the cross–channel.

Now, from Eq. (18), ${\mathit{R}}_{\mathbf{m}}$ of the new channel is:

$${\mathit{R}}_{\mathit{m}}={({\mathit{m}}_{\mathit{k}\mathit{j}}-1)}^2$$

(20)

The unknown correlation coefficient ${\mathit{r}}_{\mathit{k}\mathit{j}}$ between ${\mathit{y}}_{\mathit{j}}$ and ${\mathit{y}}_{\mathit{k}}$ is given by [35]:

$${\mathit{r}}_{\mathit{k}\mathit{j}}=\mathbf{c}\mathbf{o}\mathbf{s}\left|\mathbf{a}\mathbf{r}\mathbf{c}\mathbf{o}\mathbf{s}\left({\mathit{r}}_{\mathit{j}}\right)-\mathbf{a}\mathbf{r}\mathbf{c}\mathbf{o}\mathbf{s}\left({\mathit{r}}_{\mathit{k}}\right)\right|$$

(21)

Therefore, ${\mathit{R}}_{\mathbf{r}}$ of the new channel is:

$${\mathit{R}}_{\mathit{r}}={\displaystyle \frac{1-{\mathit{r}}_{\mathit{k}\mathit{j}}^2}{{\mathit{r}}_{\mathit{k}\mathit{j}}^2}}{\mathit{m}}_{\mathit{k}\mathit{j}}^2$$

(22)

In conclusion, in the new channel connecting $y_j$ to $y_k$ we can determine the slope and the correlation coefficient of the scatterplot between $y_j$ and $y_k$ for the same value of the independent variable $x$. Now, the availability of this scatterplot is experimentally very rare because it is unlikely to find values of $y_k$ and $y_j$ for exactly the same value of $x$, therefore cross channels can reveal relationships very difficult to discover experimentally.

In the next sections, we further developed the theory of linear channels, originally established in Reference [35] for cross channels.

3.2. Stochastic Versus Deterministic Channel

We compare a deterministic channel $k$with a stochastic channel $j$ derived from channel $k$ by adding noise. In other words, we start from the regression line given by Eq. (1) and then add noise $n$ due to the correlation coefficient $\left|r\right|\ne 1$. Therefore, from the theory of stochastic channels discussed in Section 3.1, we get:

$${\mathit{y}}_{\mathit{k}}={\mathit{m}}_{\mathit{k}}{\mathit{x}}_{\mathit{k}}$$

(23)

$${\mathit{y}}_{\mathit{j}}={\mathit{m}}_{\mathit{k}}{\mathit{x}}_{\mathit{k}}+{\mathit{n}}_{\mathit{k}}$$

(24)

$${\mathit{m}}_{\mathit{k}\mathit{j}}={\mathit{m}}_{\mathit{k}}/{\mathit{m}}_{\mathit{k}}=1$$

(25)

$${\mathit{R}}_{\mathit{m}}={({\mathit{m}}_{\mathit{k}\mathit{j}}-1)}^2=0$$

(26)

$${\mathit{r}}_{\mathit{k}\mathit{j}}=\mathbf{c}\mathbf{o}\mathbf{s}\left|\mathbf{a}\mathbf{r}\mathbf{c}\mathbf{o}\mathbf{s}\left({\mathit{r}}_{\mathit{j}}\right)-\mathbf{a}\mathbf{r}\mathbf{c}\mathbf{o}\mathbf{s}\left(1\right)\right|=\mathbf{c}\mathbf{o}\mathbf{s}\left|\mathbf{a}\mathbf{r}\mathbf{c}\mathbf{o}\mathbf{s}\left({\mathit{r}}_{\mathit{j}}\right)\right|={\mathit{r}}_{\mathit{j}}=\mathit{r}$$

(27)

$$\mathit{R}={\mathit{R}}_{\mathit{r}}={\displaystyle \frac{1-{\mathit{r}}^2}{{\mathit{r}}^2}}$$

(28)

In conclusion, in transforming a deterministic channel into a stochastic channel only the correlation noise is present, therefore the SNR is given by:

$$\mathsf{\gamma }={\displaystyle \frac{{\mathit{r}}^2}{1-{\mathit{r}}^2}}$$

(29)

Eq. (29) coincides with the ratio between the variance explained by the regression line (proportional to the coefficient of determination ${\mathit{r}}^2$) and the variance due to the scattering (correlation noise), proportional to $1-{\mathit{r}}^2$ [46].

So far, we have considered single channels. In the next section, we consider the series connection of single channels to determine the SNR of the overall channel.

4. Series Connection of Single Channels Affected by Correlation Noise

In this section, we consider a channel made of series of single channels. We consider this case because it can be found in many specialties, and because in Section 7 we apply it to specific linguistic channels.

Figure 5 shows the flow chart of three single channels in series$.$ These channels can be characterized as done in Section 3.2, i.e., only with the correlation noise, therefore, the overall channel is compared to the deterministic channel in which:

$$m=m_1{m}_2{m}_3$$

(30)

From Figure 5, it is evident that the output noise of a preceding channel produces additive noise at the output of the next channel in series. The purpose of this section is to calculate $R$ at the output of the series of channels.

Theorem. The NSR $R$ of $n$ linear channels in series, each characterized by the correlation noise–to–signal ratio $R_i$, is given by:

$$R={\sum }_{i=1}^n{R}_i$$

(30)

Proof. Let the three linear relationships of the isolated channels of Figure 5 (i.e., before connecting them in series) given by:

$$y=m_{1}x+n_1$$

(31)

$$z=m_{2}y+n_2$$

(32)

$$t=m_{3}z+n_3$$

(33)

Let $s_x^2$, $s_y^2$, $s_z^2$ the variances (power) of the variables, and let $N_1=N_{1r},N_2=N_{2r}$, $N_3=N_{3r}$ the variances (power) of the Gaussian zero–mean noise $n_1$, $n_2$, $n_3$, then the NSRs of the isolated channels are given by:

$$R_1={\displaystyle \frac{N_1}{s_y^2}}={\displaystyle \frac{N_1}{m_1^2{s}_x^2}}$$

(34)

$$R_2={\displaystyle \frac{N_2}{s_z^2}}={\displaystyle \frac{N_2}{m_2^2{s}_y^2}}$$

(35)

$$R_3={\displaystyle \frac{N_3}{s_t^2}}={\displaystyle \frac{N_3}{m_3^2{s}_z^2}}$$

(36)

When the first two blocks are connected in series, the input to the second block must include also the output noise of the first block, therefore from Eqs. (31)(33) we get the modified output variable $\breve{z}$:

$$\breve{z}=m_{2}y=m_2\left(m_{1}x+n_1\right)+n_2=m_2{m}_{1}x+m_2{n}_1+n_2$$

(37)

In eq. (37), $m_2{m}_{1}x$ is the output “signal” and $m_2{n}_1+n_2$ is the output noise, therefore, the NSR at the output of the second block is:

$$R={\displaystyle \frac{m_2^2{N}_1+N_2}{m_2^2{m}_1^2{s}_x^2}}={\displaystyle \frac{m_2^2{N}_1}{m_2^2{m}_1^2{s}_x^2}}+{\displaystyle \frac{N_2}{m_2^2{m}_1^2{s}_x^2}}={\displaystyle \frac{N_1}{m_1^2{s}_x^2}}+{\displaystyle \frac{N_2}{m_2^2{s}_x^2}}=R_1+R_2$$

(38)

Now, for three channels in series, it is sufficient to consider $R$ given by Eq. (38) as the input NSR to the third single channel to obtain the final NSR and prove Eq.(30):

$$R=R_1+R_2+R_3$$

(39)

Finally, notice that $R$ of Eq. (30) is proportional to the mean $<R_i>$:

$$R=n{\sum }_{i=1}^n{R}_i/n=n<R_i>$$

(40)

In other words, the series channel averages the single$R_i$.

In conclusion, Eqs. (30) – (40) allow studying channels made by the series of several single channels affected by correlation noise by simply adding their single NSRs.

In the next sections, we apply the theory to linguistic channels suitably defined, after exploring the database on the NT mentioned in Section 1.

5. Exploratory Data Analysis

In this second part, we explore the linear relationships between characters, words, interpunctions and sentences, according to the flow chart shown in Figure xx, of the New Testament books considered (Matthew, Mark, Luke, John, Acts, Epistle to the Romans, Apocalypse). This is the database of our experimental analysis and application of the theory of linear channels discussed in the previous sections.

Table 1 lists language of translation and language family, with total number of characters ($C)$, words ($W)$, sentences ($S$) and interpunctions ($I$).

Figure 6 shows the scatterplots in the original Greek texts between: (a) characters and words; (b) words and interpunctions; (c) interpunctions and sentences; (d) characters and sentences. Figure 7 shows these scatterplots in the English translation. Appendix B shows example of scatterplots in other languages. Table 2 reports slope $m$ and correlation coefficient $r$ of the indicated scatterplots (155 samples for each scatterplot) for each translation, namely the input parameters of our theory on communication channels. The differences between languages are due to the large “domestication” of the original Greek texts discussed in Reference [47].

The four scatterplots define fundamental linear channels and they are connected with important linguistic parameters, previously studied [34,35,36], namely:

a)

The number of characters per word, $C_P$, given by the ratio between characters (abscissa) and words (ordinate) in Figure 6(a).

b)

The number of words between two successive interpunctions, $I_P$ – called the word interval – given by the ratio between interpunctions (abscissa) and words (ordinate) in Figure 6(b).

c)

The number of word intervals in sentences, $M_F$, given by the ratio between sentences (abscissa) and interpunctions (interpunctions) in Figure 6(c).

Figure 6(d) shows the scatterplot between characters and sentences, which will be discussed in Section 7.

In the next section, we study the cannels corresponding to these scatterplots.

Figure 8 shows the probability distributions of the correlation coefficient $r$ and the coefficient of determination $r^2$, for the scatterplots: words versus characters (green line); interpunctions versus words (cyan); sentences versus interpunctions (magenta). The black line refers to the scatterplot sentences versus characters; the red line refers to the series channel considered in Section 7, which links characters to sentences.

On correlation coefficients – and consequently, on the coefficient of determination, which determines the SNR – we notice the following remarkable findings:

a)

In any language, the largest correlation coefficient is found in the scatterplot between characters and words. The communication digital codes invented by humans show remarkable strict relationships between digital symbols (characters) and their sequences (words) to indicate items of their experience, material or immaterial. Languages do not differ from each other very much being $r$ in the range $0.9753-0.9983$ (Armenian, Cebuano) and being, overall $r=0.9925\pm 0.0038$.

b)

The smallest correlation coefficient is found in the scatterplot between characters and sentences, overall $0.0140\pm 0.0027$. This relationship must be, of course, the most unpredictable and variable because the many digital symbols that make a sentence can create an extremely large number of combinations, each delivering a different concept.

c)

The correlation coefficient (and also the coefficient of determination $r^2)$decreases as characters combine to create words, as words combine to create word intervals and as word intervals combine to create sentences.

The path just mentioned in item c) describes an increasing creativity and variety of meaning, other than that of the deterministic channel.

The characters–to–words channel shows the smallest $r^2$, therefore, this channel is the nearest to be purely deterministic. It does not tend to be typical of a particular text/writer but more of a language because a writer has very little freedom in using words of very different length [34], if we exclude specialized words belonging to scientific and academic disciplines.

On the contrary, the channels words–to–interpunctions and the interpunctions–to–sentences are less deterministic, a writer can exercise his/her creativity of expression more freely, therefore these channels depend more on writer/text than on language. Finally, the big “jump” from characters to sentences gives the greatest freedom.

In conclusion, humans have invented codes whose sequences of symbols making words cannot variate very much for indicating single physical or mental objects of their experience. To communicate concepts, on the contrary, a large variability can be achieved by introducing interpunctions to form word intervals and word intervals to form sentences, the final depositary of human basic concepts.

Figure 9 shows the probability distributions of the slope $m$. The black line (only partially visible because it is superposed to the red line) refers to the scatterplot sentences versus characters; the red line refers to the series channel that connects sentences to characters, discussed in Section 7.

On slopes, we notice the following important findings:

a)

The slope of the scatterplot between interpunctions and sentences (magenta line) is the largest in any language– overall $0.3795\pm 0.0755$ – and determines the number of word intervals, $M_F,$contained in a sentence in its deterministic channel.

b)

The slope of the scatterplot between interpunctions and words (cyan line) determines the length of the word interval, $I_P$, in its deterministic channel.

c)

The slope of the scatterplot between words and characters (green line) determines the number of characters per word, $C_P$, in its deterministic channel. As discussed below, this channel is the most “universal” channel because, from language to language $C_P$ varies little, compared to other linguistic variables.

d)

The smallest slopes are in the scatterplots between characters and sentences, overall$0.0140\pm 0.0027$. For example, in English there are 519,043 characters and 6590 sentences (Table 1); now, according to Table 1, the deterministic channel predicts $0.0128\times \mathrm{519,043}\approx 6644$ sentences just $+0.8$% difference from the true value.

As reiterated above, the slopes describe deterministic channels. As discussed in Section 6, a deterministic channel is not “deterministic” for what concerns the number of concepts, because the same number of sentences can communicate different meanings by just changing words and interpunctions. What is “deterministic” is the size of the ensemble.

In the next section, we model single linguistic channels, i.e. channels not yet connected in series from the linear relationships shown above.

6. Single Linguistic Channels

In this section, we apply the theory developed in Section 3.2 to the scatterplots of Section 5, therefore, to the following single channels:

(a)

Characters–to–Words.

(b)

Words–to–Interpunctions.

(c)

Interpunctions–to–Sentences.

These single channels are modelled like in Figure 2, Figure 3, they are affected only by the regression noise. $\mathsf{\Gamma }$ is obtained from Eq. (29) and drawn in Figure 10. Table 3 reports mean and standard deviation of $\mathsf{\Gamma }$ in each channel.

From Figure 10 and Table 3, we notice the following interesting facts:

a)

Languages show different $\mathsf{\Gamma }$ due to the large degree of domestication of the original Greek texts [47].

b)

$\mathsf{\Gamma }$ decreases steadily in this order: characters–to–words, words–to–interpunctions, interpunctions–to–sentences. A decreasing $\mathsf{\Gamma }$ says, relatively, how much a channel is less deterministic.

c)

Words–to–interpunction and interpunctions–to–sentences have close values, therefore they show similar deterministic channels.

d)

Most languages have $\mathsf{\Gamma }$ greater than that in Greek. This agrees with the finding that in modern translations of the Greek texts domestication prevails over foreignization [47].

e)

Finally, we can consider $\mathsf{\Gamma }$ as a figure of merit of a linguistic channel being deterministic: the larger $\mathsf{\Gamma }$ is, the more the channel is deterministic.

Figure 11 shows histograms (37 samples) of $\mathsf{\Gamma }$ for each channel. The probability density function of $\mathsf{\Gamma }$ can be modelled with a Gaussian model (therefore, $\mathit{\gamma }$ is a lognormal stochastic variable) with mean and standard deviation reported in Table 3. Figure 12 shows the probability functions of $\mathsf{\Gamma }$ which show, again, differences and similarities of the channels.

In conclusion, the large $\mathsf{\Gamma }$ of the characters–to–words channel, in any language, indicates that the transformation of characters into words is the most deterministic.

In the next Section, we connect the single channels to obtain the series channels modelled in Figure 5 and study them according to the theory of Section 4.

7. Series Connection of Linguistic Channels Affected by Correlation Noise

Let us connect the three single channels to obtain the series channel shown in Figure 5 and apply the theory of Section 4. We first show the results concerning the theory of series channel, and then we compare the single channel characters–to–sentences to that obtained with the series of single channels.

Figure 13(a) shows the single NSRs and the series NSRs in linear units for each language; Figure 13(b) shows the corresponding $\mathsf{\Gamma }$(dB), partially already reported in Figure 10. We can notice that in the sum indicated in Eq.(30), the NSR of the characters–to–words channel is negligible compared to the other two NSRs. For example, in English (language no. 10)$\mathit{R}=0.0152+0.1059+0.1120=0.2331\approx 0.1059+0.1120=0.2179$, therefore $\mathit{R}\approx 0.22$ against $\mathit{R}\approx 0.23$. In general, ${\mathit{R}}_1\ll {\mathit{R}}_2,{\mathit{R}}_3$ so that the characters–to–words channel can be ignored, to a first approximation, because it is about $1/10$ of the other two addends in Eq. (30).

For the characters–to–words channel, Figure 14(a) shows the slope calculated from the scatterplot between characters and sentences (Table 2) and the slope given by Eq. (30). The agreement is excellent, in practice the two values coincide (correlation coefficient 0.9998). Figure 14(b) shows scatterplot between the correlation coefficient calculated from the scatterplot between characters and sentences (Table 2) and that calculated by solving Eq. (29) for $\mathit{r}$ after calculating $\mathit{\gamma }$ from Eq. (39). In this case, the two values are poorly correlated (correlation coefficient 0.3929). Finally, notice the difference between the probability of $\mathsf{\Gamma }$ calculated by solving Eq. (29) for $\mathit{r}$– red line in Figure 12 – and that calculated from the available scatterplots and regression line (Table 2), black line. The smoother red curve models more accurately the relationship between characters and sentences than the available scatterplot shown in Figure 6(d), because $\mathit{R}$ is proportional to the mean value of the single channels, see Eq. (40).

In conclusion, $\mathsf{\Gamma }$ calculated in a series channel linking two variables is more reliable than that calculated from a single channel/scatterplot between the two variables.

In the next section, we apply the theory of cross–channels of Section 3.1.

8. Cross Channels: Language Translations

In cross channels we study how the output variable $y_k$ of channel $k$ relates to the output variable $y_j$ of another similar channel $j$ for the same input $x$, therefore we apply the theory of Section 3.1. In this new channel, we can determine the slope and the correlation coefficient of the scatterplot between $y_k$ and $y_j$ for the same value of the independent variable $x$, therefore cross channels can reveal relationships more difficult to discover experimentally.

From the database of the NT texts and the scatterplots of Figure 6, we can study at least three cross channels:

a)

The words–to–words channel, by eliminating characters, therefore the number of words are compared for the same number of characters.

b)

The interpunctions–to–interpunctions channel, by eliminating words, therefore, the number of words intervals are compared for the same number of words.

c)

The sentences–to–sentences channel, by eliminating interpunctions, therefore, the number of sentences are compared for the same number of word intervals.

Now, since these channels connect one independent variable in one language to the same (dependent) variable in another language, they describe very important linguistic channels, namely translation channels and they can be studied from this particular perspective. Therefore, cross channels in alphabetical texts describe the mathematics/statistics of translation, as we first studied in Reference [35].

Figure 15 shows the slope $m_{kj}$ and the correlation coefficient $r_{kj}$ by assuming Greek as language $k$, namely the reference language, for the three cross channels We can notice the following:

a)

For most languages, $m_{kj}>1$ in any cross channel, therefore most modern languages tend to use more words – for the same number of characters –; more word intervals –for the same number of words – and more sentences – for the same number of word intervals – than Greek. In other words, the corresponding deterministic channel (the channel characterized a multiplicative slope) is significantly biased compared to the original Greek texts.

b)

The correlation coefficient $r_{kj}$ is always very near unity. Therefore the scattering of the data around the regression line is similar in all the three cross channels.

Figure 16 shows the findings assuming English as reference language. In this case, we consider the “translation” from English into the other languages [35]. Clear differences are noticeable:

a)

Words–to–words channel: for most languages $m_{kj}\le 1$. The multiplicative bias is small, language tend to use the same number of words of English. This was not the case for Greek. The correlation coefficient $r_{kj}$is practically unity for all languages. In other words, modern language tend to use the same number of words English, for the same number of characters, therefore domestication of the alleged translation of English to the other languages is moderate, compared to Greek or Latin (see languages 1 and 2 in Figure 16(a)).

b)

Interpunctions–to–interpunctions channel: the multiplicative bias $m_{kj}$is strong, as in is Greek, therefore, the deterministic cross channels are different from language to language. The correlation coefficient $r_{kj}$is more scattered than in Figure 15, and different from language to language. Curiously, in the channel English–to–Greek, $m_{kj}\approx 1$, no bias The correlation coefficient $r_{kj}$ is similar to that of the sentences–to–sentences channel.

c)

Sentences–to–sentences: $m_{kj}\le 1$ for most languages, $r_{kj}$ is similar to that of the interpunctions–to–interpunctions channel.

Since similar diagrams can be shown when other modern languages are considered as the independent language – not shown for brevity – we can conclude that the translation from Greek to modern languages show a high degree of domestication, due especially to the multiplicative bias, namely to the deterministic channels not to the stochastic part of the channel. In conclusion, the translation from a modern language into another modern language is mainly done through deterministic channels. Therefore, the SNR is mainly determined by $R_m$.

This conclusion is visually evident in the scatterplot between $X=\sqrt{R_m}$ and $Y=\sqrt{R_r}$ shown in Figure 17, where a constant value of $\mathsf{\Gamma }$ traces an arch of circle [34]. It is clear that $\sqrt{R_m}\gg \sqrt{R_r}$, in other words, in the three cross channels, $\mathsf{\Gamma }$ is dominated by $R_m$, in agreement with what shown in Figure 15, Figure 16.

Finally, Figure 18 shows mean value and standard deviation of $\mathsf{\Gamma }$ in the three channels, by assuming the language indicated in abscissa as independent language/translation.

Notice that, overall, the probability distribution of $\mathsf{\Gamma }$ can be modelled as Gaussian with mean value and standard deviation reported in Table 4 (for its calculation see Appendix C). Notice that cross channels have larger $\mathsf{\Gamma }$ than the series channels (Table 3), because “translation” between two modern languages use mostly deterministic channels.

Figure 19 shows, as example, the modelling of the words–to–words overall channel.

Now, we conjecture the characteristics of the three channels for an indistinct human being, by merging all values, as done with words in Figure 19.

Figure xx shows the Gaussian probability density functions and their probability distributions of the overall $\mathsf{\Gamma }$ in the three channels calculated with the values of Table 4. These distributions refer, therefore, to channels in which all languages merge in a single digital code. In other words, we might consider these probability distributions as “universal”, typical of humans using plain text.

From Figure 18, Figure 19 and Figure 20 and Table 4, it clearly emerges the following “universal” characteristics.

a)

The words–to–words channel is distinguished from the other two channels, with larger $\mathsf{\Gamma }$. This channel is the most deterministic.

b)

The interpunctions–to–interpunctions and sentences–to–sentences channels are very similar both in mean value and standard deviation of $\mathsf{\Gamma }$, therefore indicating a similar freedom in creating variations with respect to their deterministic channels.

9. Summary and Conclusions

How the human brain analyzes the parts of a sentence (parsing) and describes their syntactic roles is still a major question in cognitive neuroscience. In References [2,3,33], we proposed that a sentence is elaborated by the short–term memory with three independent processing units in series: (1) syllables and characters to make a word, (2) words and interpunctions to make a word interval; (3) word intervals to make a sentence.

This approach is simple but useful, because the multiple processing of the brain regarding speech/text is not yet fully understood but characters, words and interpunctions – these latter needed to distinguish word intervals and sentences – can be easily studied in any alphabetical language and epoch. Our conjecture, therefore, is that we can find clues on the performance of the mind, at a high cognitive level, by studying the most abstract human invention, namely the alphabetical texts.

The aim of the present paper was to further develop and complete the theory proposed in Reference [2,3,33], and then apply it to the flow of linguistic variables making a sentence, namely, the transformation of: (a) characters into words; (b) words into word intervals; (c) word intervals into sentences. Since the connection between these linguistic variables is described by regression lines, we have analyzed experimental scatterplots between the variables.

In the first part of the article, we have recalled and further developed the theory of linear channels, which models stochastic variables linearly connected. The theory is applicable to any field/specialty in which a linear relationship holds between two variables.

We have first studied how the output variable $y_k$ of channel $k$ relates to the output variable $y_j$ of another similar channel $j$ for the same input $x$. These channels can be termed as “cross channels” and are fundamental in studying language translation.

Secondly, we have studied how the output of a deterministic channel relates to the output of its noisy version. A deterministic channel is not “deterministic” for what concerns the number of concepts, because, for example, the same number of sentences can communicate different meanings by just changing words and interpunctions. What is “deterministic” is the size of the ensemble.

Then, we have studied a channel made of series of single channels and have established that its noise–to–signal ratio $\mathsf{\Gamma }$ is proportional to the average of the single channel noise–to–signal ratios.

In the second part of the article, we have explored, experimentally, the linear relationships between characters, words, interpunctions and sentences, in a large set of the New Testament books. We have considered the original Greek texts and their translation to Latin and to 35 modern languages because, in any language, they tell the same story, therefore it is meaningful to compare their translations. Moreover, they use common words, therefore, they can give some clues on how most humans communicate.

The characters–to–words channel is the nearest to be purely deterministic. It does not tend to be typical of a particular text/writer but more of a language because a writer has very little freedom in using words of very different length.

On the contrary, the channels words–to–interpunctions and the interpunctions–to–sentences are less deterministic, they depend more on writer/text than on language.

The signal–to–noise ratio $\mathsf{\Gamma }$ is as a figure of merit of the deterministic channel. The larger $\mathsf{\Gamma }$ is, the more the channel is deterministic.

In conclusion, humans have invented codes whose sequences of symbols making words cannot variate very much for indicating single physical or mental objects of their experience. On the contrary, to communicate concepts, a large variability is achieved by introducing interpunctions to make word intervals and word intervals to make sentences, the final depositary of human basic concepts. Future work should be devoted to non–alphabetical laguages.