Equivalent Processors Modelling the Short–Term Memory

1. A Short–Term Memory with Three Equivalent Processors?

Humans can communicate and extract meaning both from spoken and written language. Whereas the sensory processing pathways for listening and reading are distinct, listeners and readers appear to extract very similar information about the meaning of a narrative story heard or read, because the brain assimilates a written text like the corresponding spoken/heard text [1]. In the following, therefore, we consider the processing of reading a text – and also writing that text because a writer is also a reader of his/her own text – due to the same brain activity. In other words, the human brain represents semantic information in a modal form, independently of input modality.

How the human brain analyzes the parts of a sentence (parsing) and describes their syntactic roles is still a major question in cognitive neuroscience. In References [2,3], we proposed that a sentence is elaborated by the short–term memory (STM) with two independent processing units in series (processors), with similar size. The clues for conjecturing this input–output model emerged by considering many novels belonging to the Italian and English literatures. In Reference [3] we showed that there are no significant mathematical/statistical differences between the two literary corpora, according to the so–called surface deep–language parameters, suitably defined. In other words, the mathematical structure of alphabetical languages – digital codes created by the human mind – seems to be deeply rooted in humans, independently of the particular language used or historical epoch [4–10] and can give clues to model the input–output characteristics of a complex and partially inaccessible mental process still largely unknown.

The first processor was linked to the number of words between two contiguous interpunctions, variable indicated by$I_p$ – termed word interval (Appendix A lists the mathematical symbols used in the present article) – approximately ranging in Miller’s $7\pm 2$ law range [11–39]. The second processor was linked to the number $M_F$of$I_p$’s contained in a sentence, referred to as the extended short-term memory (STM), ranging approximately from 1 to 6. These two units can process a sentence containing approximately a number of words from $8.3$ to $61.2$, values that can be converted into time by assuming a reading speed. This conversion gives $2.6~19.5$ seconds for a fast–reading reader [21], and $5.3~30.1$ seconds for a reader of novels, values well supported by experiments [22–37].

The E–STM must not be confused with the intermediate memory [38,39]. It is not modelled by studying neuronal activity – recalled below in Section 2 –  but only by studying surface aspects of human communication, such as words and interpunctions, whose effects writers and readers experience since the invention of writing. In other words, the processing model proposed in References [2,3] describes the “input–output” characteristics of the STM by studying the digital codes that humans have invented to communicate.

The modeling of the STM processing by two units in series has never been considered in the literature before References [2,3]. Now, the literature on the STM and its various aspects is very large and multidisciplinary, but nobody – as far as we know – has never considered the connections we found and discussed in References [2–10]. Moreover, a sentence conveys meaning, therefore the theory we are further developing in the present article might be one of the necessary starting points to arrive at the Information Theory that will finally include meaning.

Today, many scholars are trying to arrive at a “semantic communication” theory or “semantic information” theory, but the results are still, in our opinion, in their infancy [40–48]. These theories, as those concerning the STM, have not considered the main “ingredients” of our theory, namely $I_P$ and $M_F$, parameters that anybody can easily calculate and understand in any alphabetical language, as a starting point for including meaning, still a very open issue.

The aim of the present article is to further develop the model on input–output characteristics of the STM by including another processor, with a small capacity (from 1 to about $3~4$ cells), set in series before the two processors already mentioned. This first processor memorizes syllables therefore its output is a word.

In other terms, in the present article we propose what we think is a more complete input–output model of the STM made with three processors in series, which independently process: (1) syllables to make a word, (2) words and interpunctions to make a word interval; (3) word intervals to make a sentence.

Figure 1 sketches the signal flow–chart of the proposed three processors. Syllables $S_1$, $S_2$,…$S_k$ are stored in the first processor – from 1 to about $3~4$ items, as shown in the following – until a space (a pause in speaking or reading) or an interpunction is introduced (vertical arrow) to fix the length of the word. Words $P_1$, $P_2$,…$P_j$ are stored in the second processor – approximately in Miller’s range [11] – until an interpunction or a pause (vertical arrow) is introduced to fix the length of $I_P$. The word interval $I_P$ is then stored in the third processor – from about 1 to 6 items – until the sentence ends with a full stop (or a pause), a question mark or an exclamation mark (vertical arrow). The process is then repeated for the next sentence.

Since the second processor (modelled by $I_P$) and the third processor (modelled by $M_F$) are discussed in detail in References [2,3], the purpose of the present article is mainly to study and define the statistical characteristics of the newly added first processor and confirm the presence of the second and third processors.

However, since syllables are fundamental to conjecture and dimension the capacity of the new processor, first Section 2 summarizes what is mostly known about the brain processing of syllables according to cognitive neuroscience.

The article will then proceed as follows. Section 3 recalls and defines the deep–language parameter; Section 4 reports statistics of syllables and their connection with characters and words; Section 5 shows the statistical independence of $S_P$, $I_P$ and $M_F$; Section 6 defines the details of universal input–output STM model drawn in Figure 1; Section 7 examine the human difficulty in reading texts, measured by a universal readability formula and Section 8 summarizes the main conclusion of the article and proposes new investigations. Appendixes A–F report useful material and further data.

2. Syllables Brain Processing

Humans can produce a remarkably wide array of word sounds to convey specific meanings. To produce fluent speech, however, a structured succession of processes is involved in planning the arrangement and structure of phonemes in individual words [49–51]. These processes are thought to occur rapidly during natural speech in the language network involved in word planning [52–56] and sentence construction [57–59].

Scholars of cognitive neuroscience try to understand speech processing units in the brain through non–invasive means. They have reached the conclusion that speech processing works on multiple hierarchical levels, which involves the coexistence of various higher–order processing windows, all of which are important.

Speech information is conveyed at least in two specific timescales involving syllabic and phonemic information. Syllabic information extends over relatively long interval (~200 milliseconds), whereas phonemic information extends in shorter intervals (~50 milliseconds) [60,61]. These two timescales define the lower and upper boundaries of speech intelligibility [62]. To understand speech, the brain must extract and combine information at both timescales [61, 63, 64]. Whereas the neural dynamics underlying the extraction of acoustic information at the syllabic time scale has received extensive attention, the precise neural mechanisms underlying the parsing of information at these two timescales are not fully understood [65–70].

The investigation of the syllabic timescale and its functional role in speech perception is an active area of research. The main acoustic temporal modulation in the speech signal approximates the syllabic rate of the speech stream [69–72] and occurs in the so–called (neural) theta range $~3\mathrm{t}\mathrm{o}~10$ Hz, i.e. $3\mathrm{t}\mathrm{o}10$ syllables per second. During speech listening, neural activity tracks the speech rhythm flexibly. This tracking phenomenon seems to contribute to the segmentation of speech into linguistic units at the syllabic timescale [63, 73, 74]. The syllabic rate is the strongest linguistic determinant of speech comprehension and, as long as the syllabic speech rhythm falls approximately within the theta range, speech comprehension is possible [71,74].

Finally, syllables are placed at an intermediate stage between letter perception and whole–word recognition. In alphabetic languages, syllables can be defined as the phonological building blocks of words. This suggests that syllables are functional units of speech and text perception. Research on visual word recognition carried out in French, Spanish and English, show that syllables are relevant processing units [75–77].

In conclusion, the multiple processing of the brain regarding speech/texts is not yet fully understood but syllables, the basic processed constituents of words, can be easily observed and studied in any alphabetical language by reading, for example, literary texts of any language and epoch. This conclusion supports our simple input–output modelling of three processors in series, whose sizes are estimated from a large sample of alphabetical texts, as discussed in the next section.

3. Data Base of Literary Texts

To obtain clues on the first processor modelling the transformation of syllables into words – and also the confirmation of the other two processors already studied, now referred to as the second and third processor – we consider a large selection of the New Testament (NT) books ‒ namely the Gospels according to Matthew, Mark, Luke, John, the Book of Acts, the Epistle to the Romans, the Book of Revelation (Apocalypse) – in the original Greek versions and in the translations into Latin and 35 modern languages, a sample size large enough to give reliable statistical results, already studied in References [5,8,78–80] to which the reader is referred for more details. Table 1 lists the languages of translation. The statistics reported are discussed in Section 3.

The rationale for studying NT texts is twofold.

First, these texts are of great importance for many scholars of multiple disciplines. Although the translations are never verbatim, they strictly respect the subdivision in chapters and verses of the Greek original texts – as they are fixed today, see Ref. [81] on how interpunctions where introduced in the Greek original scriptio continua – and can be studied therefore at least at these two different levels (chapters and verses), by comparing how a deep−language variable changes from translation to translation.

Notice that in the present article “translation” means also “language” because we deal only with one translation per language – it is curious to notice that there are tens of different translations of the NT in English or in Spanish [80] –, but notice that language plays only one of the roles in translation, being the addressed linguistic culture of the audience another one. A “real translation” − the one we always read − is never “ideal”, i.e. it never maintains all deep–language mathematical characteristics of the original text, because “domestication” of the foreign text (Greek in our case) prevails over foreignization [82,83].

Secondly, these translations address always general audiences, with no particular or specialized linguistic culture, therefore the terms used in any language are common words so that the findings and conclusion we can reach refer to general readers, not to those used to reading specialized literature, like essays or academic articles. Therefore, our conclusions will refer to an indistinct human reader

For our analysis, as done in Reference [78], we have chosen the chapter level because the amount of text is sufficiently large (a total of 155 chapters per language) to assess reliable statistics. Therefore, for each translation/language we have a database of $155\times 37=5735$ samples, sufficiently large to give reliable statistical results on deep–language parameters, which we briefly recall, or define, in the next section for reader’s benefit.

4. Exploratory Data Analysis of Syllables

In this section we explore the relationships between syllables, words and characters – for the relationships involving the other deep–language parameters, see Refs. [4,5,6,7,8,9,10,78] – for French, Italian, Portuguese, English and German, namely the languages for which we consider the full data banks (NT). Notice, however, that the conclusion drawn for these languages are valid also for the others, because similar results can be reported for Matthew only, not shown for brevity.

Figure 2 shows the scatterplot between words and syllables (Figure 2a), and between characters and syllables (Figure 2b) for English. We can see a very tight relationship, measured by a very high correlation coefficient, as Table 3 shows.

Figure 3 shows the scatterplot between $S_P$ and words (i.e., the ratio between ordinate and abscissa of the samples shown in Figure 2a), and between $C_S$ and characters (the ratio between ordinate and abscissa of the samples shown in Figure 2b). The spread around the mean value is very small. The same results are found for the other languages, as it can be seen in Appendix B. Table 3 reports slope and correlation coefficient for the indicated languages, which clearly indicate that the linear relationship between syllables and words, or between characters and syllables, is very tight.

In fact, notice that $C_P\approx a{S}_P$, with $a$ constant, and very large correlation coefficient, hence very small spread (see Table 3). For example, in English $a=3.275$and $r=0.9968$, therefore $r^2={0.9968}^2=0.9936$, i.e. $99.36$% of the variance of the characters is due to the linear relationship and only $0.64\%$ to scattering (“noise” compared to the deterministic linear relationship).

In other words, syllables are so tightly linked to characters so that if they are not available, the characters can be used instead to assess statistical properties. Figure 4 shows a modified first processor that considers now the relationship between syllables and characters: a syllable is coded with characters $C_1$, $C_2$,…$C_n$ – from 1 to about $3~4$ items, see Table 3, until a space or an interpunction is found (vertical line).

For example, the scatterplot of $S_P$ versus other linguistic variables must have approximately the same statistical properties of the scatterplot of $C_P$ versus these variables. Therefore, we can estimate correlation coefficients involving $S_P$also for the languages for which syllables count is not available (Table 1), by considering $C_P$.

It is useful to notice that since syllables and characters are extremely correlated, $S_P$ is rarely considered as an ingredient of readability formulae. The historical founder of studies on readability formulae – Rudolf Flesch (1911–1986) – he first did consider also syllables for measuring the difficulty of reading a text, but he ended up in replacing $S_P$ with $C_P$ in his famous readability formula, Flesch’s formula for English [84]. Also the universal readability formula proposed in [85] does use $C_P$, not $S_P$.

In the next section we model the deep–language variables to confirm the modelling of the second and third processors defined in References [2,3] – established by studying English and Italian Literatures – and for defining the first processor, by studying the NT texts in the languages listed in Table 1.

5. Statistical Independence of ${\mathbf{S}}_{\mathbf{P}}$

6. Universal Input–Output Model of STM

From the mean values and standard deviations (conditional values) concerning the languages listed in Table 1, we can calculate the unconditional log–normal probability distributions of the linguistic variables, as discussed in Section 6 and Appendix D. This exercise is useful to define the statistical characteristics of the “universal” human STM memory, according, of course, to the simple input–output model shown in Figures 1, 4.

Table 4 reports fundamental statistics of the deep–language parameters and Figure 8, Figure 9 and Figure 10 show their log–normal probability densities. Table 5 reports probability ranges corresponding to $\pm 1$, $\pm 2$, $\pm 3$standard deviations of the log–normal models (Table 4).

From Table 5, we expect 99.73 % of the samples to fall in the range: (a) $1.23~3.14$ syllables for the first processor, or equivalently$1.93~3.80$ characters, values that are well within the ranges given by neuroscience studies (Section 2); $3.22~11.63$ words in the second processor, a range that includes Miller’s range given by his $7\pm 2$ “law” [11]; $1.46~7.21$ word intervals in the third processor.

It is also interesting to notice that the length of sentence, $P_F$, is in the range $7.41~36.66$ words. In other terms, this is the number of words that humans mostly use to express a full concept. The number of concepts can be very large. For example, by considering permutations with no repetition (words are not repeated), we get the range $7!=5040$ to $36!=3.7199\times {10}^{41}$, fantastic numbers. Of course, they include sequences with no meaning for the languages listed in Table 1, but they might express meaning in a language still to invent.

Finally, is also interesting to notice the range of characters that make a word, $C_P$, namely $3.01~7.65$. Therefore, to name everything humans use a very limited number of alphabetical symbols, mostly from 3 to about 8, but the number of words that can be invented can be very large. With 26 letters, they range from ${26}^3=17576$ to ${26}^8=2.0883\times {10}^{11}$, numbers that include, of course, sequences of letters (words) never found in the actual languages. For example, in English there are approximately $1.6\times {10}^6$ different words (this number refers to words that are different at least for one character), therefore there is still a large room for new words, $~{10}^5$ times more. Similar numbers are found also for the other languages.

Now, another interesting issue can be studied. These humans, when they read a text – assuming they have the culture to understand what they read – what reading difficulty do they find? Next section addresses this issue by first recalling what readability formulae are used for and then applying a universal readability formula.

7. Human Difficulty in Reading Texts and Readability Formulae

First developed in the United States [90,91,92,93,94,95,96,97,98,99], a readability formula allows to design the best possible match between readers and texts. A readability formula is very attractive because it gives a quantitative judgement on the difficulty or easiness of reading a text. Every readability formula, however, gives only a partial measurement of reading difficulty because its score is mainly linked to characters, words and sentences length. It gives no clues as to the correct use of words, to the variety and richness of the literary expression, to its beauty or efficacy, to quality and clearness of ideas or give information on the correct use of grammar. The comprehension of a text – not to be confused with its readability, defined by mathematical formulae – is the result of many other factors, the most important being reader’s culture. In spite of these limits, readability formulae are very useful because – besides their obvious use – they give an insight on the STM memory of readers/writers, as we suggested in References [2,3,85].

We first recall the universal readability formula [85], and secondly we find some universal human features linked to the deep–language parameters discussed in Section 3.

8.1. The Universal Readability Formula Contains the Three STM Equivalent Processors

Many readability formulae have been studied for English [95], practically none for other languages; therefore, in [85] we proposed a universal readability formula – referred as the index $G_U$ – based on a calque of the readability formula used in Italian [100]– applicable to any alphabetical language, given by:

$$G_U=89–10k{C}_P+300/P_F–6\left(I_P–6\right)$$

(8)

$$k=<C_{P,ITA}>/<C_{P,Lan}>$$

(9)

In Eq.(8), $<C_{P,ITA}>$ is the mean value of $C_P$ found in Italian texts, $<C_{P,Lan}>$ is the corresponding mean value found in the language for which $G_U$ is calculated. In Appendix A of Ref.[9] we showed that if $G_U$ is calculated by introducing in Eq.(8) mean values, i.e.. ${<C}_P>$, ${<P}_F>$ and ${<I}_P>$, then its value is smaller than ${<G}_U>$ calculated form the samples.

Very important, and contrarily to all other readability formulae, $G_U$ is strictly connected with the three STM equivalent processors. It contains the second processor directly, trough $I_P$; it contains the first processor indirectly through $C_P$ – strictly linked to $S_P$ (Sections 4, 5) –; it contains the third processor trough $P_F$, which is very well correlated to $M_F$, as it was shown for Italian Literature (correlation coefficient $0.937$) and English Literature (correlation coefficient $0.914$), see Figure 3 of Ref.[3].

In the present article, ${<C}_{p,ITA}>=4.48$ (see Table 1); for ${<C}_{p,Lan}>$ we assume the unconditional mean value of a “universal reader” discussed in Section 7, hence ${<C}_{p,Lan}>=4.73$, therefore in Eq.(9) $k=0.947$.

By using Equations (8) and (9), the mean value ${<kC}_P>$ is forced to be equal to that found in Italian, $4.48$. The rationale for this choice is the following: $<C_P>$ is a parameter typical of a language (see Table 1) which, if not scaled, would bias $G_U$ without really quantifying the reading difficulty of readers who in their own language are used to read, on average, shorter or longer words than Italian readers do. The scaling, therefore, avoids changing $G_U$ only because a language has, on the average, words shorter or longer than Italian.

From the probability distributions of the deep–language parameters discussed in Section 7, we can theoretically calculate the probability distribution of $G_U$ by assuming the statistical independence of $C_P$, $I_P$ and $M_F$. Instead of doing complex analytical calculations to arrive at a mathematical formula, we have estimated it by doing a Monte Carlo simulation of independent log–normal samples of $C_P$, $I_P$, and $P_F$, according to Table 4.

Figure 11 shows the probability density function (blue line) obtained with many simulations (300,000). Since $G_U>0$ and specifically $G_U\gg 0,$ we can model the central part of its experimental density (blue line) as Gaussian, with the “experimental” mean value $<G_U>=63.24$ and $s_{G_U}=11.92$ (black line; from this model, the probability of $G_U<0$ is negligible).

8.2. Readability Index and Universal Schooling of Humans

An interesting exercise is to link $G_U$ to the number of school–years – i.e., the only quantitative information that can give clues on reader’s reading skill – necessary to assess that a text/author is more or less difficult to read than another text/author. We do so according to the Italian school system, assumed as a common reference, Figure 12 (redrawn from Ref. [100]).

This assumption does not mean, of course, that readers of any other language must attend school for the same number of years of the Italian readers, but it is only a way to do relative comparisons, otherwise difficult to do from the mere values of $G_U$. In other words, the number of school–years should be intended as the “equivalent” schooling and be used only as a common reference.

According to this chart, the combination of readability index and school–years that fall, for example, above the line termed “easy” corresponds to texts that the reader is likely to find “easy” or “very easy” to read. The same interpretation holds for the other cases.

Now, from the probability distribution shown in Figure 11 and the “decoding/comparison” chart shown in Figure 12, we can do the following exercise: let us estimate the probability that a universal reader finds a text – extracted from the body of the universal literature written in any alphabetical language – “very easy”, “easy”, “difficult” or “very difficult”.

Figure 13 shows the result of these exercise by reporting in ordinate the probability (%) that the readability index is greater than a certain value, $P\left(G_U\right)$– hence, this is the probability of finding texts easier to read than those of the given $G_U$ – and in abscissa the number of school–years.

As the number of school–years increase, readers can read more texts with the same label. For example, with only 5 years of school (elementary/primary school), the reader finds 8% of the texts “easy” to read and the 92% texts left, with a continuous transition, “difficult”, “very difficult” or “almost unintelligible”; with 8 school–years the percentage becomes 61% (39%) and with 13 school–years 98% (2%). In other words, with 13 school–years practically all texts should be easy to read.

In conclusion, the chart shown in Figure 13 can be used for two diverse aims. The first aim is to guide a writer to establish approximately reader’s school–years necessary to read the text with a “built–in” $G_U$; in this case, the horizontal (red) line $y=G_U$ can cross diverse areas of texts declared “very difficult”, “difficult”, “easy”, therefore indicating a spread of readers with diverse schooling who will experience diverse difficulty. The second aim is to guide a writer to write a text with a given $G_U$ suitable to the schooling of the intended readers (magenta line). Finally, notice that whatever the combination, it always implies considering the STM equivalent processors. In other words, every reader/writer must fall somewhere in the chart of Figure 13.

8. Final Remarks and Conclusion

In the present article we have further developed the model on the input–output characteristics of the STM by including a processor that memorizes syllables to produce a word. The final model is made of three equivalent processors in series, which independently process: (1) syllables to make a word, (2) words to make a word interval; (3) word intervals to make a sentence, schematically represented in Figure 1, 4.

This is a very simple but useful approach because the multiple processing of the brain regarding speech/texts is not yet fully understood but syllables, characters, words and interpunctions – these latter used to distinguish words and sentences – can be easily observed and studied in any alphabetical language. These are digital codes created by the human mind that can be fully analyzed by studying the literary texts written in any language and belonging to any historical epoch, as we have presently done with the translations of the New Testament.

We have considered these texts because they always address general audiences, with no particular or specialized linguistic culture, therefore the terms used in any language are common words. Therefore, the finds and conclusion we can draw refer to indistinct human readers, not to those used to reading specialized literature, like essays or academic articles.

The deep–language parameters – linked to syllables, characters, words and interpunctions – of indistinct human readers/writers can be specifically defined with probability distribution functions that provide useful ranges on the codes that humans can invent. Their application to a universal readability formula can provide also the distribution of readers and texts that these readers can read with a given “built–in” difficulty – measured by the universal readability index –, as a function of their schooling.

Future work should be done, very likely along these same research lines, on non–alphabetical languages like, for example, Chinese, Korean and Japanese.