Knowledge and Information in Epistemic Dynamics

1. Introduction

Artificial Neural Networks (ANN) are the epicenter of artificial intelligence. They were introduced in the seminal paper [26], where neurons contain discrete values, and synapses connecting neurons are equipped with functions transforming the values of afferent neurons into the values of efferent ones [28]. An important shift in perspective emerged after the seminal book by Donald Hebb [3,14], altering the functional perspective to a dual vision, where synaptic plasticity forms the basis of learning processes. Namely, in the following seminal works [16,17,20,36,37,38,43,44], neurons became places containing functions, and the synapses expressed arrows labeled with real values, called weights, denoting connection strengths. Under very general assumptions, neural networks become, in this form, universal approximators of all continuous functions between hyperspaces of real numbers [8,18]. Given that many behavioral competencies are representable in such functions, the role of neural networks is a crucial basis for any computational model of complex human abilities.

Synapse modification through training by examples was modeled by Machine Learning algorithms, which update weights to improve the behaviors of neural networks toward the acquisition of functions related to specific competencies. [16,17,20,27,29,36,37,43,44]. In this way, artificial neural networks became a model inverting Turing’s paradigm. Namely, Turing machines and equivalent formalisms are programmed to compute functions. On the contrary, ANNs equipped with machine learning methods are trained by pairs of values (input, output) from which they discover the function underlying the received pairs. The main idea in this field was the back-propagation method, rooted in the theory of function optimization, which provides the right weights to ensure a desired functional behavior of a given neural network. According to this algorithm, errors committed, with respect to the expected output, are sent in the opposite direction that the computation goes along the network. In such a way, a criterion of weight updating determines error decreases by reaching, at the end of the training phase, the right weights associated with the expected behavior [1,2,22,24].

Recently, ANN technology [30] realized Turing’s visionary hypothesis [42] of a talking machine through networks with dimensions of neurons and synapses comparable to those of the human brain, producing chatbots, machines generating conversations (in many natural languages) of a level comparable, in many aspects, to that of typical human conversation. From this perspective, the crucial role of language emerged as an essential aspect in the construction of cognitive processes. Based on this ability, neural networks trained by machine learning methods can solve many complex problems and have become the starting point of a new era in human technology.

The paper explores the new perspectives opened by LLM transformer models, but in a direction that is opposite to the standard one. What do transformer models tell us about cognition? Can we define general notions of cognitive systems that shed new light on intelligence, suggesting new models, and giving rigorous definitions of fundamental psychological concepts? These findings would promote reciprocal advancements in neuropsychology and artificial intelligence.

2. Probabilistic Versus Epistemic Information

In his famous booklet [40], Claude Shannon founded the mathematical theory of information. He proposed a measure based on probability. Given a stochastic variable X assuming values in a set A with probabilities ${\left\{p_a\right\}}_{a\in A}$, the information quantity of a value a is given by

$$-log\left(p_a\right)$$

where log is the logarithm in base 2. The intuitive idea is that the information content is inversely proportional to probability, and is additive for joint and independent occurrences of values (with a probability that is the product of the single probabilities). Then it is natural to use logarithms, which change products into sums:

$$Inf(a,b)=-log\left(p_{(a,b)}\right)=-log(p_a\times p_b)=-log\left(p_a\right)-log\left(p_b\right)=Inf\left(a\right)+Inf\left(b\right).$$

Just at the beginning of his paper, Shannon links the measure of information to the measure of uncertainty, because the two concepts are two faces of the same coin. Namely, the a priori probability corresponds to the information we gain when the value occurs, and, at the same time, to the loss of uncertainty when the value has occurred, so that ignorance decreases in a measure that coincides with the gained information.

In this probabilistic framework, Shannon develops many important informational concepts that play crucial roles in communication. However, other important aspects are completely absent in Shannon’s perspective. Namely, the primitive notion of information suggests that something informative for a person is completely meaningless for another, or even something very informative in a given context may become meaningless in a different situation. The usual intuition perceives a strong connection between the information and the knowledge of the receiver of that information, and this knowledge changes continually as new information is received. Can we find an epistemic value of information and quantify it in relation to a knowledge state of a given communication? The interaction and interdependence between information and knowledge require a different setting, where knowledge is an active component establishing the values of data, rather than a result of their accumulation.

First, it is important to distinguish between data and information. Data are events containing information, but the information is independent of them. Namely, data can be encoded into other data without changing their information content. Therefore, data are elements that carry the information they “contain". Of course, this is a circular description that does not clarify which elements are or are not data. In the probabilistic perspective, any stochastic variable is an information source, without any involvement of the agent receiving the data generated by the source. Conversely, the approach we will present inverts the usual perspective by assuming a knowledge entity whose inputs are considered data and whose elaboration provides new data, changing the knowledge entity. Encoding and decoding are the key concepts of an intrinsically functional perspective, where knowledge becomes the “internal" encoding of data within a system, which, after this encoding, acquires new competencies, that is, functions, in data processing and interaction with the external world.

3. Cognitive Systems

In a series of papers and books [32,33,34,35] (among many others), Jean Piaget and his school of developmental psychology investigated the conceptual organization of mathematical and physical intuitions in children and adolescents during the learning processes of educational curricula.

A schema emerged, which could be expressed by the dichotomy assimilation/accommodation. Comprehension is very similar to food ingestion. An external concept enters the student’s mind, but to be effective and productive, it needs the assimilation to the structure of the mind, in the state it is, and in a form that can be significant for the internal organization of the mind when the acquisition happens. In the case of food, it is reduced to minimal components by means of suitable enzymes, and these components provide the substances and energy necessary for organs, tissues, and cells to perform their functions. An analog mechanism acts in comprehension. Data of many possible formats (textual, visual, …) when entered into a cognitive system need to be “encoded" in forms coherent with the internal structure of the system. This internal encoding process is called assimilation. But assimilation requires a reciprocal mechanism, called accommodation, according to which the cognitive system receiving the new data reacts with suitable adjustments of its structure for better assimilation, internal coherence, and adequacy.

This approach will be the main inspiration for the following definition.

3.1. A General Definition

We define a cognitive system as a dynamic structure on an event-state space. This is a measurable space $(S,M,p)$ where S is a state space, for example, a Hilbert space of features, and M is a $\sigma$-algebra of subsets of S, and p is a probability measure [11].

In the space state S, an event is a pair $(A,t)$ where $A\in M$ is a subset of states that may occur at time t (the notion of event occurrence may be further specified in many possible ways, according to parameters of localization/distribution in the considered state space).

Intuitively, a cognitive system develops in time by interacting with its external world through several levels of representation of the reality it perceives. The information content of data coming from the outside is given by the costs that the internal representation requires during the assimilation and accommodation phases of this representation.

Representing objects, events, and states of affairs gains the possibility of discovering relations, making inferences, predictions, and decisions for fruitful interactions with the external world.

The essence of these abilities is the notion of mathematical function, going back to Euler’s definition in Introductio in Analysin Infinitorum (1748), and formalized in by Alonzo Church in 1930 years in terms of $\lambda$-expressions, that is, a map sending objects of a set to images of them, in such a way that operating on images gives an advantage with respect to operating directly on the objects. A map of a city is a simple way to understand it. On a map, we can choose the best way to connect two places by exploiting the quick overview of a reduced and synthetic representation of streets. When symbols or symbolic structures map objects, functions are named codes. Typically, a code is a function from strings over a finite alphabet of symbols, codewords, into a set of encoded objects. Coding theory is a widely investigated subject, central in Information Theory [7]. We do not enter into specific details about codes, but want to stress their critical role in cognition.

Any cognitive system assumes states. Therefore, at any time, it is placed at a point in a multidimensional space. Some of these states are knowledge states, that is, projections in subspaces, with representational roles.

Coding, possibly articulated in parts and levels, is the main ingredient of cognitive dynamics. Considering a cognitive system only in terms of states and codes is an “extensional" approach, because it prescinds on the physical realization of states (molecular, biomolecular, cellular, circuital, …). In the following, we adopt this viewpoint and will show that, although it is very general and abstract, it allows us to derive interesting properties.

The main intuition of the following definition of a cognitive system is based on internal states, external states, and knowledge states. Internal and external states have to be disjoint to express the self-non-self distinction.

Definition 1.

A cognitive system Γ has five components:

$$\Gamma =(In{t}_t,Ex{t}_t,Kn{l}_t,{\gamma }_{enc},inf)$$

$In{t}_t,Ex{t}_t,Kn{l}_t,$ are sets of states of an assumed event space, where t ranges in a set T denoting the lifetime of Γ (Γ is equipped by an internal clock, $t\in T$), and ${\gamma }_{enc}$ is an encoding function, we will abbreviate with γ;

$In{t}_t$ and $Ex{t}_t$ are non-empty sets of the internal and external states of Γ; $Kn{l}_t$ is the set of knowledge states, where $Kn{l}_t\subseteq In{t}_t$;

The self-non-self distinction holds:

$$In{t}_t\cap Ex{t}_t=\varnothing$$

(1)

γ i s the knowledge encoding function:

$$\gamma :Kn{l}_t\to In{t}_t\cup Ex{t}_t$$

(2)

such that, for any argument x, the non-idempotence condition holds:

$$\gamma \left(x\right)\ne x$$

(3)

$inf$ is a measure of information, the epistemic information of an input datum d. It is the sum of two components:

$$inf\left(d\right)=\mu \left(\gamma \right(d\left)\right)+\eta \left(\gamma \right(d\left)\right)$$

(4)

the μ component expresses the assimilation cost of d. The η component expresses the corresponding accommodation cost, which may require a reorganization of Γ to reach some consistency and adequacy features of its global structure. □

A state is accessible only if it is a $\gamma$-image of a knowledge state, and this request holds even for knowledge states. In other words, accessibility requires, in any case, a true representational level, through a knowledge state that is different from the state it represents.

In a cognitive system, $Kn{l}_t$ may increase and enrich over time; the costs of data acquisition from the system may change dramatically during its lifetime, depending on the construction of its knowledge. Being $Int$ and $Ext$ disjoint, also their counterimages are disjoint, hence no confusion arises between the knowledge encodings of internal and external states, in symbols:

$${\gamma }^{-1}\left(In{t}_t\right)\cap {\gamma }^{-1}\left(Ex{t}_t\right)=\varnothing$$

External states are based on input data expressing the information coming to $\Gamma$ from the outside. A part of the internal state is available to the outside and constitutes output data of $\Gamma$.

Example 1

(Shannon’s Information Source). Let us consider a basic cognitive system that takes in input a sequence al symbols over a finite alphabet and assimilates them internally by a knowledge encoding γ that constructs a table of binary encodings of symbols according to a Huffman code, where more probable symbols receive shorter representations [7]. The external states are input sequences to the system; the internal states are tables of pairs (symbol occurrence, binary representation), and the knowledge states are the binary codewords of input symbols with their occurrence multiplicity. □

If the assimilation cost is the length of the binary representation, and the accommodation cost is null, the epistemic information of symbol i results in $-log{p}_i$ with $p_i$ the frequency of i in the input sequence. This corresponds exactly to Shannon’s notion of information quantity.

By classical information theory, we know that the average information of symbols in the sequence corresponds to Shannon’s entropy, and no code can exist that has a shorter average information. This means that Shannon entropy is an inferior limit to the average Shannon’s information quantity, and Huffman codes are optimal codes for this information measure. In other words, Shannon’s information quantity is a measure of a basic cognitive system, where knowledge encoding minimizes the average length of codewords (and no assimilation costs are considered).

Shannon’s framework is simple yet powerful enough to develop numerous informational concepts, making it very useful in many contexts. However, it is too simple to express the complex interaction of cognitive character, where the knowledge of a cognitive agent and its history of development over time play crucial roles in evaluating the informational content of data.

Example 2

(Artifical Neural Network). An ANN is a cognitive system with the following components.

Internal States: the weights, biases, and activation states of all neurons at a given time.

External States. The external states are the input vectors provided to the network, representing observations from the environment.

Knowledge States: a subset of the internal states, the part of the system’s state that “encodes" past training data. In this case, the memory of knowledge is not simply a storage of copies, but a deeper structure of values that, in the context of network connections, synthesizes the experience, providing the competencies acquired by the ANN during its training history.

Knowledge Encoding: the function γ of the forward-pass computation of the network. It takes an input vector and maps it to a set of activated neurons, which represent its internal representation.

Assimilation Cost: the computational cost of the forward pass, or the cost of computing embedding vectors related to linguistic inputs in a LLM transformer model (in a chatbot case).

Accommodation Cost is where the model shines. It is the cost of the backpropagation algorithm and the subsequent weight updates. This is the computational effort required to adjust the knowledge states (weights) to minimize the error between its prediction and the new data point. A data point that is consistent with the network’s current knowledge will have a low error and, therefore, a low accommodation cost. A data point that is a "surprise" or an "outlier" will have a high error, triggering a large weight adjustment and thus incurring a high accommodation cost. □

ANN training is a continuous process of assimilation (forward-pass) and accommodation (backpropagation) that defines the information content of each training sample. The dual-cost model assimilation-accommodation captures the full complexity of learning as a continuous process, defining the information content of data.

3.2. Incompleteness of Epistemic Reflexivity

Reflexivity is a fundamental phenomenon of logic and computation. It occurs when there is a function f that maps a proper subset of A into the whole set A (expansive reflexivity), or when a set A is mapped into a proper subset (contractive reflexivity). If A is a finite set, then this is impossible; therefore, reflexivity is strictly related to infinity and an essential notion in the foundations of mathematics [9,25]. There are many forms of reflexivity. For example, recurrence is a case of reflexivity, where a new value of a function is defined in terms of previously defined values. Reflexive patterns constitute the logical schema of many paradoxes (Liar paradox, Russel paradox, Richard’s paradox …) from which fundamental logical theories stemmed [39].

A reflexion over a set A is a function f from a proper subset B of A to A. The reflexion is complete if it is surjective, that is, if all elements of A are images of elements of B; otherwise, it is incomplete. A set A is reflexive, or Dedekind infinite, when a 1-to-1 function exists between a proper subset B of A and A. Any infinite set is reflexive [25], and no finite set can be reflexive. A cognitive system, even if it is physically finite in any instant of its life, is potentially infinite when its states are defined on values that range over infinite sets.

According to the definition of a cognitive system $\Gamma$, we know that $Kn{l}_t\subseteq In{t}_t$. Now we show that the encoding $\gamma$ does not completely cover the internal states, that is, the set of $\gamma$-images is strictly included in the internal states. There are internal states that are not encoded by any knowledge state: these states are inaccessible (to the knowledge of $\Gamma$). This result resembles typical results in mathematical logic, and this emphasizes its general relevance. The relationship with classical results of mathematical logic will be discussed below.

Theorem 1.

In any cognitive system Γ, the knowledge encoding γ is an incomplete reflexion:

$$\gamma \left(Kn{l}_t\right)\subset In{t}_t.$$

Proof. Let $E={\gamma }^{-1}\left(Ext\right)$ and $I={\gamma }^{-1}\left(Int\right)$ (time parameter t is omitted, because it does not impact the following reasoning). Of course, sets E and I are subsets of $Knl$, and $Knl=I\cup E$, $I\cap E=\varnothing$. Let us consider the biggest subset A of I such that $\gamma \left(A\right)=A$:

$$A=\{s\in I|\exists q\in A:\gamma \left(q\right)=s\}.$$

The states in A are “autoreferential", because they are $\gamma$-images of states in A. Let U be the set of knowledge states that are neither in E nor in A. Then, $A\cap E\cap U=\varnothing$, and:

$$Knl=A\cup U\cup E.$$

If any state s of U is a $\gamma$-image of some state $q\in U$, then, by the definition of A, the inclusion $U\subseteq A$ holds, but this is impossible because A and U are disjoint sets. Therefore, if $U\ne \varnothing$, some of its states are not $\gamma$-images of (other states of) U. These states can neither be $\gamma$-images of A nor $\gamma$-images of E, because the $\gamma$-images of A are in A and the $\gamma$-images of E are external states; therefore they are inaccessible. If $U=\varnothing$, then all the internal states in $Int-Knl$ would be inaccessible, because, again, these states can neither be $\gamma$-images of A nor $\gamma$-images of E ($\gamma$-images of A are in A and the $\gamma$-images of E are external states). If also $Int-Knl=\varnothing$, for the same reason, all the states in E are inaccessible. Therefore, in all possible cases, surely in $\Gamma$ there are internal states that are inaccessible.

□

It is surprising that from very general requirements on cognitive systems, it follows the existence of inaccessible internal states. In other words, we can conclude that a cognitive system has to include an internal part of “ignorance" that is necessary for keeping a coherent distinction of the self-non-self distinction.

The incompleteness theorems of mathematical logic are at the origin of computer science. When mathematicians defined logical calculi of predicate logic that could deduce all the logical consequences of some given axioms [6,15], they also soon discovered that, in predicate logic, for axiomatic theories reaching some logical complexity, there exist propositions p such that either p or $\neg p$ cannot be deduced from the axioms (first Gödel’s incompleteness theorem [6]).

Kurt Gödel [12] invented the arithmetization of syntax for encoding propositions of Peano’s arithmetic by numbers, within the axiomatic system $PA$ formulated in predicate logic. His celebrated theorem asserts that any theory including Peano Arithmetic is logically incomplete because, under any interpretation of the theory, there are true propositions that cannot be deduced as theorems of the theory. In $PA$ for any proposition p there is a number $\left[p\right]$ encoding p. Gödel determined a formula $D\left(x\right)$ of $PA$ such that $D\left(\right[p\left]\right)$ is true in $PA$ if and only if p is deducible in PA. This is a case of reflexivity that we could say autorefentiality, where numbers encode propositions on numbers. The original proof by Gödel was based on one of the most interesting cases of reflexivity. It was given by constructing in $PA$ a formula $\mathbb{A}$ expressing, via the D predicate, its undeducibility in $PA$ (a transformation of the Liar Paradox sentence, going back to Epimenides, VI Century A.C.). Let ⊢ be a symbol expressing the deducibility and $⊬$ the undeducibility. Let us assume that $PA$ is coherent, that is, for no formula $\varphi$ of $PA$, both conditions $PA⊢\varphi$ and $PA⊢\neg \varphi$ can hold.

If $\mathbb{A}$ were deducible in $PA$, then $PA⊢\mathbb{A}$, that is, $\mathbb{A}$ would be true, so $\mathbb{A}$ would not be deducible, whence $PA⊬\mathbb{A}$.

If $\neg \mathbb{A}$ were deducible, then $PA⊢\neg \mathbb{A}$, then $\neg \mathbb{A}$ would be true, so it is not true that $\mathbb{A}$ is undeducible, therefore $PA⊢\mathbb{A}$, which implies $PA⊬\neg \mathbb{A}$. In conclusion:

$$PA⊢\mathbb{A}\Rightarrow PA⊬\mathbb{A}$$

and

$$PA⊢\neg \mathbb{A}\Rightarrow PA⊬\neg \mathbb{A}.$$

This means that both $\mathbb{A}$ and $\neg \mathbb{A}$ cannot be deducible, but one of them has to be surely true, then there exists a true formula of $PA$ that is not deducible in $PA$.

The construction of $\mathbb{A}$ is very technical, so we prefer to present another proof that relates the logical incompleteness to Turing’s notion of computability.

Theorem 2

(Gödel’s logical incompleteness). In Peano Arithmetic $PA$, there are true arithmetic propositions that cannot be deduced within $PA$.

Proof. Any recursively enumerable set A of numbers [28] can be represented in PA (by expressing the computation of the Turing machine generating it). This means that there exists a formula $P_A$ such that $P_A\left(n\right)$ is deduced if and only if $n\in A$. Moreover, Alan Turing [41] defined (using a diagonal construction due to Cantor) a recursively enumerable set K of numbers that is not decidable, because no Turing machine exists that can always establish, in general, in a finite number of steps, whether a given element belongs to it. Therefore, there exists a number a such that it is impossible to decide in a finite number of computation steps whether $a\in K$ or $a\notin K$. This means that both $D\left(\left[P_K\left(a\right)\right]\right)$ and $\neg D\left(\left[P_K\left(a\right)\right]\right)$ are not deducible in $PA$, otherwise simulating $PA$ deductions with a Turing machine computations (easily realizable with Gödel arithmetization) we could decide the membership of a to K. Of course, at least one of the two formulae above is surely true in $PA$. Therefore, there are true propositions of $PA$ that cannot be deduced in $PA$. □

We now provide a proof of the theorem above using the definition of a cognitive system, which demonstrates the power of such a general definition. A theory is logically coherent if, for no formula p, both p and $\neg p$ are deduced in the theory. An extension of a theory $\mathbb{T}$ is a theory ${\mathbb{T}}^{\prime }$ such that all the formulae deducible in $\mathbb{T}$ are also deducible in ${\mathbb{T}}^{\prime }$. The theory ${\mathbb{T}}^{\prime }$ is a proper extension, since in ${\mathbb{T}}^{\prime }$ can be deduced formulae that are not deduced in $\mathbb{T}$. A coherent theory is logically complete if all its true formulae are deducible in it, that is, if for any formula p of the theory either p or $\neg p$ is deducible in the theory. A theory that is not logically complete is logically incomplete.

Theorem 3

(Logical incompleteness via Cognitive Systems). True formulae of any logically coherent theory that is an extension of $PA$ cannot coincide with the formulas deducible in the theory.

Proof. The proof follows from the possibility of defining $PA$ as a cognitive system. True formulae of $PA$ are its internal states, while false formulae of $PA$ are its external states. Knowledge states of $PA$ are the deduction formulae of $PA$, expressed by the deduction predicate D. The formula $D\left(\right[p\left]\right)$ encodes the true formula p, while $\neg D\left(\right[p\left]\right)$ encodes the false formula p.

Even if they are not relevant in the proof, we mention that in $PA$ as a cognitive system, the time could be that of the theory extension steps (see later on); the assimilation cost could be some measure of the complexity of deductions; and the accommodation cost could be that of reorganizing the theory (adding new axioms) to prove a true proposition that cannot be deduced.

From the theorem of epistemic incompleteness, we can deduce that knowledge states cannot encode all the internal states. In $PA$ as a cognitive system, this implies that the true formulae cannot coincide with the deducible formulae of $PA$, because internal states of $PA$ do not coincide with those represented by knowledge states of $PA$. □

Another proof of Gödel’s incompleteness follows from the deduction predicate D and the existence of extensions of $PA$. The following argument is interesting in cognitive terms, because it shows the positive role of incompleteness as a guarantee of enrichment.

Theorem 4

(Logical incompleteness via Theory Extensions). The logical incompleteness of theory $PA$ follows from the fact that it can be coherently extended.

Proof. Let us assume that $PA$ is logically complete, then its true formulas coincide with the deducible formulae of $PA$.

Now, let q be a formula that cannot be deduced from the Peano axioms, then $\neg D\left(\right[q\left]\right)$ is true, so it is deducible in $PA$. If we extend $PA$ by adding q to it, the formula $D\left(\right[q\left]\right)$ is true in the new theory and deducible in it, but the theory becomes incoherent. This means that if $PA$ were complete, it cannot be extended to any coherent theory. On the other hand, we know that if $PA$ is coherent, it can be extended to another coherent theory (including, for example, negative numbers). This implies again that true and deducible formulae of $PA$ cannot coincide, that is, $PA$ is not complete. □

Figure 1 can be applied to a formal theory, in accordance with its interpretation in terms of a cognitive system. The internal circle represents deducibility formulas, the white circle true formulas, and the external border false propositions.

Figure 1. A diagram that illustrates the reflexivity of a cognitive system. The internal circle is the knowledge states, the white oval is the set of internal states, while the border around it is the set of external states. The arrow is the encoding of knowledge states.

Figure 1. A diagram that illustrates the reflexivity of a cognitive system. The internal circle is the knowledge states, the white oval is the set of internal states, while the border around it is the set of external states. The arrow is the encoding of knowledge states.

Figure 2. The structure of internal states. The subset E contains the encoding of external states. The subset A contains the autoreferential states, and $I=A\cup U$ the encoding of internal states. Some of the subsets $Int-Knl$, $U,E$ are not empty and contain inaccessible states.

Figure 2. The structure of internal states. The subset E contains the encoding of external states. The subset A contains the autoreferential states, and $I=A\cup U$ the encoding of internal states. Some of the subsets $Int-Knl$, $U,E$ are not empty and contain inaccessible states.

3.3. Epistemic Levels

The given notion of a cognitive system is very general. In real cases of such systems, we must consider their internal structure, which is often articulated in a vast number of functional modules, organized at multiple levels [24]. The knowledge states and their encoding are surely a crucial aspect of the cognitive architecture. It is natural to assume a notion of knowledge level as the number of iterations of the encoding $\gamma$ given in our definition. Namely, a knowledge state s encodes an internal state $\gamma \left(s\right)$, but even this state can encode at a higher level of representation the state $\gamma \left(\gamma \right(s\left)\right)$. Higher knowledge levels enhance the connection and elaboration of data processing. Moreover, it is also reasonable that $\gamma$ is not a simple function, but it is the union of many encoding functions, which act in dependence on the parts and on the levels of the states to which they apply.

Encodings, as well as all the “objects" of knowledge, are functional modules, which can be encoded by real vectors that generalize embedding vectors, say feature vectors. In this way, they become data at further knowledge levels. Thus, cognition is intrinsically reflexive, in a way analogous to arithmetic and all fundamental mathematical theories.

Transformer LLM models demonstrate [4,5] that comprehension is expressed through the values dynamically generated during data processing, which essentially consist of embedding vectors in multidimensional Hilbert spaces. The further knowledge levels encode data into other semantic spaces, adding levels of internal comprehension to more complex meanings. When modules are encoded by feature vectors f.v., structures resembling formal theories, f.v.t., realized by suitable functional modules, provide operations and relations, generating other kinds of feature vectors too f.v.t.v, which again encode, at a superior level, functions of lower levels. In this, some details are lost, but features appear that express new relationships.

Therefore, meanings of a cognitive system are not static values memorized in some locations, but rather processes of localization in abstract spaces that geometrize data and their relations in corresponding "visualizations" within those spaces. Based on the first encoding of embedding vectors, learning/training mechanisms acquire specific competencies by reducing the errors between computed and expected functions that realize such competencies. When competencies are acquired, a cognitive system reaches the first step in constructing its structures.

In the superior levels, cognitive systems elaborate a representation of the external world and define finalities and strategies based on self-knowledge. Any level may include many sublevels; however, the three main levels, illustrated in Figure 3, can be denoted as: the operative level of competencies; the coordinative level, where the elements of the operative level and their relations are represented and integrated as different parts of a global competence; the directive level, where the system elaborates tasks and strategies, possibly organized in a hierarchy of finalities. The most superior level encodes a representation of itself, that is, a full consciousness. The intelligence of the system emerges as the ability to coordinate and integrate the functionalities of all cognitive levels, by finding the best representations, and by finalizing its behavior to the situations of its life. In the superior levels, a cognitive system needs to incorporate theories or similar structures.

The system of Figure 3 consists of functional modules. Arrows are channels sending feature vectors. represented by full squares. From the outlined perspective. Some feature vectors encode functional modules. In this way, a cognitive system can get knowledge of its own functionalities.

The encoding mechanisms and knowledge levels are not prefixed according to an external project elaborated by a designer; rather, they emerge following an internal principle that propagates and generalizes the transformer approach toward a multilevel perspective. Implementations of this generalization in artificial systems remain an open problem for future research. However, the key point toward multilevel transformers is the possibility of recognizing modules as functional units and encoding them through suitable feature vectors. Figure 4 shows a possible way to encode a functional level of an inferior level by an expression of weights and bias, which can be translated into a feature vector (the functional circuit of the module is expressed down, on the right, where circles are a sigmoid function).

Thus, the encoding of functional modules upward propagates the basic transformer mechanism.

In the middle level, embedding vectors give meaning to words and discourses. Therefore, feature vectors exploit embedding vector semantics to describe functional modules and introduce reflexive knowledge about themselves.

Logical operations underlying the logic of natural language are functions [23]; therefore, they are realized by functional modules. This means that chatbots, suitably extended, could, in principle, be able to “understand" the mechanisms on which their comprehension is based. In other words, the encoding of functional modules is a crucial step to increase the reflexivity of cognitive systems. Understanding how to realize the encoding of functional modules is a key research topic for both artificial intelligence and theoretical neurophysiology.

This scenario requires new levels of training. Training by example is too limited. Training by reasoning is the method that induces a cognitive system to organize its structures toward a multilevel transformer representation of its internal knowledge. It has to be based on natural language and developed through conversational activities. The semantic spaces that underlie the different knowledge levels add new dimensions according to a Chinese boxes mechanism where a single coordinate may refer to a point encapsulating the features of some hidden semantic space.

4. Conclusions

Our general definition of a cognitive system resonates with several lines of research in machine learning and cognitive science.

The idea that knowledge is encoded in distributed representations within neural networks has been a core tenet of connectionism [38]. Our framework provides a formal way to link this internal representation to information. Moreover, the “manifold hypothesis", according to which neural networks learn to represent data on dimensional manifolds [13], is related to our idea of knowledge as an internal representation. Embedding or feature vectors exist in these manifolds, and the acts of accommodation are the processes of deforming and refining them to fit better observed data.

The concept of “information gain" is often defined as the expected reduction in uncertainty that a data point can provide. Our framework reformulates this, where the “most informative" data are those with the highest accommodation cost, as they force the most significant and beneficial structural changes to a “knowledge gain". This aligns with approaches that seek to discover and learn from “surprising" inputs [31].

Our model offers a unifying framework to bridge the gap between abstract cognitive principles and the concrete mechanics of modern artificial intelligence. What is apparent from the discussion developed so far is that AI could be a unique occasion for suggesting general perspectives in cognition. On the other hand, these general viewpoints could suggest new models of AI based on the speculations emerging at a very general level.

Future research on cognitive systems intends to develop specific themes relevant to artificial intelligence and to neuropsychology. The most crucial theme is extending the transformer approach toward many levels of encoding.

Another important investigation topic is overcoming the present limitations of chatbots in dealing with complex logical deduction, in long implication chains involving abstract concepts [5,10,19]. In previous sections, it has been shown that artificial neural networks and formal theories are, in essence, different forms of cognitive systems. Then we could internalize formal theories into suitable neural networks to obtain more powerful cognitive systems, overcoming the present logical limitations [21,23].

Finally, we focused on cognition, but we know that it is only a part of a more complex psychological unit, where other components are present and interact, in the construction of a “person", with at least three different aspects: cognition, emotion, and volition. Emotion is the fuel of the “motor" providing the causes of the behavior; the volition provides the finalities, and the cognition provides the instruments and methods to realize the finalities. Mathematical analyses of the interaction and integration of these components can suggest new perspectives and new models for further comprehension and realization.