Discover AI knowledge to preserve Cultural Heritage

: Documenting cultural heritage by using artiﬁcial intelligence (AI) is crucial for preserving the 1 memory of the past and a key point for future knowledge. However, modern AI technologies make use of 2 statistical learners that lead to self-empiricist logic, which, unlike human minds, use learned non-symbolic 3 representations. Nevertheless, it seems that it is not the right way to progress in AI. If we want to rely on AI 4 for these tasks, it is essential to understand what lies behind these models. Among the ways to discover AI 5 there are the senses and the intellect. We could consider AI as an intelligence. Intelligence has an essence, 6 but we do not know whether it can be considered “something” or “someone” . Important issues in the 7 analysis of AI concern the structure of symbols -operations with which the intellectual solution is carried 8 out- and the search for strategic reference points, aspiring to create models with human-like intelligence. 9 For many years, humans, seeing language as innate, have carried out symbolic theories. Everything seems 10 to have skipped with the advent of Machine Learning. In this paper, after a long analysis of history, the 11 rule-based and the learning-based vision, we propose KERMIT[1] as a unit of investigation for a possible 12 meeting point between the different learning theories. Finally, we propose a new vision of knowledge in AI 13 models based on a combination of rules, learning and human knowledge.


16
For centuries, humanist scholars have contributed to the preservation of memory and 17 knowledge of the past [2,3] by analyzing symbolic documents with symbolic minds and, today, 18 they are seeking the help of artificial intelligence (AI) to speed up their analyses. Currently, 19 AI "minds" are dominated by non-symbolic, obscure statistical learners, which have cancelled 20 symbols in their controlling strategies. Humanist scholars can hardly control these statistical 21 models by using their knowledge expressed with symbols. 22 There are then two theories of how to manipulate knwoledge: the current empiric trend in 23 AI and the nativist theory of Chomsky [4][5][6]. The first ones, the empiricist models, do not pay 24 attention to form such as the difference between a noun or a verb because they do not have a  However, the idea that thoughts and sentences can be represented as vectors, flat and 33 meaningless, rather than as complex symbolic structures such as syntactic trees [4] makes the 34 Transformers-models a very good tester of the empiricist hypothesis. 35 Recently, many works have shown that knowledge acquired only from experience, as is 36 done among the Transformers-models, is superficial and unreliable [8,9] and they adhere to hooks 37 that emerge from the statistical nature of the model [10]. Nevertheless this is nothing new, since 38 had already been pointed out by Lake and Baroni in [11]. They argued that RNNs, which are 39 after all the parents of Transformers-models, generalize well discretely when differences between 40 and computation that have been poured into them. However, this is a clear sign that it is time to 48 dust off old approaches.

49
With the aim of studing these phenomena, we propose KERMITsystem [1] (Kernel-inspired 50 Encoder with Recursive Mechanism for Interpretable Trees) as a meeting point between the two 51 theories of knowledge. The purpose is to embed the long symbolic-syntactic history in the modern 52 Transformer architecture. In order to investigate the origin of knowledge and the achievable 53 performance, a long reflection on the role of knowledge in artificial intelligence is made during 54 this work.

55
The contributions of our paper are as follows: firstly, we will try to give "form" to the origins 56 and development of artificial intelligence. (sec.2). After investigating the long history, KER-57 MITsystem will be introduced as a unit of investigation (sec.3). Finally, limitations, weaknesses 58 and future developments will be highlighted in a wide-ranging discussion (sec.4).

60
In the last few years, there has been an increasing interest in the digitization of Cultural ideas are present in the human mind before birth. Therefore, "seeking and learning are in fact 77 nothing but recollection". Plato tried to explain the innactivity of knowledge in the human brain 78 by defining it as a "receptacle of all that comes to be" [15]. In this space, matter takes form and 79 symbols take on meaning thanks to the ideas and thoughts innately embedded in the human brain.  On the other side of the coin, Aristotle in the "Physics" revised the ideas of his mentor 92 Plato on the difference between "matter" and "form" [21]. Aristotle broke down the theories formal axiomatization of logical reasoning, which, added to a "tabula-rasa" knowledge, allows 99 the human being to think, and it can be seen as a physical system, a precursor idea of the ML. In [24]. The thought that unites them is something like a "tabula-rasa", for which our knowledge 105 comes from experience, provided through the senses, arguments reminiscent of the ML approach. representations that are independent of the task. Although these models achieve extraordinary 113 results, knowledge from experience alone seems to be not enough. The statistical learners are 114 very good students as long as we talk about superficial and "simple" tasks. However, when the 115 bar is raised and the task becomes more difficult, the inability of the statistical learners emerges 116 [10]. Moreover, it seems that the knowledge acquired by the Transformers-models is superficial 117 and unreliable [8,9].

118
To briefly summarise, there seems to be a huge gap between nativist and empiricist theory.

119
The gap between nativists and empiricists is also transmitted in the representation of the world 120 because while the former make strong use of symbols, the latter use dense vectors. In the field  Actually, we will find out that it is not entirely true, because the gap is not so huge. In fact, 125 it seems that human beings have an innate mechanism, ready to adapt, as we will see in sec.3.3 126 and consequently also representations are not so radical and firm as several representations can 127 coexist at the same time.

128
In order to test this hypotheses, we there is a middle way between the two streams of thought as mentioned in sec. 3.3.

215
In order to understand the ingredients in depth we can define the knowledge as a function: In function 1 there is a kind of background knowledge, called initialization knowledge 216 denoted by k, in contrast, e denoting knowledge derived from experience. Regarding r and t are 217 the representation of the innate part and the representation non innate part. Finally, m indicates 218 the mechanisms and the underlying algorithms used to arrive at K.

219
In a tabula rasa radical scenario, it would set k,i, t and r to zero, set a to some extremely 220 minimal value (e.g., an operation for adjusting weights relative to reinforcement signals), and seems that no variable is truly absolute zero, at the same time it is rare to find an architecture in 237 which all these variables coexist at the same time.   To answer these questions we propose an approach to discern how much innateness might 275 be required for AI. In order to find an answer we would be to create synthetic agents that do 276 difficult tasks, with some initial degree of innateness, achieve state of the art performance with 277 those tasks, and then iterate, reducing as much innateness as possible, ultimately converging on 278 some minimal amount of innate machinery. This strategy is close to the one proposed by Silver et 279 al.

280
In order to carry out this test, we need to build machines that are partially innate and 281 therefore, able to learn from experience, as machine learning paradigm, but at the same time act 282 by human hand. This architecture should be similar to Pat-in-the-Loop[50], a system that allows 283 humans to input rules into a neural network. The results must be interpretable, so humans must be 284 able to understand the decisions made by the system. In order to guarantee the explainability of   Finally, we concluded this work by proposing an innateness test. This test will enable 300 researchers to investigate the amount of innateness needed to achieve better performance on a task.

301
Furthermore, once you have quantified the necessary innateness you can start testing on KERMIT 302 and no longer just by using these two parts as encoders, but by using them with learnable weights.

303
The point of this paper is that, on the one hand, the issue is difficult to solve, and on the 304 other, the balance between the two approaches has become, across the whole field of machine 305 learning, seriously distorted. It is time for AI to take nativism more seriously.