A Comprehensive Survey of Cognitive Graphs: Techniques, Applications, Challenges

The realization of the third-generation artificial intelligence (AI) requires the evolution 1 from perceptual intelligence to cognitive intelligence, where knowledge graphs may not meet 2 the practical needs anymore. Based on the dual channel theory, cognitive graphs are established 3 and developed through coordinating the implicit extraction module and the explicit reasoning 4 module as well as integrating knowledge graphs, cognitive reasoning and logical expressions, 5 which have achieved successes in multi-hop question answering. It is desired for cognitive graphs 6 to be widely used in advanced AI applications such as large-scale knowledge representations 7 and intelligent responses, promoting the development of Al dramatically. This review discusses 8 cognitive graphs systematically and elaborately, including basic concepts, generations, theories 9 and technologies. Moreover, we try to predict the development of cognitive intelligence in the 10 short-term future and further enlighten more researches and studies. 11


14
The previous few decades have witnessed the dramatic development of artificial 15 intelligence (AI). Broadly speaking, there have been three major stages during the 16 evolution of AI [1], as can be seen in Figure 1. The first stage is computational intelligence, which is owing to the fast computing 18 and mass storage capacities of computers. With the maturity of technologies such as grid 19 computing, distributed storage and quantum storage, the computing power of machines 20 has far exceeded that of human beings and laid a solid foundation for the next stages. 21 The second stage is perceptual intelligence, which is the current stage of AI. Perceptual 22 of the deep learning system and knowledge graphs. single-paragraph Q&A. However, in multi-hop questions, this method suffers from 107 "short-sighted retrieval". This means that the relevance between the text of last few 108 jumps and the question is very low, which is actually difficult to be directly retrieved, 109 resulting in a poor effect. In addition to retrieval problems, there are also two challenges 110 lying ahead, which are explainability and scalability. 111 Grounded on the dual process theory, an ideal cognitive graph can contribute to 112 all the three challenges significantly. It is an iterative framework to build the cognitive 113 graph step by step. As for the example of "Who is the director of the 2003 film which has 114 scenes in it filmed at the Quality Cafe in Los Angeles?", the overview procedure of the 115 cognitive graph is shown in Figure 5. 116 Models based on System 1 extract question-related entities from paragraphs to build 117 the cognitive graph and generate semantic vectors for each node. Then the relevant 118 paragraphs about new extracted entities are retrieved or just indexed from Wikipedia. 119 Meanwhile, models based on System 2 carry out reasoning based on semantic vectors 120 and compute clues to guide the extraction of System 1. After several iterations, System 2 121 selects a node as the predicted answer based on the reasoning results. Figure 6 shows 122 the detailed procedure of cognitive graph. 123 System 1 and System 2 can be established by various types of models. Since the 124 cognitive graph is initialized with entities extracted from questions, it is crucial to seek 125 out a powerful module to extract useful entities and generate semantic vectors for each 126 node. Recently, BERT [31] has been proved to be a successful language representation 127 model. Therefore, BERT is designed to serve as System 1. The input of System 1 consists 128 of three parts, including the question, the "clue" found in the previous paragraph and  the Wikipedia document about an entity x (for example, x is the movie "Old School"). 130 The goal of System 1 is to extract the "next hop entity name" and "answer candidate" in 131 the document. For example, as shown in Figure 5, from the "quality café" paragraph, 132 "old school" and "gone in 60 seconds" are extracted as the entity names of the next jump. 133 These extracted entities and answer candidates will be added to the cognitive graph 134 as nodes. In addition, System 1 will calculate the semantic vector of current entity x, 135 which will be used as the initial value of relational reasoning in System 2. Owing to 136 the inductive bias of graph structure, GNN has presented remarkable performances on

164
As shown in Figure 8, for an intractable semantic retrieval question without any 165 entity mentioned, the cognitive graph finally gets the answer "Marijus Adomaitis", It is well known that human cognition can successfully integrate the connectionist (brain-177 inspired) and symbolic (mind-inspired) paradigms, where the language is a compelling 178 case in point. To build an intelligent cognitive graph, it is urgent and indispensable to 179 develop a framework that can routinely acquire, represent, and manipulate knowledge, 180 simultaneously using the knowledge in the service of reasoning logically like humans.

181
Thus, as shown in Figure 9, three core technical supports are needed as prerequisites 182 for building cognitive graphs: 1) large-scale knowledge graphs to support intuitive 183 knowledge expansion; 2) reasoning mechanisms to conduct complex reasoning and 184 make analytic decisions; 3) large-scale pre-trained natural language generating models 185 to explain the inference process and express the reasoning results in a human-friendly 186 way.
187 Figure 9. The Cornerstone of Cognitive Graph. On the way to build cognitive graphs, three core technical supports are needed as prerequisites for building a cognitive graph: large-scale knowledge graphs to support intuitive knowledge expansion; reasoning mechanisms to conduct complex reasoning and make analytic decisions; large-scale pre-trained natural language generating models to explain the inference process and express the reasoning results in a human-friendly way.

188
The knowledge graph is regarded as an important cornerstone in the transformation 189 from perceptual intelligence to cognitive intelligence. The techniques applied in knowledge graph building mainly include knowledge 206 graph construction and knowledge graph representation.

207
The overview framework of knowledge graph construction is shown in Figure   208 10. As can be seen, the whole framework mainly consists of four parts, which are 209 data acquisition, information acquisition, knowledge fusion and knowledge processing 210 respectively. 211 Figure 10. Overview of knowledge graph construction framework. The whole framework mainly consists of four parts: data acquisition, information acquisition, knowledge fusion, and knowledge processing [36].

212
Data Acquisition is the cornerstone of knowledge graph, whose goal is to extract 213 structured data from unstructured or semi-structured data. to ensure the quality of the knowledge base after quality assessment. Knowledge 246 processing mainly consist of ontology construction and quality evaluation. The task of quality assessment of knowledge base is usually carried out together 260 with the entity alignment task. Its significance is that the credibility of knowledge can be 261 quantified, and the quality of knowledge can be effectively guaranteed by retaining the 262 higher reliability and abandoning the lower confidence.

264
The knowledge graph representation is also called knowledge graph embedding.

265
The key idea is to embed entities and relationships into a low-dimensional continuous

519
The method of reasoning based on distributed representation is to map entities 520 and relationships to low-dimensional space vectors, and to use semantic expressions for 521 reasoning. The advantage is that it fully utilizes the structural information in the knowl-522 edge graph, and the method is convenient to extend for large-scale knowledge graphs.

523
The disadvantage is that this kind of approach does not consider prior knowledge when to fuse multi-source information and multiple methods to further improve reasoning 553 performance will also become a major research direction in the future. Among them, the 554 fusion mode, that is, how to fuse, is a major difficulty.  To make a machine have cognitive intelligence, it not only requires the machine to 610 understand the data, process the data, and make decisions through cognitive reason-611 ing, but more importantly, the machine is supposed to have the ability to express the 612 reasoning results in a way that humans can understand. Therefore, how to make the 613 machine generate natural language in line with human understanding is a crucial aspect 614 of cognitive intelligence.

615
Along the way to cognitive intelligence, NLG plays a crucial role: it is responsible 616 for converts a cognitive system action into a human-understandable response. Therefore, 617 the response is supposed to be fluent, adequate. NLG has significant influence on users' 618 experience.

619
In this section, we comprehensively review the concept, key technologies, problems 620 and challenges and future research directions of NLG.  Markov chains are the earliest algorithms for language generating. It predicts the 641 next word in a sentence from the current word. For example, the model is trained in the 642 following two sentences: "I drink coffee in the morning" and "I eat sandwiches with tea".

643
The probability of "coffee" after "drink" is 100%, and the probability of "eat" and "drink"   In order to correctly predict the next word "Spanish", LSTM will pay more attention to 669 "span" in the previous sentence and use cell to memorize it. As the sequence is processed, 670 the cell stores the acquired information, which is used to predict the next word. When a 671 period is encountered, the forgetting gate will realize that the context of the sentence   can be helpful in understanding forward words. Early language models could be trained 737 from left to right or right-to-left, but the two could not be conducted at the same time.

738
Masked language model: Humans understand language with contexts in mind.

739
BERT cleverly utilized the idea of filling in the blanks, put forward the masked language 740 model to achieve a two-way transformer.        is insufficient. We need to extract confidence values to rank extracted rules. In this way,

961
It has been argued eloquently that to build a semantical, explainable and ultimately 962 trustworthy AI system, one needs to pay attention to a lot of aspects, such as integrated  In essence, the crucial innovation of cognitive graphs is to reduce the information 972 loss during the construction of the graphs, transfer the pressure of information process-973 ing to retrieval and natural language understanding algorithms, and retain the graph 974 structures for explainable relational reasoning.

975
In the future, it is necessary to focus on how to capture structural information and 976 learn rule knowledge at the same time, so as to improve the performance of cognitive 977 graph reasoning. In the big data era, large-scale, diverse forms, scattered distribution, dy-978 namic changes and low-quality data features bring new challenges to AI technologies. It 979 is necessary not only to learn the distribution representation of data from the perspective 980 of perception but also to interpret the semantics of data from the perspective of cognition.

981
The research and development of cognitive graphs that integrate core technologies such 982 as common sense knowledge graphs, cognitive reasoning and logical expression will 983 become the key to the breakthrough of the next generation of AI technologies. Given 984 the fast pace at which developments occur both in industry and academia, we feel it is 985 helpful to point to potential future directions. reasoning about what will happen surrounding you. Obviously, you cannot reliably 992 make plans.

993
As mentioned previously, deep learning is essentially based on a "big data for small 994 tasks" paradigm, which has a demand for massive amounts of data in a single narrow 995 task. Yixin Zhu [140] proposed "small data for big tasks" paradigm which is capable of