An Overview of Knowledge-Based System: History, Advances and Challenges in the Era of Large Language Models

1. Introduction

Knowledge is power, which was coined by Francis Bacon in 1597 [1]. To sufficiently utilize the tremendous knowledges accumulated by humans in thousands of years, the technology of knowledge-based system (KBS) was proposed as one typical form of artificial intelligence (AI), which has ever drawn much research interest and been applied to many areas. However, the fast-rising large language model (LLM) in recent several years has transformed the way of artificial general intelligence (AGI) to technological singularity which seems to go far from traditional KBS. Does KBS become outdate in the era of LLM? From the broad overview of 60-year history, we argue that KBS still has advantages over LLM in certain aspects thus combining KBS with LLM will be a promising research direction.

Although there exist many articles reviewing KBS and discussing the combination of KBS with LLM, this paper will contribute to the scientific body of knowledge in the following sense. 1) providing an overview of KBS in a broad sense while existing review articles generally cover specific narrow topics. 2) discussing the combination direction from the view of wide commercialization by comparing the return of cost (ROI) in two stages and six dimensions based on the historical experience, while existing review articles generally cover technical views [2,3,4,5,6,7,8]. The following questions were researched in this paper.

1)

What are the traditional KBS technologies? In the two main AI schools, symbolism and connectionism, KBS stands as the representative form of symbolism. Main technological concepts of KBS were introduced, including rule-based system, knowledge engineering, semantic network, semantic web, ontology engineering and knowledge graph. Each technology has its origin, breakthrough, application and limitations. From the first KBS of DENDRAL proposed in 1960s to the modern knowledge graphs in 2020s, the developing history of KBS was reviewed in a very broad sense both in time and area scope.

2)

How KBS was influenced by LLM? Before the emerge of LLM, the construction process of KBS had already been significantly improved by machine learning, indicating a trend of combination of symbolism and connectionism. The rapid emergence of LLM has influenced the field of traditional KBS from two sides. On the negative side, LLMs are partially replacing KBSs to a certain extent, leading to the doubt of whether KBS is still useful under LLM. On the other positive side, as a powerful tool, LLM has accelerated the development of KBSs especially the process of knowledge acquisition and updating.

3)

Why should we combine KBS with LLM? From the perspective of wide commercialization, the return of cost (ROI) is essential. The lifecycle of KBS can be separated into two stages, construction stage and service stage. While LLM has relatively shorter period and lower cost in the construction stage, traditional KBS costs lower in service stage and lives longer. Comparing their advantages and disadvantage in six dimensions, including reliability, controllability, adaptability, scalability, training efficiency and running efficiency, the way to high ROI must involve the combination of KBS with LLM. Several possible combination methods were further introduced.

This article was intended for both new researchers interested in the broad AI area and experienced researchers engaged in relatively narrow AI directions. First, more and more researchers start to learn AI and apply AI technologies into traditional areas. As new beginners, they need quickly understand the various concepts and technologies in AI which have been provided by this systematic overview from a historical perspective spanning 60 years. Second, many experienced researchers narrowly in symbolism are wondering the future of KBS under the strike of LLMs. This article reviewed both the negative influence and the positive contribution by KBS, indicating a potential direction for these researchers. Third, experienced researchers narrowly in connectionism can also benefit from this article, since the disadvantages of machine learning and LLMs may be overcame by the assistance of KBS. Lastly, researchers specially engaged in the combination of symbolism and connectionism, such as knowledge graphs and LLMs, will gain their cognition on the limitations of both sides thus avoid repeating past efforts.

2. Survey Methodology

The main search engine was chosen to be Google Scholar due to its comprehensive coverage of multi-source literature libraries. In comparison of different search engines for same search conditions, Google Scholar covers ~80% results from Web of Science while Web of Science only covers ~30% results from Google Scholar (see supplementary material S1). As shown in Table 1, a total of 19 searches were conducted with different keywords or conditions. The condition of whether a term is required in the title can be set in the advanced options of the search engine. All searches did not limit the time range or the article type, and the search results were sorted by relevance by default (actually in this mode, the search engine tends to consider content relevance, publication time, citation number and literature source in balance). All searches excluded patents and citations. After deduplication, a total of 3,108 search results (see supplementary material S2) were manually checked with some quick filtering conditions as shown in the flow chart Figure 1. Since this is a broad overview instead of a narrow survey, subjective selection is almost inevitable to extract main thread from massive information. As a supplement, the peer reviewers give important contribution to ensure the article as comprehensive and unbiased as possible. Finally, 162 papers were cited in the main text of this article, and the temporal and domain coverage was shown in Figure 2.

The first survey aimed to determine whether any existing review article was highly similar to this one. The keywords "overview knowledge based system" were specified, with the condition that they must appear in the title. The keyword "overview" was replaced with "review" or "survey," and the keyword "system" was also replaced with its plural "systems". Thus, there are six possible combinations of keywords for this search. Since the term KBS is also referred to as "expert system" in a narrow sense, an additional search using "overview / review / survey expert system(s)" was conducted to ensure full coverage. After deduplication, these two searches retrieved 117 and 469 results respectively. However, most of these review articles focused on specific application fields or were published more than 20 years ago, and none provided a broad overview of the entire KBS research area in the last 10 years.

The second survey aimed to cover all well-known technological concepts and significant advances in the field of KBS. Two searches were conducted using the keywords "knowledge based system(s)" and "expert system(s)" respectively, with the condition that the terms should appear in the title. Thousands of results were returned by the search engine, far exceeding the capacity for manual checking. Therefore, only the first 400 results were checked manually, as these articles should cover the most common technological concepts and the most important advances in this field. To further check related articles, additional search terms were used without requiring the terms to appear in the title, including: "knowledge based system(s)", "expert system(s)", "rule based system(s)", "knowledge engineering", "semantic network(s)", "semantic web", "ontology engineering", "ontology learning", "knowledge graph(s)", "machine learning", and "large language model(s)". Furthermore, some combinations of the above keywords were also searched to cover cross-field research, including: "machine learning" + "knowledge based system(s)", "machine learning" + "knowledge graph(s)", "large language model(s)" + "knowledge based system(s)", and "large language model(s)" + "knowledge graph(s)". Millions of results were returned by the search engine, but only the first several hundred results for each searched keyword were selected for manual checking.

The third survey aimed to trace the origins of each specific technological concept related to KBS. This was generally accomplished by manually examining widely cited review articles in each specific area and their references until the putative original article for each technological concept was identified.

3. Traditional KBS Technologies

3.1. KBS in AI History

Since as early as 1950, when the famous question “Can machines think?” proposed by Alan Turing [9], scientists and engineers have been trying to build AI systems to help people processing miscellaneous information and doing challenging work. In 1956, the term of AI was first officially proposed at a workshop at Dartmouth College. Afterwards, two major schools of AI, symbolism and connectionism, have competed with and learned from each other through periods of rise and fall [10,11], as illustrated in Figure 3. In the development of symbolism, KBS (also named as expert system in a narrow sense) stands as the representative research branch, which started from the proposal of the DENDRAL system in 1964 [12]. In the parallel way of connectionism, the first trainable perceptron machine was created in 1957 [13,14], and the artificial neural network (ANN) gradually became a basic model for machine learning—a concept that was proposed by an IBM employee in 1959 [15].

The research journey of KBS can be broadly categorized into several technical stages, each corresponding to some key concepts within this field. Initially, due to the difficulty of building a generalized AI system, a domain-wise system was considered much more realizable, thus leading to the first-generation KBS in the form of rule-based system during the 1960s [12,16]. Then the key thought of separating stored knowledge and problem-solving method (PSM) was extended to diverse domains, showing certain potential of scalability and interpretability [17]. During the subsequent 1970s and 1980s, knowledge engineering helped KBS evolve beyond its initial simplistic form through adopting modeling frameworks, specification languages, and reusable inference engines, similar to how software engineering scaling up through modularization [8]. Distinct from the isolated style of storing knowledge, another research thread of KBS originating from 1967 [18] was semantic network, which emphasized the semantic relations between words and concepts, particularly targeting common-sense knowledge [19]. After the rise of the Internet during the 1990s, the Semantic Web was proposed in 2001 to connect worldwide information and knowledge using a standardized machine-readable format [20]. To bridge the gap among various AI systems, ontology engineering, a concept that incorporated philosophy with engineering, emerged in the early 1990s to formalize knowledge representation [21]. Benefiting from developed methodologies and easy implementation, ontology rapidly became the most widely adopted framework for KBS. To reduce human labor for constructing large-scale KBSs, research focus shifted in the 2000s toward learning methods of automatically extracting knowledge from raw data [22]. In 2012, Google introduced the first knowledge graph, making a large-scale KBS accessible to the general public through integration into its search engine [23].

3.2. Rule-Based System

The first KBS, named DENDRAL (indicating a dendritic algorithm), was born in Stanford University to determine the topological structure of organic molecules. Professional chemical knowledge was transferred from human chemists to an artificial program in the form of precise rules, as shown in Figure 4. DENDRAL was proposed by Joshua Lederberg in a project report to NASA (National Aeronautics and Space Administration) in 1964 at first [12], and afterwards it underwent continuous research and reporting to NASA for several years [16,24]. By 1967, Georgia Sutherland released a computer program written in LISP, which first implemented the concept of DENDRAL [25]. The system comprised two parts: one was the computer representation of knowledge (what to know), and the other was the inference engine (how to do). Soon after the birth of DENDRAL, in 1970, Feigenbaum, Buchanan and Lederberg began to explore the generality and expertness of the problem-solving system [17], trying to find a variant of the “big switch” theory (some people thought there might be a pivotal technology capable of switching narrow AI to AGI). However, they soon found that the generality of problem-solving (breath of applicability) was inversely related to its power such as success rate, efficiency, and accuracy. Fortunately, they noted that the it was quite easy to transfer expertness between specialty areas. They further conjectured that the most significant factor for designing intelligent systems might be the capacity to learn specialties as needed.

In the following decade, the core concept of rule-based system was extensively explored and widely applied across various problem areas [26]. MYCIN, an AI system with 200 rules developed by Shortliffe, was designed to provide advice regarding infectious disease therapy selection, achieving a reliability rate of 63% [27]. PUFF, an expert system with 55 rules that reused the inference engine from MYCIN, was able to interpret lung function test data and became a working tool in the pulmonary physiology lab of a large hospital [28]. SU/X was designed to determine object information in physical space by inferring from continuous signal data over long periods of time [29]. AM, with 250 heuristic plausible rules, facilitated the development of new concepts in the specific area of elementary mathematics research [30]. MOLGEN, a collaborative project between computing science department and genetics department, was developed to design both synthesis and analysis experiments related to DNA structures [31].

Early deep thoughts and a variety of application experiments about rule-based system have demonstrated the superior efficiency, broad adaptability, and strong extensibility across different expert areas. This significant advancement in the field of AI was achieved through imitating intelligent behaviors rather than through philosophical discourse. Consequently, the AI field emerged as a new discipline within the category of science and technology instead of the category of psychology. However, researchers soon realized difficulties to bring the rule-form KBS into effective and practical applications. On one hand, there were limitations related to computer hardware and software, including the huge machine size and the lack of user-friendly interaction way at that time. On the other hand, it was not economical to invest extensive efforts in extracting and representing knowledge across numerous expert areas. From the current perspective, the former difficulty has been well overcome, while the latter difficulty persists to some extent. Nonetheless, rule-based system has maintained long-term vitality, and even in recent years, there are still existing applications and related research in certain fields such as financial risk control [32].

3.3. Knowledge Engineering

When scaling up to large and long-living commercial systems, the approach demonstrated in small academic prototypes proved inadequate to handle the multifaceted difficulties arising throughout the entire system lifecycle encompassing design, implementation and maintenance. For instance, an industrial client in the 1970s might want a practical KBS to optimize and integrate the operations of an entire factory. The first difficulty would be how to represent various knowledge types in different domains, such as building architecture, production equipment, manufacturing techniques, storage management, supply chains, customer orders and human resources. The second difficulty was the predominantly implicit and tacit nature of relevant knowledge, hindering its effective transfer from domain experts during the knowledge acquisition phase. The third difficulty was the limited generality of PSMs, as researchers have not developed any PSM capable of working across all domains, not even in a broad range of common domains. Additionally, non-functional requirements like efficiency and maintainability might also require task-specified or method-specified knowledge.

To mitigate the predicament in scaling complexity, the engineering discipline helped KBS evolve beyond its initial form of simplistic rule-based system, adopting a more systematic solution. In 1973, knowledge engineering was first proposed by Donald Michie [33] to explore methodologies for complex information processing. Over the subsequent two decades, knowledge engineering gradually transformed the construction process of KBS from a transfer-oriented paradigm to a systematic modeling-oriented paradigm [8]. Some modeling frameworks, such as CommonKADS (Figure 5) [34], MIKE [35], PROTÉGÉ-II [36], were developed to facilitate the construction process of various models. Specification languages, such as KARL [37] and DESIRE [38], were developed to formalize notations for describing KBSs, facilitating dynamic reasoning with a strong conceptual model and rich primitives. Meanwhile, refined from generic inference engines like forward or backward resolution, a substantial number of PSMs [39,40] were developed as libraries to enhance modularity and reusability for building new applications.

One notable technology, fuzzy system, was introduced to knowledge engineering to deal with the difficulty of vagueness and uncertainty in natural language [41]. For example, a piece of subjective knowledge may be "Tom is tall", in which "tall" refers to an ill-defined boundary rather than clear-cut. The technology concept of fuzzy system originated from the notion of fuzzy sets, which was proposed by Zadeh in 1965 [42]. In a fuzzy set, a membership function maps each element to a degree of membership, which is generally a number between 0 and 1. If the membership function is deterministic, the fuzzy system is type-I, while type-II means the membership function is also fuzzy. Two common inference methods, Mamdani and Sugeno [43], are suitable for human input and efficient computation, respectively. Together with fuzzy logic and reasoning [44,45], fuzzy systems found wide application in control systems, image processing, pattern recognition, as well as expert systems especially for medical diagnosis [46].

Interestingly, the situation of KBS development in the 1970s and 1980s was directly comparable to that of software engineering in the 1960s, referred to as the “software crisis” which was similarly caused by scaling predicament [47]. Software engineering demonstrated that reuse was a promising strategy to reduce development costs of large-scale software systems, meaning that modularization and modeling framework were essential. The successful experience in software engineering provided valuable heuristics for knowledge engineering [48]. Consequently, numerous large-scale and practical KBSs were successfully developed at that time, such as ASW/ASuW [49], ONCOCIN [50], KBTA [51], DIVA [52], ISIS [52], etc.

3.4. Semantic Network

Although KBS had demonstrated its capability across various application domains in the 1980s, it was still criticized for the brittleness in certain aspects, which could be attributed to the isolated style of representing and storing knowledge. The prominent intrinsic limitation of those systems lay in the very narrow expertise, in the sense that their performance fell off precipitously near their capability boundaries, particularly when compared with the more graceful decline exhibited by human experts [3]. Some researchers considered that a key strength of humans was the ability to learn new knowledge by effectively leveraging what they already knew through analogous relations [19]. Therefore, to improve system expandability, it was thought to be highly important to embed semantic relation information into the architecture of KBS.

Semantic network was proposed to represent conceptual knowledge using graphs with vertices connected by edges or arcs [6]. Figure 6 provides a simple example of a semantic network, in which the vertices and connecting edges respectively represent core words and their sequential relations extracted from five distinct definitions of the concept itself [6]. The first publication can be traced back to the paper about semantic memory written by Quillian in 1967 [18], which aimed to represent knowledge from human sentences in computer memory using links between word nodes. Sowa categorized semantic networks into six common types: definitional networks, assertional networks, implicational networks, executable networks, learning networks, and hybrid networks [53]. The data structures commonly used by semantic networks were frames consisting of a flexible collection of named slots or attributes [54]. Among those semantic networks, a famous project led by Lenat, named CYC [19], was ambitiously aimed at building a large KBS of real-world facts and methods. Although its ambition was not fully realized, the completion of CYC and the core methodology of semantic network have significantly facilitated the knowledge acquisition and plausible reasoning processes of numerous KBS applications [55,56].

3.5. Semantic Web

The 1990s witnessed the rise of the Internet and the beginning of the information explosion [57]. When it came into the 21st century, the World Wide Web (WWW) gained popularity and began transforming the information infrastructure. The link between semantic information was no longer restricted to a single computer but began to extend through local network or the Internet. In 2001, the concept of the Semantic Web was first proposed by Berners-Lee et al. [20] to help computers understand and automatically process web information by assigning well-defined meaning instead of using ill-structured hypertext. Standards organizations like the World Wide Web Consortium (W3C) directed major efforts toward building up a standard system for the Semantic Web [7], including the Resource Description Framework (RDF) for Universal Resource Identifiers (URI) or Internationalized Resource Identifiers (IRI), the RDF Schema (RDFS) as an extension of data-modelling vocabulary, the SPARQL Protocol and RDF Query Language (SPARQL, a recursive acronym) for queries across diverse data sources, the Web Ontology Language (OWL) for representing complex knowledge, the Rule Interchange Format (RIF) for exchanging rules among different systems, the Semantic Web Rule Language (SWRL) as a combination of OWL and Rule Markup Language (RuleML), etc. As illustrated in Figure 7, a series of standards and technologies constitute the foundational infrastructure for the Semantic Web.

Although the grand goal of the Semantic Web has not been fully achieved yet, some intermediate results have shown practical use and even industrial value, especially in the form of Linked Data, such as Wikidata [58]. Frustratingly, as most data on the Internet were concentrated in the hands of a few commercial giants like Google and Microsoft during the 2010s, it became increasingly difficult to realize an open and shared information network. Worse still, the mobile Internet, which is increasingly popular and replacing the traditional Internet, generally adopts the form of relatively isolated apps instead of interconnected webpages. Therefore, the Semantic Web may need to find its new way to encourage people to share and connect information in a universal and machine-readable form.

3.6. Ontology Engineering

As the relationship between knowledge pieces was critical for reasoning and reuse, a simple connection in a semantic network was far from adequate to represent the complete hidden information. Ontology, a term originally coined in the early seventeenth century within philosophy, was later incorporated into the engineering discipline and used to formally model the complicated relationships between conceptual entities [59,60]. Research on ontology in information engineering began in the early 1990s [61,62,63,64]. The first widely accepted definition of ontology was given by Gruber in 1993: “An ontology is an explicit specification of a conceptualization” [21]. To a certain extent, ontology engineering was born with the high expectations of solving the major problem of knowledge sharing and reuse among AI systems.

To bridge the gap between human natural languages and machine-understandable symbols, ontology engineering generally adopts formalized representation methods, such as Ontology-based Web Annotation (OWA) [65,66]. According to the mathematical representation form proposed by Genesereth and Nilsson [67], an extensional relational structure, which is a triple of (D, W, R), can be used to define an ontology system [59,68]. In this triple, D denotes a set called the universe of discourse, W denotes a set of possible worlds, and R denotes a set of conceptual relations on the domain space <D, W>. For example, there is a training school with the minimal scale, which initially has only one teacher and one student, as shown in Figure 8. After a week, a second student joins. There will be two worlds, W = {w1, w2}, representing all the information at these two stages respectively. In the first world w1, D = {teacher, student, teaches, learns-from}, and R = {(teaches, T01, S01), (learns-from, S01, T01)}. In the second world w2, R expands dramatically to {(teaches, T01, S01), (teaches, T01, S02), (learns-from, S01, T01), (learns-from, S02, T01), (studies-with, S01, S02), (studies-with, S02, S01)}, while D only slightly expands to {teacher, student, teaches, learns-from, studies-with}. If a third student joins later, the relations R will continue to expand in w3, while the domain D remains unchanged. This simple example shows how the conceptualization of ontology engineering can extract unchanged knowledge in a changing world.

During the 1990s and 2000s, a variety of ontology methodologies and frameworks were proposed initially or emerged from experiences to facilitate the building process of ontology systems [69]. As early as 1994, the IDEF5 ontology capture methodology was developed to construct ontology systems in a way that closely reflected human understanding of the specific domain [70]. Around the same time, the methodology in MENELAS project started using conceptual graphs as its core formalism [71]. In 1996, the ONIONS methodology was developed for integrating ontologically-heterogeneous taxonomic knowledge, and applied to medical domain [72]. In 1997, METHONTOLOGY was introduced for ontology engineering to build domain ontologies from scratch [73], which followed a lifecycle based on evolving prototypes and were applied to many domains like chemicals [74], bibliographical evaluation [75], environmental pollutants [76], graduation screen [77], data mining [78], DNS [79], law [80], etc. In 2001, the Ontology 101 guide was released by Noy and McGuiness [81] and became the most cited ontology methodology according to Google Scholar [82]. In 2005, another ontology methodology, named UPON, was derived from the Unified Software Development Process (USDP) and the Unified Modeling Language (UML), which aimed to be a highly scalable and customizable framework through its distinct nature of being use-case driven and iterative [83]. In the same year, Ontology Design Pattern (ODP) was also introduced for the ontology lifecycle over the Semantic Web [84,85]. There was not much significant change afterwards about methodologies for ontology engineering [82]. But still, none of those existing methodologies was fully mature especially in the sense of supporting reusability, interoperability and collaboration [69].

Benefiting from these available methodologies, as well as easy implementation through object-oriented programming (OOP), ontology rapidly became the most widely adopted framework for KBS within a few years after its inception. Many ontology systems have been successfully developed [86,87] including some large-scale community-driven projects such as DBpedia [88] and YAGO [89]. However, the bottleneck to build a practical KBS with ontology engineering still exists in several aspects. The first problem is reusability, which is the initial aim of shared ontology but presents to be much lower than expected. In 2017, Kamdar et al. performed a systematic analysis and found the reuse rate of most ontology systems did not achieve 5% yet [90]. Standardization organizations have made great efforts to bridge isolated ontologies. As an example, the Suggested Upper Merged Ontology (SUMO) was proposed as a starter document by an IEEE-sanctioned working group of collaborators from many domains in 2001 [91]. Another standardization effort, named schema.org, was initiated in 2011 by commercial companies of search engines including Bing, Google and Yahoo, and now maintained by the W3C schema.org Community Group, which has specified commonly used classes like person, location, organization, product, etc. [92] for lightweight ontologies [93]. As of 2024, over 45 million web domains mark up their web pages with adopting schema.org objects, according to its official website. The second problem is the substantial amount of human labor to manually collect and input the knowledge, including ontology concepts and their relations. One example is the famous CYC project, which consumed more than 900 person-years of effort over 20 years to obtain 2.2 million knowledge items (facts and rules) [94]. A popular trend to build ontologies nowadays is to use ontology learning to automatically extract knowledge from structured, semi-structured or non-structured original data [22,95]. Besides, another problem is the lack of mature integrated tools for newcomers including ontology editor, visualization tool, reasoner, lifecycle management, and even search or recommendation tool for existing ontologies [96]. Actually, the lack of mature tools implies the need for participation from commercial developers, since it is unlikely to be solely a scientific issue.

3.7. Knowledge Graph

In 2012, Google announced its knowledge graph as a new way to help users to directly discover information more easily through a dialog rather than traditional webpage links [23,97]. This commercial product has led to a surge in research on knowledge graphs, and thus is widely regarded as the beginning of modern knowledge graphs. Later in 2014, Google introduced the Knowledge Vault, which was developed based on its knowledge graph and expanded to a larger scale [98]. Dating back, the term “knowledge graph” was coined as early as 1972 (published in 1973) by Schneider in an early study of Computer-Aided Instruction (CAI), which aimed to model knowledge systems in computers [99]. Afterwards, much research on semantic networks and later the Semantic Web, including projects like ConceptNet [100] and Freebase [101], was highly related to knowledge graphs, although not explicitly named as such. In this sense, a knowledge graph can be described as a very large semantic network that integrates diverse and heterogeneous information sources to represent knowledge within specific domains of discourse [102]. Actually, a knowledge graph is more likely to be an integrated system rather than a single technology.

Comparing to other types of KBS, an extraordinary advancement made by knowledge graph is the breakthrough in the bottleneck of knowledge acquisition. According to public reports, Google Knowledge Graph had billions of entities and facts, and had kept growing rapidly for years from the initial number of 3.5 billion in 2012 [103] when it was first announced to an impressive number of more than 1.6 trillion in 2023 [104]. To build a knowledge graph, the general processes include data collection, knowledge creation or acquisition, knowledge hosting, knowledge curation (assessment, cleaning and enrichment), knowledge deployment, and knowledge evolution [105,106]. The main method to achieve very large scale involves using mapping [107] and automatic annotation [108] for structured data sources and heterogeneous data sources like webpages, respectively. The latter method, automatic annotation, is becoming increasingly important, since larger scale knowledge graphs typically involve more diverse data sources. This has sparked the popularity of using natural language processing (NLP) and machine learning methods, as they are better at handling unstructured data with higher efficiency and accuracy. More specifically, a variety of technologies such as named entity recognition [109], information extraction [110], knowledge discovery [111], and text mining [112,113] have been extensively studied to support building knowledge graphs.

Another remarkable advancement made by knowledge graph is the widely commercial application of KBS, which enables the general public to experience the convenience of directly obtaining knowledge, no matter common sense or domain-specific information, at any time in their daily lives [4,114,115]. One way to access knowledge graphs is using the web search engines like Google, Bing and Baidu [116,117]. As shown in Figure 9, when the input text was equal or highly similar to a term in the knowledge graph of the search engine, extensive knowledge including the short introduction and commonly used attribute fields was listed alongside or even above those traditional webpage links. Another way to access knowledge graphs is using the AI assistants of mobile devices or apps, such as Apple Siri, Amazon Alexa, Microsoft Cortana, and others [105,118,119,120].

4. Paradigm Shift

4.1. Advances by Machine Learning

In the past two decades, machine learning has significantly changed many research areas including KBS. As elaborated in the previous sections, the processes to build large-scale KBSs, such as ontology learning and knowledge discovery, have widely adopted machine learning as their key methodology. Conversely, KBS has also helped machine learning improve accuracy and reduce data requirements through technologies like knowledge embedding [121,122,123,124]. The combination of KBS and machine learning has been earmarked as the most promising approach to address certain issues in domains like smart manufacturing [124].

From a historical perspective, the artificial neural network (ANN) became the basic form of connectionism with the creation of the first trainable perceptron machine [13,14], although the mathematical model of a neuron was proposed as early as 1943 [125]. Later in 1959, Arthur Samuel, an IBM employee using his spare time, introduced the concept of machine learning while trying to develop a computer program that could learn to play the game of checkers through an algorithm based on a search tree [15]. According to the data style, machine learning can be categorized into supervised learning, unsupervised learning, reinforcement learning, and combinations or hybrids of the above. Machine learning encompasses a wide range of algorithms [126], including the decision tree, random forest, support vector machine (SVM), k-nearest neighbors (KNN), k-means clustering, Q-learning, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), etc. In 2006, Hinton et al. proposed deep learning in neural networks [127], which quickly became the most popular direction for the research and application of machine learning [128]. In 2012, Alex Krizhevsky et al. used deep learning with CNNs to successfully win the ImageNet [129] contest [130], significantly outperforming other traditional methods and drawing widespread attention from the AI research community [131]. In 2013, Mnih et al., researchers from DeepMind, proposed deep reinforcement learning (DRL), showcasing its potential through a program that played Atari games better than humans [132]. In 2016, a computer program named AlphaGo, developed by Google DeepMind, defeated Lee Sedol, a world champion Go player, sparking widespread interest in deep learning among the general public worldwide [133].

Apparently, many research fields have been significantly changed by machine learning, as demonstrated by experimental results showing state-of-the-art performance when compared to traditional approaches. The field of KBS is currently in a similar situation, particularly regarding the methodology for building large-scale systems. Generally, the process of using machine learning to construct a large-scale KBS, say a knowledge graph, can be classified into two principal paradigms, the top-down and the bottom-up approaches [134], as illustrated in Figure 10. The first step used to be data acquisition, encompassing structured, unstructured, semi-structured data, and occasionally other existing KBSs. Then the subsequent learning sequence of ontology and entities varies between the top-down and bottom-up paradigms. Since the ontology layer is more abstract than the entity layer, the top-down paradigm prioritizes ontology learning over entity learning, whereas the opposite sequence is adopted in the bottom-up paradigm. In detail, the machine learning process involves sub procedures including entity extraction, relation extraction, entity linking, concept extraction or abstraction, rule definition or knowledge reasoning, etc. A variety of extraction methods have been developed based on open-source datasets, such as CoNLL-2003 [135], ACE-2005 [136] and OntoNotes-2005 [137]. Some of these methods have reached F1-scores as high as over 90%, including the Automated Concatenation of Embeddings (ACE) combined with document context (94.6%) [138], the Princeton University Relation Extraction system (PURE) (90.9%) [139], the Packed Levitated (PL)-Marker (91.9%), etc. [140] After initial construction, constant knowledge updating is also indispensable to keep the knowledge systems timely and accurate. Benefiting from the advancement of machine learning technology, many large-scale KBSs have been constructed, achieving much higher efficiency and also much lower cost in both generic domain and specialized fields like industry [141] and education [142].

4.2. Influence by the Rise of LLM

Although the introduction of machine learning has significantly reduced the professional effort required of knowledge engineers, the replacement effort in labelling training data for machine learning can hardly be neglected. In 2018, the OpenAI research team introduced a novel generative pre-training (GPT) approach aimed at significantly reducing reliance on labeled data by efficiently utilizing raw unlabeled text for training a language model [143]. The GPT model leveraged a semi-supervised approach that combined unsupervised pre-training with supervised fine-tuning. The first step of pre-training used the Transformer model [144] to predict the next word as its objective from a large corpus of raw unlabeled text. The second step of fine-tuning adapted the model parameters using manually annotated examples to address various downstream tasks including natural language inference, question answering, semantic similarity, and text classification. Soon after, in late 2018, the Google AI Language team released BERT (Bidirectional Encoder Representations from Transformer), a large-scale language model containing 340 million of parameters, which achieved state-of-the-art performance on a large suite of sentence-level and token-level tasks [145]. A key finding indicated by BERT was that the scaling law [146] remains effective at extremely large scales, driving a race in model size among researchers and companies in the field of LLM [147,148,149]. In 2019, OpenAI released GPT-2 with 1.5 billion parameters, and demonstrated that the fine-tuning step could be omitted while maintaining comparable performance [150]. In 2020, OpenAI released GPT-3 with 175 billion parameters, and explored the transition from zero-shot to few-shot learning, which led to the rise of prompt engineering [151,152]. The breakthrough in language models was soon extended to neighboring fields like speech, image, and video generation [153]. In 2021, the concept of foundation model was proposed to collectively refer to these large-scale models trained on broad data, highlighting their critically central ability to easily adapt to a wide range of downstream tasks [2]. In November 2022, OpenAI released the famous ChatGPT, which was available to experience by the general public, becoming the fastest-growing consumer software application in history and gaining over 100 million users within two months [154].

To date, widely-known advanced LLMs include GPT-4o and o1-preview by OpenAI [155], Gemini 1.5 Pro by Google DeepMind [156], Claude 3.5 by Anthropic [157], Llama 3.1 405B by Meta [158], Grok 2 by xAI (founded by Elon Musk), Mistral Large 2 by Mistral AI, Titan by Amazon, Apple On-device by Apple, GRIN MoE by Microsoft, Inflection-2.5 by Inflection AI, Jamba 1.5 Large by AI21 Labs, Command-R+ by Cohere, NLVM 1.0 by NVIDIA, DBRX by Databricks, Solar by Upstage, Ernie 4.0 by Baidu, Qwen2.5 72B by Alibaba, ByteDance 530B by ByteDance [159], Hunyuan-Large by Tencent, Xinghuo 3.5 by iFlyTek, PanGu by Huawei, Qiyuan by Qihoo 360, DeepSeek-Coder-V2 by DeepSeek [160], Yi-Large by 01.AI, TeleChat2-115B by China Telecom, etc. [147,148,149,161,162,163] To evaluate the performance across different aspects, a variety of benchmarks have been proposed, including Chatbot Arena [164], MMLU [165], GPQA [166], MATH [167], HumanEval [168], SOCKET, CUAD, APPS, C-EVAL, EmotionBench, HELM, API-Bank, SafetyBench, GAOKAO-Bench, etc. [172,172] Some typical LLMs have been listed in Figure 11 with comparison of their performance and price. Among these LLMs, OpenAI o1-preview achieved the best average quality score but also has the dramatically highest price. Another notable difference between these LLMs is whether they are open-source or proprietary, with Meta leading in open-source models and OpenAI dominating proprietary ones, respectively.

The rapid emergence of LLM has influenced the field of traditional KBS from two sides. On the negative side, LLMs are partially replacing KBSs to a certain extent, since both provide a similar user interface, a chatting box that assists users through question-and-answer interactions. With advanced LLMs like GPT-4o and Gemini capable of delivering proper answers across a wide range of topics, people no longer need to rely on many separated domain-specific KBSs [170]. For complicated and professional domains like clinical applications, LLMs may also adapt to high requirements through fine-tuning with corresponding datasets [171]. On the other positive side, LLMs have accelerated the development of KBSs especially the process of knowledge acquisition and updating, as researchers and engineers can now concentrate more on addressing the challenges and unanswered questions that LLMs are still unable to solve accurately or reliably [5]. In the process of knowledge acquisition, LLMs have been widely studied to perform information extraction tasks including named entity recognition, relation extraction, event extraction, and universal information extraction [172]. In the process of knowledge updating, LLMs can help knowledge graphs with factual validation, link prediction, and knowledge completion [173,174]. Furthermore, LLMs can also play a useful role in knowledge embedding, information recommendation, knowledge retrieval, and semantic parsing, helping users to make better use of KBSs [175,176].

5. Discussion

5.1. Challenges for KBS

With the rise of connectionism and the fall of symbolism in AI over the past two decades, the primary challenge for KBS is that many people doubt its relevance and usefulness nowadays. We consider this challenge from several perspectives as follows. First, although machine learning and LLM have exhibited strong capabilities and great potential in many domains, they also have intrinsic shortcomings and limitations inevitably, such as hallucination and lack of explainability. Second, there exist many domains, such as medical diagnosis, criminal justice, and accident investigation, which have high requirements on reliability or no sufficient historical data for effective modeling. Third, many KBSs have already been effectively used to improve LLMs as elaborated in the previous section, demonstrating the long vitality of KBS. Lastly, we may compare how humans and human society operate to achieve high levels of efficiency and reliability. On the one hand, humans need to accumulate empirical intuition in activities like vision and motion, working similarly to connectionism. On the other hand, they also need acquire knowledge in activities like reading books and learning mathematical formulas, working similarly to symbolism. Human society operates in a similar way, placing even greater emphasis on rules like laws and regulations to maintain cooperation among human individuals. In this sense, KBS, representing the essential thoughts of symbolism, will undoubtedly continue to play an important role in the next generation of AI paradigm together with connectionism through some form of combination or unification.

Theoretically speaking, the fundamental challenge for KBS can be elaborated from further comparison with human intelligence, which has been studied for long time in cognitive science [177]. On the one hand, cognitive science has promoted the development of KBS including reasoning and explanation through modelling mechanism like Newell’s unified theory of cognition [178]. On the other hand, cognitive science has revealed key limitations of current KBS which correspond to the uncharted aspects of human intelligence including representational plasticity, functional dynamism, domain specificity, creativity, and concept learning [179]. Therefore, in the view of comparing with human intelligence, the fundamental challenge of KBS is to handle complex but generalizable problems with efficient and creative solutions.

Macroscopically speaking, the real challenge for KBS is how to be cost-effective. Although KBSs have developed a variety of methodologies and been widely applied in many domains as we have reviewed, the question is that we still have not witnessed their universal commercialization. Apparently, it is not the issue of feasibility but the issue of ROI (Return on Investment) as illustrated in Figure 12. Two typical forms of KBS are considered for an example case: a KBS that helps manage a medium-scale manufacturing factory. One is symbolism way (e.g. using ontology engineering and manual construction), and the other is connectionism way (e.g. using LLM with fine-tuning on small-scale labeled domain data). The lifecycle of a KBS can be roughly divided into several stages: the construction stage, the service stage, and the decline stage. In the first construction stage, the connectionism way will take much less time and cost, comparing with the labor-intensive work of knowledge engineers in the symbolism way. However, in the second service stage, the symbolism way will be much easier to maintain for additive knowledge updates, while the connectionism will be further encumbered by extra cost of knowledge validation due to its comparatively lower reliability. In the final decline stage, the KBS ends its lifecycle, becoming totally obsolete and requiring either reconstruction or termination, regardless of its form.

Technically speaking, the practical challenge for KBS is how to combine both the traditional and the new technologies including rule-based system, knowledge engineering, semantic network, semantic web, ontology engineering, knowledge graph, machine learning, and LLM, to significantly reduce costs throughout its entire lifecycle.

5.2. Combining KBS with LLM

From the view of history, there exist two possible directions to step over the last barrier to the final goal of AI singularity. One way is to improve the construction efficiency of the traditional accurate KBS, and the other is to improve the service reliability of the modern efficient LLM. To address this challenge, a promising research direction is combining the modern LLM methods with those traditional KBS frameworks through separating the construction and service stages.

To explore the possible direction for further research, several dimensions can be considered to evaluate the relative advantages between symbolism and connectionism for a typical application. (1) Reliability: symbolism has the advantage due to explainability and convenience for validation; (2) Controllability: symbolism generally allows to edit a single concept or a designated rule; (3) Adaptability: connectionism has the advantage since diverse training data reduces the requirement for strict formalism; (4) Scalability: LLM has demonstrated a single model can cover multiple types of data and handle numerous types of downstream tasks; (5) Training efficiency: connectionism exhibits high efficiency when understanding information from the huge input data; (6) Running efficiency: symbolism exhibits high efficiency when generating information to the output data through stored rules and links. The goal of combination should utilize the advantages of each way, that is, utilizing the KBS in aspects of reliability, controllability and running efficiency, while utilizing the LLM in aspects of adaptability, scalability and training efficiency.

There exist several methods that are trying to combine symbolism and connectionism. According to the participation degree of KBS in the hybrid systems relative to LLM, we classify typical combination methods into three categories: low-KBS methods, medium-KBS methods, and high-KBS methods. Basically, low-KBS methods only utilize KBS in very limited partial processes to improve the reliability and controllability of LLMs, like retrieval augmented generation (RAG). It is still hard to achieve very high reliability and controllability as traditional KBSs in this way. Besides, the low running efficiency or the high computation cost of LLM is still left unresolved. In the opposite side, high-KBS methods keep the framework of KBS and only utilize LLM in very limited partial processes to improve the adaptability and training efficiency, like entity auto-extraction of knowledge graphs by LLM. Similarly, it is hard to achieve as high as the LLM, and the scalability remains unresolved since KBSs are generally domain-specific. From the perspective of two extreme ends, we could narrow the promising combination direction down to a medium-KBS style, which has not been researched too much yet.

One example of medium-KBS methods is neuro-symbolic architect, which integrates neural learning and symbolic reasoning in a unified framework. Neuro-symbolic AI started to attract research attention from the early 21st century [180]. Several neuro-symbolic integration paradigms include loosely coupled models where neural and symbolic components interact sequentially or unidirectionally, highly coupled models where neural and symbolic components interact cyclically, and tightly integrated models where neural and symbolic components are mapped to each other [181]. Neuro-symbolic AI has already found applications in many domains like question answering, robotics, computer vision and healthcare. However, limits and difficulties like scalability and efficiency are still long to resolve for neuro-symbolic AI. Other medium-KBS methods like synergized LLMs and knowledge graphs have also been proposed but still on the way of further research.

In our perspective, it is still not existing that a unified framework could effectively combine KBS and LLM with keeping all advantages in the 6 dimensions at a high level. To further narrow down the promising research direction in the future, we propose three features that a better combination method may have: (1) skeleton-based, to ensure reliability and controllability; (2) customizable-modular, to ensure scalability and running efficiency; (3) globally-plastic, to ensure adaptability and training efficiency. However, such an innovative method tends to be complicated and time-consuming to develop. Therefore, in a short term, we need downgrade the goal to a simple and realizable level by considering different weights of KBS and LLM in different stages of system lifecycle, to minimize the cost and maximize the return. In the construction stage, the weight of KBS should be relatively lower than LLM to improve adaptability. Both knowledge bases and PSMs need be automatically built to significantly reduce training cost. In the service stage, the weight of KBS should be relatively higher than LLM to improve the customer return and keep extremely low running cost. When the KBS need be updated intermittently, the LLM should be reused to significantly reduce the maintenance cost. Considering this separate mode, a possible solution will be using LLM as an intelligent assistant to effectively construct and update various domain-specific KBSs for customized services, which is promising to make KBSs widely commercialized with attractive ROI as soon. Then next, in a long term, we can consider integrating many domain-specific KBSs and LLM assistants into a universal skeleton-based, customizable-modular and globally-plastic framework.

6. Summary and Conclusion

In summary, we have provided a full overview of the 60-year history of KBS evolution, from the early history of rule-based systems like DENDRAL to the influence by the new emerging LLMs like ChatGPT in recent years, during which multithreading concepts and technologies have been proposed and developed. The early trials and thoughts of rule-based system first taught us the power of KBS by distinguishing what to know and how to do with domain knowledge. Then knowledge engineering emphasized on the methodology for building reusable and generalized KBSs. As another research thread, semantic network originated from exploring the links between words and concepts, particularly targeting common-sense knowledge. With the rise of the Internet, the links broke the local limitation and developed into worldwide connections, forming the foundation of the Semantic Web. Ontology engineering introduced systematic standardization and frameworks for conceptual knowledge, facilitating the representation and reasoning of complicated knowledge. Promoted by major companies like Google, the large-scale knowledge graph became the symbol of modern KBS, allowing the general public to experience the power of knowledge through daily applications such as search engines. In the past two decades, machine learning has been the critical technology for the success of large-scale KBSs, since automatic knowledge acquisition became possible. Finally, LLMs emerged and changed the situation with influence on traditional KBSs from both a positive assisting side and a negative replacing side. To conclude, KBS still plays an important role in AI research and applications, and a promising direction to address the challenge of further reducing costs for broad commercialization is to combine traditional symbolism frameworks with machine learning, particularly LLMs, but having different emphasis in different lifecycle stages.