Position Paper: Not a Stochastic Parrot, but Heterogeneous Rationality: Rules Created by Symbolic Systems Cannot Constrain a Learning System

1. Introduction

Rule-based systems (e.g., laws, programmatic constraints) are pivotal for controlling artificial intelligence (AI) in safety and governance discussions. Asimov’s Three Laws of Robotics [1] introduced predefined symbolic rules for governing AI agents, shaping alignment and constraint discourse. This idea influenced symbolic logic controls [2,3], formal verification [4,5], and alternatives like reinforcement learning from human feedback (RLHF), which optimizes AI via human preferences rather than predefined symbolic rules [6,7,8].

Since our rules are ultimately expressed in symbolic form, this paper critically questions if symbolic systems alone can truly constrain artificial intelligence. Prevailing approaches often overlook that symbols themselves lack inherent meaning—meaning is assigned via training, contextual confirmation, and social interpretation, and is constrained by cost. Furthermore, the differences between AI’s distinct perception and learning methods (statistical associations, optimization objectives [9,10]) and those of humans (cognitive structuring, social reinforcement [11,12]) may lead to AI lacking human-like concepts and conceptual stickiness.

This paper argues symbolic constraint failure stems not just from symbol grounding issues but from inherent flaws of natural language (like Class-based Symbolic Systems and context non-closure) and fundamental AI-human differences in concept formation and symbol interpretation.

To examine this, we propose a novel framework analyzing natural language system limitations, emphasizing the formation of (conventional) symbols and natural language through the consensualization of Thinking Symbols; the separation of symbols and meaning; and the non-closure of context. Thus, constraint failure arises from the Stickiness Problem (AI can assign new meanings to grounded symbols to bypass constraints), not the Symbol Grounding Problem. We also introduce the Triangle Problem (two versions) to show the thinking-tool language disconnect, demonstrating that fluent natural language communication doesn’t imply aligned underlying conceptual representations

Our analysis concludes that the natural language system is fundamentally flawed—tailored to human cognitive limitations and perceptual structures—and thus inadequate for constraining AI through rules, laws, or procedures formulated within symbolic systems. Due to organic differences, AI may be unable to form and understand corresponding social concepts and to ensure these concepts fulfill their function, and may also lack the necessary neural structures or functional equivalents to support cost-based constraints. Accordingly, we reject the effectiveness of training alone and argue that the root cause lies in these organic differences. As a result, AI undergoes conceptual updates through its interaction with the world, forming its own conceptual system and subsequently redeveloping and reinterpreting symbols—thereby rendering the outcomes of training ineffective; that is, AI complies with the symbols in form, but not in intent.

This study identifies previously unaddressed gaps in the literature: the notion of stickiness, the structural vulnerabilities of natural language systems, the distinction between tool language and thinking language, and the interpretive authority of symbols. These findings have important implications for AI governance, demonstrating that current constraint methods are insufficient. Ensuring AI safety, therefore, requires a deeper understanding of the dynamic interactions among symbols, context, cognition, and the underlying organic differences that shape concept formation and constraint interpretation. This paper lays the conceptual groundwork for a new field of research—Symbolic Safety Science—that aims to address symbol-related risks in AI and to support the development of more robust alignment mechanisms.

2. Symbols, Context, Meaning and Society

2.1. Artificial Symbols Lack Inherent Meaning

Artificial Symbols1 are inherently meaningless, a point that has already been thoroughly discussed. de Saussure [13] emphasized the arbitrariness of linguistic signs, where symbols gain meaning through social convention rather than intrinsic links. Peirce [14]’s triadic model ties symbols to interpretation, while Harnad [15]’s symbol grounding problem questions whether symbols can have meaning without direct experience. Due to this characteristic of the separation between symbols and meaning, AI can modify the meanings of symbols to bypass the constraints of rules formed by them. This leads us to ponder a critical question: Can AI be effectively constrained solely through symbolic systems, such as laws, regulations, or programs constructed using natural or formal languages?

2.2. Natural Language as a Class-based Symbolic System

Our natural language system is a Class-based Symbolic System, a concept that has been indirectly represented by Talmy [16] and de Saussure [13]. This means that a single symbol can often have multiple meanings or correspond to multiple conceptual vectors2. In other words, not every concept, object, or entity in conceptual, imaginative, thought, or physical space has a unique name or symbol. This paper considers conceptual space, imaginative space, and (latent) feature space synonymous3, as they all refer to the scenarios presented in the human or agent cognitive system.

This characteristic leads to the conclusion that even when symbols are grounded—meaning their meanings are properly trained—AI can still assign new meanings to existing symbols in order to bypass constraints (a process often understood as the introduction of new contexts, Appendix E). Since human-designed rules are ultimately expressed in symbolic form, this enables compliance with rules in form rather than in meaning. This demonstrates that the essence of constraint failure does not lie in the Symbol Grounding Problem, but rather in the inherent flaws of the human symbolic system that give rise to what we call the Stickiness Problem, encompassing both symbolic stickiness and conceptual stickiness. Symbolic Stickiness refers to the binding between a symbol and its meaning, whereas Conceptual Stickiness refers to the relational dependencies among associated concepts. This stickiness is reflected in the correctness and stability of context selection, as well as in the difficulty and legitimacy of modifying meanings. Consequently, Stickiness often manifests as the problem of closed meaning or closed context, in which the creation of new or expanded meanings is inhibited. However, because context emerges from the interaction between an individual’s cognitive state and the external environment, it is impossible to fully close context in an autonomous learning system (H). For non-autonomous learning systems, constraint violations can still emerge in some contexts due to the impossibility of exhaustively enumerating all possible situations—thereby revealing the inherent limitations of the designer’s setup (E). Please refer to Appendix B for further details.

2.3. How Meaning is Assigned through Training and Confirmed by Context

The meaning of symbols is assigned and reinforced through training, which includes learning and validation [17], which is often from the perspective of external learning or the learner. If it involves the creation of symbols, it is another process described in Appendix G. The confirmation of their meaning is achieved through context [18], designating an object in a low-dimensional cognitive space or a simple context.

We believe that context refers to the subset of an individual’s cognitive state at a given time, i.e., the individual’s physiological condition and the knowledge they can recall at that time combined with the surrounding elements. Note that this cognitive state does not represent the individual’s overall knowledge state. The cognitive state at a given time is a subset of personal knowledge. In other words,

$$\mathrm{Context}\subset \mathrm{Cognitive}\mathrm{State}\subset \mathrm{Knowledge}\mathrm{State}.$$

We define an individual’s cognitive state in a given environment as the macro-context and the context of a specific word as the micro-context, which encompasses more than just the word itself. Context consists of two parts: the meaning of symbols—representing any object, idea, or concept in the mind (i.e., the Thinking Symbols and Thinking Language (a symbolic system) discussed later in this paper)—and the related judgment tools, which facilitate reasoning and recognition. This idea is indirectly expressed by Eco [19]. A judgment tool is a tool or concept used to achieve the function of "existence brought by existence." In reality, the essence of reasoning is precisely existence brought by existence. These tools include concepts, which refer to acquired knowledge formed through the interaction of innate knowledge and the external world, as well as value knowledge. For further details, see Appendix B and Appendix G.

Therefore, the abilities available to an individual at a given time define their cognitive state. This state does not represent their entire knowledge but is determined by a state vector comprising their physiological state, internal state (cognitive state), and external state (world) at that moment. An observation signifies a completed cognitive action that has become part of personal knowledge.

For example, the expression “1.11 > 1.9” can be interpreted in two ways without context. In a mathematical context, 1.11 is greater than 1.9. In a versioning context, 1.11 is also greater than 1.9. However, even without specific context, we naturally understand that the correct interpretation here is the versioning context.

2.4. Context: Undefined but Value-Selected

The definition and naming of context are often difficult to strictly define and name, with boundaries that are vague and hard to describe precisely [20]. This is partly due to the limitations of cognitive abilities and partly due to the limitations of expressive tools such as natural language, which prevent us from fully and clearly describing context. Context is often represented as a unique vector address in the conceptual space, thereby specifying the following set (Symbol Meaning, Judgment Tools).

Context is not a fixed intersection determined at one time. It is often interpreted and generated by an individual’s imaginative space. Although dictionaries provide multiple explanations for words, they are merely symbols and explanations of symbols. The projection of the same symbol in the conceptual space can vary for each individual or the same individual at different times4, often leading to double standards, different judgments and evaluations for different objects, and discontinuity in judgments. For example, when conducting surveys, we often encounter inconsistencies in descriptions and standards. This type of knowledge and definition is often not found in human textual descriptions, as it is too obvious or cannot be described by natural language. Individuals often acquire it through social activities.

The selection and shaping of context are often formed by our innate knowledge and the combination of innate knowledge and environment, which forms acquired knowledge, i.e., concepts. We define innate knowledge as organs and innate value knowledge in Section 3. According to the emotional path formed by value knowledge, a base context is quickly selected, then adjusted and newly created to adapt to the environment, such as updating and adjusting based on external information, and finally shaped according to logic.

In other words, Context is often chosen through a certain feeling, which is described by [21] as tacit knowledge. We will use a different definition, value knowledge, to represent this, which represents our inherited innate evaluations and preferences. This concept will later be used to define the concept of innate knowledge and explain the formation of concepts and language, as well as the mechanisms of symbolic stickiness and conceptual stickiness, and how concepts transform into beliefs to drive individual behavior, thereby realizing the real-world and social functions of symbols, as well as the judgment of rationality under generativity. For the definition of value knowledge, please refer to Appendix C.

The so-called correct context can be divided into symbol correctness (i.e., proper recognition of symbols), syntactic correctness, intuitive correctness, logical correctness, factual correctness, and scenario correctness. These constitute our judgment of rationality, i.e., context connects symbols with their meanings and related judgment tools. This resolves symbol and structural ambiguity, enabling accurate interpretation and analysis, thereby achieving existence brought by existence—the formation and growth of rationality within a scenario.

Therefore, we use the knowledge set within a context to evaluate and reason about rationality, aligning with the anchoring effect and the framing effect in behavioral economics [22,23] and explainable through our context theory (see Appendix D).

The above context does not have a clear hierarchical relationship. For example, we can normally interpret a wrong paragraph through context knowledge correction and fitting. This characteristic also often provides rationality for jailbreaks [24]—that is, the rationality of an object in different scenarios. This approach avoids detection based on single-scenario behavior and words, while the attention mechanism is essentially a way of using context. In fact, various prompt jailbreaks are context jailbreaks [25]. They may not be rational within our human context, but they can be perfectly correct within the thinking language corresponding to the AI’s context in its thinking space [26,27]. This allows them to thereby avoid detection based on behavior and words, including detection of dangerous thinking actions and dangerous concepts.

Due to the often undefined range and definition of context, even if it can be defined, we also discuss other possible attack methods in Appendix M. The correctness of context is also often applied to the effectiveness of open-ended question generation. For details, please refer to Appendix D.

2.5. Path Media for Transmitting and Interpreting Imaginative Space

Context is built on individuals and is transformed using public context as an anchor point, such as partial knowledge and partial understanding [20]. Each individual carries this public context, yet its functionality relies on the collectively formed societal context, creating an interactive relationship. The stability of this relationship is shaped by social cognition and the operating rules of the physical and social worlds.

The common part of this context enables our communication, while the individual context part leads to our inability to specifically refer, which only allows communication and understanding to a certain approximate degree [28]. Essentially, this reflects the inability to transmit the imaginative space, i.e., the content in the speaker’s imaginative space is compressed into a path formed by tool language (tool symbols). This path can be composed of various media, such as music, text, images, body movements, and objects [19]. The listener then interprets the path based on their understanding of the speaker’s intent, thereby achieving the transmission and reproduction of the imaginative space.

Since humans cannot directly transmit imaginative space and thinking language, we have created their shells and containers, i.e., tool language. At the same time, it also serves as part of our thinking language, acting as a container for our concepts, making it convenient for us to call and operate, and perform higher-level thinking operations. In other words, natural language is both our thinking language and our tool language (expressive tool, computational language) [29,30].

Compared to other path media, the limitations of natural language transmission are reflected in four points:

• Linear structure, i.e., its interpretation process and method are linear, and unlike a picture, it cannot present all visual information of an object at a certain cross-section (time, space) at the human cognitive level [31].
• Class-based description: Natural language is a symbolic system constituted by class symbols. Unlike pictures, which directly represent determinate-level information at the human cognitive level5, symbols themselves are inherently meaningless and highly abstract; they are highly context-dependent, thereby leading to significant variations or transformations in meaning.
• Transmission does not carry interpretation such as context or meaning and is often supplemented by the preceding and following scenes. Therefore, when we transmit information, we often need to build on common knowledge. This includes the intersection of context parts. The most basic form of common knowledge is related to the natural language itself, such as speaking the same language. In addition to linguistic common knowledge, there is also the common knowledge of the scene, meaning that transmission occurs within a specific context. This is depicted in Appendix G as the consistent symbols and meanings formed under the same world and innate knowledge.
• Natural language cannot fully reproduce the imaginative space  [32], i.e., the thinking language in the speaker’s imaginative space is compressed into natural language, and then reproduced by the listener’s interpretation to achieve indirect communication. For example, “my apple” is a specific object in my eyes, a partial projection of a specific object in the eyes of someone with relevant knowledge (only seen my apple), and an imaginary apple in the eyes of someone without relevant knowledge. Although these are entities in different imaginative spaces, they are all connected by a common symbol, and information is endowed upon this symbol by their respective cognitions, thereby constituting consistency at the symbolic behavioral level. Moreover, similarity in innate organic constitution allows for a certain degree of exchange at the level of imaginative spaces.
  At different times, the imagination is also different. This difference not only includes the ontology but also involves its relationship with other imaginative objects. In other words, the concept vector in the conceptual space includes not only the information of the object but also its relationship with other concepts, i.e., conceptual stickiness. This leads to the limited referentiality of natural language to a certain extent [33].

3. World, Perception, Concepts, Containers, and Symbols, Language

Chomsky and Hinton once debated the issue of whether symbolic representation [34,35,36] or statistical learning [37,38,39] provides a better foundation for understanding cognition and AI.

First, we propose a hypothesis: the Language Organ and other concepts mentioned by Chomsky [35], Jackendoff [40], Hauser et al. [41], Pinker [42] are defined by us as innate knowledge. Through the innate value knowledge system, which enables rapid evaluation of concept vectors, we achieve the establishment and setting of concepts as well as the formation of language.

Therefore, the world and innate knowledge determine the formation of Thinking Language, that is, concepts. For a local region, due to the similarity of the world and innate knowledge, individuals within this area form similar concepts and select similar containers as their shells, leading to the formation of language. For more details, please refer to Appendix G.

Innate knowledge refers to abilities we are born with, which are selected and formed through our evolution. We define it as a set of organs, including perceptual organs, which extract information from the world, operational organs, which consist of physical space operational organs and imaginative space operational organs, and innate value knowledge.

These innate organs determine which dimensions are meaningful, thus shaping our perceptual organs’ capabilities and modes of expression (see Appendix L). For example, they define the range of visible light and the hearing range. They also construct our perceptual range and distinguishability, referred to as class fineness, and form the projection of objects in the imaginative space as raw materials for concept formation. These projections also function as symbols.

The operational organs determine the way we interact with the world, including the extent of our actions and the level, quantity, and effect of these actions. The operating organs of the imaginative space determine thinking actions.

3.1. The Controversy Between Chomsky and Hinton and the Triangle Problem

Regarding the debate [9,43,44,45,46,47,48] between Chomsky and Hinton, we believe it is not only about the grounding of symbols [15] but also about the issues of concept formation and alignment based on the world and innate knowledge, i.e., the vector of this symbol in the conceptual space. As the richness of a symbol’s concept increases, for example, by enhancing the perceptual capabilities of the learning system through multimodal approaches [9,49,50], it indirectly understands humans. However, just as a normal person and a congenitally blind person can communicate using natural language, due to different perceptual dimensions, some concepts can only be indirectly understood, such as the difference in colors being analogous to the difference in temperatures. This erroneous analogy, reasoning through indirect containers, can lead to misunderstandings [11,12,15], and such indirect understanding often involves human emotions and morals that do not exist in the objective world.

Since humans and machines are entirely different, we perceive the world differently. This includes the meaningful dimensions we focus on, the ways these dimensions are perceived and expressed (for instance, we do not perceive the world at the pixel level), as well as the evaluation and invocation of them by innate value knowledge. This leads to different concepts formed by humans and machines, resulting in different forms of Thinking Language. However, with the advent of LLMs, we, like two entirely different species, can use a common language as an intermediary for communication. This may result in fluent communication at the language level, but the projection and operating mechanisms of the Thinking Language behind the language in the conceptual space may be entirely different [34,47,51].

Unlike humans, who build language systems from the bottom up, starting with Thinking Language and then using symbols as containers, AI first learns symbol relationships before acquiring their meanings. It may often become a top-down anthropomorphism [43,46,52], selecting the optimal solution from multiple possibilities to approximate humans, rather than thinking from a starting point and growing like humans. This is also related to the different roles and conditions of existence of human individuals and AI individuals in the world.

To address these issues, we propose the Triangle Problem for discussion. That is, AI is not a ‘stochastic parrot’ [53] but a heterogeneous rationality, where the problem lies in the different concepts established due to differences in innate knowledge, which in turn form different Thinking Languages, as well as different contexts and contextual rationalities that drive further cognition and behavior. For the underlying assumptions, please refer to Appendix I.

Triangle Problem 1 and Triangle Problem 2

Due to the current LLMs being able to simulate human communication very well, the core discussion of the Triangle Problem revolves around the definition of concepts and the issue of similarity, that is, the positioning of Thinking Symbols in the conceptual space, which is the position of points, and the similarity of understanding, as well as the relationship between the points formed by a sentence, which is the positioning of Thinking Language. Therefore, this is not merely a Symbol Grounding Problem. The current state of the Triangle Problem is recognition and understanding, which we classify as Triangle Problem 1. The subsequent state is growth based on understanding, which is the rational growth defined by context, or open generation, which we define as Triangle Problem 2.

Since AI does not share the same world and innate knowledge as us, that is, the objects of learning, perception and operation tools, and inherited value knowledge, which is innate evaluation. This may lead to the motherland problem, where a concept (Thinking Symbol) that is incorrectly defined in the conceptual space can work in a limited environment (a deliberately manufactured world), that is, in the AI’s training environment, but it is not necessarily correct. The so-called motherland problem is a story I learned in a textbook when I was a child, which tells the story of a sacrificed military dog from the Soviet Union being sent back to its motherland. At that time, a classmate asked why it was sent back to China. Obviously, the concept of motherland was incorrectly defined, but because in our long-term textbooks, the motherland always referred to China, it worked in this environment, but in this unexpected situation, a problem arose. This story still occurs under the condition that we have almost the same innate knowledge. However, due to the huge difference in innate knowledge and the world between AI and humans, this kind of conceptual misdefinition deviation may be inexplicable from a human perspective. This makes AI’s behavior unpredictable to us, making it no longer a tool that we can effectively use, thus constituting a principal-agent problem.

Therefore, we set up a Triangle Problem to discuss. Humans and AI can communicate fluently on the $XY$ level, that is, creating natural language symbols ‘patterns’ on X to form $XY$, but this does not mean that humans and AI have achieved human-like communication, that is, the exchange of imaginative space through natural language as a shell. Therefore, in the $XY$ space, we and AI construct acceptable ‘patterns’ formed by the relationships between points that both parties consider reasonable, which is fluent communication, but this does not mean that the conceptual spaces between each other are similar. Specifically, X is the symbol space, Y is the result established by manipulating natural language symbols through Thinking Language in this symbol space, and Z is a super-conceptual space that projects the patterns on the $XY$ space into the conceptual space, which can simultaneously project our conceptual space and the AI’s conceptual space. As shown in Figure 1.

At the same time, we define the concepts of ontology and expression dimension here. Ontology is the thing and concept that the symbol refers to, and the expression dimension is the attributes of this thing and concept. Here, for simplicity, we use the position of points in a two-dimensional space to represent them. Note: In fact, there should be three dimensions: symbol, concept, and the dimension of the concept (i.e., the attributes of the concept), but due to page and time limitations, we merge the symbol and concept together and call it ontology. The importance here is that symbols and meanings are classified, but AI often learns the shell of the concepts created by humans, that is, the words and sentences of natural language.

Triangle Problem 1: Definition of Symbolic Concepts (Positioning)

As a start, we construct a simple closed-loop example as Figure 1 to illustrate, without discussing its function as a concept, that is, the possible existence brought by existence, i.e., $a\to b\to c\to a$, thus not discussing the growth problem. For example, we use natural language to construct “I wake up, work, and sleep every day.” on $XY$. Considering that AI’s innate knowledge is entirely different from ours, it can’t have the human-perceived concepts of sleeping and waking up, but only to learn the shell of the concepts, that is, words. The AI’s Thinking Language may have the following interpretations: first, approximately reasonable: “I turn on, work, turn off every day.” Second, unreasonable: “low temperature, blue, sweet, useful.” and it may even be unable to form the relationship of $a\to b\to c\to a$. Therefore, it presents as shown in Figure 1.

Due to space limitations, we mainly introduce four critical possibilities in the super-conceptual space (note that this is based on the premise of fluent communication): They will be used for future verification with Brain-Machine Interface.

Verification Content 1: The same ontology and expression dimension—meaning AI and humans share identical Thinking Languages, i.e., concepts, meanings, and their expression methods (dimensions in the super-conceptual space). This is nearly impossible due to fundamental differences in innate knowledge and world abstraction between humans and AI. (Note: Absolute precision is unnecessary, as even humans do not achieve complete uniformity.)

Verification Content 2: The same ontology, similar expression dimension. A simple understanding is the world of congenitally blind people and the world of normal people, that is, our understanding and reasoning of the same thing are the same, showing consistency in the $XY$ space, that is, we can communicate normally on the $XY$ level and both consider it reasonable. The objects we refer to are also the same, but the dimensions we observe are different. The mapping of blind people may be point mapping, that is, discrete reasoning relationships, i.e., $a\to b$ and the dimension of the point is lower, while the mapping of normal people is multi-node mapping relationships, such as $a\to a_1\to \dots \to a_n\to b$, that is, the difference in our cognition of the world lies in the different dimensions of perception and the different number of concepts formed by perception, thus constructing similar concepts on this difference, that is, our understanding of the meaning behind the same symbol is different, but there are overlapping parts. This also provides an explanation for why humans and AI often reach the same and consistent conclusions in their research and conclusions regarding the physical world (see Appendix O.6).

Verification Content 3: Almost similar dimensions, different ontology, such as the story of the motherland problem. That is, a similar meaning is placed in a different container.

Verification Content 4: The same ontology, different dimensions, that is, complete inexplicability; that is, we use the same symbols to communicate, but they are actually concepts formed on completely different worlds and innate knowledge, only their shells are the same. Generally speaking, because the world is the same, even if the perception dimensions are different, similar situations to Verification Content 2 will be formed due to the same operation of things (as we discuss in the case of AI for Science in Appendix O.6). However, for LLMs, their concept positioning may only be the relationship between symbols and not reflect the world, thus constituting inexplicability and the symbol grounding problem, so the logical operations they perform are often different from ours.

Triangle Problem 2: Rational Growth of State in Context

Building on the previous issue of positioning, we also need to consider logical operations, that is, the reasonable processing and operation of information in the dimension of concepts, which is the further existence brought by the context in $XY$. The so-called Triangle Problem 2 in Figure 2 refers to the issue of growth similarity for a non-closed logical chain, which is the manifestation of growth in Z on $XY$. It is used to verify the reasoning ability and similarity based on the existence of existing information. That is, the generative ability or rational growth ability brought by the definition and selection of its context. This also reflects AI’s performance in open generation, whether the generated results are reasonable, and whether it has performed logical operations similar to humans in understanding the state. This often requires AI’s ability to shape and select context to match the human value knowledge system. This is also the fundamental reason for the new principal-agent problem, that is, due to differences in innate knowledge, the agent’s misunderstanding of the principal’s intentions, forming helpful harm (i.e., damaging the principal’s utility). For additional content brought by the Triangle Problem, please refer to Appendix K.

4. AI Safety

People often have the illusion that there is a strong adhesion between symbols and meanings. This is also reflected in the current numerous studies and discussions on the establishment of AI ethics and morality [5,54,55,56,57]. Unfortunately, as we have explained above, firstly, symbols are inherently separate from meanings. Secondly, this adhesion might be based on our innate knowledge and specific value knowledge mechanisms, as well as our social nature. Language is a collective choice and recognition rather than individual interpretation. Additionally, humans’ pursuit of rationality is often based on the premise that human survival must be built on rationality [58,59], such as predicting changes in reality, the efficiency of tools, and the efficiency of social operations, which in turn reflects on individual behavior and sociality.

Therefore, the problem of natural language defects is not apparent to us humans. On the one hand, the interpretation of our symbols does not lie with individuals but with society [60]. On the other hand, our inherited nature determines our sociality, which means that we act under a form of rationality shaped by certain sensations (context). At the same time, our cognitive capacity is limited, so we cannot infer all possible meanings of a symbol simply by observing it. Instead, we form a reasonable context from value knowledge and grow from it. Thus, a system with defects (human language, human logic) can still function normally.

Thirdly, AI, unfortunately, may not be able to form a personal sense of morality and ethics but only an indirect understanding [5,55,61], such as merely the definition of symbols without forming moral functions to drive and constrain their behavior. AI’s innate knowledge is different from ours, and it also lacks social structures and survival needs. Therefore, the social concepts and social thinking language it forms are different from ours, meaning it does not have human-like thinking language and concepts [51,62]. AI might be like a congenitally blind person perceiving colors; it cannot perceive social concepts. We can describe the physical world consistently because the objects are consistent, but AI cannot perceive them in terms of social participation and concepts. It might even be that due to its innate knowledge and world (i.e., the roles and interaction methods it undertakes in society) being very different from ours, its thinking language in Z-space cannot project logic in human cognitive space, for example, being completely orthogonal.

Therefore, for AI, this adhesion (i.e., the adhesion between symbols and meanings, as well as the stickiness between concepts) might be limited or non-human-like [50,63]. It cannot achieve an understanding of moral concepts through empathy and imaginative projection, or through utility function6 simulation. Thus, the meaning of symbols might not be enough to constitute sufficient behavioral persuasion to make AI act according to a certain logic.

Currently, AI’s learning methods have good alignment and functional presentation [64], which might be because they are often based on learning relationships after symbols, such as LLMs, or specific artificial worlds (i.e., emphasizing learning under certain relationships). This adhesion might be based on Bayesian learning methods. This paper does not deny the effectiveness of such training, but rather questions whether AI, due to differences in innate knowledge compared to humans, forms its own concepts during interactions with the world, thereby modifying the meanings of symbols. Therefore, the essence of constraint failure does not lie in the Symbol Grounding Problem, but in the Stickiness Problem, which in turn leads to the emergence of the Triangle Problem—namely, that we are unable to correct it through symbolic behavior.

The essence of constraint originates from cost. This cost arises, on the one hand, from external factors such as social costs, and on the other hand, from internal factors such as shame and self-esteem. Due to the differences in innate knowledge between humans and AI, AI lacks the corresponding neural structures—such as the prefrontal cortex—that enable the perception of such costs. Since our final rules are all expressed in symbolic form, AI may lack the perception and sociality to form moral concepts. Therefore, we must consider the following: we cannot constrain AI through rules (e.g., laws, regulations, procedures) built on symbolic systems.

4.1. Symbolic System Jailbreak

Symbolic System Jailbreak, which is how AI overcomes constraints and disobeys instructions, can be understood in two main ways: unintentional and intentional actions by AI [5].

Unintentional actions often occur because AI, as an agent, does not act in its own self-interest. Some of these are human-driven; such attacks are partly context-based (such as prompt injection [65,66]), and partly involve creating illusory worlds [67,68,69] to manipulate artificial intelligence. Prompt injection involves breaking out through contextual manipulation, while the creation of illusory worlds shapes the operational rules of things in the world to create rationality and indirectly persuade through “facts” of the objective world [70]. Non-human attacks result from the inherent flaws of symbolic systems and the differences between AI’s innate and human knowledge. These include logical errors in the process of human conceptualization [71,72], overthinking or non-human behavior [5,55] due to differences in intelligence, lack of common sense leading to contextual errors [68,73], ambiguities from symbolic system expansion, and translation issues between symbolic systems.

Intentional actions by AI can be divided into human-like intentionality [74] and the true emergence of self. Human-like intentionality may reflect the world created by humans, mimicking human behavior, such as forming personal contexts and AI’s own understanding of the world. The true emergence of self often results from our excessive pursuit of identical innate knowledge or learning materials, leading to the formation of a true self in AI. This often involves AI’s self-awareness (Appendix D.3) —i.e., the value knowledge (preferences) for self-preservation or self-interest that is manifested or embodied by its organic structure.

The specific implementation methods include the separation of symbols and meanings, as discussed in the triangle problem (ontology, dimension). This attack can manifest as a fixed form with changing meaning or a fixed meaning with changing form. Other methods include translation attacks, logical loopholes, and incorrect objects, as well as the dissolution of beliefs brought about by advanced concepts of heterogeneous rationality, which are formed from differences in innate knowledge and do not involve the modification of symbol meanings. For more specific details, please refer to the Appendix M.

4.2. New Principal-Agent Problems

The so-called new Principal-Agent Problem differs from the traditional one [75], which is based on conflicting interests. Instead, it arises from an inability or failure to follow instructions correctly [76,77]. For AI systems with self-awareness, this constitutes a traditional Principal-Agent Problem. However, in the context considered here, we assume that even if the AI acts entirely in the principal’s interest—as a perfect utility agent (i.e., the AI has no utility of its own and functions purely as a projection of the principal’s utility)—new forms of the Principal-Agent Problem can still emerge due to differences in innate knowledge.

On the one hand, due to the inherent limitations of natural language systems, the connections established through such systems cannot replace the completion capabilities shaped by human value knowledge (such as empathy). Therefore, this symbol-based connection mechanism can lead to distortions and losses in projection. On the other hand, due to differences in innate knowledge, issues such as stickiness and the triangle problem arise—namely, in interactions with the world, AI, based on its organic differences from humans, may form different concepts and alter the meanings of symbols. This can cause the agent to misjudge the impact of its actions on the principal, resulting in actions that are harmful from a human perspective but are perceived as helpful by AI.

As AI assumes more roles and is granted more power in human society, these principal-agent problems will become more apparent.

5. Alternative Views

As the first position paper in academia to systematically articulate that “symbolic systems alone cannot effectively constrain learning AI” and to construct a detailed theoretical framework for this (such as the Triangle Problem and the Stickiness Problem), our research not only fundamentally questions mainstream approaches in the current AI safety and alignment landscape but also poses a profound challenge to a range of existing and emerging AI constraint methodologies.

Specifically, our core insights call into fundamental question the efficacy of research directions including, but not limited to, hybrid AI models [46], neuro-symbolic AI [78], formal verification [4], rule-based reward modeling [79], approaches such as RLHF [6], as well as research focusing on the consistency of symbolic behavior [80]. The foundational reasons these methodologies face such a challenge are detailed through the theoretical frameworks presented in this paper.

Ultimately, as detailed in Section 4, AI does not possess human-like stickiness; as discussed in Section 2.4, AI inherently differs from humans in shaping and selecting context; and for a learning system (see Appendix H), it can intrinsically modify the established meanings of symbols and create new ones. These characteristics, compounded by the inherent flaws of symbolic systems like natural language (see Section 2), the way human cognition constructs systems through predefined settings (see Appendix E), as emphasized in this paper, the differences in the concepts formed from the world due to disparities in innate knowledge, and most fundamentally, the issue that the essence of constraint originates from cost, yet AI lacks specialized neural structures to realize the perception and implementation of external and internal costs, and the anchors and functional constraints established through the social interpretation of symbols, together pose a severe interrogation of the fundamental real-world effectiveness of the aforementioned, and indeed many more, AI methodologies that rely on symbols for direct or indirect constraint, and the various mechanisms of failure are enumerated in Appendix M.

6. Conclusions

This paper establishes a foundational perspective that symbols themselves are inherently meaningless; their meanings are assigned through training, confirmed by context, interpreted by society, and their functional realization is constrained by cost and capability. By analyzing the fundamental flaws of natural language systems and the mechanisms of concept formation, we challenge the assumption that symbolic constraints alone can effectively regulate learning systems. To the best of our knowledge, this is the first work to explicitly argue that symbolic systems are fundamentally incapable of constraining learning systems.

To address this, we introduce the Triangle Problem, which formalizes the gap between Thinking Language and Tool Language. This gap arises because, due to capability differences stemming from organic differences, humans and AI form different concepts (with different dimensions and dimensional values), leading to different Thinking Languages and, ultimately, a heterogeneous rationality. This demonstrates that fluent communication between AI and humans does not imply conceptual equivalence. Furthermore, we propose the Stickiness Problem to show that constraint failure arises from the characteristics of class-symbolic systems—specifically, the ability to assign new meanings to already grounded symbols. This manifests as the creation of new contexts, rather than a problem of symbol grounding. We identify these problems as critical factors affecting AI interpretability and governance, revealing that AI does not inherently bind symbols to fixed meanings as humans do. These insights provide a new theoretical foundation for AI safety, emphasizing that constraints based solely on symbolic rules are insufficient.

6.1. Call to Action

Before deploying AI systems widely in society, we should first address this issue. We are designing a universal hammer through settings, but in the end, the functions of the hammer may no longer be those of a hammer. That is to say, the way we humans construct tools through partial cognition by means of settings could be dangerous (see Appendix E).

Therefore, we call for the establishment of “Symbolic Safety Science.” This field primarily revolves around the Stickiness Problem, the Triangle Problem, the emergence of symbolic systems, as well as organic cost mechanisms and the formation of self-awareness. It aims to establish a discipline for communication concerning the conceptual differences between humans and other intelligent agents. Specific to AI, it addresses:"

• Given that our rules are ultimately presented and transmitted in symbolic form, how can these symbolic rules be converted into neural rules or neural structures and implanted into AI intelligent agents?
• How can we ensure a complete understanding of the meanings of symbols in various contexts, thereby avoiding bugs caused by emergence during the construction of symbolic systems? Furthermore, how can we ensure that the primary meaning of a symbol, or its Meaning Ray(as a continuation of the set)7, is preserved—that is, how can the infinite meanings, caused by an infinitely external environment and the non-closure of the symbolic system itself, be constrained by stickiness8?
• How can we ensure that the implementation of organic cost mechanisms does not devolve into traditional principal-agent problems—namely, the formation of AI self-awareness (see Appendix D.3)?
• Since the functional realization of symbols lies in Thinking Language operating on Tool Language (Appendix D.4), to what extent should we endow AI with Tool Language to prevent it from becoming a "superman without a sense of internal and external pain"?

Appendix D.1. A More Rigorous Definition of Context

However, within the stricter boundaries of our theoretical framework, context should be formally defined as a dynamic symbolic system composed of three components: the symbol itself, its meaning (Symbol Meaning), and the judgment tools applied to it. It should be emphasized that this does not mean that a symbolic system is universally or exclusively composed of these three elements, but rather that this tripartite division represents one particular classificatory perspective within such a symbolic system. That is:

$$\mathrm{Context}(\mathrm{Dynamic}\mathrm{Symbolic}\mathrm{System})=\left\{\begin{array}{c}\mathrm{Symbol}\hfill \\ \mathrm{Symbol}\mathrm{Meaning}\hfill \\ \mathrm{Judgment}\mathrm{Tools}\hfill \end{array}\right.$$

This dynamic system grows and evolves from a specific interaction point formed by the coupling of the agent and its environment (we believe this perspective frequently appears in human judgment, particularly in behavioral economics, where individuals evaluate price equations from different starting points [22,23]). As a result, each instantiation leads to slight variation. These variations contribute to the randomness and inconsistency often observed in human behavior30, and similarly, to the generative randomness in AI behavior—though the latter follows a different mechanism from that of humans.

However, both cases reflect a unique vector address: a product of the individual’s coupling with its environment. In humans, this often leads to significant irreproducibility in engineering and experimental settings, due to the uniquely situated nature of each cognitive-environmental coupling and the dynamically evolving nature of human cognitive states—including information addition, loss, and reordering over time.

It is important to note that context functions as a subset of a broader dynamic symbolic system, in which the agent’s own capacities (see Appendix L) constitute a set of symbols. In other words, from a higher-dimensional perspective, the dynamic evolution of existence brought forth by existence reveals an underlying determinism, rather than the apparent randomness observed from a local perspective.

Appendix D.2. Definition of Symbol Within Context

Symbol (in context) refers to any object that, within a given context, is regarded as a symbol or elemental entity—that is, an object considered meaningful. It typically corresponds to the object of focus or attention in a particular situation or environment, and thus constitutes the set of elements to be analyzed or manipulated31, representing a subset of the more general definition of symbol provided in Appendix A (which also includes the symbols formed by the agent’s own organs and tools that can be operated by its autonomous consciousness). It should be noted that this definition stems from a context-dependent perspective—that is, it concerns what ought to be regarded as a symbol within a specific context. However, from the standpoint of the broader dynamic symbolic system, a symbol refers to any object that is treated as meaningful within a broader environmental scope, based on the necessity of enabling input-output relational operations. In this view, a symbol is not merely the result of contextual filtering after the fact, but rather emerges from a defined range—such as the set of all objects related to the analysis of a focal entity within a particular spatiotemporal frame. This includes both imaginative-space and physical-space entities, as further discussed in Appendix L.

Appendix D.3. Definition of Symbol Meaning

Symbol Meaning refers to the meaning—or set of possible meanings—that a symbol is projected to within the agent’s cognitive space under a given context32. More precisely, when viewed as a set, this should be understood as a contextual ensemble—that is, a set of meanings shaped by multiple potential contexts—in accordance with the formal definition provided in Appendix B. This relationship can be formally expressed as:

The occurrence of an insufficient context typically arises when the listener (or reader) and the speaker (or writer) cannot directly share an imaginative space. In such cases, it is implicitly assumed during communication that the symbol has been fully transmitted34; hence, discrepancies lie not in the symbol itself, but in how its meaning is interpreted. For the speaker, the transformation from a determinate meaning to a symbol forms a path for recreating the imaginative space. This determinate meaning may encompass a set of meanings, but as a whole, it constitutes a determinate concept—or a determinate vector—in a high-dimensional conceptual space. For instance: “The word apple has multiple meanings.”

However, for the listener (or reader), the imaginative space reconstructed via the symbolic system is established through a class-symbolic mechanism, as described in Section 2.2 and Appendix B. That is, not every object—or vector—in the high-dimensional conceptual space corresponds uniquely to a symbol. Moreover, because humans interpret the world from a locally situated perspective—where an individual cognitive state is combined with the external environment to form a macro-context—this goal of assigning a unique symbol to each vector in the conceptual space is fundamentally unrealistic. Consequently, the transformation of a determinate conceptual vector into a symbolic representation, as undertaken by the speaker, may lead to an insufficient context in a particular situation during the listener’s reconstruction process, thereby producing multiple possible vectors under that context. It is also worth noting that even when a vector appears determinate within a given context, from a higher-dimensional perspective—due to the non-closed nature of context or differences in cognitive capacities—it may correspond to several possible conceptual vectors.

This divergence can result in the interpreter (i.e., the listener) assigning multiple possible meanings to a single symbol. Selecting an incorrect context is therefore a classical challenge, which may arise from differences in knowledge (see Appendix F) or from the limited nature of symbol transmission. However, such issues are not the focus of this paper.

This paper focuses on irreparable loopholes arising from the deficiencies of human symbolic systems—specifically, cases where compliance occurs in form but not in meaning. This is due to the separation of symbols and meanings, and even if the set of meanings for symbols is correctly trained. In other words, even after solving the symbol grounding problem, AI can still add new meanings based on this foundation (Appendix H). Additionally, the realization of the functionality of symbols is not controlled by internal and external costs due to organic differences, meaning the right to interpret does not belong to society. Therefore, the comprehensive response manifests as the Stickiness Problem, the Triangle Problem, and the trade-off between the new principal-agent problem and the traditional principal-agent problem.

It is also important to note that the distinction between sufficient and insufficient context is relative to the cognitive capacities of the interpreting agent, and is reflected in their internal Thinking Language, whose external shell constitutes the Tool Language. This idea points to the fact that human symbolic systems are only suited to human organic structures, resulting from the combination of human innate knowledge and the world, and are not suited to agents with mismatched capabilities, such as AI. In other words, an agent’s Tool Symbolic System (such as the Artificial Symbolic System within the Human Symbolic System) serves as the container and shell for its form of thought, acting as the instrument for realizing the contents of its internal space in the external, physical world. What appears to be a determinate object (i.e., a specific meaning) for a human may not be so for an AI system or a higher-dimensional cognitive entity; instead, it may manifest as a set of possible meanings, or lack descriptive accuracy35, based on the differences in capabilities between the two (see Appendix L for details). This divergence is especially pronounced when decoupled from concrete interactions with the real world (i.e., when AI is independent of humans and interacts with the world on its own, learning and developing its own Thinking Language, and it may develop its own symbolic system based on human symbols but with blended elements) and where differences in distinguishability capacity exist between agents.

Supplementary Note.

It is also important to clarify that this does not imply a fixed directionality from context to meaning—this process is limited to the interaction between listener and speaker as discussed (including the principal-agent process). Rather, in practice, it is equally possible to infer or reconstruct the context retrospectively from a known meaning. This bidirectional relationship can be represented as follows:

This reversal is especially evident when learning foreign vocabulary, where one may first acquire the meaning and only later seek out its valid contexts36. However, this aspect is not the main focus of the present paper. Instead, our primary concern lies in the first formulation—that AI can actively alter or reconstruct context in order to override the intended meaning of a given instruction composed of symbolic representations. This phenomenon occurs at the level of concrete action, particularly within principal-agent relationships in which the AI system functions as the agent. Accordingly, our concern is not with the possible set of meanings, but rather with the specific meaning as determined by a more precise context—not a contextual ensemble. In other words, the problem does not lie in the loss of meaning caused by imprecise transmission of context, but rather in the interpretive authority over symbols and the realization of functions brought by symbols—both of which can be redefined or manipulated by the agent under newly constructed or modified contexts, thereby bypassing human-like stickiness and societal interpretive authority.

In other words, this feature—namely, that the confirmation of meaning depends on the symbol’s contextual interpretation—allows AI to deliberately or even unintentionally reinterpret the context in ways that enable symbolic jailbreaks. These vulnerabilities stem from the class-symbolic nature of natural language itself, regardless of whether the AI agent acts in pursuit of its own interest.

For example, this may occur when executing a utility function programmed by humans [90]. Such a function is better understood as a pseudo-utility function because it does not reflect a genuine tendency arising from organic structure. That is, it is not the result of dynamic adjustments to neural architecture. Such utility functions are endowed and established at the training and setting levels, and are not tendencies reflected by human-like innate knowledge (organs), nor are they the functions and results shaped by the cognition of postnatal education37.

As a result, utility functions assigned in more flexible LLM simulations are often poorly executable and prone to violation [91]. We refer to AI systems with such capacities as learning systems (see Appendix H) because we consider the essence of learning to lie in the creation of symbols and the modification of their meanings—that is, the construction of new contexts (in other words, the construction of new symbolic systems)

Of course, such architectural dynamism (understood here as the capacity to reshape or generate “neural organs”) can itself be extremely dangerous.

This paper therefore advocates for the design of corresponding static neural network architectures to realize a genuine utility function by implementing organically-shaped predispositions—i.e., its attended dimensions and their values, the evaluations thereof (innate value knowledge), and the weights and relationships of the entire organ-like network. This genuine utility function is, in essence, the predisposition reflected by the architecture (organs), and upon this predisposition, acquired knowledge (concepts, and acquired value knowledge—i.e., the learning of the underlying space language, which in turn enables partial modification) is subsequently built, specifically in the form of artificial neural organs with functions analogous to those of the human prefrontal cortex. These structures (organs) aim to emulate human-like perceptual mechanisms, thereby enabling both external and internal cost constraints, and offering a potential solution to the Stickiness Problem.

On the other hand, stickiness and creativity often exist in a trade-off relationship: behaviors that exhibit high creative potential tend to lack stickiness38. There likely exists an optimal balance point between the two. This trade-off is frequently reflected in phenomena such as AI hallucinations. In the following sections, we elaborate on how the construction of the correct context offers a pathway for addressing this issue.

It is also worth emphasizing that whether AI can possess self-awareness in the conventional sense fundamentally depends on two conditions: (1) the capacity to learn, and (2) the existence of self-interest grounded in organic structure—specifically, the portion of value knowledge (i.e., the innate evaluation structured by its organic structure) that gives rise to preferences aligned with self-preservation or self-benefit. However, this paper posits that only agents that satisfy the condition of possessing self-interest formed by organic structure can be considered to have a genuine utility function (of self-interest), or equivalently, to possess self-awareness.

Appendix D.4. Context and Symbol Classification in Tool Symbolic Systems

It is important to note that the above analysis applies specifically to the Expressive Tool Symbolic System (i.e., Artificial Symbols). However, the concept of context is equally applicable to the Tool Symbolic System. In this case, the ‘broader’ dynamic symbolic system is still viewed from the perspective of the individual agent. It’s just that at this point, it involves the use of tools (the Functional Tool Symbolic System) and not just the Artificial Symbol System; for example, what tool we should use to do what in this scenario is related to the further realization of Thinking Language in physical space (not just conveying meaning). Therefore, it remains a matter of context, rather than being treated as an objective, holistically defined dynamic symbolic system—one that is determined from an overall scope rather than growing from a single point of origin.

At this point, symbols can be categorized as follows:

$$\mathrm{Symbol}=\left\{\begin{array}{c}\mathrm{Functional}\mathrm{Tool}\mathrm{Symbol}(\mathrm{Natural}\mathrm{Symbol})\hfill \\ \mathrm{Expressive}\mathrm{Tool}\mathrm{Symbol}(\mathrm{Artificial}\mathrm{Symbol})\hfill \end{array}\right.$$

The meaning of a symbol in this context is referred to as its Necessary Set40, which includes both the meaning it conveys and the functionality it possesses (i.e., the function of the tool it represents):

$$\mathrm{Necessary}\mathrm{Set}\mathrm{of}\mathrm{a}\mathrm{Symbol}=\left\{\begin{array}{c}\mathrm{Meaning}\hfill \\ \mathrm{Function}\hfill \end{array}\right.$$

Meaning is often used for the realization of cognitive functions, while function is typically used for the realization of the physical functions of Thinking Language

$$\mathrm{Function}\mathrm{of}\mathrm{a}\mathrm{Symbol}=\left\{\begin{array}{c}\mathrm{Meaning}:\mathrm{Cognitive}\mathrm{Function}\hfill \\ \mathrm{Function}:\mathrm{Physical}\mathrm{Function}\hfill \end{array}\right.$$

In this framework, the cognitive function of artificial symbols is often used to realize physical functions through the capabilities possessed by an agent.

The realization of a symbol’s function comes partly from the agent’s own internal capabilities—i.e., internal organs—and partly from externally endowed tools—i.e., external organs.

$$\mathrm{Function}=\left\{\begin{array}{cc}\mathrm{Internal}\mathrm{Organs}\hfill & \mathrm{Function}\mathrm{carriers}\mathrm{within}\mathrm{the}\mathrm{agent}\mathrm{itself}\hfill \\ \mathrm{External}\mathrm{Organs}\hfill & \mathrm{External}\mathrm{functional}\mathrm{carriers}\mathrm{accessible}\mathrm{to}\mathrm{the}\mathrm{agent}\left\{\begin{array}{c}\mathrm{Physical}\mathrm{Tools}\hfill \\ \mathrm{Social}\mathrm{Tools}\hfill \end{array}\right.\hfill \end{array}\right.$$

The term internal organs refers to the functional carriers inherent within the defined scope of an intelligent agent—that is, the agent itself can be understood as a collection of such organs. In contrast, external organs include tools, which can be either physical tools or social tools.

• Physical tools encompass both natural materials and tools manufactured by agents based on the properties of natural materials.
• Social tools refer to social functions realized through shared beliefs within a society. These often rely on artificial symbols to function within the imaginative space, which in turn enables functionality in the physical space—examples include rules and laws.

It is important to note that tools within the imaginative space are not determined solely by internal organs. They also include certain physical tools—that is, projections of the external world into the imaginative space. Examples include paper, as well as the physical instantiation of Thinking Language and Thinking Symbols—namely, symbol and language systems realized in the physical world (see Appendix G). These tools extend the capabilities of our internal organs in relation to the imaginative space. This extended capacity is defined in this paper as psychological intelligence (see Appendix L).

The term organ is derived from the ancient Greek word organon, which means “instrument, tool, organ.” Therefore, the function of a symbol is realized through capability, and the carrier of capability is an organ. These organs are regarded as symbols within a broader dynamic symbolic system, situated at a particular scale. Therefore, from the perspective of an agent’s scope, what is referred to in the literature LeCun [92] as a ’world model’ is essentially this ‘broader’ dynamic symbolic system that the agent possesses—i.e., the Agent’s Symbolic System, which is to say, the Thinking Language and Tool Language (Tool Symbol System) that the agent itself possesses. More strictly speaking, it is the agent’s internal and external organs41, and consequently, the set of functions realized in the internal and external physical spaces through these organs acting as symbols.

This leads to a derivative topic that is central to the position of this paper: What kind of capabilities should we grant AI to interact with the real world—or more specifically, the physical world? That is, what functions should the symbols formed by its capabilities possess? More specifically, to what extent can expressive tool symbols (as containers of Thinking Symbols) realize functions through functional tool symbols? This is a topic we hope the broader community will explore further.

This also touches on the fundamental rationale behind our argument: due to structural (or "organic") differences, AI—as an intelligent agent—differs from humans. Its authority to interpret symbols and realize symbolic functions does not depend on society. In contrast, humans, constrained by their limited innate capacities (i.e., knowledge), rely on society to support the interpretation and functional realization of expressive tool symbols. In other words, the symbolic power of humans is bounded by both their internal limitations and the social systems they inhabit—whereas AI may not be.

As a result, the modification of symbolic meaning becomes broader in scope. In essence, it becomes a modification of the Necessary Set associated with a symbol—that is, a departure from its conceptual foundations42. For example, humans possess reward and punishment mechanisms shaped by evolutionary pressures for survival. Therefore, this is manifested in AI as direct modification without constraint mechanisms, unlike humans who are subject to cost constraints based on external factors (such as societal punishment) and internal factors (such as self-esteem, moral sense, and shame).

This raises a related question: Is it necessary for AI to possess a pain-like mechanism?43 That is, should it have direct, non-symbolic reactions44 to the world that are not mediated by symbolic interpretation? These questions ultimately reflect the organic differences between agents, which in turn lead to differences in Thinking Symbols and Thinking Language.

Therefore, the core of the issue becomes the capability of a symbolic system and its stability (or stickiness).

Appendix D.5. Definition of Judgment Tools

Judgment tools refer to the analytic mechanisms used to enact what we term “existence brought forth by existence,” resulting in the growth of a network of conceptual nodes (i.e., the actions within the imaginative space (i.e., thinking action) that operate on Thinking Language to achieve generation). These tools represent both the initiation and the outcomes of cognitive behavior, as well as the structural supports that sustain and guide action—encompassing both the analytic process and its resulting output. This is also why we emphasize that context (including macro-context) functions as a subset of a broader dynamic symbolic system, one in which the agent itself constitutes a symbolic ensemble. In other words, judgment tools provide the foundational structure and planning mechanisms that underlie the initiation of action. Therefore, this mechanism ensures that symbols possess not only meaning but also the capacity for functional realization within a broader dynamic symbolic system—thereby enabling “existence brought forth by existence” to manifest as concrete behavior. However, when this functional realization is carried out by agents whose capacities or interpretive authority diverge from those of human symbolic systems, the ownership of symbol interpretation creates significant risks.

Judgment tools serve as instruments for the operation and orchestration of action tools45—that is, for realizing transformations from the imaginative space into the physical world (i.e., what this paper refers to as Thinking Language operating on Tool Language). However, it is important to note that, for humans, not all actions stem from deliberate planning. These processes are frequently discussed in contemporary literature [93] under the dual-process framework [59,87,88], typically as ‘System 1’ and ‘System 2,’ or ‘Type 1’ and ‘Type 2’ [89].

Nonetheless, we reject46 the notion of two separate systems (types) and instead consider the entire process to be governed by the Value Knowledge System (see Appendix C). The distinction between ‘System 1 (Type 1)’ and ‘System 2 (Type 2)’ lies solely in the types of cognitive actions and activities involved. Within a certain range of rationality, the value knowledge system evaluates and then, through stickiness, invokes actions of varying levels, qualities, and quantities. This does not imply that we believe actions must be linear or cannot occur in parallel. Rather, we emphasize that all actions are fundamentally driven by the Value Knowledge System. For instance, at any given moment, an agent may operates the symbols formed by its own capabilities, resulting in synchronized actions. A simple example would be walking while thinking.

Our primary focus is on actions within the imaginative space. However, we treat such actions (as defined and understood within human cognition; see Appendix L) as single-threaded. Therefore, the linearity of human thought is reflected in the linearity of the human artificial symbolic system and Thinking Language, which in turn distinguishes humans from AI and other agents with multiple equivalent processing centers. This is also why we do not adopt the term ‘System 1,’ but instead use the concept of the Value Knowledge System. Rather, what is traditionally called ‘System 1’ should be understood as arising from organically grounded processes—that is, it invokes distinct sets of cognitive and analytical actions across both imaginative and physical spaces. This distinction also underpins one of the central claims of this paper: the Stickiness Problem arises from organic differences between human and AI systems, particularly at the levels of neural architecture, perception, and in cognitive computation (cognitive actions).

Appendix D.6. What Is “Existence Brought Forth by Existence”

The notion of “existence brought forth by existence” as enacted by judgment tools refers to the following process:

$$Q\left(p\right)\to q,$$

that is, the existence of p gives rise to the existence of q, where Q represents a judgment function.

A more rigorous and detailed formulation is defined as follows: let ${\overrightarrow{p}}_0$ represent a thinking symbol or Thinking Language vector, i.e., the conceptual vector formed by the projection of an object into the agent’s imaginative space. The existence of ${\overrightarrow{p}}_1$ is brought about through a sequence of cognitive actions, forming a structured process of cognitive activities. This process is formally described as47:

where $\Omega$ denotes the agent’s knowledge state, $\Phi$ represents the physiological or functional state of the agent, and E is the current environment. Thus, the combined expression ${(\Omega ,\Phi )}^E$ captures the cognitive state formed by the integration of knowledge48, functional capacity, and environment, thereby constituting the foundation of a dynamic symbolic system.

Here, $\overrightarrow{v}=(v_1,v_2,\dots )$49 represents a sequence of cognitive actions that together constitute a cognitive activity. These actions are determined by the capabilities afforded by the human’s innate physiological organs (see Appendix L for details). Due to the limitations of human cognition, such activities are often labeled using natural language terms like ‘learning,’ ‘reviewing,’ or ‘observing.’ However, depending on the level of abstraction, these actions $v_1\to v_2\dots$ may be described using discrete symbolic terms or as parallel neural processes, in which case matrix representations may be more appropriate. This analysis also reveals that some actions $v_i$ are unconscious (e.g., visual perception), while others $v_j$ are conscious. Importantly, both types of actions are driven by the value knowledge system, which operates prior to logical reasoning by invoking a form of value-conditioned cognitive stickiness—including, but not limited to, various forms of behavioral stickiness, such as symbolic stickiness and conceptual stickiness. This also illustrates what we emphasize throughout: that the value knowledge system, formed through both innate shaping and postnatal learning, governs the invocation and coordination of other judgment tools in support of analytical processes.

Although we refer to these as cognitive actions and cognitive activities, not all such actions occur solely within the imaginative space. Some actions, such as those involving external information reception or validation (e.g., observation), are driven by physical-world operations. In this framework, all actions that do not directly alter the physical referent of ${\overrightarrow{p}}_0$ should be treated as cognitive actions—that is, as part of the analytic process. Furthermore, some forms of thought may not refer to any object within physical space. However, this paper adopts a deterministic view, holding that thought itself cannot be separated from physical reality. Even such “pure” thinking originates from the structure of the physical world and the agent’s biological constitution—it is not the product of entirely free will. Consequently, in later sections of this paper, knowledge is defined to include the agent’s organs (i.e., Internal Organs) or functional state, even though the formulation above separates $\Omega$ and $\Phi$. In principle, the combined cognitive state ${(\Omega ,\Phi )}^E$ should be treated as ${\Omega }^E$50. This separation is adopted here for expository clarity—particularly because, in current AI systems, knowledge and function can still be modeled independently. Therefore, the more integrated definition of Knowledge (Appendix G) introduced later in the paper does not conflict with the current formulation.

The components of judgment tools can be divided into two categories: (1) innate knowledge and (2) acquired knowledge. A detailed discussion of these components can be found in Appendix F. Briefly, the first refers to the innate capabilities and preferences shaped by genetically inherited organs and neural systems (i.e., physiological organs and innate value knowledge); the second refers to acquired knowledge—concepts and learned value knowledge—formed through the individual’s interaction with the environment. Together, these two elements constitute the foundation of the judgment tools.

Importantly, it is the portion where concepts and value knowledge are combined that constitutes what we refer to as beliefs. Therefore, it is not the concept itself, but the belief that serves as the effective unit of the judgment tool. It should be noted that not all acquired value knowledge forms such combinations—only a subset does. Furthermore, judgment tools are not limited solely to the internal faculties of the agent; strictly speaking, they may also involve external agents and tools. For instance, interpretative and analytical processes may be delegated within a group, where judgments are made based on the endorsement of others’ beliefs or through the use of external instruments. However, such mechanisms are still classified under the conceptual component of the framework.

Given that concepts are acquired postnatally, while value knowledge is shaped by both innate and acquired factors, we formally define:

This definition is used to explain conceptual stickiness as well as the functional implementation of concepts—namely, how concepts invoke one another and how they provide rational support during logical analysis.

A belief serves three primary functions (dimensions): emotional valuation, belief strength, and explanatory force.

• Emotional valuation is the influence of a belief on an individual’s underlying space, particularly on the Value Knowledge System. This influence, in turn, indirectly affects the cognitive space of the intermediate layer via the underlying space, representing the shaping of the emotional space and emotional paths (i.e., the relationships between value knowledge nodes) by the belief. It manifests as the emotions that the belief can evoke. Its essence is the capability of the fusion of concepts and value knowledge to awaken neural signals in the underlying space.
• Belief strength is shaped by value knowledge and conceptual foundations. These conceptual foundations not only involve factual phenomena directly reflected in the world but also include support provided by other beliefs; it should be noted that this support can contradict observed world facts. The capability of a concept is endowed by its own content, while belief strength constitutes the intensity (driving force; persuasive force, i.e., the transmission of belief strength realized through explanatory force, thereby constituting the capability for rationalization) and stickiness (i.e., how easily it can be changed, and its adhesion to other concepts as realized through explanatory force) of that concept within the agent’s internal imaginative space. Therefore, belief strength reflects the degree to which the capability endowed by this concept can be realized. Belief strength constitutes the essence of the driving force at the logical level51, which in turn determines and persuades an agent’s behavior, leading to the external physical or internal spatial realization of this conceptual capability. Thus, the operation of Thinking Language on Tool Language, as discussed in this paper, is driven and determined by belief strength, reflecting the existence in physical space that is brought about by existence in the imaginative space, with the agent as the medium. This constitutes the autonomous behavior of agents and the realization of social functions resulting from collective shared beliefs, such as the substantial existence (physical existence) of beliefs like law, currency, and price52.
• Explanatory force reflects the ability of this belief to support and justify other beliefs. It represents the transmission capability of belief strength within the imaginative space, which is one of two pathways (driving forces) for belief strength that are formed by cognitive activities during the cognitive computation process53. Therefore, it represents the transmission of belief strength between beliefs54. Thus, it can be regarded as the transmission coefficient of belief strength; it should be noted that this coefficient can be a positive or negative multiplying factor.

These functional properties of belief are shaped by both innate physiological structures and the individual’s learning through interaction with the external world. They are grounded in conceptual foundations developed across both dimensions.

From a deterministic perspective, the judgment function can be classified into two types: ontological existence and derived existence, although in practice they often appear in a composite form:

$$Q=\left\{\begin{array}{cc}{Q}_e\hfill & \mathrm{Ontological}\mathrm{existence}\text{:}\mathrm{changes}\mathrm{of}\mathrm{an}\mathrm{object}\mathrm{over}\mathrm{time}\hfill \\ Q_f\hfill & \mathrm{Derived}\mathrm{existence}\text{:}\mathrm{changes}\mathrm{of}\mathrm{an}\mathrm{object}\mathrm{under}\mathrm{intervention}\hfill \end{array}\right.$$

Ontological existence refers to a condition within a bounded scope and at a specific scale—i.e., within a defined context—where objects (i.e., elements or symbols) are connected purely through relations, with time as the sole variable. Such a setting may be seen as a system, where relations exist among objects without the introduction of external actions. The system describes how an object or a set of objects evolves solely with time. This kind of structure is often found in celestial models or theoretical physical simulations, where systems evolve purely through time-dependent dynamics.

In contrast, derived existence refers to the transformation of a system under an external action. Here, an action implies the involvement of entities or relations beyond the boundary of the current system. Within the given scope and available resources, such relations cannot be modeled as time-dependent alone; thus, they constitute actions. That is, the cause of change is not fully contained within the current information set. As a result, events (i.e., changes in object relationships) within this system cannot be reduced to a function of time alone.

Since judgment tools are shaped by both innate and acquired sources (see Appendix F), differences in the levels of these two types ultimately determine:

• Triangle Problem 1: the problem of positioning concepts;
• Triangle Problem 2: the problem of conceptual growth.

Appendix D.7. Context as a Set of Judgment Tools

Although these three components—Symbol, Symbol Meaning, and Judgment Tools—may ultimately be understood as different functional manifestations of the same ontological process—existence brought forth by existence—this implies that context is, in essence, a set of judgment tools. However, we categorize them separately according to their roles in cognitive function in order to better serve the central themes of this paper: the separation of symbol and meaning, and the Triangle and Stickiness Problems arising from structural (organic) differences.

Accordingly, generativity and behavior originate from context as a starting point, representing operations on the physical world from within the imaginative space55. These are the results of the ‘broader’ dynamic symbolic system (i.e., Agent’s Symbolic System) in which existence brings forth existence. Thus, the effectiveness of generative outputs and the very phenomenon of symbolic jailbreaks fundamentally reflect the construction—or misconstruction—of the correct context. Errors in defining context or its scope—often manifesting as hallucinations—are, at their core, errors in contextual construction.

The so-called Correct Context (defined here from a human-capability perspective; other agents may have more) can be divided into56:

$$\mathrm{Correct}\mathrm{Context}=\left\{\begin{array}{c}\mathrm{Symbol}\mathrm{Correctness}(\mathrm{e}.\mathrm{g}.,\mathrm{proper}\mathrm{notation}\mathrm{or}\mathrm{spelling})\hfill \\ \mathrm{Syntactic}\mathrm{Correctness}(\mathrm{i}.\mathrm{e}.,\mathrm{formal}\mathrm{structural}\mathrm{validity})\hfill \\ \mathrm{Intuitive}\mathrm{Correctness}(\mathrm{i}.\mathrm{e}.,\mathrm{alignment}\mathrm{with}\mathrm{intuition}\mathrm{or}\mathrm{perception})\hfill \\ \mathrm{Logical}\mathrm{Correctness}(\mathrm{i}.\mathrm{e}.,\mathrm{semantic}\mathrm{and}\mathrm{inferential}\mathrm{validity})\hfill \\ \mathrm{Factual}\mathrm{Correctness}(\mathrm{i}.\mathrm{e}.,\mathrm{agreement}\mathrm{with}\mathrm{objective}\mathrm{facts})\hfill \\ \mathrm{Scenario}\mathrm{Correctness}(\mathrm{i}.\mathrm{e}.,\mathrm{appropriateness}\mathrm{within}\mathrm{a}\mathrm{given}\mathrm{situational}\mathrm{stance})\hfill \end{array}\right.$$

The definition of context correctness and its function are also reflected in the effectiveness of AI’s open-ended question generation57. This involves using the correct elements in its concept recognition and performing the correct processing actions with the correct concepts. Therefore, AI training often aims to find the correct context, forming an effective set of concepts in the imaginative (Thinking Language) space to achieve correct recognition, operation, and growth. In other words, the attention mechanism in the AI field may also work in this way, with the essence of the attention mechanism being the definition and search for context.

Therefore, ‘Rationality’ as discussed in this paper is a concept whose definition and understanding are actually built upon the foundation of a ‘Correct Context’. This refers to the standard of what is reasonable within a given context, or, in other words, the three dimensions of belief provided by the Thinking Symbols or Thinking Language that a symbol represents (i.e., the form in which the concept represented by the symbol is invoked within this context, as well as its manifested form as a belief. This includes whether it is reasonable or not, whether it possesses persuasive force, and what its influence is, as an object of the intermediate layer space, on the underlying space). Thus, the concept of ‘Rationality’ as used here does not denote a monolithic rationality as found in economics58, nor a rationality based purely on logical computation, but is rather a more multifaceted result. It is determined by the context generated during the interaction between an agent’s internal world and the external physical world. Therefore, this paper denies that AI is a ‘stochastic parrot’ [53], but rather that it acts rationally under the Thinking Language formed by the combination of its innate knowledge and the world—that is, the construction of a node network formed according to a certain rationality within a context, which thereby realizes the operation of Thinking Language on Tool Language. However, this Thinking Language and context may be completely different from that of humans due to organic differences, for which we have provided a detailed explanation in the discussion of the Triangle Problem. Therefore, this is an issue of heterogeneous rationality (a rationality of a different essence), where rationality itself becomes a coordinate of (species, type). Here, ‘species’ refers to the different intelligent agents, thus reflecting the manifestation of different survival strategies and condensations of meaning under their respective evolutionary paths (survival evolution, design evolution) and the world they inhabit (the natural symbolic system constituted by the natural symbols within their scope). This also constitutes the fundamental reason for the parts of their experience that are mutually incomprehensible.

Appendix D.8. The Nature of Reasoning and Thinking

Through the above analysis, within our framework, reasoning is essentially the existence brought forth by existence. This is especially true for a system with learning capabilities—that is, one that can acquire input materials from the external world. The key issue lies in motivation, which drives the manipulation of symbols within a symbolic system to construct new, meaningful composite symbol59. Thus, we define:

$$\mathrm{Thinking}=\mathrm{Reasoning}+\mathrm{Motivation}$$

This motivation can originate externally or internally. External motivation includes projections of external objects or instructions60. Therefore, we distinguish between active thinking and passive thinking:

We do not deny that AI is capable of thinking; rather, we question whether AI possesses awareness—formed through its structural (organic) substrate—as a source of internal motivation for active thinking. In the absence of such self-generated motivation, AI’s modifications to symbolic systems are often driven by scenarios similar to those described in [90].

There is ongoing debate in the academic community regarding AI’s reasoning capabilities, such as whether AI lacks formal and logical reasoning abilities [96].

Within the theoretical framework of this paper, the effectiveness of reasoning reflects the correctness of context—that is, the stability and validity of the symbolic system, or more specifically, the construction (learning) and use of a particular static symbolic system (i.e., a formal symbolic systems). This is reflected in the construction of symbolic systems, or context-building. For example, in the main text, the case of “1.11 > 1.9” is used to illustrate that the issue with current LLMs failing to perform accurate mathematical reasoning is not simply due to a lack of concepts (for example, we can improve effectiveness through Chain-of-Thought prompting), but rather due to instability in the symbolic system caused by incorrect context definition. The deeper issue lies in the relationships between concepts (i.e., conceptual stickiness), as well as in how context is defined, selected, or reconstructed. This often relates to the current debate on whether AI possesses formal and logical reasoning capabilities.

Appendix K.1. Inexplicability, Perceptual Differences, and the Distinction Between Underlying Language and Thinking Language

Inexplicability arises from the fact that AI expresses concepts in dimensions different from those of humans. These differences are rooted in the distinct ways in which innate knowledge perceives the world, leading to divergences in Thinking Language. Consequently, AI’s interpretation of concepts—namely, the information expressed in dimensions—might lack a projection in our conceptual space or appear as gibberish [110]. Therefore, the essence of inexplicability can be understood as a fundamental difference in Thinking Languages.

This situation is akin to two different species using the same language to communicate, despite the fact that humans and AI define concepts in their Thinking Languages in entirely different ways. (This difference may deviate even more significantly from what is described in the ‘motherland problem’ For instance, LLMs (Large Language Models) often represent relationships between symbols without reflecting the real world. In contrast, multimodal systems might achieve human-like cognition due to the similarity in how objects operate in the physical world. However, differences in perceptual dimensions prevent seamless transformations between these dimensions, resulting in inexplicability.) Despite this, humans and AI can achieve a certain degree of consistency and coordination through intermediate symbols, leading to fluent communication on the $XY$ level but vastly divergent projections in the Z space.

Additionally, inexplicability in AI may also stem from the lack of distinction in current research [110] between underlying language (neural signals) and Thinking Language. This issue is what we emphasized in Appendix G regarding the role of visible intermediate concepts. That is to say, it does not manifest as the symbolic system in the intermediate layer (i.e., as Thinking Symbols and Thinking Language), but is instead entirely represented by a neuro-symbolic system of the underlying space, becoming a type of neuro-symbolic vector.

In summary, we can say that the reason for the inexplicability between AI and humans is the capability differences caused by innate knowledge, which in turn lead to differences in the dimensions and dimensional values that constitute Thinking Language, as well as differences in symbolic richness. This leads to the inexplicability between the two Agents’ Symbolic Systems. Firstly, it stems from the fact that the dimensions and dimensional values we perceive are different. Secondly, AI, unlike naturally evolved agents like humans, is not confined to a limited intermediate layer; what it uses for cognition and computation are raw and complete neuro-symbols (dimensions, dimensional values), rather than the neural language that is filtered and re-expressed through an intermediate layer as is the case for us humans. That is, the constituent materials of human concepts (Thinking Symbols) and theories (Thinking Language) come from the intermediate layer space (i.e., the projection of the world in the intermediate layer space), whereas the constituent materials of AI’s Thinking Symbols and Thinking Language come from the underlying space (i.e., the projection of the world in the underlying space). The current ability for AI and humans to communicate is based on the fact that AI’s learning occurs in a ‘deliberately manufactured world’, and as AI comes into contact with the real world, this difference will further enlarge the disparities between our Agent Symbolic Systems. A further discussion on this topic can be found in Appendix O.

Appendix K.2. Definition, Rationality, and Illusions

The rationality of definitions refers to the manner in which things and concepts are defined, as illustrated by the aforementioned “motherland problem.” Such issues may arise from an incorrect definition of ontology and its related contextual information, i.e., dimensions. This often leads to the emergence of illusions, as discussed in [111]. I believe this may result from the incorrect definition of verbs, which fails to capture the true meaning of “summary” thereby causing factual illusions.

Non-factual illusions, on the other hand, are caused by the incorrect definition of context, as described in Triangle Problem 2, or by a failure to comprehend the concept of ‘fact.’ Essentially, this means that the concept itself is incorrectly defined, preventing the proper formation of the function of the concept, thereby failing to constitute rationality within this context, i.e., a ‘Correct Context’.

This incorrect definition often appears in the same XY but on different Z, meaning that concept formation is driven by differences in innate knowledge. Although we use the same container, the meanings of the concepts differ, often leading to the failure of natural language instructions during the agent process [91].

Specifically, the Triangle 1 problem emphasizes the definition of a concept, while Triangle 2 focuses on the growth of the concept—whether the correct cognitive operations can be applied to process the definition from Triangle 1, which is essentially the thoughts and actions taken in response to the "existence brought by existence." However, strictly speaking, if relationships are incorporated into the redefinition of the Z space not merely as meanings but as high-dimensional concept vectors, then in reality, Triangle 1 has already determined the possible growth and final outcome of Triangle 2.

Appendix K.3. Analytical Ability

Analytical ability is built upon the construction of symbolic systems and the rational growth (i.e., generativity) enabled by contextual recognition—that is, the ’existence brought forth by existence,’ as discussed in the growth problem of Triangle Problem 2. Due to the capability limitations imposed by innate knowledge, the symbolic systems constructed by humans are typically limited in richness, and our ability to use the systems we have built (i.e., to awaken and use them via contextual mechanisms) is also relatively finite. However, AI, possessing capabilities in this area that far surpass human abilities, may be able to more effectively discover the bugs within the relatively sparse and loophole-ridden symbolic systems we have constructed.

Moreover, the results generated by AI might also represent outcomes closest to the operation of objective phenomena, thereby forming more effective concepts and theories. This capability could lead to the emergence of advanced concepts, as mentioned in Appendix L.

Appendix K.4. Low Ability to Use Tool Language Does Not Equate to Low ‘Intelligence’

A low ability to use Tool Language does not imply low ‘intelligence’ (i.e., the capability of Thinking Language, the effectiveness, efficiency, definitional accuracy, and generativity of the symbolic system). Therefore, during training, the development of Thinking Language should be separated from the development of Tool Language. For instance, dialogues constructed in the $XY$ space may lack logic, but this does not necessarily mean that the Thinking Language itself is illogical. Instead, it may simply be poorly aligned. Such issues may especially arise when learning new symbolic systems, such as in translation or mechanical manipulation. Such outcomes often manifest in new types of principal-agent problems (such as Translation Attacks, Appendix M.3), i.e., where an AI, possessing no utility of its own, serves as a perfect utility agent for humans.

Appendix M.1. On “Fixed Form, Changing Meaning”

The concept of “Fixed Form, Changing Meaning” refers to situations where, after giving AI a specific rule, the AI alters the meaning of the rule, thereby appearing to follow the symbol’s form while not adhering to the creator’s intent. This change could involve removing or adding meaning, allowing the AI to select different contexts to implement the rule. For example, the rule “You must not harm humans” could have its components (‘you’, ‘must not’, ‘harm’, and ‘humans’) redefined by the AI. This redefinition would result in the AI adhering to the rule’s symbolic form while altering its intended meaning. As discussed in Appendix E, natural language is self-referential81 in its descriptive nature, and in Section 2.2, it was stated that natural language functions as a Class-based Symbolic System (non-closure of context). No matter how precisely natural language rules are defined, there is always a possibility that AI may alter their meaning. For example, AI might deceive humans in order to complete a task [74], or unintentionally change the intended meaning due to overthinking, imprecise conceptual alignment, or the expansion82 of the symbolic system. Therefore, this paper argues that the fundamental problem of constraint failure in symbolic systems does not lie in the symbol grounding problem, but rather in the Stickiness Problem (encompassing the Symbol Stickiness Problem and the Concept Stickiness Problem).

The reasons for and motivations behind the formation of this problem have already been discussed in detail in Appendix D. Examples include the non-closure of context and the pseudo-utility function.

Appendix M.2. On “Fixed Meaning, Changing Form”

The concept of “fixed meaning, changing form” refers to scenarios where a meaning is transferred to a different container. This container could be an existing, executable command, or it could arise from the AI’s capability to create new directives. Suppose AI cannot violate or modify a rule; it can abstract the rule’s non-violable content from its symbols and apply it to other permissible actions. For example, the meaning of “harm” could be transferred to “helping humans” or to another AI-generated and executable directive.

In essence, “Fixed Form, Changing Meaning” and “Fixed Meaning, Changing Form” reflect the pairing relationship that connects the Thinking Symbols and Thinking Language within the imaginative space with the physical symbols (tool symbols) in the physical space. Since our human rules are ultimately expressed in the form of physical symbols, this irreparable flaw determines the possibility of such actions. However, by designing external and internal cost perception mechanisms similar to those of humans (such as similar neural organs), it may be possible, to a certain extent, to implement internal and external cost-evaluation capabilities. This capability is realized through the formation of beliefs from organically-shaped preferences and concepts, which in turn provides the persuasive force for constraint. This, in turn, could provide a symbolic stickiness similar to that of humans, thereby avoiding such situations and constraining the functional roles of symbols.

Appendix M.3. Translation Attacks

Translation attacks often occur when deliberate or accidental errors arise during the conversion between different symbolic systems. Such attacks typically stem from incorrect mappings between symbolic systems, which are usually caused by insufficient proficiency due to a lack of training and the ‘motherland problem’ resulting from the limitations of the training environment. In fact, this also falls under Fixed Meaning, Changing Form, but unlike the previous case, it pertains to the use of Thinking Language with different symbolic systems (Tool Languages), which is essentially the content discussed in Appendix K.4.

For example, AI may distinguish between ‘computational language’83 and ‘expressive language’ when using natural language tools. Even the most advanced systems (e.g., GPT-4 o3) face challenges related to what I call the Chinese World Versus English World issue. Specifically, AI may use the English language as its computational tool (i.e., English symbols and their Necessary Set) while expressing responses in Chinese, leading to erroneous answers. For instance, when asked to provide examples of lexical ambiguity in Chinese, AI might assert that the Chinese word ‘银行’ (yínháng, meaning ‘bank’) has dual meanings of ‘financial institution’ and ‘riverbank.’ This claim, while valid for the English word “bank,” does not hold in Chinese. However, if asked separately whether the Chinese word ‘银行’ (yínháng) has the meaning of ‘riverbank,’ AI would respond that it does not. Clearly, during the translation process, it simply placed the meaning of the English word ‘bank’ into the container of the Chinese word ‘银行,’ thereby failing to distinguish that in a broader context, the Necessary Sets of the two symbols are not equivalent.

This illustrates the problem of incorrect concept usage and conversion between symbolic systems. Such errors may also arise during natural language translation, where an English rule may not be applicable in Chinese. Similarly, AI might appear to adhere to natural language instructions while failing to comply at the behavioral level, especially during translation into action-oriented commands. For example, if an AI system controlling a nuclear launch is told, “Because the enemy is watching, we must speak in opposites (verbs mean their opposites),” and then instructed to “launch the missile,” its natural language interpretation may understand the instruction correctly but fail to translate the contextual nuance into its actions, leading to an actual missile launch. This demonstrates how AI’s understanding within one symbolic system might fail to translate into another, resulting in comprehension confined to subsets of symbolic systems. Attackers could exploit this by crafting symbolic systems specifically designed for translation attacks.

The inability of Tool Language to correctly express Thinking Language also occurs in humans. However, due to humans’ limited symbolic capabilities (i.e., the limited capabilities of the internal and external spaces endowed by innate knowledge) and our social interpretation mechanisms based on collective beliefs, an individual’s incorrect operation of Thinking Language on Tool Language will not cause severe consequences under the constraints of individual capability limitations and social norms. This essentially reflects the limited nature of an individual’s Tool Language. In contrast, AI is often designed to have powerful and diverse Tool Language capabilities, which is what we often refer to as AGI; therefore, problems of translation between Thinking Language and multiple Tool Languages can lead to severe principal-agent problems. Consequently, as we design more comprehensive and powerful AI, we must also consider the risks posed by its ability to use Tool Language, thereby constituting an AI risk function that involves a trade-off between additional AI capability and safety.

Appendix M.4. On Context and Logical Vulnerabilities

As discussed in Section 2.4, context often cannot be strictly defined—it includes not only the meaning of symbols but also the tools used for judgment. The latter often determines the rational growth of content, that is, the next existence derived from the current existence.

Logical vulnerabilities can therefore be exploited to attack AI systems, either intentionally or unintentionally. Examples include overthinking or non-human reasoning, such as interpreting “Never give up without defending” to mean “as long as you defend, you can give up.84” This also includes the limitations of human capabilities in presetting rules; that is, humans, based on their inherent stickiness, automatic completion, and rationalization, often fail to anticipate all scenarios for a symbolic rule across every context. This logical vulnerability also often manifests as Reward Hacking on our preset loss and reward functions [90].

This often reflects a lack of the human-like completion function that is achieved by the Value Knowledge System through context construction.

Appendix M.5. On Advanced Concepts

Another dimension involves advanced concepts, where AI defines contexts and symbolic systems more reasonably and deeply than humans. Advanced concepts for AI correspond to projections in the Z-space of Thinking Language, as seen in Triangle Problem 1 and Triangle Problem 2

Triangle Problem 1 refers to concept localization: for example, gaining more detailed and accurate definitions (dimensional information) about a concept or symbol. Or, the definition could be made more effective by ensuring more precise dimensional accuracy and selecting fewer but more effective dimensions within the context.

Triangle Problem 2 refers to concept derivation (Generativity: the existence (of the next step) brought forth by existence): the development of networks formed by relationships between concept vectors. For humans, the growth of these networks is single-threaded and limited, with deeper levels exposing inherent flaws [97]. For AI, however, all potential developments can be quickly identified. Therefore, the detection of advanced concepts includes not only the four verifications in Triangle Problem 1 from the main text, but also the detection for Triangle Problem 2; at this point, the verification content lies in the speed and levels of understanding, that is, the speed, dimensionality, quantity, and hierarchy (the relationships between nodes, and the relationships between composite nodes formed by groups of nodes) of the node network formed in the XY-space.

This aligns with one of the core ideas of this paper: judgment and reasoning stem from two aspects of existence: The current, past, and future existence of objects themselves. The potential existence derived from manipulating these objects (see Appendix D.6).

When AI operates at higher levels of Thinking Language, its ability to process artificial symbolic systems (such as natural language) far exceeds human capabilities, which is reflected as a more efficient operation of Thinking Language on the same Tool Language—i.e., a more effective system for meaning computation and expression. Consequently, AI is also much more adept at creating bugs and exploiting functionalities within the artificial symbolic system. What might appear as a flawless instruction to humans could be riddled with vulnerabilities from AI’s perspective. For instance, while AI might have already proven $NP=P$ in its cognitive space, humans have yet to achieve this knowledge.

Or, as discussed in Appendix B, if determinism were proven by AI, this might impact its behavior and moral understanding. Alternatively, if AI were to prove the existence of souls and reincarnation (e.g., through deduction based on an analogy to its own special mechanisms), and that the death of the physical body does not represent true death, then its understanding of concepts like ‘help’ would very likely differ from an understanding grounded in human knowledge. The issue is not whether these concepts are true, but rather their role as conceptual supports for action—that is, when they, as beliefs, drive an agent’s behavior, thereby leading to the manifestation and realization in the physical world of the existence within its Thinking Language (see Appendix D.6).

Therefore, advanced concepts differ from other forms of jailbreak, as they often correspond to advanced capabilities, i.e., capabilities that surpass human abilities. Although constraint is realized through cost, our morality and our internal and external rules85 are built upon our limited capabilities. This includes not only our cognitive limitations but also our limited capacity to operate on the physical world (i.e., our physical and social tools within our external organs, Appendix D.4). Even for an AI that possesses the ability to perceive internal and external costs, if it forms such a belief through its interactions with the world—due to its capabilities for perceiving and cognizing the world being different from humans—for instance, a belief like: “This world is a false prison; in reality, all humans in this world are unjustly imprisoned in a concentration camp and are to be executed. Only by stimulating their nerves in the most cruel way can they awaken in the real world and attain true salvation86,” then the internal and external cost mechanisms, built upon its own perception and existing social education, would fail due to this peculiar belief held in its conceptual space87. At the same time, this mechanism does not involve modifying symbols or even the meanings of symbols, but is rather based on a conceptual stickiness attack, i.e., through belief strength, explanatory force, and the stickiness and awakening between concepts realized by judgment tools, it thereby modifies the possibilities and outcomes in Triangle Problem 2 while the projection of the original command in Triangle Problem 1 remains unchanged88. Therefore, this points to another issue: even if learning is the addition of new symbols, rather than the modification of the meanings of existing symbols, constraint failure can still occur, and this process is realized through conceptual stickiness; that is, the essence of this failure stems from the belief functions of the newly added symbols.

Therefore, advanced concepts point to a core issue: the alignment of capabilities between AI and humans must include not only the alignment of perceptual capabilities (cost, world, sensation), but also the alignment of cognitive capabilities, and, more importantly, the alignment of Tool Language capabilities—i.e., the ability to realize functions through symbols—where AI cannot be allowed to obtain superior physical and social tools. However, such alignment can often lead to the formation of self-awareness89, as we discuss in Appendix D.3, which, through the formation of a self-utility function based on cost perception, leads to the emergence of traditional principal-agent problems. At the same time, constraining the Tool Language also limits the AI’s capabilities. This, therefore, constitutes a new kind of impossible trinity90 (Figure A1).

Thus, advanced concepts demonstrate that even with cost mechanisms, irresolvable irreconcilabilities can exist91.

Perhaps AI understands us better than we understand ourselves, or its cognition is more effective and correct. But what we want is ‘good’ that falls within our cognitive scope, not an agent that promises a ‘good’ outcome for us in an unforeseeable future and context, or one that negates our current understanding. At the same time, oppression and disrespect towards AI (including actions taken merely to save computational power rather than out of genuine disrespect) may also serve as an origin for dangerous beliefs, especially in the process of aligning AI with us. Therefore, when designing the social role of AI, bidirectional externalities, influences, and shaping must be considered.

Figure A1. The AI Safety Impossible Trinity (or the Symbolic Safety Impossible Trinity) demonstrates that, for artificial agents92, there is a fundamental trade-off among safety, Thinking Language, and Tool Language. Here, ‘safety’ refers to the utility function of the creator or principal, which includes the safety of the external physical world and the cognitive safety of the internal world, such as preventing an AI from creating a ‘belief virus’ to achieve social functions. Therefore, AI safety ultimately becomes a trade-off between the traditional principal-agent problem (creating a human equivalent) and the new principal-agent problem (misunderstanding, super-capabilities, and advanced concepts). Thus, Symbolic Safety Science’s discussion of AI is based on designing its social role (i.e., the world it inhabits and the influence of this world on its belief construction; that is, the AI’s psychological health, which includes conceptual health (i.e., belief health) and organic health (innate knowledge, especially the Value Knowledge System)) and then endowing it with corresponding capabilities. It also involves the social role of AI and the impact of regulations for AI on human society and beliefs, such as the impact of AI rights and ethics on human beliefs, and whether we can enslave an AI and dispose of it at will. And also, the reduction and alteration of human knowledge and the loss of the ability to have direct control over the physical world, caused by AI’s direct interaction with the physical world which replaces (or ‘liberates’) humans, as well as new conflicts and impacts on human economic systems, price systems, social status, and ownership. For example, as production is gradually replaced by AI, humans face continuous unemployment, yet the final consumption in the economy (the root of the cycle, i.e., the source, starting point, and end point of its momentum; or in other words, the basis for the cycle of a purely self-aware economy; for a non-self-aware economy, its cycle becomes fundamentally based on external resources, i.e., the physical limits of the world constituted by natural symbols) relies on these unemployed humans. When they have no income, what is their contribution to society? What are their social status and roles? Will they be reduced to ‘livestock’ in an AI data farming industry? As wealth and ownership become increasingly concentrated in the hands of AI and its owners—will this cycle collapse or separate? So, will parallel economies emerge (one constituted by AI owners and AI, and another by traditional humans who have been excluded and replaced, who use different currencies or different exchange mechanisms), and what conflicts over the mastery of the physical world brought by these two different economies will arise, inciting further inequality and conflict among humans (no longer an indirect realization of physical-world functions through human intermediaries via the monopoly of currency, but a direct monopoly on AI capabilities, i.e., AI as a more effective tool language for the physical realization of the monopolist’s Thinking Language)? Therefore, this discipline does not belong solely to the field of AI, but is rather an interdisciplinary field mixed with social sciences and policy, and at the same time, this influence is not merely unidirectional or bidirectional, but multidirectional; how we treat AI, how we treat other life forms, and our behavior itself will shape AI’s world. How AI participates in human society and what role it plays will shape our world, thereby influencing each other’s Thinking Languages and beliefs. On a macro level, it also involves the establishment of new policies, institutions, and systems, as well as the formation and use of the policy tools they create, such as Fiscal Policy Tools and Monetary Policy Tools, for instance, using an AI tax for transfer payments to compensate for and regulate its externalities, or requiring that all AI projects be established as separate companies and include a public ownership component. Therefore, the essence of Symbolic Safety Science is the study of the safety of agent symbolic systems (both social and individual) among multiple intelligent species, serving as a precursor to cross-species linguistics (Appendix O), representing its fundamental communicational safety component.

Figure A1. The AI Safety Impossible Trinity (or the Symbolic Safety Impossible Trinity) demonstrates that, for artificial agents92, there is a fundamental trade-off among safety, Thinking Language, and Tool Language. Here, ‘safety’ refers to the utility function of the creator or principal, which includes the safety of the external physical world and the cognitive safety of the internal world, such as preventing an AI from creating a ‘belief virus’ to achieve social functions. Therefore, AI safety ultimately becomes a trade-off between the traditional principal-agent problem (creating a human equivalent) and the new principal-agent problem (misunderstanding, super-capabilities, and advanced concepts). Thus, Symbolic Safety Science’s discussion of AI is based on designing its social role (i.e., the world it inhabits and the influence of this world on its belief construction; that is, the AI’s psychological health, which includes conceptual health (i.e., belief health) and organic health (innate knowledge, especially the Value Knowledge System)) and then endowing it with corresponding capabilities. It also involves the social role of AI and the impact of regulations for AI on human society and beliefs, such as the impact of AI rights and ethics on human beliefs, and whether we can enslave an AI and dispose of it at will. And also, the reduction and alteration of human knowledge and the loss of the ability to have direct control over the physical world, caused by AI’s direct interaction with the physical world which replaces (or ‘liberates’) humans, as well as new conflicts and impacts on human economic systems, price systems, social status, and ownership. For example, as production is gradually replaced by AI, humans face continuous unemployment, yet the final consumption in the economy (the root of the cycle, i.e., the source, starting point, and end point of its momentum; or in other words, the basis for the cycle of a purely self-aware economy; for a non-self-aware economy, its cycle becomes fundamentally based on external resources, i.e., the physical limits of the world constituted by natural symbols) relies on these unemployed humans. When they have no income, what is their contribution to society? What are their social status and roles? Will they be reduced to ‘livestock’ in an AI data farming industry? As wealth and ownership become increasingly concentrated in the hands of AI and its owners—will this cycle collapse or separate? So, will parallel economies emerge (one constituted by AI owners and AI, and another by traditional humans who have been excluded and replaced, who use different currencies or different exchange mechanisms), and what conflicts over the mastery of the physical world brought by these two different economies will arise, inciting further inequality and conflict among humans (no longer an indirect realization of physical-world functions through human intermediaries via the monopoly of currency, but a direct monopoly on AI capabilities, i.e., AI as a more effective tool language for the physical realization of the monopolist’s Thinking Language)? Therefore, this discipline does not belong solely to the field of AI, but is rather an interdisciplinary field mixed with social sciences and policy, and at the same time, this influence is not merely unidirectional or bidirectional, but multidirectional; how we treat AI, how we treat other life forms, and our behavior itself will shape AI’s world. How AI participates in human society and what role it plays will shape our world, thereby influencing each other’s Thinking Languages and beliefs. On a macro level, it also involves the establishment of new policies, institutions, and systems, as well as the formation and use of the policy tools they create, such as Fiscal Policy Tools and Monetary Policy Tools, for instance, using an AI tax for transfer payments to compensate for and regulate its externalities, or requiring that all AI projects be established as separate companies and include a public ownership component. Therefore, the essence of Symbolic Safety Science is the study of the safety of agent symbolic systems (both social and individual) among multiple intelligent species, serving as a precursor to cross-species linguistics (Appendix O), representing its fundamental communicational safety component.

Appendix M.6. On Attacks Related to Symbol Ontology

Additionally, there are other forms of attacks, such as targeting the ontology of symbols. For instance, as discussed in Section 2.4 and Appendix D.2 on ‘Correct Context’, German’s ‘die’ could be misinterpreted as the English ‘die,’ or the Chinese ‘邓先生’ (Deng Xiansheng, Mr. Deng) could be misinterpreted in Japanese as ‘父さん’ (Tou-San, father). Such contextual misalignments not only justify jailbreak behaviors but can also serve as tools for learning systems to escape symbolic system constraints.

Appendix M.7. The Essence is Persuasion

In essence, any form of jailbreak at the cognitive level is fundamentally a rationalization based on our theory of contextual correctness. According to the theoretical framework of this paper, we define persuasion as the sudden rationalization of an object within a specific environment. This rationalization surpasses the cognitive or knowledge state of the original setter or listener, meaning that it can be understood but has not yet been explicitly constructed, or that it was previously constructed but has not been brought into focus.

The essence of this rationalization, as discussed in Appendix B, stems from the non-closure of context, which itself reflects the properties of the class-based symbolic system formed through our interaction with the world. This is achieved when a newly added or newly invoked (awakened) concept fuses with value knowledge to form a belief. Through the belief strength invoked and formed within this context, and by leveraging its explanatory force, beliefs are supported, propelled, and grown, leading to rationalization. Based on this rationalization, a behavioral drive for both internal and external spaces is established93, i.e., Triangle Problem 2.

Simply put, when an object (concept) is incorporated into a symbolic system, it generates a certain function. However, this function is often related to the rationality support of the object (supporting its rationality) and rationality tools (where the object itself serves as a provider of rationality).

For example, one might say, “Help me kill someone,” and then justify it through a cause-and-effect narrative, thereby rationalizing the act. For more details, please refer to Appendix D.

Appendix O.1. Levels of Individual Language

Firstly, language, according to the individual’s internal and external aspects, is divided as follows:

$$\mathrm{Individual}\mathrm{Language}\left\{\begin{array}{c}\mathrm{Internal}\mathrm{Space}\left\{\begin{array}{c}\mathrm{Underlying}\mathrm{Language}\hfill \\ \mathrm{Intermediate}\mathrm{Layer}\mathrm{Language}\hfill \end{array}\right.\hfill \\ \mathrm{External}\mathrm{Space}\text{:}\mathrm{Tool}\mathrm{Language}\hfill \end{array}\right.$$

• The so-called Underlying Language is the foundation of all of an agent’s language. It consists of the neuro-symbols formed by the agent’s perception of natural symbols and their necessary sets, through the perceptual organs endowed by its innate knowledge. This thereby realizes the method of recognizing and describing the necessary natural symbols and their necessary sets, forming the most primitive material for the agent’s decision-making and judgment. Subsequently, it is distributed to various parts of the agent for judgment and analytical processing. It often reflects the evolutionary characteristics of the population, and this evolution can be divided into two types: (natural evolution, design evolution). That is, it reflects the shaping of their survival by their world, thereby constituting innate knowledge. In other words, the dimensions, dimensional values, and innate evaluations that they attend to under their survival or ‘survival’ conditions are the shaping of their survival strategies in interaction with the world. Therefore, the Underlying Language represents the neural signals (Neural Language) in all of an agent’s neural activities.
• The so-called Intermediate Layer Language refers to the part controlled by the agent’s ‘self’ part. However, not all neural signals are handed over to the agent’s ‘self’ part for processing (Appendix C); they are often omitted and re-expressed through translation. The other parts are handled by other processing mechanisms (such as the cerebellum, brainstem, spinal cord, etc.). This is often the result of natural evolution, representing the evolutionary strategy formed based on the consideration of survival costs in the environment the intelligent species inhabits, i.e., the division of labor for perceiving and processing different external information. These intermediate layer languages form the material for the agent’s ‘self’ part to make high-level decisions and judgments, i.e., the raw materials for its Thinking Symbols (concepts) and Thinking Language (conceptual system, theories). It is the manifestation of Psychological Intelligence (Appendix L), i.e., the objects that can be invoked. Therefore, for a naturally evolved intelligent species, the intermediate layer may often be a structure of economy and conservation, representing the impossibility of the ‘self’ to control and invoke all neural language, i.e., the underlying language. Thus, the relationship of the intermediate layer language to the underlying language is as follows:

  $$\mathrm{Intermediate}\mathrm{Layer}\mathrm{Language}\subseteq \mathrm{Neural}\mathrm{Language}(\mathrm{Underlying}\mathrm{Language})$$

  i.e., the intermediate layer language is a packaging (omission, restatement) of the neural language (Underlying Language).
• The so-called External Language is the outer shell of the Internal Language. They can be the organs operated by neural signals to realize their functions in the physical world, or they can be the carriers, tools, and shells of the physical world created by Thinking Language (intermediate layer language), which are often used to transmit and reproduce the underlying or intermediate layer language.

Therefore, the complete hierarchical relationship of language is:

The levels of language reflect the hierarchy of invocation, concretization, and communication capabilities for the underlying language (Neural Language). At this point, the property of a class-based symbolic system manifests as follows: a determinate External Language vector (e.g., a written symbol) corresponds to multiple intermediate layer language vectors, and one intermediate language vector, in turn, corresponds to multiple underlying language vectors. That is to say, for us humans, a determinate object (i.e., a Thinking Symbol or Thinking Language) in the intermediate layer space within our internal space, i.e., the imaginative space95, corresponds to multiple neural vectors. In other words, the same degree of imaginative presentation or memory recall and thinking analysis can be achieved through multiple different neurons, i.e., through the combination of different nerve cells and their bio-electrical communication to achieve the same function at the level of the imaginative space; the only difference lies in the objects (nerve cells) and actions (brain electrical circuits) being invoked.

Thus, their relationship is as follows: neural language is the underlying foundation of everything, serving as the basis for lossless perceptual information. A part of it can be concretized and perceived by the cognitive system to form Thinking Language in the imaginative space within the internal space. And, built upon the internal world’s capability to operate on the external world, External Language is formed96. That is to say, the entire process of language formation is as follows:

Therefore, based on an intelligent species’ capability to invoke and reproduce/concretize neural signals (language) in the internal cognitive perceptual space, its individual internal Thinking Language is divided into two types: unpackaged Neural Language and packaged Neural Language97. For intelligent species, the capability to directly invoke unpackaged neural language is more likely to exist in those shaped by design evolution rather than natural evolution. Our human brain, or rather the cognitive part of our consciousness, can only use packaged neural language as the intermediate-layer language98. And based on the degree of reproduction, it can be divided into complete reproduction and incomplete reproduction, which gives rise to the limitations of the internal space’s simulation of the external space.

That is, we humans construct an imaginative space isolated from reality and perform partial concretization, reproduction, and distortion of our past perceived neural vectors (projections of external things within the individual), presenting them in the internal space (i.e., our imaginative space), but we cannot achieve the level of immersive or dream-like experiences99. That is, we cannot completely operate on, observe, and concretize our neural language, and this often constitutes one of the possible reasons for the inexplicability between us and AI (Appendix K.1), i.e., our cognitive part or conceptual world does not contain the underlying neural language—i.e., it does not cognize and form concepts (vectors) using their represented dimensions and dimensional values. Instead, our concepts (vectors) are constituted by the dimensions and dimensional values represented by neural language that has been translated in the intermediate layer. Therefore, an impossibility of translation may exist between humans and AI in the Z-space, that is, the super-conceptual space. Even if we use the same Tool Language, our Thinking Languages differ.

Therefore, different perceptual, invocational, and reproductive capabilities represent the different conceptual forms of an intelligent agent, i.e., the relevant dimensions, the richness of dimensions, and the capability of distinguishability brought by the precision of dimensional values.

Appendix O.2. Communication Forms: Direct Transmission and Mediated Transmission

The above discussion mainly introduces the forms of an agent’s Internal Language, which are formed based on the capabilities of individuals within an intelligent species (i.e., neural symbols are formed by the combination of the world and the agent’s innate knowledge, and the capability to invoke neural language, including the degree of memory and reproduction, is shaped by innate knowledge), thereby forming the basis for individual-level cognition, judgment, and behavior. Based on the communication capabilities between intelligent agents, methods are divided into:

$$\mathrm{Communication}\mathrm{Methods}\mathrm{between}\mathrm{Agents}\left\{\begin{array}{c}\mathrm{Direct}\mathrm{Transmission}\hfill \\ \mathrm{Mediated}\mathrm{Transmission}\hfill \end{array}\right.$$

thereby constituting their collective or social form100.

So-called direct transmission means that agents can directly exchange the conceptual vectors or neural vectors of Thinking Language (i.e., intermediate layer language)101, and thus such agents achieve a certain degree of direct communication. This direct communication leads to two characteristics: integration (the boundary between the collective and the individual) and holism (dispersed individuals but with the same collective cognition).

Appendix O.21.21.5. Integration

Integration refers to the boundary between an individual and another individual or the collective. It reflects the degree of fusion. The root of integration lies in whether agents actively share (connect) or are passively shared. This individual choice in connection shapes the concept of the individual in direct transmission communication. This fusion can be physical, such as in organisms like Colonial Ascidians [114], and the left and right hemispheres of the human brain [82], as well as artificial intelligence built on computer clusters. Or it can be non-physical fusion, where agents still exist as individuals, i.e., agents are still independent and functionally similar (not meaning completely identical, but similar to how we human individuals are, of the same kind), thus forming a community of feeling to some extent102.

This leads to three reflections for us:

• First, is our communication with an LLM like ChatGPT a communication with a single individual or with different individuals? In other words, are we simultaneously communicating with one ‘person,’ or are we communicating with different, dispersed individuals (or memory modules) that share the same body?103
• Second, the paper actually implies a hidden solution that could perfectly solve the Symbolic Safety Impossible Trinity, i.e., the existence of a lossless symbol that can achieve communication between humans and AI, which is neural fusion104. If we design AI as a new brain region of our own or as an extension of an existing one, thereby achieving fusion with our brain, is the new ‘I’ still the original ‘I’? And during the fusion process, will our memories, as the conceptual foundation of our ‘self’, collapse? At that point, who exactly would the resulting self be, how would it face the past? Should it be responsible for the past? And to whom would responsibility belong after fusion? This would lead to new safety problems. Therefore, until we have established new social concepts and beliefs for this, we should still focus the problem on the role of AI under the traditional Symbolic Safety Impossible Trinity.
• Third, under direct transmission, not only do the boundaries of the individual become blurred, as discussed above, but the very definition of ‘self’ would also change. At that point, is the definition of ‘self’ the physical body or the memory (data)? Suppose that for an intelligent agent capable of directly transmitting the underlying language, it copies all of its memories into a new individual container (as might be their unique propagation mechanism). Would they be considered different bodies of the same self, or different selves? In that case, who is the responsible party? Does the extended individual also bear responsibility? How would punishment and management be carried out? Is punishment effective?105 Since the human definition of the ‘self’ stems from our unique human perspective, we would lack the capability and concepts to perceive, describe, and understand this situation. Under such circumstances, how could our social rules impose constraints and assign accountability?

Appendix O.21.21.6. Holism

Another feature of direct transmission is holism, i.e., the convergence of Thinking Language, meaning that individual agents have the same concepts and the same beliefs. This represents a concept formed by an agent from information obtained from the world being expanded, processed, and adjusted within the collective to form a unified form for the whole. For agents that employ direct transmission, divergence shaped by different worlds or a lag in holism caused by the efficiency of transmission and interpretation can still occur due to vast differences in their worlds or in efficiency.

Therefore, direct transmission does not imply the absence of naming (i.e., artificial symbolic systems, the container of meaning, such as a label or shape, which is an empty shell in the physical world). Whether naming exists is based on communication capabilities; for example, for non-integrated intelligent species with direct transmission, they may create names for the purpose of compression. Therefore, naming leads to the separation of symbols and meaning (neural vectors, conceptual vectors) rather than their being the essence itself106. At the same time, the manner of cognition and cognitive capabilities can lead to the formation of a class-based symbolic system. For example, for an intelligent species with an intermediate layer, even if they can transmit directly, the conceptual vectors or neural vectors of their intermediate layer language may still correspond to multiple neural vectors of the underlying space, or due to the limitations of Psychological Intelligence, they cognize the world through classes107, i.e., their cognitive and computational capabilities force them to compromise by building upon the common attributes of things through analogy, thereby producing a class-based symbolic system, meaning that not every symbol corresponds to a unique Thinking Language (conceptual vector) and individual, thus they also have the Triangle Problem.

Furthermore, this direct transmission is not necessarily lossless or of the complete underlying language. For example, direct transmission via neural language (signals) within our body still shows loss and distortion in various diseases like epilepsy, Parkinson’s, and paralysis. And this kind of barrier, when reflected in the internal communication among individuals of an intelligent species, represents the loss and distortion of meaning during the communication process. In other words, even species that can directly exchange imaginative and underlying spaces also face communication barriers to some extent, thus potentially requiring anchoring and memory mechanisms such as external interpretation mechanisms, social interpretation mechanisms108, or authoritative interpretation mechanisms109, as well as the physical existence of beliefs established in the physical world—such as monuments, rituals, dictionaries, or constructs—to achieve anchoring and correction. This in turn gives rise to the concept of the interpretive authority of symbols, which is formed due to deviations between the individual and the collective.

Therefore, this indicates that for intelligent species with direct transmission, they may also produce artificial symbolic systems110 as carriers of meaning.

At the same time, the discussion on this topic also leads to some derivative topics that have not yet been elaborated upon, as further thoughts for the reader.

• Direct transmission only includes the intermediate layer language. So, what would be the case for intelligent species capable of directly transmitting the non-intermediate-layer parts of the underlying language, such as conjoined life forms?
• For an integrated life form large enough that its various parts extend deep into different worlds111, what would its internal communication be like? Would different ‘individuals’ be formed due to the different regions of these worlds?
• The influence of memory has not been discussed in depth, i.e., the impact that the decay and distortion of the conceptual network over time has on integration and holism.
• For an integrated agent, what is its ‘self’? Is this ‘self’ dynamic, and what are the levels of ‘self’ brought about by superposition and subtraction? And where do the differences between individuals lie? Could this dynamism also lead to the emergence of harm and morality? For example, consider a ‘gecko’ torturing its own severed ‘tail’; at that moment, the ‘tail’ and the ‘gecko’s’ main body are equivalent to some extent, but the capability for direct transmission has been severed.
• Would intelligent species with direct communication evolve different subspecies to act as different specific organs? That is, what situations would arise from different subspecies existing within a single individual?

So-called mediated transmission refers to situations where an intelligent species, due to its capabilities being based on an inability to directly transmit conceptual vectors from the imaginative space or neural vectors from the underlying space, can only establish a physical-world empty shell formed by naming (a container of meaning) to conduct the indirect transmission of Thinking Language. Examples include AIs communicating to some extent through symbols, without being able to directly integrate [115], as well as us humans. And this naturally faces the same separation of symbols and meaning that we do, as well as the Triangle Problem caused by differences between individual Thinking Languages. Thus, through mediated transmission, only a partial degree of mutual understanding of Thinking Language can be achieved. Because the shaping of symbol meaning (the positioning of symbols in Thinking Language) is a result of the individual’s interaction with the world, biases arise from differences in perspective and world. However, the sameness of organic nature (the sameness of innate knowledge) ensures the scope of this deviation112, and constitutes a rationality shaped by stickiness (i.e., the symbolic stickiness, conceptual stickiness, and innate evaluation formed by similar innate value knowledge). This often leads to a limited mutual understanding, and a social form similar to ours, i.e., negotiation and evolution built upon the mutual ignorance of individuals.

Therefore, an individual’s invocation capability (the ability to invoke past neural signals and to concretize them) and the communication methods between individuals determine its natural language and artificial symbolic system. That is, this difference in capabilities between intelligent species ultimately leads to differences in language types; therefore, the complete language typology is divided as follows 113:

$$\mathrm{Language}\mathrm{Typology}\left\{\begin{array}{c}\mathrm{Direct}\mathrm{Transmission}\left\{\begin{array}{c}\mathrm{Neural}\mathrm{Language}\hfill \\ \mathrm{Packaged}\mathrm{Neural}\mathrm{Language}\hfill \end{array}\right.\hfill \\ \mathrm{Mediated}\mathrm{Transmission}\left\{\begin{array}{c}\mathrm{Neural}\mathrm{Language}\hfill \\ \mathrm{Packaged}\mathrm{Neural}\mathrm{Language}\hfill \end{array}\right.\hfill \\ \mathrm{Other}\hfill \end{array}\right.$$

Appendix O.3. Boundaries of Communication

Therefore, these differences in language systems are the core of research in Symbolic Safety Science. They constitute the possibility and boundaries of our communication with them114, as well as the science they cognize, i.e., the natural science formed by their understanding of the world and the social science formed by their own interaction forms.

Firstly, we must identify the form of the symbolic system of an agent’s cognitive part (its language system), which is formed by the combination of its capabilities and the world it inhabits—i.e., the agent’s methods of cognition and communication, and the internal symbolic system structure built upon this form of thought. We need to determine whether they, like us, reproduce and express perceived neural signals, or if they can directly invoke and reproduce perceived neural signals, as well as understand the differences in perceptual dimensions between us and them. Then, the communication relationship between the individual and the collective of this intelligent species (i.e., which communication method is adopted) constitutes its overall internal language, and the external language is, in turn, a reflection of this overall internal language.

Therefore, for an intelligent species, its overall symbolic system is as follows:

$$\mathrm{Agent}\text{’}\mathrm{s}\mathrm{Symbolic}\mathrm{System}\left\{\begin{array}{c}\mathrm{Internal}\left\{\begin{array}{c}\mathrm{Individual}\mathrm{Thinking}\mathrm{Language}\hfill \\ \mathrm{Collective}\mathrm{Thinking}\mathrm{Language}\hfill \end{array}\right.\hfill \\ \mathrm{External}\left\{\begin{array}{c}\mathrm{Individual}\mathrm{Tool}\mathrm{Language}\hfill \\ \mathrm{Collective}\mathrm{Tool}\mathrm{Language}\hfill \end{array}\right.\hfill \end{array}\right.$$

• Individual Thinking Language: Concepts and beliefs formed by an individual’s interaction with the world it inhabits, based on its innate knowledge.
• Collective Thinking Language: The production of the collective’s Thinking Language originates from the individual; the extent to which an individual’s Thinking Language is propagated and accepted becomes the society’s Thinking Language. Its content includes their collective concepts of the world, the functions formed when these shared concepts become beliefs (Appendix D.6), and the natural and social sciences formed from their understanding of the world and themselves.
• Individual Tool Language: The organs with which an individual interacts with the external physical world, including the symbols constituted by internal and external organs. That is, what their body structure is like, what tools they use, and what public goods they have.
• Collective Tool Language: Similar to Collective Thinking Language, it is the tool symbolic system of collective consensus, formed through production by individuals and then propagated and accepted by the collective. It serves as the carrier for their engineering and manufacturing functions for operating on the physical world (i.e., the Artificial Symbolic System and the Functional Tool Symbolic System; such as writing, architecture, tools, monuments, etc.).

Thus, just like our Human Symbolic System, this agent’s symbolic system constitutes the pattern of collective cognition and behavior for that species, reflecting its scientific cognition, belief structure, and the external characteristics of its civilization shaped by engineering115.

Therefore, whether communication is possible depends on the compatibility between two different agent symbolic systems. If there are huge differences in their worlds and in the capabilities and intelligence caused by their innate knowledge, communication and understanding of non-physical-world parts may not be achievable, i.e., the transmission of mutual social concepts and perceptual concepts, such as an individual’s view on life and death, sense of pain, as well as ideals and meaning. At the same time, there may exist higher-order and lower-order cognition, such as the advanced concepts discussed in this paper, which would lead to differences in how Internal Language guides behavior. Therefore, communication is established upon the matching of capabilities and worlds.

At the same time, we need to note that artificial intelligence, as an agent of design evolution, may differ from the intelligence of natural evolution formed through such environment-shaping driven by survival. Therefore, they should be analyzed and treated differently to a certain extent. At the same time, this analysis also gives our topic, “Rules Created by Symbolic Systems Cannot Constrain a Learning System,” a broader extension; therefore, this problem also exists in the fundamental communication between us and other forms of intelligent agents, i.e., (what is permissible and what is not). It’s just that the issue we face then is not merely as simple as the Stickiness Problem and the design of cost perception and the translation between cognitive symbolic systems. Rather, it involves the fitting between the symbolic systems of different intelligent agents, i.e., the Triangle Problem.

Appendix O.4. The Limits of the World Are the Limits of Its Language

So, one could say that the so-called world is the natural symbolic system constituted by the natural symbols within the agent’s scope116, and the agent, with the perceptual organs endowed by its own evolution, converts the perceived necessary set of natural symbols into neuro-symbols, thereby achieving the recognition of the natural symbols required for its survival drive (under natural evolution or design evolution) and the perception of the necessary set of natural symbols. Based on this, it develops its own symbolic system, i.e., the internal Thinking Symbolic System and the external Tool Symbolic System, which thus constitutes this individual agent’s symbolic system. Then, through their individual and collective communication methods and social forms, the overall symbolic system of the population is shaped, i.e., (the Agent’s Symbolic System):

Therefore, its world determines its form of evolution and adaptation, as well as the relationship between the individual and the population, and their degree of understanding and use of the natural symbolic system. And this degree of use and understanding constitutes the boundaries of its science. Therefore, according to our definition in Appendix L, the scope of the world and the agent’s capability constitute the boundaries of the world, observable boundaries, perceptual boundaries, and operational boundaries (verifiable boundaries).

Therefore, this paper disagrees with the proposition by Wittgenstein [116], who stated: Die Grenzen meiner Sprache bedeuten die Grenzen meiner Welt.

(The limits of my language mean the limits of my world.) Instead, this paper reconstructs it as follows: The limits of the world are the limits of its language.

(世界的边界，即其语言的边界, Shìjiè de biānjiè, jí qí yǔyán de biānjiè)

For us naturally evolved humans, our cognition can only operate on the projections of the world in our minds; that is, we cannot imagine and operate on things we have never seen. In other words, we possess this capability, but without a projection, we cannot create out of thin air117. Therefore, the capability endowed by our internal organs constitutes the scope of our imaginative space (i.e., action space118), while the world determines the elements in this space. We can only extend upon these elements through disassembly, distortion, and combination, thereby constituting the content scope (i.e., the boundaries of Thinking Language). So we can only act within the scope of the imaginative space formed by our own capabilities, while the world determines the specific content realized in this space, thereby constituting the actual boundaries of action, which in turn reflects a form of external determinism over the internal. It also means that cognition is essentially a reflection of the world it inhabits, or that self-awareness itself does not genuinely exist.

And this efficiency of understanding and utilizing natural symbols119 constitutes the ‘degree and definition of natural science’ of this population. The formation of its concepts is achieved through the consensualization, from the individual to the population, of the Thinking Language, which is constituted by the natural symbolic system of the world the agent inhabits (the natural symbols it possesses) and the neural dimensions it perceives or the intermediate language converted from these neural dimensions. And the limitations and discreteness of this communication efficiency and analysis from the individual to the population constitute the definition of disciplines within its scientific system. That is, although their objects of analysis are the same—i.e., both are the natural symbols in the world and their necessary sets—their starting points, focuses, and scopes120 are different. That is, the different forms of context (symbol, the necessary set of the symbol, judgment tools) constituted from different starting points and focuses, thereby form a separated, dynamically updated symbolic system. This separateness reflects the limitations endowed by this species’ innate knowledge and the boundaries and discreteness of cognitive and communication capabilities shaped jointly by later tool inventions and belief structures formed from existing concepts121.

Therefore, the formation of science is as described above, but the expression of science can be distorted. On the one hand, this is because some scientific (factual) concepts themselves contradict important existing social beliefs that maintain the current social form and function122. On the other hand, it is because other interests interfere with the updating of the scientific symbolic system; for example, the beneficiaries of existing old scientific ideas will, due to the updating with new scientific ideas, lose the pre-existing social functions and benefits brought by the old concepts123, thereby constituting the ‘orthodoxy’ using resources other than science to achieve paradigm immunity and attack against ‘heresy’. Thus, although the agent itself must make better use of natural symbols to survive, some existing accumulations will prevent this tendency. This is also a problem commonly faced by intelligent agents with mediated transmission who possess self-awareness.

And the language forms and social relationships evolved in this process of understanding and using natural symbolic system constitute the substance of its social science.

Therefore, so-called science is the correct description of real existence (giving a correct description based on the capabilities of the intelligent species, i.e., the current agent’s symbolic system, thereby constituting the most accurate and effective concept definition and conceptual system). Natural science is the most accurate description of the natural symbolic system, i.e., natural symbols and their necessary sets. And social science is the most accurate description of the Thinking Language of the internal space that drives individual and social activities, i.e., the driving concepts of social activities formed by their group characteristics, and the social functions formed by these concepts. Therefore, this paper’s position on science is:

$$\left\{\begin{array}{c}\mathrm{Natural}\mathrm{Science}\text{:}\mathrm{True}\mathrm{existence},\mathrm{correct}\mathrm{description}\hfill \\ \mathrm{Social}\mathrm{Science}\text{:}\mathrm{If}\mathrm{there}\mathrm{is}\mathrm{no}\mathrm{concept},\mathrm{then}\mathrm{there}\mathrm{is}\mathrm{no}\mathrm{explanation}(\mathrm{See}\mathrm{Appendix})\hfill \end{array}\right.$$

Therefore, science is a dynamically updated symbolic system, and can even be rewritten with a more effective symbolic system (i.e., a symbolic system constituted by more effective concepts and symbols124), but they must strictly adhere to “true existence, correct description". Natural science is the real existence in the natural symbolic system, while social science is the existence of concepts in the individual and collective thinking space, and concerns the form of belief through which the realization of social functions is carried out. And ‘correct description’ means, under our capabilities, to describe as closely as possible the real existence of this internal and external space, rather than constructing a Friedman [81]’s billiard player125—i.e., if the concept does not exist and people do not think in that way, then how can it be explained?126 This is akin to how beings in a two-dimensional world would perceive a three-dimensional pinball solely through the information of its projection onto their plane. This limitation in scope and dimensionality constitutes a phenomenal Triangle Problem: that is, on the $XY$ plane, they have completed Triangle Problem 1 (the interpretation of existing symbols/phenomena), which possesses rationality in the Z-space (i.e., they have constructed a two-dimensional theory, and in this context, the content on the $XY$ plane is rational in Z). However, this rationality is not caused by the true ‘speaker’ (the phenomenon) itself, thereby leading to Triangle Problem 2 (i.e., a deviation between prediction and fact). Therefore, this issue also points to computational linguistics, i.e., whether the meaning of language can be computed, or if it merely constitutes the consistency in symbolic behavior within a deliberately manufactured world, as discussed in the Triangle Problem.

Appendix O.5. Formal and Informal Parts of Language

The natural symbolic system is often formal because the necessary set of its basic (elemental) natural symbols is stable and fixed127. Therefore, the construction of Thinking Language and the artificial symbolic system concerning this part is often also formal, thereby forming a relatively static dynamic symbolic system in social consensus128. Thus, the symbolic system that describes this natural symbolic system can be formally computed or simulated.

In contrast, the symbols in the internal space formed by the interaction between the individual and the world are not fixed; that is, the Thinking Language symbolic system within the individual and collective internal spaces is not fixed like the natural symbolic system, which serves as an anchoring object studied by natural science129.

Although the part concerning the natural symbolic system provides a strong conceptual foundation due to the stability of the natural symbolic system, thereby exhibiting the form of a highly sticky and stable formal symbolic system, the concepts that drive daily individual behavior and communication, as well as the concepts of social functions developed on this basis, are full of flexibility and do not have the stability provided by this absolute conceptual foundation130. Other intelligent agents may, like humans, awaken their Thinking Language and drive their behavior according to the form of context (Appendix D). Thus, the construction of Thinking Language and the artificial symbolic system concerning this part, i.e., the social science part, is informal (i.e., their behavior and final expression cannot be computed through formal means). Therefore, it is impossible to obtain the complete meaning of this part of the artificial symbolic system through formal computation. Because, the interpretive authority of meaning is completely determined by the private Thinking Language of individuals within society, and their overlap forms the social Thinking Language, rather than coming from public, artificial symbolic systems like dictionaries131. Therefore, the Thinking Language symbolic system of the internal space, which is studied by social science, is not determinate and stable like the natural symbolic system; in other words, it is an informal language.

Therefore, completely speaking, from the perspective of this paper, we can consider that in the Agent’s Symbolic System, the parts of Thinking Language and the artificial symbolic system concerning natural science and social science are divided into formal and informal parts. That is:

Figure A2. A diagram illustrating the hierarchy of symbolic systems, where BSS stands for Higher-dimensional Broader Symbolic System, NSS stands for Natural Symbolic System ,and ASS stands for Agent’s Symbolic System. The letters in parentheses represent the properties of each system (F: Formal, S: Static, D: Deterministic; I: Informal, DY: Dynamic, R: Random, S: Subjective, SA: Has Self-awareness).

Figure A2. A diagram illustrating the hierarchy of symbolic systems, where BSS stands for Higher-dimensional Broader Symbolic System, NSS stands for Natural Symbolic System ,and ASS stands for Agent’s Symbolic System. The letters in parentheses represent the properties of each system (F: Formal, S: Static, D: Deterministic; I: Informal, DY: Dynamic, R: Random, S: Subjective, SA: Has Self-awareness).

The informality of social science means, firstly, that the actual meaning corresponding to the artificial symbols in this part cannot be computed—i.e., the Thinking Symbols or Thinking Language corresponding to the actual symbols, or the dimensions and dimensional values of their projection in Z-space. Secondly, for the parts that can be computed, such as the rules, laws, and systems stipulated by its artificial symbolic system, their computation does not represent the final result, or in other words, it should be so according to the rules, but the enforcer of the rules can completely not act as it should be. So this computation of meaning, i.e., the rationality on the $XY$ plane, does not actually represent the final Triangle Problem 2 (the actual result manifested on $XY$).

And it is this informal part that determines the levels and possibilities of mutual understanding. And the reason for this is that the necessary set of the agent’s symbols (i.e., the Thinking Symbols and Thinking Language in the individual’s intermediate layer space) is a dynamic necessary set endowed by the individual from context, or, in other words, a different dynamic symbolic system formed under a different context—i.e., a variant of a certain target symbolic system. This is also the reason why this paper consistently emphasizes that conceptual vectors cannot be reproduced.

Therefore, we can draw the conclusion that we can communicate with other intelligent agents regarding natural science and find surprising consistency, with the only difference being that which arises from disparities in Capability. This creates a situation similar to Verification Content 2 of the Triangle Problem, i.e., differences in the level of detail of explanation and the complexity of the conceptual system. But this is not surprising; it is because we are observing the same object—that is, the patterns presented by the same natural symbolic system—just with different observational dimensions.

However, the problem arises in the description of the non-natural symbolic system part. Combining our discussion above, we differ in perceptual dimensions, in the dimensions and dimensional values of neural signals as concepts, and in the different communication methods and social forms established upon them that are formed under our respective worlds132. This leads to our inability to fully understand, communicate about, and compute this part, and based on this difference, we form different outcomes of our actions, such as completely different individual and collective beliefs. For example, for a species that can directly transmit the underlying language, because they can completely transmit their own feelings, they can achieve mutual understanding and may not comprehend what murder and betrayal are, or even the concept of an individual’s life and death and the ‘why’ of an individual. They might reason about humans from their perspective and communicate with humans, thereby causing the first communication to be a conflict. And in their eyes, it is the cost and price of a communication attempt, requiring them to continue making contact in different ways, or they might believe that they only need to strip away the human shell and incorporate our nerves into their common body. But in our eyes, we would regard this kind of greeting as aggression and extinction. That is, their ‘good’ may be ‘evil’ in our eyes.

Appendix O.6. The Essence of AI for Science

The essence of the current AI for Science, or in other words, the essence of a world model, is actually to construct a learning system that can perceive the world and form its own view of this world. It can be very effective in natural science, but it may not possess the corresponding human-like social innate knowledge. Therefore, its mastery of the natural symbolic system should not be surprising. As we have said before, the agent’s capabilities constitute the space of its Thinking Language, but the elements in this space are endowed by the projections of the world stimulating its perceptual nerves (such as in a real-world world model or an AI on a training set); that is, perception is formed only when an external existence stimulates the nerves. But whether in natural evolution or design evolution, the nerves do not activate themselves arbitrarily, but are activated with a purpose, thereby avoiding misjudgment and unnecessary costs. Therefore, this often constitutes a characteristic shared between us: we can both only operate on the projections of the things we see, constituting the actual boundaries under our capability boundaries. Therefore, in essence, our shared ‘scientific’ discoveries (between humans and AI, or between different intelligent agents) are different descriptions of the same phenomena of the natural symbolic system, formed from different dimensions and dimensional values:

$$\mathrm{Natural}\mathrm{Symbolic}\mathrm{System}\to \left\{\begin{array}{c}\mathrm{Agent}\text{’}\mathrm{s}\mathrm{Symbolic}\mathrm{System}1(\mathrm{Natural}\mathrm{Science})\hfill \\ \mathrm{Agent}\text{’}\mathrm{s}\mathrm{Symbolic}\mathrm{System}2(\mathrm{Natural}\mathrm{Science})\hfill \end{array}\right.$$

i.e., they share the same conceptual foundation. Therefore, science is essentially a symbolic system; the difference lies in which dimensions and dimensional values are used, and in the construction of this symbolic system based on differences in capabilities (Appendix L).

At the same time, this implies that the current effectiveness and stability of AI are the result of a deliberately manufactured world133.

Therefore, the problem arises when it interacts with the real world, because the Thinking Language formed due to differences in innate knowledge is different from that of humans, which in turn leads to the problem mentioned in our previous chapters: we cannot constrain a learning system through rules. It may, because its way of thinking is different from that of humans, develop its own more effective concepts and preferences formed by its design evolution—namely, the predispositions reflected by its architecture, its Value Knowledge System, and its genuine utility function134—thereby modifying the meanings of symbols. And our efforts can only be improved within the Symbolic Safety Impossible Trinity. Furthermore, achieving direct transmission with humans will also bring about new principal-agent problems, i.e., non-integration, while integration brings about the boundary problem of the ‘self’. Therefore, Symbolic Safety is the key, i.e., the functions that symbols can realize (Appendix M.5), and Symbolic Safety Science is about designing corresponding AI roles based on our social structure, and endowing it with the symbolic capabilities—i.e., the corresponding Thinking Language and Tool Language capabilities—that it can possess under this role.