2. Structural Analysis: The Three Elements of Creativity
2.1. Experience Training: The “Raw Material” of Creativity
Experience as internalization of patterns. The training process of an AI system is essentially a process of “experience internalization.” By optimizing a loss function on large amounts of data, the system “compresses” statistical patterns from the data into model parameters. A trained language model encodes in its parameters grammatical rules, semantic relations, world knowledge, reasoning patterns, and other multi-level structures.
Figure 1.
Three elements of AI creativity and the emergence mechanism. Experience training, randomness, and multi-level fusion combine multiplicatively to produce structured surprise—creativity.
Figure 1.
Three elements of AI creativity and the emergence mechanism. Experience training, randomness, and multi-level fusion combine multiplicatively to produce structured surprise—creativity.
This “internalization” process is analogous to human learning. Humans accumulate experience through reading, observing, and practicing, stored in the brain as the “raw material” for creativity. A person who has never read poetry can hardly write good poetry—because their “experience library” lacks poetic patterns. Similarly, an AI system trained on a small dataset can hardly generate high-quality output—because its parameters lack sufficient patterns.
The concept of empirical density. Not all experience contributes equally to creativity. We propose the concept of “empirical density” to measure the richness of patterns a system has internalized in a specific domain:
The higher the empirical density, the richer the system’s “raw material library,” and the greater the possibility of creativity. A GPT model trained on internet-scale text has a much higher empirical density than one trained on a single book—the former possesses a richer pattern library for recombination.
The hierarchical nature of experience. Experiential patterns are not stored flatly but organized hierarchically:
Table 1.
Hierarchical organization of experiential patterns.
Table 1.
Hierarchical organization of experiential patterns.
| Level |
Experience Type |
Example |
| Surface |
Specific words, sentences, paragraphs |
“Moonlight before my bed” |
| Structural |
Grammar, rhetoric, genre |
Tonal patterns of five-character quatrains |
| Semantic |
Concepts, themes, imagery |
Association of “moon” with “homesickness” |
| Meta |
Creative strategies, stylistic features |
The technique of “expressing emotion through scenery” |
Creativity often involves cross-level experience invocation—a good poem not only uses appropriate words at the surface level but also follows rhythm at the structural level, establishes new imagery connections at the semantic level, and exhibits unique creative strategy at the meta level. This cross-level invocation is the core of “multi-level fusion.”
2.2. Randomness: The “Spark” of Creativity
Multiple roles of randomness in AI systems. Randomness plays multiple roles in AI systems, some engineering-oriented (such as regularization to prevent overfitting), but others genuinely creative. This paper focuses on the latter.
The central role of randomness in creativity is “breaking constraints”—patterns in training data constitute the system’s “experience constraint,” and if the system entirely follows these constraints, its output would be “interpolation” of training data, producing no “surprise.” Randomness introduces perturbation that causes the system’s output to deviate from the “expected trajectory” of training data, thereby producing “surprise.”
Three forms of randomness. Randomness in AI systems takes multiple forms with different effects on creativity:
1. Sampling randomness (temperature sampling). In autoregressive generation, randomness is introduced at each step when selecting the next token. The temperature parameter controls the intensity of randomness: as , the model always selects the highest-probability token (deterministic output); as , the model approximates uniform random selection. Creative output often emerges at moderate temperatures—neither too deterministic (reducing to copying) nor too random (reducing to noise).
2. Dropout randomness. During training, a fraction of neurons are randomly “deactivated.” The deeper effect of dropout is not merely preventing overfitting—it forces the system to learn “redundant representations,” where the same information is encoded across multiple neurons. This redundancy provides a foundation for creativity—the system can generate different output patterns by “partially activating” different neuron combinations.
3. Noise injection. Noise is injected into the model’s input or hidden states. Noise injection is analogous to “spontaneous activity” in the human brain—it is not a response to external stimuli but an internal “self-excited oscillation.” This self-excited oscillation plays the role of “flash of inspiration” in creativity—a random internal perturbation may trigger a novel pattern combination.
The “moderateness” of randomness. The contribution of randomness to creativity is not monotonic—too little randomness leads to “copying,” too much leads to “noise.” There exists a “creative zone”—within this range, randomness is sufficient to break the constraints of training data but not so strong as to destroy the output’s structure.
This “creative zone” is analogous to the “edge of chaos” in physics—at the boundary between order and chaos, the system retains sufficient structure while possessing sufficient flexibility. Many complex systems—from neural networks to ecosystems—exhibit maximal computational power and adaptability at the edge of chaos. AI creativity may follow a similar principle.
2.3. Multi-Level Fusion: The “Mechanism” of Creativity
Fusion as the channel for recombination. If experience is the “raw material” and randomness is the “spark,” then fusion is the “processing mechanism”—it determines how raw material and spark combine to produce something new.
“Fusion” is not simple “stitching”—placing two independent patterns side by side. Fusion is a “structural recombination”—establishing new connections between patterns at an abstract level. A creative poem is not “randomly inserting Kantian terms into a poem about quantum mechanics” but establishing deep structural correspondences between philosophical concepts and physical imagery.
Three fusion levels. Corresponding to the “multi-level recursive inference” framework of Paper 6, data fusion in AI systems also occurs at three levels:
Micro-fusion (feature level). In the feature space of a neural network, information from different sources is fused into a unified representation. For example, in attention mechanisms, representations at different positions are “weighted and fused” through attention weights to generate new mixed representations. Micro-fusion is the most basic form of fusion—it occurs in every forward propagation and provides the foundation for higher-level fusion.
Meso-fusion (sequence level). In autoregressive sequence generation, patterns from different sources are fused into a coherent text. For example, when GPT generates “a quantum mechanics poem in the style of Kant,” it must fuse Kantian philosophical concepts with quantum mechanical physical imagery within the same line of the same poem—this fusion is not simple stitching but establishing new correspondences at the semantic level.
Macro-fusion (task level). In the agent loop, information from different tasks and different domains is fused into a unified knowledge base. For example, an AI system that can simultaneously write poetry, program, and do mathematics may derive creativity from cross-domain knowledge transfer—applying the concept of “recursion” from programming to poetry, or the concept of “symmetry” from mathematics to narrative structure. Macro-fusion is the highest level of fusion—it involves cross-task, cross-domain conceptual recombination.
2.4. The Structural Relationship Among the Three Elements
The three elements are not independent but coupled through the following structural relationship:
where
F is not simple addition—there is a “multiplicative relationship” among the three elements:
Empirical density = 0 → Creativity = 0 (no raw material)
Randomness intensity = 0 → Creativity = 0 (no spark, pure copying)
Fusion depth = 0 → Creativity = 0 (no mechanism, simple stitching)
All three and within the “creative zone” → Creativity emerges
This multiplicative relationship means the three elements are “necessary conditions” for creativity—if any one is zero, creativity is zero. This is analogous to the “super-additive effect” of Paper 6—the whole is greater than the sum of its parts.