Complexity, Artificial Life, and Artificial Intelligence

1. Introduction

“The best way to understand man is by creating him”

—José Negrete-Martínez

Complexity1 has been studied since antiquity. Just to mention a few examples: Aristotle’s concept of “more than the sum of its parts” is related to emergence (See Section 6); the Sanskrit term “tantra” (interwoven) has several parallels with complexity; and ecology has always been inherently complex. There are several historical examples of what would later be called artificial life (ALife)2, with a mild surge after the publication of Mary Shelley’s “Frankenstein; or, The Modern Prometheus” in 1818 [12,111,113]. There have been many artificial creatures, first as ancient myths, then with automata (possible with clockmaking technology required to precisely measure time for making accurate maps as Europeans navigated around the planet), and in the previous century with the development of the first digital computers [15,20,118].

Still, the modern scientific study of complex systems and the field of artificial life (under that name) can be traced to the 1980s around the Santa Fe Institute (SFI, founded in 1984) and the nearby Los Alamos National Laboratory (LANL, created for the Manhattan Project) in Northern New Mexico. SFI (celebrating its ${40}^{\mathrm{th}}$ anniversary) was the first research institution to use the name “complexity”, even when there were several places where similar research had been carried out. The first conference on Artificial Life (1987) took place in Los Alamos, while the second (1990) and third (1992) were in Santa Fe, all three were organized by Chris Langton (who coined the term ALife and worked at SFI for some years) and others. In 1991, Francisco Varela, Paul Bourgine, and others, organized the first European Conference on Artificial Life, with a perspective tending more towards cognitive science. Eventually, both “schools” converged.

I will not attempt to provide a historical account of complexity, ALife, or artificial intelligence (A.I.). My purpose is to notice the similarities and differences between the three fields as they share conceptual, methodological, and philosophical approaches.

In the next section, I’ll review the historical and technological circumstances that predated the development of complexity, ALife, and A.I. In Section 3, I’ll mention common limitations that these fields face, along with the expectations they have generated. In subsequent sections, I’ll relate the concepts of interactions, self-organization, emergence, and balance to complexity, ALife, and A.I., before closing the paper with open questions.

2. Computers as Telescopes

“Where there is an observatory and a telescope, we expect that any eyes will see new worlds at once.”

—Henry David Thoreau

Why did complexity as we know it and “life as it could be” [71] become popular in the 1980s and not before or after? Personal computers. Before then, digital computing was restricted to the few research institutions that could afford the expensive equipment (and thus there were few developments that would now be considered as complexity or ALife. In the case of A.I., while there were more projects funded by governments, companies had fallen into an “A.I. winter” because of unfulfilled expectations [41]). PCs changed everything. The number of people who could exploit and explore new possibilities in information processing suddenly exploded.

As already mentioned, there were a few examples of what could be considered artificial life, e.g. [15,20,118], while Alan Turing [116], John von Neumann [119], and others were interested in the potential ability of computers of modeling the human mind. We can say that cybernetics [8,61,97,127]3 set the basis for the scientific study of complex systems, intelligence, and life. This is because cybernetics was the first transdisciplinary effort to study phenomena independently of their substrate. Systems were studied in terms of their organization, rather than in terms of their components. And because organization [9,10,100,117] can be described in terms of information [93,105], it became clear that the technology capable of increasing information processing (a.k.a. computation), storage, and transmission would be essential.

Something similar happened with fractals [75], which were named only in 1975 by Benoît Mandelbrot. Still, some examples were already proposed in the late XIXth and early XXth centuries by Cantor, von Koch, Sierpiński, and others. Even more, Gaston Julia and Pierre Fatou studied iterative functions, which can be used to construct fractals. Still, these were mainly forgotten. But Mandelbrot had a huge advantage: access to computers that could draw fractals as he worked at IBM Research4. Then, the interest in fractals exploded.

Before telescopes, no planet beyond Saturn could be detected. And our moon was the only satellite known. Galileo was able to see Jupiter’s four largest satellites with his telescope. More planets followed. Other galaxies were observed only less than a century ago as more powerful telescopes became available. The first exoplanet was detected in 1992. Now there are more than five thousand exoplanets confirmed in more than four thousand planetary systems. It was only because of these observations of exoplanets that we now know that most stars have planetary systems, even if we have yet to detect most of them.

Before microscopes, doctors were taught that disease was caused by the imbalance of “humors” (or astrological influence, from which the name of influenza comes). It took more than two centuries for the germ theory of disease to be accepted. But without seeing pathogens, how could we attempt to prevent and cure diseases caused by them? Leeches, of course5.

Before computers, we did not have the proper tools to study complex systems. Just like our vision is limited to perceive the macro and the micro, our limited cognition restricted us to dealing with only a few variables, even if we had huge blackboards. As Heinz Pagels noted, computers are like telescopes for complexity [88]. And artificial life. And artificial intelligence. All of these three have information processing at its core. Thus, we could only begin to study them once information technology reached a level where enough information could be stored, transmitted, and processed to simulate intelligence, life, and complexity [106].

Why New Mexico, “land of enchantment”? This is a trickier question. Better said, attempting to answer it has to be more subjective. Still, I can speculate that at the time there was enough talent (some Nobel prize winners) and freedom of research at LANL (for example, arXiv was created there by Paul Ginsparg in 1991). Unfortunately, as several colleagues who have worked at LANL told me, the situation changed at the Laboratory for different reasons, resulting in limited creativity and less people being attracted to it. Nevertheless, it seems to me that “back in the day”, it was remote enough so that non-mainstream ideas could be explored, but not too remote so that the successful ideas that were developed could spread.

3. Promises and Limits

“Every man takes the limits of his own field of vision for the limits of the world”.

—Arthur Schopenhauer

“The limits of my language means the limits of my world”.

—Ludwig Wittgenstein

One could naïvely think that we just need enough computational power to completely model and understand intelligence, life, and complexity. Many promises were made: robots smarter than humans in all domains, all diseases cured, genomes controlled exactly, future predicted precisely... all attempts have failed, and some researchers are still hopeful of achieving these goals with better models and faster computers. And many projects with these expectations are still being funded 6. Nevertheless, even before the first electronic computers were built, this approach was “doomed” by the limits of formal systems as proven by Gödel [55], Turing [115], Chaitin [29,31], and more.

In the late XIX^th century, Georg Cantor proposed set theory (for which he was ridiculed and ostracized), which later became the basis of modern mathematics. Paradoxes arose. Whitehead and Russell [126] attempted unsuccessfully to overcome them. David Hilbert launched a program to try to prove that mathematics was complete (all statements can be proven true or false), consistent (no contradictions), and decidable (questions posed within mathematics could be answered). A young John von Neumann, then PhD student of Hilbert, was working on this topic, and probably that is why he was the only one that understood when Kurt Gödel presented his results proving that formal systems could not be complete nor consistent. Later Turing proved that mathematics was not decidable, for which he defined the concepts of Turing machine and computable numbers. The implications of these results are that formal systems are limited in ways that have yet to be completely understood. A sign of this is that we still attempt to use formal systems for tasks that would require going beyond those limits. Still, in many cases, partial success is better than nothing at all, especially since we have yet to find a suitable alternative.

Even when adaptation is widely used [6], there is always a part of systems (axioms in the formal case, hardware or hard code in the engineering case) that cannot be changed. Still, we might argue that “real” intelligence, life, and complexity cannot change the laws of physics or chemistry, so in a sense they are also limited.

Independently on our definitions of intelligence, life, and complexity, we can say that artificial systems have yet to exhibit behavior as rich as the one of natural systems. Could this be because of the limits of formal systems? Or simply because we have yet to understand how nature changes itself?

Moreover, it might be that we want artificial systems to be simpler than natural ones. This is because we can attempt to better understand less detailed versions of natural systems.

In the case of artificial life, these limits have been evident in the study of open-ended evolution [90,107,112]. As Hernández, et al. [58] showed, undecidability and irreducibility (which might be considered as desirable or undesirable but are precisely some of the limits of formal systems) are conditions for open-endedness.

For complexity, a relevant case is that of emergence [1,16,18,102] (to be expanded in Section 6). There are several notions and flavors of emergence. In general, it can be said that emergent properties are those present at one scale (usually lower or faster, but not necessarily) and not at another scale (usually higher or slower) [50]. In particular, “strong emergence” is seen as problematic by some, since it usually implies downward causation [23,27,37,40]. This means that emergent properties at a higher scale have a causal effect on elements at a lower scale. We have yet to find a formalism that properly describes downward causation, while some argue that it does not even exist (downward causation might be apparent, an epiphenomenon, but the laws of physics explain everything)7. Could it be because of the same limits of formal systems? Nevertheless, for practical purposes, does it really matter? Even if in theory everything could be reduced to physics, in practice it is not. So, in any case, we do need descriptions at all levels to understand and face complexity.

For A.I., there have been several limits identified, one of the most relevant being that of meaning [79,98]. In principle and practice, machines can simulate in a very sophisticated way our cognitive abilities. Still, do they “really” understand [57,104]? We might say that pragmatically it does not matter. But it should, as a feature of human cognition is the ability to change meanings arbitrarily and adaptively, which again seems limited by formal systems used to implement A.I. systems. There have been impressive advances within information theory, but methods for creating semantics and understanding are still at an early stage. Some people [e.g., [3] have argued that the surprising capabilities of recent large language models could be considered as understanding, although this is still hotly debated [80].

It might be that these limits are actually a feature, not a problem. We “just” need to accept them to be able to exploit them, rather than fight against them. Imagine that mathematics (or any formal system) was consistent, complete, and decidable. as Hilbert and others hoped for. Yes, we would have “absolute truths” and certainty. But would we have creativity? Innovation? Serendipity? It seems to me that many of the features of our world (without which we would not be here) require the limits we have been so eagerly trying to eliminate.

Even when there have been impressive advances in the scientific study of complexity, artificial life, and artificial intelligence, there are several open problems that may be related to the inherent limits of formal systems. Will we be able to go beyond them?

4. Interactions

“The aim of science is not things themselves, as the dogmatists in their simplicity imagine,

but the relations among things; outside these relations there is no reality knowable.”

— Henri Poincaré

Etymologically and conceptually, we can say that the most relevant feature of complex systems is interactions [33,48]. Complexity comes from the Latin plexus, which means entwined, and has some similarities with the Sanskrit tantra. In both cases, interactions make it difficult to study or describe elements in isolation, just like threads in a fabric (which is the literal meaning of tantra). We can say that this is related to the concept of tendrel (Tibetan, Sanskrit Pratītyasamutpāda) from Buddhist philosophy, which could be translated as “interdependent origination”, “dependent arising", or simply “causation”. Tendrel notices that phenomena arise in relation to other phenomena. Nothing can be isolated, nor be caused only by itself or out of nothing. So, everything is related, directly or indirectly [43,49,99].

Neither from itself nor from another,

Nor from both,

Nor without a cause,

Does anything whatever, anywhere arise.

—Nāgārjuna, Mūlamadhyamakakārikā 1:1

Traditional science and philosophy (since times of Galileo, Descartes, Newton, Laplace...) have been reductionist, in the sense that within this paradigm we try to simplify and isolate phenomena to predict and control them [60,83]. In other words, we aim at finding fundamental “laws” and use them to obtain a priori knowledge (predict the future), reducing phenomena to the fundamental laws used to describe them. This has been extremely successful and led to impressive advances in engineering, medicine, and more. Still, this does not imply that reductionism does not have its limits nor that there might be more suitable descriptions of the world for certain purposes. Precisely when we have relevant interactions, reductionism is inadequate, as it neglects interactions and their implications.

“Reductionism is correct, but incomplete.”

—Murray Gell-Mann

There are several implications of interactions [48], but I can say at a general level that the main one is that interactions may produce information that was not present in initial nor boundary conditions. This inherently limits predictability [47], as we cannot know a priori which information will be generated. This is known as computational irreducibility [58,128,132]: there is no “shortcut” to the future, as information should be processed through interactions to reach it.

It should be noted that in practice, computational irreducibility might not pose such a challenge as it does in theory. If we are interested only about a particular context, we could potentially explore exhaustively, or at least systematically, all or several possibilities, and then a posteriori be able to describe and predict the future of complex systems, including their emergent properties and variables. Still, if we are dealing with non-stationary problems8, then even if we have a “full” understanding of a specific complex system, if the problem changes (which is not rare precisely because of interactions), it might be that new relevant information will arise and our understanding will be obsolete.

The fact that traditional tools (from reductionist science) are insufficient to study complex systems has led some researchers to seek alternatives [59,68], in part because we seem unable to address global challenges precisely because of their complexity.

The relevance of interactions and limits of predictability have been discussed mainly concerning complex systems, but they are relevant for ALife and A.I. as well. Interactions in ALife and A.I. systems can also generate novel information, limiting predictability, for better or for worse. There have been several attempts with varying degrees of success, but we still lack a general, common framework to describe, understand, and control complex systems. And it might be that such a framework could be developed within ALife or A.I., and then generalized for all complex systems.

Interactions limit the predictability of complex systems. Thus, in many cases, future information can only be knowna posteriori(because of computational irreducibility). This implies that traditional reductionist approaches and methods seem insufficient to properly understand complexity, life, and intelligence. Still, we have yet to develop widely accepted methods that show the desired sufficiency.

5. Self-Organization

“The beauty of a living thing is not the atoms that go into it,

but the way those atoms are put together.”

—Carl Sagan

There are several examples of self-organization in nature [26]: flocks, schools, swarms, herds, crowds, etc. In these examples, there is no leader or external source telling individuals what to do, but the properties of the system are a result of the distributed interactions of individuals. Thus, the study of self-organization is closely tied to complexity and information technology necessary to model it. Also, the term “self-organizing system” has its origins in cybernetics [7,9,73,117]. Nevertheless, there have been several examples of self-organization in physical and chemical systems [11,36,56,87,103].

A system can be described as self-organizing when its components interact to produce a global pattern or behavior [52]. This description can be useful when we are interested in relating multiple scales (elements and system, micro and macro), and how changes in one might affect the other (e.g., changes in individuals affect a society or changes in a society affect individuals).

If we are dealing with a complex problem, novel information can make it non-stationary, i.e., the problem changes. If the problem changes faster than the time required to find a novel solution through optimization or other traditional techniques, then the solutions will be obsolete. Self-organization can be a viable approach to develop adaptive solutions that are able to face non-stationary problems, because when the problems change, elements can adjust through their interactions [42,44].

Self-organization has been used broadly in ALife: for software (digital organisms), hardware (robots), and wetware (protocells). See  [53] for a review.

In A.I., self-organization has had a more limited use. Still, it could be argued that most artificial neural network models are implicitly self-organizing [45], as their weights (interactions) are modified during the training phase. And explicitly, Kohonen networks are self-organizing [69]. Also in robotics, self-organization has been relevant, implicitly or explicitly [91].

Self-organization can be useful when multiple scales are modeled at the same time. It has been a relevant concept for complex systems and ALife, with a potential in A.I. that has yet to be fully explored.

6. Emergence

“You could not have evolved a complex system like a city or an organism — with an enormous number of components — without the emergence of laws that constrain their behavior in order for them to be resilient.”

—Geoffrey West

The concept of emergence has certain analogies with Aristotele’s “the whole being more than the sum of its parts”, where the “more” is the emergent bit. Emergence was popular in the XIXth century [77,78], but fell out of favor in the early XXth century due to the success of reductionist approaches. But when information technology allowed the scientific study of complex systems, emergence became relevant again [17].

Still, emergence probably caused most of the confusion and skepticism around complexity in the 1980s and 1990s. In part, because some people described emergent properties as “surprising” or “unexpected”. Then, emergence should be a measure of our ignorance, because once we understand these properties, they are no longer surprising nor unexpected.

Nevertheless, there is nothing mysterious about emergence if properly described [5]. In a general way, emergent properties are those present at one scale but not at another [50]. For example, a bar of gold has color, conductivity, malleability, etc. Still, its components (gold atoms) do not have these properties, so we can call them emergent. In a similar way, it is accepted that cells are alive, but they are composed of molecules that are not alive. Whatever our definition of life, we can say that it emerges out of the interactions of molecules. It is accepted that a human is intelligent, but she is composed of cells that are not intelligent (in the same way). Whatever our definition of intelligence, we can say that intelligence emerges out of interactions of cells.

There are different flavors of emergence, some less controversial than others (see [50] for a review). For example, weak emergence [16] is about properties described by an observer, such as gliders in the Game of Life [19,20]. Still, gliders do not change the rules of the Game of Life, and we only need these rules to compute the future states of the system. Strong emergence [14,102] would be when having all information at one scale is not enough to derive information at another scale. In many cases, strongly emergent properties or information have a causal effect on the elements that produced them. For example, molecules form cells, but living cells make molecules that cannot be produced without biospheres. Also, individuals create social norms, and these norms promote and constrain the behaviors of individuals.

One could say that weak emergence is “in the observer”, while strong emergence “is real”. Some (reductionist) people do not believe strong emergence exists (e.g., [125]), as it implies downward causation, and for them only “fundamental” phenomena described by physics are real. Independently on our notion of reality, in practice, the laws of physics are not sufficient to describe, explain, and even less predict phenomena at higher scales (even fluid dynamics and chemistry, we do not have to go to life, intelligence, and culture.).

I conjecture that strongly emergent properties are not computable in practice, and that is why a lower scale description is not enough to predict them. If there is no practical way in which the properties of one scale can be described in terms of the “laws” of another, then we should validly describe those properties as emergent. Of course, this cannot be proven, for reasons similar to why a number cannot be proven to be random [30] or Kolmogorov’s complexity is not computable (in theory) [34]. Note that this approach does not rely on downward causation, , but does not prevent it [64].

For example, a person can be melted by the words of their loved one, but this cannot be derived from the laws of physics that describe the melting of matter, no matter how detailed a description one might have at the “fundamental” level. Certainly, the laws of physics are not being violated. They are simply not enough, as there is no meaning in physics [38,46].

Emergence has been a central concept for complex systems and artificial life [50]. Many ALife models have been used to better describe and understand different flavors of emergence, e.g. [21,22,62,76,82,95,114,121,123].

In A.I., emergence has been less relevant. Still, unpredictable capabilities of large language models have been recently described as emergent [124], sparking some controversy.

Emergence can be a useful concept when information is not present at one scale but is present at another. Even when it is prevalent, we lack the conceptual and formal tools to precisely speak and measure emergence in complex, living, and intelligent systems.

7. Balance

“Everything tends to a balance.”

In recent years, I have been developing a narrative of “balance”9 to bring together concepts of the scientific study of complex systems, and to communicate them to a general audience. There are several historical examples of balance from ancient cultures, and those examples show it has been a common, long-standing practice to try to avoid extremes. Still, criticality [2,11,13,32,81,84,89,96,101,108] can be seen as a type of balance between order and chaos [67,72]. Life (and computation) need some stability (order) to keep on functioning. But too much stability limits their adaptability. On the other extreme, too much variability (chaos) loses useful information. At “the edge”, evolution, life, and intelligence can emerge.

More generally, balance is a tautology, because we describe a posteriori phenomena that survived and evolved as balanced, between “too few” and “too much” change. Certainly, there can be “dynamic balance”, where the precise tradeoff varies and systems need to adapt (as exemplified by the slower-is-faster effect [51]). Also, interactions, perturbations, or noise can increase the change in a system, for which antifragility [92,110] is desirable. And we have recently shown that heterogeneity can “extend” the “balanced” region of systems [74,101].

In A.I., a well-studied balance is that between exploration and exploitation in search [35,63], also known as search in breadth or depth, respectively (when solution spaces are represented as trees). In other words, to try to find the best solution to a problem, one can exploit current solutions and try to improve them or explore completely novel solutions with the hope that some might be better than current ones. Because the best strategy cannot be predetermined, as it depends on the problem space [130,131], the precise balance between exploration and exploitation will depend on the particular problem space that is searched.

Balance also offers a promising narrative to study evolution (natural and artificial) [65,66,120], as by definition that which evolves needs to be balanced.

Phenomena that endure tend to avoid extremes, so they can be called “balanced”a posteriori(once they endured). Still, this tautology can be useful to bring together common concepts in complex systems, ALife, and A.I.

8. Inconclusion

“Being ill defined is a feature common to all important concepts.”

—Benoît Mandelbrot

I have mentioned conceptual similarities and challenges among complexity, ALife, and A.I. Still, there are many open questions.

There are no agreed definitions of complexity, life, or intelligence. But perhaps this is more a feature than a problem. If we could define one of these precisely, then we would not have so many open questions about them. And we do because their richness goes beyond our current understanding abilities. It remains to be seen whether we just need a revolution in science [59,68] to be able to understand them properly. Or it might be that there are aspects that inherently are beyond understanding as we know it [129].

In practice, there have been several relevant recent advances that have generated great expectations. Whether we consider novel forms of life, either by exploiting current ones [24,25,54,70] or exploring novel ones [28,86,94], we will take important steps to understand life on Earth and in other planets [122].

Historically, A.I. has had its cycles of expectations (summers) and disappointments (winters). We have had several years of building expectations. For example, autonomous vehicles are still “two years away” after more than fifteen years. Deep neural networks and large language models have achieved impressive performances, but in the end, they are “just” ad hoc statistical engines. It is not clear that by following the same approach something like “understanding meaning” could be achieved [79]. Still, for many practical purposes, this is not relevant. Nevertheless, there are limits to what current approaches will be able to do.

As for the scientific study of complex systems, perhaps its success will be achieved when most disciplines complete integrating their concepts and methods and adopt them as their own, so few people would speak about “complexity economics” or “biological complexity”, simply because most people would be familiar with the relevant concepts and methods. Still, there will always be a narrow space for studying complexity per se, as the study of the interactions in systems at all scales.

The limitations outlined in this paper very well might be overcome. We have no clear idea of how this will be possible, but there are several promising explorations. If further research helps to better delineate the limits of science rather than going beyond them, this will certainly be useful and will allow us to make better decisions, even if it is just by knowing what we have no way of knowing.