Effective Evolutionary Principles for System-of-Systems: Insights from Agent-Based Modeling in Vehicular Networks

1. Introduction

A system-of-systems (SoS) is a system with a highly complex structure. When the structure of an SoS undergoes transformation, further non-linear changes will occur as a result of its complexity. In order to guide the SoS to achieve directed evolution, researchers have proposed many principles to manage the evolutionary process of the SoS [1,2,3,4].

However, due to the extremely complex nature of the SoS itself, it is challenging to describe the effects of these principles during the actual engineering process, as well as their mechanisms of action [5]. This leads to a lack of understanding when the management of SoS evolution processes is considered in SoS engineering, making it difficult to ensure that the relevant measures are sufficiently accurate and effective.

To address these challenges, we examine the impacts of different SoS evolution principles on the performance of complex systems. As obtaining empirical evidence from a sufficient number of SoSs can be arduous, due to the scarceness of complex system design data, we propose an alternative approach for creating unique SoS models and simulating design processes based on empirically verified phenomena to overcome this issue. We undertook an analysis of the ABM method's efficacy utilizing a case study centered on the Internet of Vehicles. Using this method, researchers can gain valuable insights into how specific factors influence SoSs without the need for extensive empirical data.

The case study detailed in this paper examines the evolution of a telematics SoS dealing with situational awareness problems. As a typical collaborative SoS, the structure of the telematics SoS is decentralized and distributed. The core principle of telematics is interactive communication between different distributed nodes (e.g., vehicles or infrastructure) within a network, facilitating the sharing of information to achieve situational awareness of the environment. Situational awareness refers to the perception of environmental factors under certain temporal and spatial conditions, as well as the prediction of their future trends, by collecting data through on-board sensors, cameras, and other devices [6,7,8]. When a telematics SoS receives new environmental information, nodes with different devices cooperate with each other to collect, process, and transmit relevant information to achieve information sharing in the telematics SoS, thus completing the evolution process. The effect of choosing different evolutionary principles and strategies on the evolutionary performance of SoSs can be studied using the presented framework.

The problem of situational awareness in telematics SoS offers an ideal case to examine the impacts of evolutionary principles on system-of-systems (SoS) performance. The emergent evolution that occurs during the situational awareness process in telematics SoS is often unpredictable. Consequently, this paper aims to address the challenge of modeling this evolutionary process and analyzing the influence of different principles on the information sharing process in the SoS. In this article, we combine SoS evolution theory, agent-based modeling, design optimization, and research on the Internet of Vehicles (IoV) architecture, which is achieved through the following steps: (1) Generating a vehicle networking SoS, (2) simulating the evolutionary process during SoS situational awareness and behaviors of intelligent agents, (3) verifying the effectiveness of the evolutionary principles by adjusting various parameter settings.

The article delves into the intricacies of SoS evolution by examining the application of evolutionary principles and an agent-based model (ABM) in the context of vehicle networking SoS. Employing the ABM, we developed a unique model that generates complex SoS and simulates their evolutionary processes. This was achieved through the execution of Monte Carlo simulations in 150 distinct SoSs, concurrently altering their underlying evolutionary principles. The findings of this comprehensive study highlight the varying effects that different principles have on the evolution performance of SoS. This research not only contributes to a better understanding of SoS evolution, but also emphasizes the importance of selecting appropriate evolutionary principles when designing and optimizing vehicle networking SoSs.

2. Preliminaries

2.1. The Concept of SoS Evolution

In the realm of system engineering, a system-of-systems (SoS) represents a sophisticated arrangement of specialized systems that synergistically pool resources and integrate their capabilities to create a more functional and high-performing system. Within this intricate structure, a constituent system (CS) operates as a component of one or more SoSs, functioning as a complete system with its own objectives and resources. A particularly intriguing manifestation of SoSs is the collaborative SoS, which emerges when constituent systems voluntarily align themselves around a central purpose and collectively determine their implementation and maintenance standards. One salient example of a collaborative SoS is a connected vehicle SoS, designed to achieve situational awareness of the environment by harnessing the collective intelligence and capabilities of various vehicular components. This innovative approach to system integration and collaboration not only enhances the overall performance of the individual systems, but also fosters a safer and more efficient transportation ecosystem.

Evolutionary characteristics were first identified as an inherent part of SoSs by Maier [9]. From a macroscopic perspective, the SoS can be seen as continuously, but slowly evolving [10]. This evolutionary process is incessant [11], meaning that the SoS has no permanent state [12]. From a microscopic perspective, SoS evolution takes place through a series of largely deliberate preservative or adaptive interventions [3], such as upgrades to constituent systems or responses to an ever-changing environment [13,14].

In the evolutionary process of SoS, a salient challenge is that evolution is not necessarily centrally controlled, and the impetus for change may occur suddenly and dramatically [15]. Therefore, evolution can simultaneously occur at multiple levels and within multiple areas of an SoS. In practice, such evolutions within SoSs may take place in multiple places at once and on an ongoing basis. And at the same time, different parallel evolution processes may complement or disrupt each other [16]. Consequently, any unforeseen evolution in the system (SoS) may lead to undesirable emergent phenomena [2]. If such evolution cannot be effectively controlled, then the resulting emergent behavior may lead to failures within the SoS development process [17]. Accordingly, emergent evolution has been proposed to describe this unpredictable form of evolution [18]. In this paper, we examine the emergent evolution of system-of-systems from the bottom-up level.

Based on the abovementioned studies, it is clear that emergent behavior among systems can be reflected in the evolution of the whole system-of-systems and even influence the evolutionary outcome. Therefore, it is necessary to propose appropriate evolutionary measures to manage and guide this bottom-up behavior during the evolution of the system-of-systems. The validation and application of SoS principles can help managers to better design SoS frameworks and manage emergent behaviors in the evolutionary process, thus enhancing the efficiency and effectiveness of SoS evolution.

2.2. Guiding Principles for SoS Evolution

This paper focuses on four of the significant principles that guide the evolution of SoSs.

2.2.1. Facilitate Information Exchange

Facilitating information exchange is an important principle when functionally distinct systems interact within SoS [19]. Clausing argued that the impact of design reuse can be reduced and the quality of evolution improved by ensuring the effective exchange of information during system evolution [20]. Lock affirmed the positive role of information exchange, and argued that poor levels of information exchange in the interaction between member systems within an SoS can affect the efficiency and reliability of SoS evolution [2]. In addition, Carney argued that the rate of coordinated information exchange is an important aspect of the maintenance of interoperability, and that coordinated information exchange between systems allows the SoS to retain its original characteristics during the evolution process [3]. Indeed, information exchange is an important factor in the ability to build new organizations. Information exchange is the link that facilitates interaction between systems in the process of building an SoS from independent systems [21]. Many properties of systems are closely related to information exchange, including reliability and interoperability [22]. Successful information exchange is considered to be the basis for the achievement of system properties such as interoperability [21].

Information exchange can also influence emergent complexity in the organizational structure when analyzed from a micro perspective. Emergent changes that occur internally, such as unpredictable deviations, unforeseen errors, or significant constraints, can be perceived at an earlier stage through information exchange [23,24,25]. Moreover, information exchange can also motivate agents to form organizations naturally [23]. In this process, frequent information exchange has an effect on the inertia of agents—which may be due to short-term behavior—but, in the long run, may lead to evolutionary trends [26].

2.2.2. Implementing Uniform Standards

Standards can provide a common framework and terminology, making participants more likely to have the same understanding of the same problem [27]. Uniform standards can avoid ambiguity in interpretation, which may increase the level of mutual understanding between systems as they evolve [28]. For example, relying on accepted formal or informal standards in multi-agent systems can lead to mutual understanding [29]. By promoting and enhancing the level of understanding between systems, uniform standards can further influence the evolutionary performance of a SoS. Lock argued that “The agility of SoS evolution can be improved by using and implementing uniform standards that promote understanding between interacting systems” [4]. Carney et al. argued that inter-system agreement determines whether local interoperability relationships can be established [2]. Selberg and Austin argued, from a system design perspective, that the selection of uniform and widely used standards is important for SoS evolution [1].

Therefore, we argue that it will be easier for member systems to learn new knowledge and for constituent systems to adapt to dynamic change if uniform standards or protocols are implemented. A uniform standard will provide a common framework and language, making it easier for the members of an organization to identify and use the information in a database.

2.2.3. Enhancing Transparency of Information

Information transparency, in the context of organizational dynamics, pertains to the degree of information sharing and disclosure [30]. Transparency of information between systems can eliminate inconsistencies between non-verbal and verbal behaviors and build trust between systems, thereby promoting a level of mutual understanding in organizations [31,32]. For example, in human–computer interaction systems, transparency can facilitate mutual understanding between humans and agents, leading to cooperation [33]. In the area of SoS evolution, Lane and Valerdi argued that building trust and transparency between systems helps to gain the support of SoS members to effectively guide the evolution of the SoS [34]. Carney et al. argued that building trust mechanisms between systems (e.g., by establishing information transparency) is particularly necessary for each stage of SoS evolution [3].

Therefore, we argue that enhancing the transparency of information between systems is conducive to building trust between SoS members and ultimately facilitating the evolution of the SoS. In the framework presented in this study, Principle 3 (Enhancing Information Transparency) primarily operates through top-down behavioral control mechanisms, while Principle 1 (Facilitate Information Exchange) is chiefly governed by bottom-up interactive behaviors.

2.2.4. Establishing Common Goals

The term 'Common Goals' denotes the collective objectives or targeted outcomes that multiple constituents or stakeholders within a system seek to realize in order to optimize system functionality. Common goals among the members of the organization help to create synergy within the organization, thus enhancing organizational cohesion [35]. Carney et al. argued that when there is a common purpose between the constituent systems in a SoS, the closer each system is to that purpose and the greater the SoS’s ability to adapt to change [3]. When members of an organization share a common purpose, they are more motivated to seek consensus and, thus, collaborate and cooperate more effectively [36].

Therefore, we argue that establishing a common purpose between constituent systems within a SoS increases the SoS’s ability to adapt to change, which ultimately facilitates the evolution of the SoS.

2.3. Agent-Based Modeling

To handle the intricacy of SoS evolution, an agent-based model (ABM) can be used to simulate the SoS, representing agent behaviors and SoS alterations [37]. This technique can be used to simulate the attributes of an SoS (i.e., its types, behaviors, and capabilities) [38]. As a “bottom-up” technique, an ABM uses agents to represent the domain being modeled [39]. Then, organizational behaviors may emerge as the cumulative result of individual behaviors. It should be noted that the evolution of a SoS is a complex, non-linear process. And an ABM provides just such a model, requiring each agent to behave in a stochastic, non-linear manner and possess a non-linear ability to adapt over time [40]. Characterized by being autonomous and flexible [41,42], An ABM explains the actions and interactions of agents with a view to assessing their effects on the system as a whole, as in the Kauffman NK model [43].

The autonomy and flexibility of Agent-Based Modeling (ABM) make it highly suitable for modeling a wide range of distributed SOS. In the context of this research paper, the vehicle SoS under study represents a typical distributed SOS, thus making it an ideal candidate for the application of ABM methodology. The pioneering work in utilizing ABM methodology for vehicle research can be attributed to Nagel and Schreckenberg [44], who developed a stochastic discrete automation model incorporating Agent-Based Simulation (ABS) to simulate individual vehicle nodes. Since then, ABM has garnered significant attention in the realm of vehicular systems research. Notable contributions include studies on traffic theory by Nagel and Flötteröd [45] and Zhu et al. [46], vehicular transportation simulations conducted by Zou and Levinson [47], Levy et al. [48], and traffic control investigations by Bosse [49], Bui and Camacho [50], Wang and Lv et al. [51].

And regarding the information sharing problem addressed in this paper, Shang Wenlong and Ke Han et al. employed ABM to establish a transportation network model and a vehicle SoS communication model. Their study aimed to explore the impact of tourism information sharing on road networks[52]. Furthermore, the issue of information sharing within vehicular networks using the ABM approach has been explored by Mahdi and Hasson [53], Zia and Shafi et al. [54], and Rathee and Garg et al. [55]. It is important to note that this paper distinguishes itself from previous vehicular system studies by focusing on the intricate interaction behaviors among vehicle nodes and attempting to simulate the unpredictable emergent evolution observed in vehicle SoS.

At the same time, this paper examines the suitability of ABM for organizational evolution. In this paper, each individual of an organization interacts with the others and the environment through various behaviors. It has been well-documented that the foundational level of an organization can be elucidated by examining the interactions between actors, whether individuals or groups, and their environment [56]. And such characteristics of interactions play a crucial role in managing the emergent evolution of SoS. Thus, we anticipate that ABM can capture these behavior characteristics within an organization and facilitate research into SoS evolution issues.

In fact, several articles have utilized the ABM approach to examine intricate organizational structures of SoS evolution. Peppard and Breu et al. used ABM techniques to analyze the evolutionary patterns of the formation mechanisms of collaborative SoSs by simulating the assembly processes of molecules under natural conditions [57]. These results were used by engineers to improve the design and management of SoSs. Besides, Sindiy and DeLaurentis et al. modeled a distributed SoS and used an SoS to define time-varying performance metrics and an agent-based model to simulate the evolution of these architectures [58], while Nikolic and Dijkema used ABM techniques to model industrial and infrastructural development in seaport areas, combining the feedback of knowledge processes with SoS evolution simulation [59]. While ABMs are beginning to be used for SoS-related research, there have been few efforts to improve SoS theory-related aspects [36]. The use of ABM to test SoS theory in this paper represents a continued expansion of this field.

3. Methodology

This section presents the conceptualization and implementation of the model utilizing the ODD (Overview, Design concepts, Details) protocol [60].

3.1. Overall Model Structure

In this paper, the subject is an abstract system consisting of a number of agents in three categories (vehicle, infrastructure and mobile device) . These agents can absorb internal and external changes (e.g., changes in the environment or changes in the mission) through different behaviors and interactions, leading to emergent evolution of the SoS. It is assumed that these three categories of agents can adapt spontaneously to the dynamics of the SoS. The structure of the agent-based model is illustrated in Figure 1.

3.2. SoS Evolution

The case assumed in this paper is the evolution of an SoS when dealing with situational awareness problems in telematics. The telematics SoS consists of vehicle nodes, infrastructure nodes (e.g., traffic lights, surveillance cameras, road sensors, wireless network base stations), mobile device nodes (e.g., portable devices such as smartphones and wearable devices), and cloud servers. The core principle of telematics is interactive communication between various distributed nodes (e.g., vehicles and infrastructure) within the network and information sharing, in order to obtain situational awareness of the environment. Situational awareness refers to the perception of environmental factors under specific temporal and spatial conditions, as well as the prediction of their future trends, by collecting data from on-board sensors, cameras, and other devices.

In this model, three facilities in the same area form the Telematics SoS. To achieve situational awareness of the area, different behaviors occur at each node (i.e., constituent system) in response to changes in the environment. The constituent system is represented by an agent. In the experiment, initial relationships between constituent systems in the same sector or in different sectors are established to simulate the relationships between systems in the SoS.

When the Telematics SoS receives new environmental information, the nodes carrying different devices cooperate with each other to collect, process, and transmit relevant information for information sharing in the Telematics SoS, thus completing an evolutionary process. Specifically, in the Telematics SoS, each node within it undergoes independent evolution in response to changes in the external environment through interactive behaviors, which eventually leads to evolution of the SoS.

In this research, the fundamental design concepts of the model primarily pertain to the implementation of information sharing principles. Consequently, the agents in the experimental setup were represented as binary strings comprising numerical values that encapsulate information content. This representation serves as a means to depict their knowledge sets, which follows a well-established modeling approach. The environment and the behavior of the agents in the experiments change these strings and, thus, the properties of the agents. The schematic diagram of the knowledge set is shown in Figure 2.

The Telematics SoS is in a constantly changing environment, where external changes fall into three categories: Natural environment changes, man-made environment changes, and vehicle status changes. Natural environmental changes refer to changes in the external natural environment which have an impact on vehicle performance and driving safety (e.g., weather changes, road conditions, and so on). Man-made environmental changes refer to the impacts of urban planning, population flow, road reconstruction, and other factors on vehicle driving. Vehicle state changes refer to changes in the operating state of the vehicle itself (e.g., engine failure, tire leakage, and so on). Different changes add different knowledge values to the agent’s knowledge set, and the agent’s behavior has different effects under different changes. These knowledge values are assigned to all agents and ultimately affect the evolutionary direction of the SoS.

For example, if a change in the human environment is applied to a class of facilities, an agent in that class will set its knowledge value about the “human environment” to 1, meaning that it senses and receives knowledge relating to the change. At this point, the knowledge values of the other agents are set to 0, indicating that they are not receiving knowledge related to that change. As the SoS evolves, the knowledge values of the other agents about the change will eventually change to 1, indicating that the SoS as a whole has fully absorbed the change.

3.3. Agent Behaviors

The agents in this paper are designed as state machine models. In particular, the agents have two states: No knowledge set and existing knowledge set. Agents in the no knowledge set state are transformed to the knowledge set state when they are affected by three environmental changes. The agent state transition diagram is shown in Figure 3.

Five spontaneous behaviors are set for the agents in the model, with reference to the behavior of the self-directed system during the evolution of the SoS: Communication, negotiation, learning, cooperation, and competition. When the knowledge set exists, there are two possible states for the knowledge value of a certain bit in the knowledge set: the knowledge value is either 0 or 1. When the knowledge value is 0, if any of the five interactive behaviors affect the knowledge value, then it will be transformed to 1. The state transition diagram for the knowledge value is shown in Figure 4.

Agents perform the established behaviors spontaneously and participate in the evolution of the SoS. The five spontaneous behaviors of agents are described below.

Communication refers to the communication between various nodes within the telematics SoS through information transfer, such as vehicle–road cooperation between vehicles and road facilities, communication between vehicles, and data exchange between vehicles and cloud servers. Through communication, different nodes can understand each other’s status and needs, thus improving the efficiency and safety of the whole system.

Learning means that the nodes in the Telematics SoS continuously collect and analyze a large amount of data by interacting with the cloud server, as well as using algorithms to improve their own performance and adaptability to the environment. For example, monitoring devices can predict future traffic conditions by analyzing factors such as traffic flow and congestion, then adjust their own monitoring behavior based on these predictions.

Negotiation is the process of reaching a common decision between two or more independent nodes through interaction. For example, nodes must decide among themselves how to allocate resources such as bandwidth, processing power, and so on. The nodes need to adjust their respective behaviors to adapt to environmental changes or to optimize certain metrics (e.g., reducing latency or lowering power consumption) among themselves. During the negotiation process, nodes need to send messages to each other, explain their intentions, exchange preferences, and reach an agreement. The result of negotiation can be an agreement, an allocation of resources, or a change in the way that the nodes behave.

Competition refers to the behavior of nodes competing for limited resources. In the Telematics SoS, individual nodes require access to resources such as data, storage space, and network bandwidth to perform their tasks. As these resources are limited, resource competition between nodes can occur; for example, multiple nodes send data to the central server at the same time, which may result in insufficient bandwidth, ultimately affecting the quality and speed of data transmission.

Cooperation is the act of working together among nodes to achieve a common goal. In the Telematics SoS, individual nodes must work together to accomplish the overall situational awareness task. For example, in self-driving cars, individual sensors need to work together to obtain information about the environment and aggregate this information to the central controller for analysis and processing, which enables the autonomous driving function.

Overall, the behaviors of communication, cooperation, competition, negotiation, and learning are essential factors in the evolution of an SoS. In this model, these behaviors interact and influence each other, acting on the knowledge sets of the agents and ultimately shaping the characteristics and evolutionary direction of the SoS.

3.4. Principle

The ultimate goal of this study is to investigate the roles of the four principles mentioned in Section 2.3 in the evolution of the SoS. In the model described herein, a crucial design concept is the impact of these principles on the likelihood of an agent's behavior and consequently the evolutionary dynamics of the system.

In the considered model, the evolutionary principles of facilitating Information Exchange influence the communication and negotiation behaviors in the SoS. On one hand, communication is the fundamental behavior within an organization, and is also a form of information exchange. Encouraging internal information exchange within an organization can improve the quality and fluidity of communication [61]. On the other hand, moderate and accurate information exchange can significantly improve the negotiation performance without significant cost to the negotiators who initiated it, resulting in more mutually beneficial negotiation outcomes [62].

In the considered model, the principle of implementing uniform standards can influence learning and competitive behaviors in the SoS. We found that applying uniform and standardized work principles effectively reduces costs, has a positive impact on team member learning, and provides a basis for sustainable team improvement [63]. Furthermore, researchers have found that uniform standards can regulate competition patterns and, thus, energize the entire organization [64]; however, an excessive reliance on standards can also lead to certain monopolistic phenomena [65].

In the considered model, the principle of data transparency can affect cooperation and communication behaviors in the SoS. Information transparency is considered, in some studies, as a tool that helps stakeholders to perceive information as relevant and timely, providing a reliable picture of the organizational reality [66]. The principle of transparency enhances communication between stakeholders and, thus, allows more valuable information to be conveyed [67]. In addition, information transparency enhances transparency within an organization, leading to the involvement of members in collaboration and the creation of certain collaborative mechanisms [68].

In the considered model, the principle of establishing common goals can affect cooperation and negotiation behaviors in the SoS. Researchers have concluded that there is a positive relationship between the degree to which members of an organization agree on a common goal and the effectiveness of collaboration toward that goal [69]. Thus, effective collaboration requires a shared, common goal [70]. In addition, what is negotiated among members is a limited common goal. A common goal can create a sense of trust between negotiators [71]. From another point of view, maximizing the common goal effort between the two parties is the focus of each negotiation [72].

In the initial model, the different behaviors have a defined probability of occurrence, which are validated to allow the system to evolve in a balanced way, while different principles are established to influence the probability of occurrence of the behaviors. In this way, evolutionary principles are studied indirectly in this paper.

3.5. Indicators

In order to assess the impacts of different principles on the system, this subsection introduces misalignment as a reference indicator for the SoS evolutionary process in simulation experiments. Furthermore, to provide a complete picture of performance, this subsection introduces the evolutionary time (ET), degree of variation (DOV), and cost as performance assessment metrics.

Misalignment refers to the degree of mismatch between the knowledge sets of the member systems in the system. We argue that, as the system evolves, differences between member systems will continue to emerge, where higher differences will negatively affect organizational performance. Based on the ABM model constructed in this section, misalignment is viewed as a difference in knowledge between different types of entities in the evolutionary process.

$${RMSE}_{VNs,INs}=\sqrt{\frac{{\sum }_{i=1}^n{{(X}_{VN,i}-X_{IN,i})}^2}{n}}.$$

(1)

The formula for calculating this indicator is shown in Equation (1), where $X_{VN,i}$ on the right side of the formula denotes the mean value of the i^th knowledge value of the vehicle node (VN) and $X_{IN,i}$ denotes the mean value of the i^th knowledge value of the infrastructure node (IN). Therefore, ${RMSE}_{VN,IN}$ on the left side of the formula indicates the root-mean-squared error between the vehicle node (VN) and the infrastructure node (IN); that is, the difference in value between the two facilities. The smaller the ${RMSE}_{VN,IN}$, the smaller the difference between the two facilities and the smaller the negative effect of evolution on the SoS.

The total root-mean-squared error between the three types of facilities is denoted as $Misalignment$, as shown in Equation (2), where ${RMSE}_{VNs,IN\mathrm{s}}$ denotes the difference between vehicle nodes (VNs) and mobile facility nodes (MDNs); ${RMSE}_{INs,MDNs}$ denotes the difference between infrastructure nodes (INs) and mobile facility nodes (MDNs); and ${RMSE}_{VNs,MDN\mathrm{s}}$ denotes the difference between vehicle nodes (VNs) and infrastructure nodes (INs). Thus, $Misalignment$ is the sum of the root-mean-squared errors between the different facility knowledge sets, indicating the total variance value within the system at that moment. This metric is the basic metric that provides the results of the model, the role of which is to indicate the evolution of the SoS.

$$Misalignment={RMSE}_{VNs,INs}+{RMSE}_{INs,MDNs}+{RMSE}_{VNs,MDNs}.$$

(2)

Based on the misalignment base metric, the first evaluation metric of the model is evolution time (ET), which is the time taken for the misalignment base metric to return to zero (i.e., the total time required to complete the evolution of the SoS). ET describes the basic characteristics of the evolution of the SoS. If the ET is smaller, the time required for SoS evolution is shorter and the evolutionary performance of the model is stronger. As the SoS evolution process in the situational awareness problem is short, it is measured in seconds.

The second evaluation metric of the model is the degree of variation (DOV), the value of which is the integral of the variation over time (i.e., the time-weighted average of the root-mean-squared error among all knowledge sets in the model). The DOV describes the degree of accumulation of the total degree of variation over time in the Telematics SoS. If the DOV is smaller, the degree of variation in the SoS evolution process is smaller and the evolutionary performance of the model is stronger.

$$\mathrm{D}\mathrm{O}\mathrm{V}={\int }_0^{ET}Misalignment dt.$$

(3)

The third metric introduced in this study is cost, which represents the consumption of various resources during the evolution of the SoS. As knowledge transfer in an organization is supported by the consumption of various resources, we therefore assume that the implementation of each code of conduct may incur costs. The case studied in this paper is a situational awareness problem and, as vehicular networking requires the constant transmission of a large amount of data (e.g., vehicle location, speed, acceleration, vehicle status, traffic conditions, and so on), sufficient bandwidth is required to support the transmission of this data. Therefore, we use network bandwidth to represent the resource consumption in this process. The unit is Gbps, which is the amount of data transmitted in gigabits per second, in order to represent the network bandwidth occupied by the task of situational awareness.

3.6. Monte Carlo Simulation and Model Verification

Monte Carlo simulations have recently emerged as a prevalent technique for testing agent-based models (ABMs) and generating statistically significant outcomes under various evaluation metrics [73,74]. By leveraging the capabilities of Monte Carlo simulations, numerous studies have successfully modeled complex organizational relationships and gained valuable insights into the underlying dynamics that govern these systems [75,76,77].

To eliminate the stochastic nature of ABM simulations, we generated and designed several unique complex systems using Monte Carlo simulations, in order to sample the effects of specific behaviors across a large number of complex systems. To fully test the role of each principle in the evolutionary process, four experimental groups and a no-principle control group were set up, according to the four evolutionary principles detailed above. In addition, the amount of environmental change can affect the evolutionary process of the system. Therefore, the experiment was set up with three levels, according to the amount of change, which were tested separately. The experiment was run 80 times in each state, in order to eliminate the effect of random errors.

With five experimental groups, three variations, and 80 executions per combination, the ABM in this paper was run 5 × 80 = 400 times. The initial variation in each experiment randomly affected a few agents. By utilizing the Monte Carlo method, we were able to generate 400 unique and representative complex SoSs, providing a robust data set for further analysis.

Validation of ABMs is difficult but necessary [78]. Therefore, a sensitivity analysis of the simulation data between ABM experiments was required. For this study, sensitivity tests were performed on metrics that were not relevant to the purpose of the experiment, in order to select optimal control variables (see Table A1).

3.7. Time Complexity Analysis

The time complexity of Agent-Based Modeling (ABM) can pose a significant drawback as the input size of certain parameters increases. In this paper, we focus on the evolution process of the vehicular networking system as our model simulation. The initial amount of variation introduced during this evolution process plays a crucial role in a specific Telematics system. Therefore, this study aims to examine the impact of different initial variation amounts on the simulation time. The results of our tests are presented in Figure 5.

The data from the tests can be fitted to a quadratic function: $\mathrm{y}=1.0561{\mathrm{x}}^2-0.1723\mathrm{x}+55.876$, where $\mathrm{x}$ represents the initial amount of variation. By analyzing the trends, we observe that the simulation time increases quadratically with the initial amount of variation. Consequently, the estimation time complexity of the model can be expressed as $\mathrm{O}\left({\mathrm{n}}^2\right)$, where $\mathrm{n}$ represents the amount of variation introduced in the model. This implies that the model's efficiency may be compromised when handling large-scale data.

4. Results

The evolution of the SoS is shown in Figure 6, in which the basic misalignment parameter describes the evolution of the system. A summary of the information shown in the figure is provided in the following:

(1)

The misalignment metrics in all four graphs showed an increasing and then decreasing trend. The reason for this phenomenon is that the initial interactions between some of the constituent systems increased the degree of difference between all nodes in the SoS under the influence of the external environment. As the interactions continued, evolution caused the degree of difference between most of the constituent systems to decrease, eventually leading to complete evolution.

(2)

The peak misalignment values in the plot for Principle 2 (Implementing Uniform Standards) occurred earlier than those without the application of the principle. The reason for this phenomenon is that the application of this principles increased the overall efficiency of the system at an early stage and different nodes in the installation received more new information in a short period of time, thus creating differences between the self-managed systems. Meanwhile, the peak misalignment values in the plot of principle 1 (Facilitating information exchange) was significantly lower than that without the application of the principle, probably as the exchange of information between nodes somewhat mitigated the degree of difference between the self-managed systems.

(3)

Compared to the control group, the misalignment values of the SoS with different principles applied were all improved, in terms of in the rate of decline after reaching the peak. As such, the time to complete SoS evolution was also shorter in all cases. This indicates that the application of different principles can enhance the efficiency of system evolution, to some extent.

In the simulation experiments, the effects of the different principles on the evolution of the system varied. The results of the evaluation of the four principles based on the three indicators described above are shown in Figure 7. The information in the figure is summarized in the following follows:

(1)

Figure 6a shows the average evolution time of the SoS with the different principles applied. The evolution time for the SoS without applying any principles was 1181.2 s. The evolution times for the systems with Principle 1 and Principle 2 applied were close, at 989.8 s and 965.0 s, respectively (roughly 82% of the original time). Meanwhile, the average evolution time with Principle 4 applied was 954.8 s (80.8% of the original time), and the lowest evolution time was obtained with Principle 3, which was only 892.8 s (or 75.6% of the original time).

(2)

Figure 6b shows the degree of variation accumulated in the evolution of the SoS with the application of the different principles. All four principles reduced the degree of variation to a greater extent. The smallest reduction was obtained with Principle 2 (Implementing Uniform Standards), which was 83.8% of the baseline variance, while the greatest reduction in the degree of variation was achieved with Principle 1 (Facilitating information exchange), which was 72.3% of the baseline degree of variation.

Figure 6c shows the average cost of the SoS evolution process after applying the different principles. The evolution costs of the systems using Principle 3 and Principle 4 were relatively similar, at 4585.5 Gbps and 4559.7 Gbps, respectively, with very low increases in cost. Principle 2 had the highest cost of evolution (5262.3 Gbps), with a 16.4% increase compared to the baseline cost.

As can be seen from Figure 6, Principle 2 (Implementing Uniform Standards) was the least effective in improving the situational awareness model in this paper, with a small improvement in evolution performance and a large increase in cost. Meanwhile, Principle 3 (Enhancing Transparency of Information) was the most effective, significantly reducing the evolution time and variability while keeping the evolution cost low. In addition, Principle 1 (Facilitating information exchange) presented a good reduction in variance and Principle 4 (Establishing Common Goals) had the lowest cost.

5. Discussion

5.1. Elaboration of Experimental Outcomes

The model detailed in this paper provides insights into the validation of theoretical principles, allowing for study of the effects of different principles on system evolution and the implementation of complex system-of-systems modeling using the ABM approach. Overall, the experimental findings presented in this paper demonstrate variations in the impacts of the four evolutionary principles on SoS evolution.

The experimental results indicate that the Degree of Variation (DOV) of the Telematics SoS decreases significantly over time after implementing Principle 1 (Facilitating Information Exchange). This outcome corroborates the efficacy of information exchange in mitigating discrepancies during the SoS evolution, aligning with our preliminary hypothesis. Prior research has previously underscored the potential of information exchange to enhance information visibility within transportation systems [79]. Furthermore, the process of information exchange serves to eliminate obstructions between disparate systems, align the interests of stakeholders [80], and alleviate unwarranted variations within systems [81].

From a pragmatic perspective, the enhancement of information exchange is instrumental in resolving prevalent challenges within the SoS, such as conflicting interests and ambiguous accountabilities [82]. These challenges are inherently associated with the disparities present among the constituent systems. Consequently, it is plausible to postulate that the enhancement of SoS performance via information exchange is attributed to the principle's efficacy in bridging differences among system members.

In our study, we have made an interesting observation regarding the effect of Principle 2 (Implementing Uniform Standards) . Contrary to our initial prediction, we found that implementing uniform standards is actually the most costly. This finding is significant because unified standards are generally expected to reduce costs in system architecture [83]. Furthermore, our research also revealed that uniform standards are less effective in improving the performance of the SoS in our experiment. These observations raise an important question: why are uniform standards not as effective in the experimental setting designed in this paper?

We believe that this phenomenon can be attributed to the difficulty of adapting uniform standards to distributed SoSs, particularly when there are significant differences among constituent systems. This phenomenon has been acknowledged in other literature as well. In distributed systems, uniform standards may overlook the heterogeneity of individuals, leading to a decrease in overall system efficiency [84,85]. These findings highlight the need for further investigation into this issue in future research.

Our experimental findings indicate that Principle 3 (Enhancing Information Transparency) outperforms other principles in minimizing both the duration of SoS evolution and the variability encountered during this progression. We posit that this principle augments the interactive behavior among agents by modulating specific factors. Prior studies have corroborated the efficacy of augmented information transparency in bolstering trust within constituent systems [86,87,88] , especially in analogous distributed SoSs [89]. Additionally, increased transparency has been shown to bolster the efficiency of knowledge support, subsequently amplifying the innovative capacity of organizational members [90].

It is noteworthy, however, that the minimal variability observed during the evolutionary process and the shortest evolutionary duration do not necessarily correspond to the least evolutionary cost. This observation suggests that the inherent nature of enhanced information transparency might levy supplementary costs upon the organization [91]. In summation, the SoS exhibits optimal performance post the implementation of Principle 3 ((Enhancing Information Transparency). From an applied standpoint, organizational leaders can augment information transparency within the SoS employing modern information technologies, such as social networking platforms and electronic bulletin boards, or even through conventional means like performance bulletin boards [90].

In addition to these findings, the simulation results demonstrate that Principle 4 (Establishing Common Goals) has a positive impact on reducing the SoS variance and the time required for SoS evolution, aligning with the expectations before the experiment conducted. However, it is noteworthy that the establishing common goals makes the evolution of SoS significantly less costly compared to other principles. This result challenges conventional wisdom, as there is no readily apparent logical correlation between the two variables under consideration. Furthermore, there is a lack of relevant research to substantiate the relationship between common goals and costs.

Drawing upon the analyzed observations, this study posits a hypothesis delineating the nuanced role of the 'establishing common goals' in modulating the cost associated with the evolution of SoS. This modulation ostensibly occurs through its impact on a constellation of intermediary factors. Extant literature substantiates that the articulation of common goals can attenuate the risks inherent in information exchange processes, thereby cultivating a milieu of trust among constituent members [92,93]. Such trust can decrease conflict-related costs and the need for mutual monitoring within organizations. Furthermore, a consensus on common goals can enhance access to tacit knowledge, streamlining work processes and bolstering decision-making efficiency [94,95,96]. To gain further insights, we plan to conduct additional research to investigate this factor in more depth.

5.2. Evaluating the Efficacy and Limitations of the Model

In the realm of Telematics, a pioneering study on the information sharing problem was conducted by Shang Wenlong, Han Ke, and their colleagues, who employed Agent-Based Modeling (ABM) as their research approach. Their investigation centered around the utilization of the penetration rate as an evaluative metric for gauging the extent of information sharing. While this assessment method offers a broad perspective, it fails to consider the intricate interplay among vehicle nodes. In contrast to prior investigations, this scholarly article indirectly captures the interplay of diverse behaviors that are challenging to model accurately. Consequently, it substantiates the influence of various widely adopted principles on information sharing within connected vehicle systems, thereby presenting a innovation in the field.

Experimentation and validation of SoS theory is a difficult area in system-of-systems research. In addition, the ABM method is a unique tool for verification of theory. When an ABM is used as an experimental tool to study SoS problems, representing the complexity of an SoS involving multiple constituent systems is a difficult problem to solve. The research idea presented in this paper is intended to improve the modeling of agents as much as possible—for example, by incorporating multiple complex and reasonable behaviors—in order to simulate the situation as realistically as possible. On this basis, multiple agents which interact autonomously were used, which is the core advantage of the ABM approach, in order to simulate the emergent nature of the SoS and achieve an effective simulation of the SoS.

Nevertheless, it is important to acknowledge that the ABM model presented in this paper has certain limitations. As discussed in subsection 3.7, the model exhibits high time complexity, which may hinder its efficiency when dealing with more intricate problems. To address this concern, future studies will explore the application of proxy sampling methods or optimization algorithms to mitigate these limitations. Additionally, it is worth noting that certain simplifying assumptions were made in the context of this simulation experiment. To apply a model like the one presented in this paper to more complex SoS problems, researchers may need to add more realistic attributes to the agent.

5.3. Bridging Natural and Social Sciences: A Methodological Discourse

The discourse surrounding the congruencies and divergences in the research methodologies employed in the natural and social sciences has been long-standing. Hayek posited that the inherently uncertain nature of human beings, who are central subjects in social science studies, precludes the social sciences from yielding results analogous to those derived from the natural sciences [97]. This uncertainty stems from cognitive limitations, conflicting interests, diverse value orientations, and reflexivity.

Contrastingly, Popper contended that a uniform set of criteria should be applied when evaluating both the natural and social sciences [98]. Within the realm of systems engineering, there is a recurrent necessity to incorporate considerations from both technical (pertaining to natural sciences) and human (pertaining to social sciences) perspectives, especially when navigating complex systems.

Drawing upon the experimental simulation methodology prevalent in the natural sciences, which emphasizes categorical identification, this study aligns with Popper's perspective to scrutinize social science theorems. The objective is to facilitate a transition from quantitative analysis to a more qualitative approach. It is important to acknowledge that the principles examined in this research are inherently challenging to fully validate or falsify due to the fundamental uncertainty associated with human behavior, a challenge frequently encountered in social science research.

Given this backdrop, the present paper proposes a novel approach to validating social science theories leveraging Agent-Based Modeling. This preliminary exploration seeks to foster a foundation for further empirical investigations in subsequent research endeavors.

6. Conclusions

A system-of-systems is a complex system with a high degree of unpredictability in its evolutionary process. In order to improve the performance of SoS evolution and to achieve a guided evolutionary process, many studies have proposed principles for SoS evolution. However, these principles remain at the theoretical level and lack experimental data to support them. In this study, an agent-based model of the SoS evolution process was developed against the background of an SoS handling the situational awareness problem in vehicular networks, and an attempt was made to validate and study the SoS evolution principles.

The results of the simulation indicate that the application of all four evolutionary principles can enhance the evolutionary performance of the telematic SoS, but with varying effects. Specifically, promoting information exchange between constituent systems can successfully minimize the degree of variation during SoS evolution, while establishing a common objective among constituent systems can substantially reduce the cost of SoS evolution. Regarding the evolutionary problem in the model, the implementation of uniform standard in collaborative SoS seems less effective due to its disregard for individual heterogeneity. Among the considered contexts, improving high information transparency within the SoS performs optimally among the four strategies and significantly shortens the time required for the evolution of the telematics SoS.

The aim of this study was to validate the principles of SoS evolution proposed in previous studies, in order to provide evidence for the SoS theory through experimental model and data. The contributions of this paper serve to further deepen understanding of the SoS evolution process through experimental results, as well as providing researchers with a reference for improving SoS evolutionary principles. However, it is important to note that the current model being investigated suffers from high time complexity and oversimplification, as discussed in this paper. To address these limitations, we plan to optimize the developed ABM model in the future to enhance the Computational Conclusion presented in this study.