Snapshot-Based Analysis of Distributed Organizational and Technical System

1. Introduction

Construction companies, petrochemical companies, and airports are classified as large-scale organizational and technical systems (OTS) [1,2,3]. In OTS, control is implemented using hierarchical distributed control systems. To achieve goals, numerous parallel technological operations and business processes are executed, requiring synchronization and consuming significant energy resources. To optimize energy resources at all hierarchical levels, an up-to-date picture of the OTS’s state is necessary [4]. To analyze the system’s state, it is necessary to regularly collect, transmit, store, and process large volumes of heterogeneous data using information systems (IS). The data is then aggregated in ERP systems, CRM systems, and a unified company data warehouse (Data Lake). Based on historical data and machine learning algorithms, a generalized assessment of a company’s performance indicators can be formed, and forecasts for their changes can be developed.

Given the complexity of the OTS, management is carried out based on goal decomposition within a hierarchically organized control system (HCS). Each HCS level addresses its own tasks: at the planning level, large volumes of data are processed, effective forecasts are generated, and the company’s strategy is formed. At the coordination level, tasks are optimized for resource allocation between subsystems to effectively complete production tasks. At the execution level, various methods are used to manage the trajectory of technological and business processes, taking into account accepted constraints. Management tasks at all levels are implemented using the company’s information system (IS), which stores current information on the company’s business processes. Business process models are represented as diagrams, for example, based on the BPMN (Business Process Model and Notation) specification [5,6,7]. The complexity of assessing the state of all processes within a company lies in the need to generate a large dataset reflecting the intermediate states of these processes. This requires the development of intermediate layers for processing distributed data [8].

Typically, so-called snapshots in the form of a set of dashboards are used to assess the current state of the enterprise. In an information system of a company, snapshots are used to restore IS in the event of a failure [9,10,11]. The generation of snapshots utilizes distributed algorithms and multi-agent technologies, which are part of IS and, therefore, consume memory, communication channels, and processor time, ultimately increasing the energy consumption of IS.

A snapshot simultaneously displays data on changes in system resources, task statuses, and information flow across all subsystems of the distributed system. Additionally, a generalized timestamp is generated, enabling the implementation of procedures for creating (stitching) a comprehensive picture of the system state (a snapshot). Note that if a general time scale is not feasible, other scales can be used, such as a scale for the rate of risk propagation within the system. When analyzing the dynamic state of a system, successive snapshots on a given scale are compared to see the general dynamics of changes in system indicators and the rates of change depending on the stages of the system lifecycle [12,13,14].

The advantages of the method under discussion include the ability to obtain an objective picture, as the snapshot reflects the real situation (as is), eliminating human error in reporting [14]. Thus, when solving management problems, methods from distributed systems control theory based on big data processing can be used. The method is applicable to systems where processes occur in parallel or in different geographic locations. In the event of system failures, the method allows for localization of the subsystem experiencing a prolonged delay or loss of control. Snapshot analysis is in demand when assessing the company’s status by stakeholders, company management, and potential shareholders. Next, we will consider the features of the proposed approach.

2. Information System Snapshot

In the context of information technology and large-scale infrastructure, a system snapshot (large-scale snapshot) means capturing and preserving the complete state of a large, distributed system at a specific point in time [15]. Because distributed information systems include microservices, computing clusters, and distributed databases, creating a snapshot is technologically complex. It requires coordination of various processes across hundreds or thousands of distributed nodes that collect, transmit, store, and process large amounts of data, thereby capturing the state of elements, subsystems, and business processes within the system. The main challenge in creating snapshots of large systems is coordinating the procedures for creating snapshot elements over time. The essence of the problem is that in distributed systems, there is no single signal used to synchronize processes; each process has its own “lifetime.” For example, if the state of business process A is captured at 12:00 PM and business process B at 12:01 PM, the snapshot may be generated during the time interval when business process A sent data to business process B. Our snapshot will contain the state of business process A before it sent the data, and the state of process B after receiving the transaction from business process A. This leads to an inconsistent state of the processes, which distorts the snapshot. The solution to this problem is to generate a consistent set of local process states that together form the current global state of the analyzed system. Several different types of snapshot generation methods are used. We will consider them below.

Information systems use various methods and algorithms for creating snapshots.

Copy-on-write (CoW) is the basic storage technology for most snapshots. It involves several key steps:

• When a snapshot is requested, the system creates metadata on the existing data storage block;
• If an application attempts to overwrite a data block, the system intercepts the write.
• It first copies the old data to a separate storage “snapshot”;
• It then allows writing to the primary location allocated for the snapshot.

The advantage of this approach is that snapshots are created instantly, since the data is not initially copied. Data is transferred to the storage only when data changes. When creating a company snapshot, one must be aware of the large volume of data, which requires special measures when saving the snapshot.

The Chandy-Lampore algorithm. This is a classic algorithm for generating distributed snapshots in information systems. The basic idea is that the system records a consistent global state without stopping the application itself. A specific process (the initiator) records its state and sends a special message (a token) to all other processes with which it communicates. When a process receives a token, if it has not yet committed its state, it commits it and forwards the token to all its neighbors. It stops processing regular messages until it receives tokens on all its incoming channels. As a result, the collection of all recorded states forms a consistent global snapshot. Forming a snapshot of the company’s state requires the development of a middleware software layer, which requires additional computing resources.

Snapshot Strategies in Cloud Environments. Modern cloud platforms (Kubernetes, AWS, Azure) use orchestration to manage snapshots. A key approach for ensuring data integrity in stateful clusters (e.g., PostgreSQL) is the “Freeze and Sync” strategy. This strategy is implemented in several stages:

• Quiesce. The orchestrator puts the application or database into a mode that guarantees data integrity (e.g., read-only mode or using internal backup mechanisms) to temporarily stop writing to the primary node;
• Snap. The system ensures that all replicas have fully reached the primary node. With asynchronous replication, this requires explicit verification of transaction log positions;
• Snapshot. After synchronization is confirmed, the orchestrator simultaneously initiates the creation of storage snapshots on all cluster nodes;
• Resume. After completing the snapshot creation, the system returns the application to normal operation, allowing data writing.

When creating a snapshot of a distributed company, synchronization of the generated parts of the overall company snapshot is necessary.

Log-Based Snapshots. In large systems (e.g., DynamoDB, Cassandra), frequent full snapshots are impractical due to high disk I/O load and long execution times. Instead, a Point-in-Time Recovery (PITR) strategy is used:

• Baseline creation: the system periodically creates one full baseline snapshot (e.g., once a week);
• Continuous change recording: in parallel, all write operations are continuously recorded in a dedicated log. The system stores the change stream (transaction log, WAL, or commit log) in durable storage;
• Recovery process: To restore the state to a specific point in time, the latest available baseline snapshot is first deployed. The system then sequentially replays the stored transaction logs up to the specified point in time.

This approach allows for the restoration of the system state, should the need arise, without the high costs of creating full copies of the data. Therefore, when designing or administering an information system that uses snapshots, it is important to consider various factors that influence the cost of creating a snapshot of a large system.

So, creating a snapshot of a large system is a balance between performance (maintaining system operability) and consistency (ensuring data suitability for creating a distributed system snapshot). These methods and algorithms form the basis for creating snapshots of a company’s processes, and in this case, they can be classified as an intermediate algorithmic (software) layer, the implementation of which requires significant resources from the company’s IS.

3. Analyses of Hierarchical Control System Based on Snapshots

Up-to-date information on the state of technical systems and their subsystems during operation allows for the assessment of the system’s resource and the adoption of appropriate decisions when planning repairs and maintenance. In the case of distributed systems, information on their state allows for the reorganization of the operation of their subsystems to ensure optimal resource allocation when necessary. The lifecycle of a large system can vary from several years to decades. Operation of such systems is entrusted to qualified technical personnel, whose task is to maintain the operability of the large system, ensuring the efficiency and technological safety of all processes within the system. This is achieved through periodic inspections, scheduled repairs, and replacement of failed components and subsystems. Given the complexity of a large system, a significant period of time is required for support team members to accumulate production experience. Continuous professional development and team spirit play a significant role in this process. Evaluation of team member performance is also required and is part of the system state analysis procedure.

Figure 1 shows the architecture of a hierarchical control system S, where L₃ is the strategic control level, L₂ is the coordination level, L₁ is the executive level, and P is the level of production processes, elements, and subsystems (the control object). The flows of control and feedback signals are represented by arrows.

Next, we will consider the main stages of analyzing the state of system S and snapshot generation methodology.

Step 1. Generate a graph G_T = (V_T, E_T), which is a model of the hierarchical level P. V_T is the set of vertices representing the nodes, subsystems, and business processes of this level, and E_T is the set of connections, such as information transmission channels.

Step 2. Determine the set of critical elements V_TR⊂V_T and the set of connections E_TR between V_TR elements, i.e., those elements whose failure could lead to the division of the system into several unrelated subsystems, thereby compromising the integrity of system S.

Step 3. Periodically analyze the state of V_TR and E_TR elements (based on snapshots over a given time period Δt).

Step 4. Generate graphs for levels L₁ and L₂. At the L₂ coordination level, for example, teams of performers are formed and tasks are distributed within the team in an optimal manner, taking into account the status of team members (experience, qualifications, etc.). Assignment problems are solved, i.e., assigning performers (machines) to perform tasks. For large systems, various optimization algorithms must be applied to solve assignment problems.

Let’s now consider an example for system S, where it is necessary to manage the distribution of computing resources between hierarchy levels. We will choose the energy consumption level E as the generalized indicator of resource usage.

Figure 2 schematically represents a company’s hierarchical control system, where I_p is the input material flow, O_p is the output product flow, P is the enterprise (control object), E₁ is the electrical energy used to generate control command flows U₃ and feedback information flows I₁ by the executive level L₁, E₂ is the electrical energy used to generate control flows U₂ and information flows I₂ by the coordination level L₂, and E₃ is the electrical energy used to generate control flows U₁ and information flows I₃ by the strategic level L₃. In Figure 2, uncontrolled data flows are highlighted in gray, taking into account interference, time delays, etc. Based on the obtained data, a basic snapshot of the system’s state in normal operation is generated, represented as a block matrix A₀ (see Figure 2,a). The states of block matrix elements are highlighted in color: green – normal state, yellow – potential failure state, red – failure state. A snapshot is generated at times t+1 and t (see Figure 2,b). If the state of layer L₁ and data transmission channel I₁ changes, the system saves the changes and generates a snapshot (see Figure 2,c).

Complex systems execute multiple business processes. Let’s now consider the specifics of generating a snapshot of a business process’s state.

4. Creating a Business Process Snapshot

When creating business process snapshots, it is necessary to use business process description models that allow the system state to be assessed in terms of the quality of the business processes running in the system. Models based on IDEF diagrams or BPMN diagrams can be used for this purpose. These models are developed by systems analysts who have up-to-date information on the structure and nomenclature of the company’s core business processes. The company’s information system (the enterprise’s ERP system) is then developed based on these approved (reference) models. BPMN diagrams are widely used to describe business processes, using which a sequence of processes, subprocesses, and activities is formed. Figure 3 shows an example of a BPMN diagram for business process #k. The business process includes six activities (A, B, C, D, F, F) and two decision-making blocks (Gateway One and Gateway Two). As a result of execution, business process #k can take three states (End Event One, End Event Two, End Event Three).

Based on this diagram, a software module for supporting this business process is designed: implementation, interfaces, etc. Software implementations have been developed that allow for analyzing the state of business processes based on this notation [ ]. The model shown in Figure 3 can be represented as a directed graph G (see Figure 4), where the following notation is used: node s – business process initialization; node l₁ – activities A, B, C; node l₂ – activities D, E; node l₃ – activity F; node e – business process completion. The edge weights of the graph reflect the quantitative characteristics of the frequency of transitions from one node of graph G to another. This representation of the model allows the use of graph theory for analyzing the business process model.

A generalization of these two models is the DRAKON process description model [16,17] ]. Let’s now examine the details of using DRAKON for describing business processes and developing dynamic models, as well as the specifics of its use in generating snapshots of the system state S [18].

4.1. Building a Business Process Model Based on the DRAKON Language

Visual modeling models based on business process description languages such as IDEF, BPMN, and others have become widely used to describe business processes in the OTS. When designing information systems within a process-based approach, these models are used to implement algorithms for supporting technological and organizational processes within a company [19].

Various mathematical models, such as finite-state machine models, can be used to analyze discrete processes (business processes) [20]. Their construction requires statistical data sets reflecting the state of the machine and the probabilities of transitions from one state to another [21,22].

Figure 5 shows a finite-state machine model of a business process with four states:

M = (S, P),

(1)

where:

S = (s_i, i = 1…4) is the set of business process states,

P = (p_i,j, i = 1…4, j = 1..4) is the set of probabilities of transitions from one business process state to another.

If the ability to collect data on the model states and transitions from state to state at a given point in time is provided, the business process can be described as a finite-state machine.

DRAKON is a visual modeling language for dynamic processes that utilizes the concept of states and a tree-like structure for implementing computational processes. This is convenient for solving the problem of transitioning from describing business processes to representing them as a finite-state machine model.

An algorithmic model describing the business process “Process #k” in the DRAKON language is shown in Figure 6. The states of the business process are represented by a set of states. When analyzing the execution logic of a subprocess, a transition is selected that transfers the business process to a new state:

M = (S, P, R, V, C),

(2)

where:

S = (State #1, ..., State #n-1, State #n) is the specified set of possible states of the business process “Process #k”,

P = (Subprocess #1.1, ..., Subprocess #n-1.1, Subprocess #n.1) is the specified set of possible subprocesses,

R = ({if (•)}) is the set of transition conditions for selecting the next process or subprocess,

C = (C#i) is the set of memory blocks, where i = 1-n.

It should be noted that state control flows are subject to the following rules. Internal control flows on the diagram branches progress from top to bottom, while for the outer loop, control flows from left to right. The business process execution algorithm itself includes several parts, represented by vertical branches. Each business process may include subprocesses, designated by the branch name at the top of the branch. The bottom of the algorithm execution branch contains the names of subsequent branches of possible transitions that are formed during the business process implementation. The DRAKON-based business process representation is intuitive for analysts and facilitates process analysis by managers when monitoring the modeling and implementation of algorithms in the information system, as well as during the algorithm verification process.

Next, we will consider the basic steps for creating a state snapshot.

4.2. Creating a Snapshot of a Distributed Business Process System

The time complexity of generating a snapshot of a large system is related to analyzing the state of multiple nodes and subsystems, collecting and processing data on their states (Figure 6) [23,24]. While we previously discussed the basic algorithms for generating a snapshot of an information system’s state, we note that these algorithms can be implemented in middleware or as functions of the core data processing algorithms within the ERP system itself [25,26]. When generating a snapshot of a business process, it is important to develop an automaton model with distinguishable states of the business process. Reliable protocols for transmitting data on the current state of the business process are required. If necessary, visualizing business process states in a given state space of indicators can be addressed [27].

Figure 7 shows a generalized diagram of the state of a business process system as a stochastic graph, the vertices of which change their positions in a given metric state space:

G = (B_ij,V_ij), i = 1...n, j = 1...m,

(3)

where:

B_ij is a business process (graph vertex) in the space D(R_k,R_m), V_ij are the connections between processes (graph edges) that arise during interactions between business processes, and R_i are the business process quality indicators.

Let’s consider the algorithm for generating a graph snapshot.

BPS (Business Process Snapshot) Algorithm

Start

Stage 1. Building an automata-based model of business processes in the DRAKON language.

Stage 2. Defining a set of metrics in a given business process space.

Stage 3. Selecting a data exchange protocol between business processes.

Stage 4. Generate a data array for generating a snapshot.

Stage 5. Generate a graph describing the business process interaction model.

Stage 6. Analyze the snapshot

End

Figure 8 shows an example of a snapshot of the business process system state at times t_k and t_k+1, generated based on the algorithm for generating a snapshot of the business process system state presented above. The dynamic trajectories of the business processes at time t_k+1 form subgraphs, for example, G_s=(B20_k+1,B1_k+1, B3_k+1), illustrating the joint state of the business processes in an arbitrary quadrant of the R₁R₂ state space. When selecting control actions, the goal of control may be to ensure the system’s movement within a given business process state space [28].

4.3. The Problem of Analyzing the Quality of System State Snapshots

The problem statement in the context of system state snapshot analysis S boils down to finding a balance between the accuracy (completeness) of the system state assessment reflecting the current system state and the cost of generating the snapshot. We choose the following as the objective function reflecting the accuracy of snapshot generation at time i:

f_i(x)= S_{i real} (x_i) - S _{i snap}(x_i)_.,

(4)

where S_{i real} and S_{i snap} are data structures reflecting the state of processes in system S at a given moment.

Problem: It is necessary to minimize function F over a given interval of a selected scale, while respecting technological and budget constraints:

$$F=\min {\displaystyle \sum _{i=1}^n(S_{real}^i}-S_{snap}^i)\to 0$$

(5)

where:

Sⁱ_real is the actual state of the system at a given i-th moment on the selected scale,

Sⁱ_snap is the state recorded in snapshots at a given i-th moment on the selected scale.

We introduce constraints that take into account the accuracy of the estimate and the cost of generating a snapshot:

• Snapshot generation frequency, where the parameter q is determined by the process dynamics. If snapshots are generated without taking into account changes in the entropy of processes, this leads to a loss of information about intermediate stages of the process state. As the snapshot generation frequency increases, communication channels are overloaded, and the cost of generating an overall picture increases;
• Delay in synchronizing the processes of generating snapshots of the system state, i.e., the maximum allowable time difference between snapshot generation in different nodes (subsystems) of the system S;
• The cost of generating snapshots, reflecting the costs of storing and processing the data arrays required to generate snapshots, given the specified budget value C_max.

In distributed systems, when determining the optimal interval for generating a sequence of snapshots, the following are taken into account:

• The probability of detecting critical errors in processes P_f over a given time interval;
• The additional load on the computing nodes of the information system when the E_snap snapshot is generated.

It should be noted that when analyzing the state of a system S, it is necessary to develop a system state monitoring protocol, i.e., determine which subsystems (nodes), with what detail, and how often to take snapshots. This allows for the history of business processes to be reconstructed if necessary, i.e., it is necessary to ensure the ability to reconstruct the entire value chain based on a set of snapshots.

Distributed systems use metrics that reflect not only the pace of process execution but also the consistency of all subsystems within the distributed system. Let’s consider the main metrics used for process analysis below.

Metrics based on process execution time (time-based metrics). These metrics typically include:

• Cycle Time. This is the time required to complete a process from start to finish. Snapshots allow one to determine on which node (subsystem or department) a process is “stuck”;
• Waiting Time (Idle Time). The portion of time a request or resource is idle between execution stages. In distributed systems, this is caused by information transfer delays or differences in execution rates;
• Throughput. The number of completed business processes per unit of time;

Quality-based metrics:

• First Pass Yield. The percentage of processes that pass through all system subsystems (nodes) without being returned for rework;
• Rework Rate. The frequency of business process reversion to the previous stage. Snapshots help identify “loops,” i.e., subsystems (nodes) that most often cause failures in distributed work.

Resource and technical metrics:

• Resource Utilization. Reflects the state of a subsystem, i.e., whether some subsystems (nodes) are overloaded while others are idle. Optimization here leads to optimization of the load between subsystems (nodes);
• Synchronization Gap. An important metric for system snapshots, it reflects the difference between the actual time an event occurred and the time it was reflected in the overall system monitoring information system. For example, if we select a metric (e.g., Cycle Time) and use snapshot analysis to find critical paths in a process chain, then the solution to the optimization problem will involve changing the process parameters (redistributing resources, changing the transition logic) to improve the selected metric.

5. Example of a Solution for Optimizing Energy Consumption in HСS

Let’s consider an example of generating control actions based on analyzing snapshots of the company’s HСS status. The HСS distinguishes three management levels: the strategic level, the coordination level, and the execution level. Control actions and decisions are transmitted via communication channels, and information about the results of control actions is transmitted to higher levels [29,30].

Consider the case of an overload of the data transmission channel to the executive level L₁ from control object P. The causes of the overload may include failures of subsystems of control object P, the occurrence of various critical situations, etc., which necessitates increasing the frequency of sensor polling at level P, which in turn necessitates increasing the data transmission channel capacity. Expanding the data transmission channel, for example, by using additional channels, increases the load on network equipment, storage systems, and data processing. Let’s set the task of optimizing the overall energy consumption of the company’s HMS in the face of critical situations, taking into account the need to redistribute electricity between HMS levels. Figure 9,a shows the generalized architecture of the HMS; a snapshot of the state allows one to determine the overload of the I1 data transmission channel (the block is highlighted in red); in this case, the L₁ control level receives information with a delay (the block is highlighted in yellow), which leads to a decrease in the effectiveness of control actions.

Figure 9, b shows the timing diagrams, where C_c is the change in the I₁ data transmission channel throughput, R is the risk change curve associated with the delay of relevant information, S_n are the snapshot generation stages, and E₁, E₂, and E₃ are the energy consumption curves for the company’s HCS control levels. We use Distributed Model Predictive Control (DMPC), a control method used for managing distributed systems. This method enables the implementation of control at various hierarchical levels by local control systems and coordinates their actions while taking into account global constraints. The method is designed for managing large-scale systems consisting of multiple interacting subsystems.

Figure 10 and Figure 11 show the results of modeling the response of the HCS energy consumption management system at various levels, which helps maintain the overall HCS energy consumption level.

6. Conclusions

This article examines the problem of analyzing the state of large-scale organizational and technical systems. Control is assumed to be performed by a hierarchical distributed control system. To achieve the control goal, numerous parallel technological operations and business processes are performed, requiring synchronization and consuming significant energy resources. To optimize energy resources at all hierarchical levels, an up-to-date picture of the organizational and technical system’s state is necessary. The concept of a snapshot of the organizational and technical system’s state is introduced, allowing for an assessment of the system’s state within the framework of selected process performance criteria and the selection of required solutions within the entire control system. Procedures for constructing a snapshot are discussed. The use of DRAKON as a design language is proposed. An example of energy consumption optimization for a hierarchical control system is considered.