Integrating Statistical Simulation and Optimization for Redundancy Allocation in Smart Grid Infrastructure

1. Introduction

The advent of the Smart Grid, akin to previous technology revolutions in telecom and the Internet, marks a crucial milestone in modernizing our electric grid. With its implementation, we harness technology to enhance the efficiency, reliability, and affordability of electricity distribution. This transformation shifts our electric system from a centralized, producer-controlled network to a more interactive, consumer-centric model.

Addressing the grid's declining reliability, marked by a surge in outages, the Smart Grid becomes imperative. Currently, these interruptions cost Americans an estimated $150 billion annually. Furthermore, with a projected 30% increase in nationwide electricity demand by 2030, investments of around $1.5 trillion over the next two decades are essential for infrastructure development [1]. By fostering this transition to a smarter grid – a process already underway – and eventually adopting the Smart Grid, electricity will become more affordable, and our environment will benefit from reduced impact. During this transformative period, ensuring fairness, cost-effectiveness, and adequate customer protection will be paramount. The Smart Grid represents a significant leap forward, utilizing data in megabytes to move megawatts of electricity efficiently and reliably into the 21st century [2]. A key feature of the Smart Grid is its ability to conduct continuous self-assessments, allowing it to prevent disruptions proactively rather than merely reacting to them.

Ever-increasing demand coupled with increasing energy prices have prompted the energy industry to develop intelligent strategies for energy tracking, control, and conservation [3]. Electricity disruptions like blackouts can trigger cascading failures affecting banking, communications, and security, particularly in winter when heating is crucial. A Smart Grid enhances power system resilience, ensuring preparedness for storms, earthquakes, and emergencies. Its bidirectional communication reroutes power automatically during outages, minimizing their impact. Smart Grid technology swiftly detects and isolates outages, prioritizing essential services for swift recovery. By integrating customer-owned generators, vital facilities remain operational during crises. Moreover, it addresses aging infrastructure, boosts energy efficiency, raises consumer awareness, and enhances national security, utilizing locally sourced, resilient electricity. On the other hand, interoperability among various grid components, data handling and management across wide geographies with different environmental conditions pose challenges for the traditional smart grids [4].

As systems become increasingly complex and critical to enterprise, the need for optimal redundancy becomes more important to business continuity. Systems failures can stem from a wide variety of causes (refer to the survey article [5]) ranging from the large-scale natural or human-caused disasters that can disrupt an entire region due to a simple electronic part failing. In a recent article, Wang et al. [6] noted that “diversification is one of the most effective approaches to safeguard multitier systems against attacks, failure, and accidents. However, designing such a diversified system poses challenges due to factors such as stochastic user and attacker behaviors, combinatorial-explosive solution space, and multiple conflicting design objectives.” The authors further noted, “diversification is the application of different means for performing a required function. It is currently the best defense against attacks on the flaws of any particular software component.” Smart Grid redundancy is a diversification technique that aims to prevent information technology in the occurrence of a disaster. Smart Grid heavily depends on the IT infrastructure for operations such as cloud computing and edge computing.

In additional to the tens of millions of computers and servers heavily depend on the reliability of IT infrastructure using cloud computing paradigm, the development in the 5G cellular network, Internet of Things (IoT) applications of smart home, smart city, smart transportation of auto-driving has made edge computing an indispensable infrastructure to connect cloud and end users. The emergence of the IoT coupled with the advances in energy management sphere has resulted in the concept of the smart grid as the Internet of Energy (IoE). Krishnan and Jacob [7] propose a hybrid technique in developing an Energy Management System (EMS) for distribution system with IoT framework. As a popular technology, IoE integrates various forms of energy and leverages the internet to collect, organize, optimize, and manage energy networks. Mishra and Singh [8] study energy management techniques in smart cities using IoE in an effort to reveal improvements in clean energy processes.

Edge computing utilizes the resource of cloud servers to direct the data and computing services to a real-time low-latency system at the edge of a network. For example, 5G network in the edge computing infrastructure provides high-bandwidth access to end users on location services, augmented reality, video analytics and data caching [9]. Unlike the data centers of cloud computing, the servers in edge computing of IOT systems must be located close to the end users in order to provide a real-time high bandwidth and low-latency services. Thus, redundancy allocation in edge-to-cloud computing focuses on the network structure of the end user community such as critical nodes and links of social connectivity [10]. The network structure of the user community can be revealed via community detection methods [11]. ML models are popular methods of community detection in edge computing [12,13]. The IoT systems follow the power law distribution [14,15,16,17]. Patsidis et al. [18] employ an architecture which includes edge-cloud communication to extract data-driven insights from microgrids.

In an article addressing the economics modellings of information security, Gordon, and Loeb [19] underscored that an information set is characterized by the loss conditioned on a breach occurring, the probability of a threat occurring, and the vulnerability, defined in the model as the probability of a threat once realized. Interruptions in the services of large-scale service-oriented companies can potentially result in losses amounting to hundreds of millions of dollars per day in revenue. Consequently, these companies must implement specific disaster recovery or disaster avoidance strategies [20].

Cost is always a concern and is part of the redundancy model. Organizations face the challenge of determining the financial allocation for disaster recovery and, within that, the proportion to be allocated to Smart Grid redundancy. Due to the costly nature of redundancy allocation resources companies must select the best options they have to protect their assets [21,22]. It is economically efficiency to protect the critical nodes and links with redundancy resource. Multiple mathematical models have been proposed to identify the critical nodes and links in the Smart Grid (see Table 1).

The mathematical models above encounter challenges when it comes to identifying critical nodes and links in larger-scale problems. Recently, statistical models have been applied to identify the critical nodes and links in very large networks such as social networks and biology networks [27]. These models encompass both model-based and distribution-based methods. They utilize the topological features of the embedding networks for model training and validating the outcomes of critical nodes and links (See Table 2).

Once the critical nodes are identified, redundancy resources are allocated based on the importance of components (nodes) in smart grid infrastructure. The redundancy allocation problem (RAP) is typically formulated with two alternative objectives: maximizing the reliability within the budget constraints or minimizing the system costs to satisfy the minimum system reliability. Kulturel-Konak et al. [30] proposed two integer programming (IP) models to address these alternative RAP objectives and pointed out that these IP models can be converted into 0-1 IP with additional binary decision variables. Shao [31] introduced a 0-1 integer programming solution formulation for RAP. However, this model is highly non-linear and complex. The author provided a specialized dynamic programming procedure for obtaining optimal solutions.

This paper presents a novel linear integer programming approach to address RAP based on the generalized assignment problem (GAP). Over the past few decades, a variety of theoretical and heuristic findings have emerged for GAP, which can be readily applied to the Smart Grid redundancy allocation problem (see for example, [32,33], for a recent survey). Devi et al. [34] conducted an extensive literature review and classified 280 papers on RAP according to the methods employed. Our model here is more general than GAP with applications extending to various scenarios, including multi-skilled workforce assignment [35], the assignment of unmanned aerial vehicles (UAV) [36], and optimal preventive maintenance scheduling in manufacturing environment for related applications and heuristics, see the recent paper by [37] and its references). Two early development of the model regarding multi-skilled workforce applications with some heuristics also may be found in [38,39]. The rest of the paper is organized as follows. In section 2 we first present a framework of combining statistical simulation and optimization model to identify the critical nodes in the edge computing infrastructure using a power grid system as an example, then optimize the redundancy resource allocation with a linear integer programming model. In section 3, we report the computational results in critical nodes. Section 4 summarizes conclusions drawn from this study.

2. Materials and Methods

2.1. Statistical simulation and optimization framework

Statistical simulation for critical nodes is based on the random matrix theory that uses the probability distribution of eigenvalues such as the Tracy Widom (TW) distribution [29].

The critical nodes detection problem can be formulated as a special case of clustering, where critical nodes are assigned to a particular cluster, while the remaining nodes form some disconnected clusters. The number of singleton clusters increases as the number critical nodes increases.

The largest eigenvalues of the adjacency matrix associated with the critical nodes cluster has the TW distribution. Instead of using parametric bootstrap to estimate the TW distribution, it is computationally efficient to run a few simulations to compute the mean and the variance of the distribution. Figure 1 illustrates the statistical simulation-optimization framework used in this study.

To compute the eigenvalues of the graph network, we use spectral clustering method. Initially, the Laplacian matrix is computed, followed by generating eigenvectors. The eigenvectors form an n by n matrix, where each row represents a node, and each column stores an eigenvalue. These eigenvalues are sorted incrementally with those close to zero being removed.

Once the eigenvalues are sorted, the largest k eigenvectors are chosen and stored in a new n by k matrix, which is subsequently normalized. We can assess the TW distribution on the new matrix of eigenvalues. If the matrix of eigenvalues follows the TW distribution, the clustering method described below is applied to the normalized matrix of eigenvalues to obtain the labels and scores of each bank.

The simulation yields a normalized matrix of eigenvalues that can be used to compute the value of signed weight $w_{ij}$ on each edge in the cluster, the critical nodes of that cluster can be computed with the following Integer Programming (IP) optimization model.

$$\mathrm{IP}:\mathit{max}\hspace{0.33em}{x}_0={\sum }_{i=1}^{n-1}{\sum }_{j=i+1}^n{w}_{ij}{\sum }_{k=1}^{c\_max}{x}_{ik}{x}_{jk}$$

(1)

s.t.

$$\sum _{k=1}^{c\_max}{x}_{ik}=1\hspace{1em}\hspace{0.33em}i=1,n$$

(2)

Denoted by $x_{jk}$ equal to 1 if node i in cluster k and c_max as the maximum number of clusters formed. After the clusters are formed, then the critical nodes are identified using a connected node pairs (CNP) reduction model to minimize the total connectivity of the computer network.

$$\mathrm{Minimize}F\left(n_1,\cdots ,n_L,L\right)=\frac{1}{2}{\sum }_{l=1}^L{n}_l\bullet (n_l-1)$$

(3)

denoted by $n_l$, $l$= 1, . . ., L, and the number of nodes in each cluster. Once the critical nodes are identified, resources can be allocated to improve the resilience of the network.

2.2. Optimization model for redundancy allocation

Before we present a general linear IP model for GRAP, the notations for optimization model is given as follows:

Parameters and Variables:

D: number of potential disasters +1 (the last one for no disaster occurring),

$p_d:$ probability of disaster d occurring, $p_d\in \left(\mathrm{0,1}\right)$ and ${\sum }_{d=1}^D{p}_d=1,$M: number of components in Smart Grid needs to perform,

$w_m$: importance weight of components (nodes) in Smart Grid m, $w_m\in \left(\mathrm{0,1}\right)$and ${\sum }_{m=1}^M{w}_m=1,$$n_m:$ number of solutions (assets) available for component (node) m to select from,

$X_{mi}:$ 1 if solution $i\in \left\{1,\dots ,n_m\right\}$ is selected for component (node) m, or 0 otherwise,

$C_{mi}:$ cost of selecting solution i for component (node) m,

$S_{mid}:$ survivability of solution i for component (node) m against disaster d,

$v_{mid}:$failure probability of solution i for component (node) m against disaster d (i.e., $v_{mid}=1-S_{mid}$).

$f_{md}\left(X_{m1},\dots ,X_{m{n}_m}\right),$ a real-valued function defined on vector $\left(X_{m1},\dots ,X_{m{n}_m}\right)$, for $m=1,\dots ,M,$ and $d=1,\dots ,D,$$U_{md}\left(Y_{md}\right),$ a utility function defined on vector $Y_{md}=f_{md}\left(X_{m1},\dots ,X_{m{n}_m}\right),$for $m=1,\dots ,M,$ and $d=1,\dots ,D,$ when $Y_{md}$ is the total contribution of applying all or some of available solutions $X_{mi},\left(i=1,\dots ,n_m\right)$.

Here, we give a general redundancy allocation problem (GRAP). As we mentioned earlier the GRAP has various of applications in different settings, (see [32,33] for explanations, examples, and several heuristic algorithms).

(GRAP)

$$\mathit{max}S={\sum }_{d=1}^D{p}_d\left[{\sum }_{m=1}^M{w}_m{U}_{md}\left(Z_{md}\right)\right]$$

(4)

s.t.

$${\sum }_{i=1}^{n_m}{X}_{mi}\ge 1,\varphi o\rho m=1,\dots ,M,$$

(5)

$${\sum }_{m=1}^M{\sum }_{i=1}^{n_m}{C}_{mi}{X}_{mi}\le B,$$

(6)

$$X_{mi}\in \left\{\mathrm{0,1}\right\},\varphi o\rho m=1,\dots ,M\mathsf{\alpha }\mathsf{\nu }\mathsf{\delta }i=1,\dots ,n_m$$

(7)

$$Y_{md}=f_{md}\left(X_{m1},\dots ,X_{m{n}_m}\right),\varphi o\rho m=1,\dots ,M,\mathsf{\alpha }\mathsf{\nu }\mathsf{\delta }d=1,\dots ,D,$$

(8)

$$Y_{mi}\ge 0,\varphi o\rho m=1,\dots ,M\mathsf{\alpha }\mathsf{\nu }\mathsf{\delta }i=1,\dots ,n_m$$

(9)

The objective function in this case aims to maximize the overall survivability of all components (nodes) against all potential disasters. Also, note that component (node) m fails against disaster d only when all of its selected solutions fail at the same time. For a component (node) m and a disaster d, we define $Y_{md}$ as the total contribution of applying all or some of available solutions $X_{mi},\left(i=1,\dots ,n_m\right)$. The utility of such solution application is equal to $U_{md}\left(Y_{md}\right)$. It is worth mentioning that GRAP is written in a generic format. When estimating the functions and parameters, one possibility is to use of game theory, see for example [40,41] for a survey.

Moreover, it is important to note that GRAP is a non-linear integer program. However, if $Y_{md}={\sum }_{i=1}^{n_m}{a}_{mi}{X}_{mi},\mathrm{for}m=1,\dots ,M,\mathrm{and}d=1,\dots ,D,$ and $U_{md}\left(Y_{md}\right)=Y_{md}$where $a_{mi}$ is a constant weight, then the objective function is separable and linear which is a special case of the generalized assignment problem (GAP) [37]. Although GAP is known to be strongly NP-hard, it is easier to solve compared to a non-linear optimization with a non-separable objective function like the GRAP. A variety of exact and heuristic algorithms are available for GAP (see for example, [28,29], for a recent survey). A variety of exact and heuristic algorithms are available for GAP (see for example, [32,33], for a recent survey).

In the following section, we will demonstrate how the GRAP can be transformed into an optimization with separable objective function, and linear constraints, converting it into a GAP. By taking logarithm from both sides of equality (8), we will have the following.

$$log\left(Y_{md}\right)=log\left({\prod }_{i=1}^{n_m}{v}_{mid}^{X_{mi}}\right)={\sum }_{i=1}^{n_m}{X}_{mi}\left(log\left(v_{mid}\right)\right)$$

(10)

and

$$-log\left(Y_{md}\right)={\sum }_{i=1}^{n_m}{X}_{mi}\left(-log\left(v_{mid}\right)\right)$$

(11)

Let $Z_{md}=-log\left(Y_{md}\right)$ and $K_{mid}=-log\left(v_{mid}\right).$ Since $0\le v_{mid}\le 1$ then we have $K_{mid}\ge 0$ and thus, we have

$$Z_{md}={\sum }_{i=1}^{n_m}{K}_{mid}{X}_{mi},\varphi o\rho m=1,\dots ,M,d=1,\dots ,D$$

(12)

Note that, we have ${\prod }_{i=1}^{n_m}{v}_{mid}^{X_{mi}}=2^{-z_{md}}$. With this in mind, the GRAP transforms into the following, where the objective function is separable and nonlinear. Please note that the last term in the objective function is constant thus can be ignored.

(GRAP)

$$Maximize{S}^{\mathrm{*}}={\sum }_{d=1}^D{P}_d\left[{\sum }_{m=1}^M{w}_m(1-2^{{-Z}_{md}})\right]$$

$$={\sum }_{d=1}^D{P}_d\left[{\sum }_{m=1}^M{-w}_m{2}^{{-Z}_{md}}\right]\hspace{1em}\hspace{1em}{\sum }_{d=1}^D{P}_d\left[{\sum }_{m=1}^M{w}_m\right]$$

(13)

s.t.

(5-7, 12)

$$Z_{md}\ge 0,\varphi o\rho m=1,K,M\mathsf{\alpha }\mathsf{\nu }\mathsf{\delta }i=1,K,D$$

(14)

Since $p_d$ for d=1,…,D, is an array of constants in the objective function and is not part of any constraint, thus in order to maximize S*, for a given d we need to optimize

$$\mathit{min}{\sum }_{m=1}^M{w}_m{2}^{-z_{md}}$$

(15)

Now, since $w_m$ for m=1,…,M, is also an array of constants and the decision variable is $Z_{md}$, for each m we need $2^{-Z_{md}}$ to be as small as possible under the constraints and thus $Z_{md}$ must be as large as possible. This proves that the RAP is equivalent to the following linear integer program.

$$\mathit{max}S\mathrm{*}\mathrm{*}={\sum }_{d=1}^D{p}_d\left[{\sum }_{m=1}^M{w}_m{Z}_{md}\right]$$

(16)

s.t. (5-7) and (9-10)

Since $Z_{md}={\sum }_{i=1}^{n_m}{K}_{mid}{X}_{mi}$, thus we can restate the RAP as the following generalized assignment problem.

$$\mathit{max}S\mathrm{*}\mathrm{*}\mathrm{*}={\sum }_{d=1}^D{p}_d{\sum }_{m=1}^M{w}_m{\sum }_{i=1}^{n_m}{K}_{mid}{X}_{mi}$$

(17)

s.t.

$${\sum }_{i=1}^{n_m}{X}_{mi}\ge 1,\varphi o\rho m=1,\dots ,M,$$

(18)

$${\sum }_{m=1}^M{\sum }_{i=1}^{n_m}{C}_{mi}{X}_{mi}\le B,$$

(19)

$$X_{mi}\in \left\{\mathrm{0,1}\right\},\varphi o\rho m=1,\dots ,M\mathsf{\alpha }\mathsf{\nu }\mathsf{\delta }i=1,\dots ,n_m$$

(20)

Constraints (19) are capacitated with the budget limit. Following our recently published method (the r-flip paper [43] and recent papers [35,44]), we implemented a r-flip local search heuristic to improve the assignment. In this r-flip heuristic, we choose r = 2,3, and 4 for the assignment of both components (nodes) and assets that components (nodes) s chose from to improve the survivability. The improvement process based on the r-flip heuristic is implemented by the Tabu Search algorithm with an embedded strategic oscillation, as detailed in .

3. Results

The statistical simulation and optimization experiments are coded in R. We chose a power grid dataset to illustrate the proposed framework (Figure 2). The dataset has 4941 nodes, 6594 edges, with a maximum distance of 45 between the pair of nodes in the graph. The minimal distance for 90% of nodes pairs is 26.

The distribution of degree nodes in power grid follows the power law distribution with p value=0.76, which is greater than 0.05, so the data follows the power law distribution [42]. Figure 3 shows the degree distribution of nodes in the power grid.

The eigenvector of nodes is obtained by using Spectrum function and the smallest eigenvalues are removed from the matrix. Following the TW distribution test, the critical nodes are identified by the optimization model (1)-(4). Figure 4 highlights the results of spectral clustering of critical nodes with large vector size. Figure 5 shows the network structure after the critical nodes are removed from the graph. The cost of redundancy allocation is computed based on the critical nodes by a heuristic algorithm. Table 3 displays the network nodes’ connectivity after the critical nodes are removed and the average nodes’ connectivity are measured by the complement of fragmentation score.

To evaluate the impact of resource allocation costs on the redundancy of critical nodes using benchmark datasets, we first identify critical nodes based on a reliability threshold ranging from 98% to 99% for the power grid system. In this study, we specifically examine power transformers as the primary components. The cost of a power transformer ranges from $600,000 to $4,000,000, with a 15-year life cycle. We randomly assign a cost between $600,000 and $4,000,000 for the critical nodes. Table 4 presents the costs associated with redundancy on critical nodes to maintain 98.79%-99.74% reliability (connectivity). A Congressional Research Service study in 2012 estimated the inflation-adjusted cost of weather-related outages at $25 to $70 billion annually (https://www.energy.gov/articles/economic-benefits-increasing-electric-grid-resilience-weather-outages). Notably, the cost of redundancy is only a fraction of the economic losses incurred due to weather-related outages.

4. Discussion

The Smart Grid incorporates proven technologies to optimize its assets – from power plants to distribution substations and critical infrastructure. These advancements lead to increased power flow through existing assets and provide utilities with precise insights, enabling them to assess the necessity for additional power plants accurately. Operational enhancements span improved load factors to reduce system losses, resulting in a net reduction in utility costs and enhanced overall efficiency.

The results of this study highlight the key strategies to improve the reliability of the Smart Grid:

• Redundancy Planning: Identify critical components in the Smart Grid infrastructure. Allocate redundancy by duplicating these components, ensuring backup systems are in place to seamlessly take over in case of failures.
• Risk Assessment: Conduct a thorough risk analysis to understand potential failure points. Allocate redundancies to the most vulnerable areas identified during this assessment.
• Advanced Monitoring: Implement real-time monitoring systems to detect anomalies and potential failures. Use data analytics to predict failure patterns and allocate redundancies accordingly.

5. Conclusions

In this paper, we presented a framework that combines statistical learning and optimization to identify critical nodes in the smart grid infrastructure. To optimize the resource allocation for critical nodes, we proposed a general redundancy allocation model based on generalized assignment problem (GAP). It includes the generalized redundancy allocation problem (GRAP) as a special case. We gave an equivalent linear GAP of GRAP.

Power outages pose an extensive list of risks, including but not limited to economy, health, and public safety. Only weather-related outages are estimated to cost between $25-70 billion to the US economy annually [45]. Thus, it is vital to develop risk assessment and quick-response plans. Combination of statistical simulation and integer programming-based optimization approach proposed in this study promises an efficient framework for managers and decision-makers in determining the critical components of smart grids and optimizing redundancy allocation for a well-planned, organized, and coordinated course of action to be followed in disaster recovery.

Another implication of this study for managers is the improved capability in assessing risks and vulnerabilities of the smart grid for redundancy allocation while using limited resources in the most efficient way. Performance of smart grids are closely related to the reliability and uncertainties involved. Thus, risk assessment to systematically detect the vulnerabilities with the potential to result in grid failures is an essential component for the future of smart grids.