A Review of Uncertainties in Power Systems - Modelling, Impact, and Mitigation

1. Introduction

The development and evolution of identified uncertainties in power systems have continued to progress over the years. Understanding this evolution is crucial for comprehending the challenges and developments in the field of electrical power generation, transmission, and distribution. The introduction provides a broad summary of the progression of uncertainty. From the late 19th century to the mid-20th century, power systems were marked by relative simplicity, featuring centralized power generation in local power plants [1]. Uncertainty is mainly demand and occasional technical failures. As the power systems grew, the unpredictability of load and outages require manual adjustments. The mid-20th century marked the growth of interconnected power grids to facilitate electricity transfer across regions [2]. As grids expanded, the uncertainty associated with power transfer, transmission losses, and voltage regulation became prominent concerns. The late 20th century saw the integration of RESs, such as wind and solar, into the grid [3]. This introduced a new dimension of uncertainty related to RES generation patterns. The late 20th century and early 21st century witnessed the liberalization of energy markets. Market dynamics, with price fluctuations and market manipulation, added an economic layer of uncertainty to the power system [4].

Advancements in technology introduced both opportunities and challenges in the 21st Century. On one hand, grid automation, advanced sensors, and smart grids improved system control and efficiency. On the other hand, these technologies introduced concerns about cybersecurity and the need for increased resilience against cyber threats. In many regions, aging infrastructure has become a significant concern, leading to uncertainty regarding the reliability and lifespan of power system components. Uncertainties raised from the integration of RESs and AMIs have been a serious threat for different Australian regions with dramatic impacts in terms of power system outages [5]. Independent system operators, transmission and distribution network service providers, regulators, and policymakers are collectively working to enhance system resilience through various approaches. AMI networks are being deployed worldwide, with penetration rates in Victoria, Australia reaching as high as 30% for solar customers in 2021 [6]. While existing security designs offer assurance against successful cyber-attacks, the potential consequences of a breach remain high. An analysis of a load drop attack on an AMI system using a cyber-physical model is given in [7].

A crucial distinction in uncertainty sources lies in their nature as continuous or discrete. Variables like renewable generation and prices, allowing any value in a range, are modeled as continuous random variables [8]. They exhibit infinite support, seen in normally distributed random variables. Conversely, discrete events such as component outages are represented by discrete random variables [9]. The choice of probability distribution depends on the time scale; short-term variations may use a Gaussian distribution, while long-term scenarios follow a Weibull distribution [10]. Uncertainty characterization also varies with decision-makers; one company perceives bids as uncertain, while the system operator views them as deterministic. Figure 1 illustrates the spatial and temporal scale of planning activities in electricity markets, providing a visual representation [11]. Nodal locational marginal price (LMP) represents the market price at a particular node or location within the power grid, indicating the cost of delivering one additional unit of electricity at a specific point in the power system based on the supply and demand conditions at that location. Uncertainty significantly grows with the expansion of the system size and longer estimated time scales. In this case, the decision variables for long-term transmission level planning and short-term local network operation are different. In long-term planning, the focus may shift towards assessing system adequacy through extreme scenarios rather than considering the entire distribution.

In the future, the primary focus could be on climate change and environmental concerns [12]. As climate change emerges as a pressing issue, there is an increasing emphasis on reducing carbon emissions. With high penetration of RESs in the power system, weather-related uncertainties have a great impact on the reliability, stability and efficiency of the power system from a wide range of aspects including unpredicted short-term events and long-term climate changes. Moreover, the consequences of the uncertainties extend far beyond mere inconveniences. They can result in extensive blackouts, leading to cascading power system failures, which can, in turn, force the utility company into bankruptcy [13]. Climate Central found that severe weather was responsible for 80% of large-scale power outages. Among these instances, 59% were attributed to heavy rainfall and thunderstorms [14]. Weather-related uncertainties have a significant impact on the entire energy supply chain, particularly power generation, transmission, and distribution [15]. Understanding these uncertainties should not only be limited to technical experts but also include policymakers and the general public to ensure informed decision-making in the energy sector [16]. In this paper, we categorize key weather-related uncertainties, examine their effects on power systems, and explore suitable methods for estimating and forecasting these uncertainties, as well as strategies for compensation and mitigation. The mitigation approaches increase the balance requirements with associated costs. Reconfiguration, adaptations in infrastructure, grid management, and energy storage operation are key components of a strategy to mitigate the effects of weather-related uncertainties on power systems [17]. Unfortunately, there is no unified control strategy to tackle all possible uncertainties. Besides, integrating uncertainty into public energy research and development decisions: the decision made by the policy makers cannot be only made for the near future but also consider the possible uncertainties. The prospects of the power system mainly involve digitalization and artificial intelligence (AI) [18]. The power system is increasingly adopting data-driven technologies and artificial intelligence AI to improve forecasting, grid management, and demand-side response, offering potential solutions to long-standing challenges.

The power system in Australia boasts a vast geographic reach accommodating diverse energy sources and decentralized population centres. As Australia strives for its unique challenges, the power system is at a juncture, requiring strategies for sustainable and resilient future development. The existing review articles for power system uncertainties lack an event-specific analysis of impacts, comprehensive modelling and mitigation approaches, and potential limitations and future directions for the newly raised uncertainties in modern power networks. Moreover, in this paper, the mitigation methods focused on emerging technologies, particularly in terms of the coordinated operation of DERs and data-driven methods such as predictive maintenance. This review illustrates the evolving nature of uncertainty in the power system, which underscores the importance of adaptability and innovation to meet the changing demands of modern electricity networks.

The rest of this paper is organized as follows. Section 2 offers a description of the impact of uncertainties including weather-related events, cyber-attacks, and asset management. Section 3 provides a comprehensive review of the existing modelling approaches for the aforementioned uncertainties. Section 4 reviews the mitigation approaches, especially the emerging technologies which may ultimately result in options that reduce the reliance on extensive powerlines, for those uncertainties. Section 5 outlines the limitations and future directions of mitigation strategies, weighing the trade-offs among reliability, economic efficiency, and computational speed. Finally, conclusions are drawn in Section 6.

2. Impact of the Uncertainties

The electricity grid faces vulnerabilities stemming from outdated system flexibility, aging assets, unpredictable weather patterns, and cyber-physical security threats. In the US, weather disasters have been a primary cause of power outages, with a total of 178 incidents from the 1980s to 2014, including 8 in 2014 alone. The cumulative damages from these events surpassed US $1 trillion [19]. Numerous industries came to a standstill for hours, if not days, and several individuals requiring specialized healthcare lost their lives due to prolonged power outages and the inability to promptly restore electricity [19].

Anticipated climate change is poised to escalate the frequency, intensity, and duration of extreme weather events [20]. While rain and floods do not directly endanger overhead transmission lines, they pose a threat to substation equipment like switchgear and control cubicles. The combination of rain with strong winds or lightning, however, can emerge as a significant threat to overhead lines. In addition, uncertainties in the evolving nature of cyber threats can reduce the reliability of critical infrastructures. These disruptive events can be classified into three categories: (1) the uncertainties that will mainly affect the power generation; (2) factors like rainfall, bushfires, and aging infrastructure that affect both power generation and one or more electrical infrastructure components; and (3) cyber-attacks which affect the power system via communication links.

2.1. Uncertainties Affecting the Generation

Incorporating solar and wind energy sources into power systems has gained growing significance in the effort to reduce carbon emissions to reach a sustainable energy future. However, the intermittency and generation uncertainties associated with these RESs could pose challenges to the operation of power systems. Since wind turbines and solar power systems were first connected to the power grid in 1941 and 1954, respectively [21,22], they have experienced rapid and continuous growth. Numerous countries have introduced incentive and tax credit programs to facilitate their achievement of approved national and local renewable portfolio standards (RPSs). Consequently, the strong growth of wind and solar power has driven advancements in wind turbine and photovoltaic (PV) technologies, culminating in reduced costs. Figure 2 illustrates the evolution of renewable energy generation from various sources between 1965 and 2022, with solar and wind generation emerging as the primary contributors to renewable energy in 2022 [23]. Solar power has gradually surpassed wind generation due to the promotion and adoption of rooftop solar panels. According to AEMO’s estimated step change scenario in their Integrated System Plan 2022, renewables are expected to make up 98% of the total annual generation by 2050, with both distributed (rooftop) and utility-scale solar and wind energy taking dominance in the generation mix [24]. Therefore, it is essential to accurately estimate the impact of high penetration of wind and solar power on the system operation and implement effective mitigation measures [25].

The impact of integrating solar and wind power into a power system is contingent on two primary factors: penetration level and system flexibility. A higher penetration level indicates that a larger portion of energy generation comes from RESs like solar and wind. This can lead to reduced reliance on fossil fuels and decreased carbon emissions, leading to a more sustainable future. However, the increased penetration level may also strain grid stability due to fluctuations in supply and demand [26]. In terms of system flexibility, it is defined as the ability to balance supply and demand in real-time operation, which is crucial when integrating solar and wind power into the grid [27]. Enhancing system flexibility can be achieved through the incorporation of ESSs, D-FCAS programs, and MG technologies, as elaborated in subsection 2.13. In general, the impacts of wind and solar power integration can be categorized as follows:

The intermittent performance of wind and solar generation may lead to suboptimal operation of conventional generation units during the unit commitment process. [28]. The cost of high-RES penetration on thermal units has been further investigated in [29,30].

The utilization of induction generators in wind turbines inherently leads to the absorption of reactive power from the grid. For fixed-speed turbines, fluctuations in reactive power output can result in voltage fluctuations. Nevertheless, modern variable-speed turbines, e.g., doubly-fed induction generators (DFIGs), can offer reactive power support through suitable interfacing methods [31].

Fast ramping and frequent start-up generation units are required to provide reserves for wind and solar power fluctuations, which could also increase the operational cost of power systems [32，33]. Moreover, the lack of system flexibility with integrated wind and solar power has the potential to trigger blackouts in the power system. In February 2021, the cold period in Texas led to the icing of the turbine blades, which prevented the turbines from operating, causing a total power outage [34]. Researchers in [35] found that over 60% of solar penetration will cause blackout due to load imbalance according to the simulation of the Kythnos power system. Consequently, protective measures are essential to maintain system frequency and prevent cascading failure events.

Short-term impacts that are associated with operational time scale are listed above, whereas long-term impacts that involve planning for peak load periods are listed below:

Transmission congestions can occur when the wind and solar generation is away from the load centre [36]. Moreover, the high RES penetration level will lead to higher transmission capacity, thereby, increasing distribution losses.

Due to the intermittency and variability of solar and wind generation, it is challenging to rely solely on these resources to meet load demand during peak hours, especially during cloudy or windless periods. In this case, the paper emphasizes the importance of demand-side ancillary services in 2.13, which can be achieved by either encouraging customers to adjust energy usage during peak hours or establishing VPP to optimize load curtailment.

2.2. Uncertainties Affecting the Network Assets

Both wildfires and rainfall have a strong impact on the overall reliability of the power system, and the frequency and intensity of these events can affect the long-term planning of power systems. Wildfire risks to the grid have emerged as a global concern in recent years. During early 2020, numerous wildfires erupted in all states in Australia destroying more than 10 million hectares of land. Strong wind associated with wildfires can cause transmission lines to sag or break, which interrupts the electricity supply over long distances. At the peak of wildfires, approximately 20,000 households experienced power supply disconnections in New south Wales [37]. Moreover, the Brazilian National Institute for Space Research detected 103,000 wildfires in the Brazilian Amazon, reflecting an annual increase of 16% [38]. Similarly, the increased frequency and duration of excessive rainfall can damage transmission lines, towers, and substations leading to power outages. The heavy rainfall could also influence the hydropower generation by increasing water flow in rivers. This paper specifically delves into the analysis of potential modelling and mitigation strategies concerning rainfall and wildfires, with a primary emphasis on infrastructure damage that encompasses the destruction of the power transmission network. It’s important to note that the examination of the impact and corresponding mitigation strategies for natural disasters, i.e., hurricanes, is beyond the scope of this study.

In addition to external factors such as wildfires and heavy rainfall causing failures, the health condition of aging assets must also be taken into account, as it is a significant contributor to common asset failures. The impact of electric asset failure can range from minor to sever, contingent upon specific circumstances. A minor failure might result in a temporary and localized power outage causing inconvenience for the customers, which has limited widespread. In contrast, a severe failure could lead to a widespread and prolonged blackout, affecting a large number of populations and causing great economic losses. Moreover, severe failures could have cascading effects, impacting various sectors including healthcare, communication, transportation, and manufacturing. Understanding the spectrum of potential consequences is crucial for developing uncertainty mitigation strategies to ensure the resilience of the power system. It should be emphasized that the consequences aspect of risk is highly case-specific for asset failure [39].

Apart from the direct consequences of asset failures, they can also precipitate social unrest and political change. For instance, the poorly implemented electric deregulation in California contributed to Governor Gray Davis losing a recall election, marked by a surge in consumer prices and rolling blackouts statewide in 2003 [40]. Following the events at Fukushima Daiichi in 2011, the political landscape in Japan remains uncertain, despite tentative steps to restart some of the nation’s reactors [41].

2.3. Uncertainties Affecting the Communication Link

According to [42], one of the unifying concepts for cyber-attack is a cause of system failure including intentional, malicious, and human-induced faults in both software and hardware. If cyber security is not managed and controlled effectively, the cyber-attack can reduce the stability of the operation network as evidenced by the U.K. power outage in 2019 [43], and the ERCOT incident in 2009 [44]. In March 2019, a cyber-attack targeted a wind plant in Salt Lake City, Utah, U.S., resulting in the loss of operator control over wind turbines [45]. Subsequently, The U.S. Department of Energy emphasized the critical necessity for dedicated efforts to identify vulnerabilities, raise awareness, and develop strategies to protect wind energy infrastructure against cyber-attacks [45].

Figure 3. Physical and communication structure of the smart grid [46].

Figure 3. Physical and communication structure of the smart grid [46].

A typical multi-layer architecture of a smart grid is demonstrated in Figure 3, including application, communication, physical, and data acquisition layers. The objective of the application layer is to participate in the energy and reserve market and control the actuators based on their status. Typically, the most significant consequences of an attack occur when the attacker gains access to supervisory control and data acquisition (SCADA) systems and initiates control actions [47]. Attackers can manipulate raw data measurements, introducing unnoticed errors into estimates of state variables such as bus voltage magnitude, phases, and line power. This risk arises when attackers exploit the tolerance for small errors within state estimation approaches, posing a significant threat to the security of the power system [48]. Moreover, the false estimation can lead to incorrect decisions making the real-time prices of the electricity market only profitable to the attackers.

3. Uncertainty Modelling

System operators, generation companies, and consumers depend on various input data to establish parameters in a mathematical optimization model, enabling them to make informed decisions. However, a large number of these parameters are uncertain. For instance, wind and solar energy generation is affected by weather forecast uncertainty and the electricity prices are influenced by the RESs generation and other participants in the market. Researchers in [31] highlight the challenges in integrating large-scale renewable energy generation into the grid due to its variability and the need for accurate forecasting models to reduce uncertainty. This paper focuses on the review of current forecasting and estimation models for weather conditions, which could serve as inputs for both uncertainty mitigation strategies and modelling processes for wildfire and aging assets.

Forecasting methods for wind and solar energy may vary depending on many factors, including the available weather information, prediction horizon and resolutions, and historical data. According to [49-52], the existing forecast methods can be categorized based on forecasting models, forecasting horizon, and performance metrics as depicted in Figure 4. The physical models are deterministic and depend on meteorological parameters, e.g., pressure and temperature. This model uses complex mathematical equations to simulate the atmosphere’s behaviour, which is complex and time-consuming [49]. Statistical models are the combinations of auto-regressive (AR) and moving-average (MA) models, e.g., autoregressive moving average model (ARMA), seasonal autoregressive integrated moving average (SARIMA), and autoregressive integrated moving average (ARIMA) [53]. These models utilize the historical data of the time series to estimate the future value of solar and wind generation. The key benefit of statistical models is their ability to provide highly accurate short-term forecasts, as indicated in reference [54]. Following [54], the formation of ARMA$\left(p,q\right)$, where $p$ stands for the number of flags in the AR model and $q$ stands for the number of flags in the MA model, is illustrated as follows:

$${\widehat{P}}_t={\displaystyle \sum }_{k=1}^p{\alpha }_k{P}_{t-k}+{\displaystyle \sum }_{k=1}^q{\theta }_k{ϵ}_{t-k}+{ϵ}_t$$

(1)

$P_t$ and ${ϵ}_t$ are the power generation and the corresponding error at time $t$, respectively, and ${\alpha }_k$ and ${\theta }_k$ are the coefficients that can be trained by the historical data. Compared with ARMA, ARIMA

added another variable $d$ which converts non-stationary data into stationary, making the model more versatile for a wider range of time series data and suitable for medium-term predictions. Moreover, SARIMA is an extension of ARIMA by capturing the seasonal variations for long-term wind and solar power prediction [55].

Machine learning models mainly include logistic regression, linear regression (LR), decision trees, k-nearest neighbors, support vector machines (SVM), artificial neural networks (ANNs). These models also belong to the realm of supervised machine learning where the output is modelled as a function of inputs. Normally, ANNs require a big data set with huge training time and the results can vary a lot depending on the input data. Compared with other ANNs, Back propagation networks (BPN) offer greater efficiency in learning and are relatively simple to be implemented. The BPN model is not only capable of making smooth predictions but also of identifying short-term patterns within time series data [49]. Researchers in [56] proved that SVM based forecasting model is more accurate than the conventional forecasting methods for solar radiation prediction. Another solution for solar and wind energy forecast is using deep learning techniques which are capable of obtaining high accuracy in various applications [57-60]. Deep learning models commonly involve using large amounts of data and complex patterns, e.g., convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM) networks. RNN-LSTM model for solar power forecast is applied in [61, 62], where the model consists of two LSTM layers, two hidden layers and one output layer The Adam optimizer is selected to optimize the weight parameters in each layer of the RNN-LSTM model due to its adaptive learning rate capability, and the activation function for the LSTM layers are hyperbolic tangent. The inputs for the solar forecast model include time, windspeed, air pressure, humidity, temperature, wind direction, and pyranometer. Multiple forecasting models can be combined to generate a hybrid model, e.g., the combination of ARIMA and ANN, to give a better prediction.

At a given location, the normalized standard deviation of forecasting errors of a single solar or wind farm tends to rise as the forecast horizon increases, and the forecasting methods that are suitable for short-term forecasts may not be applicable for long-term prediction. Therefore, the forecasting methods can be classified based on the prediction horizon including (1) short-term forecast: from a few minutes to a couple of hours ahead; (2) day-ahead forecast: 24 hours to 48 hours ahead; (3) medium-term forecast: one week to two weeks ahead; and (4) long-term forecast a month to several months ahead [63]. As the uncertainties for solar and wind power are not very large throughout the lifetime of a wind turbine, i.e., within a 10% deviation, seasonal variations are generally more predictable than annual ones [49, 52]. Besides, the solar and wind energy associated with weather systems are challenging to predict beyond a few days, whereas the diurnal variations that occur within the day are more predictable.

The performance metrics used in the current literature are mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). An overview of current solar and wind energy forecasting methods is given in Table 1.

3.1. Uncertainties Affecting the Generation

The outputs obtained from the aforementioned point forecast methods cannot be directly utilized for decision-making by system operators due to their inherent limitations in providing sufficient information. In other words, instead of using only forecasting values to solve a deterministic problem, the literature applies a stochastic approach to ensure the robustness of the dispatch decision against all possible solar and wind energy generation uncertainties. As a result, outputs from the forecasting models are employed to generate probabilistic forecasts which can be categorized as (1) probabilistic distribution; (2) distributional robust formulation; (3) independent identically distributed samples; and (4) robust uncertainty set [9]. The uncertainty parameter which is the solar and wind generation forecasting error is defined as $\delta$ in this paper. The actual solar and wind generation equals the forecast value plus the error, expressed in vector form.

$$P=\widehat{P}+\delta$$

(2)

Assuming a normal distribution for the $\delta$ is most commonly used in the current literature for the sake of simplicity. However, the actual distribution is difficult to obtain in reality, and it might not perfectly follow a predefined distribution. Researchers in [70] propose using beta distribution to represent wind forecast error instead of using the normal distribution. To overcome the drawback of defining a probabilistic distribution, a family of distributions in which the uncertainty parameter $\delta$ may fall is defined in [71]. The distributional robust formulation includes all possible distributions as the ambiguity set $B$. If we can access many possible realizations ${\delta }_s$, e.g., historical data, and assume these samples are independent and identically distributed (IID), these samples can represent the uncertainty parameter with the same probability of occurrence. The main drawback of IID is that an accurate probability distribution of $\delta$ may require consideration of a large number of samples and the historical data are normally not IID [72]. The uncertainty parameter $\delta$ can also be modelled without defining the possible distributions. Furthermore, we might lack adequate data to estimate the distribution of $\delta$. Under this situation, the robust uncertainty set $\Delta$ can be defined using a set of scenarios ${\delta }_s$ which include all possible realizations of the uncertainty or allowing continuous variations of $\delta$ but restricting these fluctuations to remain within a predefined set, e.g., elliptical sets and box-constrained sets [73]. The scenarios for robust uncertainty sets can be generated using the Markov chain Monte Carlo (MCMC) model. Motivated by [74], the researchers in [75] propose a two-dimensional MCMC model assuming the current state of uncertainty parameter ${\delta }_t$ depends on the previous state of ${\delta }_{t-1}$ the forecast value at $t$ and a constant conditional probability. In this case, the properties, i.e., auto-correlation function (acf) and probability distribution function (pdf) of the uncertainty parameter can be maintained without explicitly defining them. However, due to the intermittency and the forecast cannot reach 100% accuracy, the solar and wind generation uncertainties cannot be entirely tackled by relying only on the forecast methods.

3.2. Wildfires and Rainfall

A crucial motivation for enhancing grid resilience is the prediction of the grid state [76], representing how the grid or its components perform for various weather events, e.g., heavy rainfall and wildfire, and it can influence the grid operating conditions. The grid state can be presented by the binary variable $I$ as below:

$$I=\left\{\begin{array}{c}-1; Outage state\\ 1; Noraml state\end{array}\right.$$

(3)

Grid state serves as an essential input factor for the decision-making process of power system planning and operation. Before implementing measures to mitigate the impact of weather events, various strategies can be employed to estimate the potential grid state based on prevailing weather conditions in each region. Data-driven methods, particularly machine learning (ML) models, offer an effective means to model the intricate correlation between the grid state and the associated weather conditions. These methods demonstrate efficiency in terms of time and resource requirements, as they do not rely on a physical model. For instance, a predictive approach based on linear regression models was introduced in [77], aiming to forecast the electrical infrastructure damages caused by heavy rain and storm events.

The reliability, i.e., failure rate, of transmission and distribution facilities is significantly affected by the environment where they are located. As the weather prediction methods have been illustrated above, in this subsection, we conducted an analysis of the correlated characteristics between weather patterns and outage distributions to characterize both rainfall and bushfire events. A novel method is introduced to establish a correlation between the severe weather risk index and the power network model, incorporating geographical information about the transmission lines [78]. However, it’s worth noting that a long transmission line may traverse multiple weather regions, potentially encountering different weather conditions in each region.

For instance, lines 7-8 and 15-16 cross two different weather regions in Figure 5. Consequently, the influence of weather conditions on a single line will differ from one region to another. Precisely defining the weather-affected failure rate of high-component components is crucial for modeling the impact of weather on the entire power infrastructure. Two common approaches exist for defining the failure rate of a transmission line crossing multiple regions. The first approach involves employing the weighted-average method, where the summation of the point-specific failure rates along each line segmentation is calculated based on an incremental multiplier of failure rate (IMFR) [79]. In the second category, the failure rate for the entire transmission line was determined by selecting the highest failure rate from any single point of a line [80].

To depict the correlation between electric component failure rate and weather information, the risk for wildfire and rainfall needs to be estimated. The rating systems for wildfires are emphasized in this subsection, where different standard is selected for each country. The Canadian Forest Service Fire Weather Index (FWI) is employed to assess the distribution of wildfire risk across a power grid. While, in Australia, the forest fire danger index (FFDI) is used widely to indicate the likelihood of wildfire in various regions. FFDI encompasses measures of dryness based on rainfall and evaporation, temperature, wind speed, and humidity, without considering the potential risk from fuel management and lightning [53]. According to Figure 6, the risk of wildfire in a certain region can be classified into six categories including low-moderate, high, very high, sever, extreme, and catastrophic based on the Australian standard.

The impact of wildfire on the electricity supply of the transmission lines crossing the corresponding weather region can be quantified by constructing the fragility curve of the sub-transmission line failure rate with the weather conditions within the region. The fragility curve for the line failure rate and restoration time under varying risk levels can be achieved by a parametric method assuming that $p_0$ is the nominal line failure rate. The line failure rate remains constant when the FFDI is below $11$, and the slope of the fragility curve is $s$ when the FFDI is greater than 11. Therefore, the fragility curve of the line failure rate can be formulated as [81]:

$$p\left(FFDI\right)=\left\{\begin{array}{c} p_0, FDDI\le 11\\ \min \left\{s\left(FFDI-11\right)p_0+p_0,1\right\}, FDDI>11 \end{array}\right.$$

(4)

3.3. Asset Management for Aging Assets

Condition-based risk management (CBRM) can be employed to analyze the health conditions for assets within the network and predict the potential risk to renew in optimal time. It operates as a structured framework that amalgamates data, engineering knowledge, and experience, creating a transparent, robust, and repeatable decision-support tool. The CBRM process unfolds through a series of sequential steps [82]: 1) define health index (HI) to indicate the asset conditions; 2) derive the correlation between HI and the probability of failure (POF) of the assets; 3) employ the knowledge of degradation process to estimate the future condition and performance of assets, where HI and operation conditions are two main factors at this step; 4) assess consequences of failure (COF) to evaluate the consequences in specific categories, including safety, environmental and financial; 5) establish a risk model that combines POF and COF to quantify the risk and give priority to each asset; and 6) evaluate the optimal interventions in terms of risk and recalculate the new POF and COF after the risk reduction process. Overall, Steps 1-3 pertain to asset condition and performance, offering a systematic process to identify the relationship between the condition and Probability of Failure (POF). Steps 4-6 provide the assessment of COF and enable the quantification of risk when combined with POF. The relationship between the defined HI and POF at step 2 is depicted in Figure 7, where the curve is constructed by curve fitting technology given the function of the calibration curve. HI is affected by normal asset life, current age and conditions, and the degradation processes of the asset, and a higher health index indicates a lower condition.

3.4. Cyber Attacks Detection

Cyber-attacks can influence critical power system operations, including state estimation, automatic generation control, voltage control, and energy market processes.

Attacks on automatic generation control (AGC): the purpose of AGC is to adjust the power output of the generators within an area to maintain the frequency and power exchange via the tie line. Given that the time resolution of AGC control signals operates on a timescale of seconds, it cannot afford to utilize elaborate data validation algorithms [84].

Attacks on state estimation: state estimation is crucial for operational decisions for a smart grid. The motivation for leading attacks on state estimation varies from causing blackouts to gaining financial profit from the market.

Attacks on the energy market: cyber-attacks on the energy market can be achieved by denial of service or injecting wrong data, e.g., jamming of price signals.

Attacks on voltage control: Compared to the system frequency, voltage stability is a major concern in long distribution networks. Load tap changing (LTC) transformers are widely applied to improve voltage stability in response to load variation. The control topology for LTC makes it vulnerable to man-in-middle attacks.

Various methods have been employed to detect the abnormal conditions for each of the above-mentioned system operations. Most research works have been done using ML algorithms, e.g., K-means, Q-learning, and generative adversarial networks (GANs) to generate malicious data for attacks. In [85], SVM-based technology and statistical anomaly detection are applied to identify the false data injection, which outperforms the statistical approach. Researchers in [86] propose a conditional deep belief network (CDBN) detection strategy that extracts temporal features from the measurements of distributed sensors, which is robust against different environment noise levels. Dynamic Bayesian network is applied in [87] to detect unobservable attacks, i.e., too many measurements are discarded in the system. In [88], a physics-data-based detection method is proposed to detect the cyber-attack on TLP transformers to ensure solar farm security.

4. Mitigation Approaches

This paper first introduces several common mitigation approaches based on the operational procedures for each type of uncertainty mentioned in this study and subsequently focuses on the most emerging technologies including microgrid, demand-side management, and battery control strategies, potentially offering alternatives to extensive powerline reliance in high-risk environments. It is worth noting that apart from the operational approaches discussed in this paper, hardware updates can be considered, e.g., undergrounding the distribution and transmission lines, to increase the robustness of the components to sever weather conditions.

4.1. Uncertainties Affecting the Generation

The objective of this paper is not to offer a comprehensive tutorial on any specific method. Instead, we explore the essential attributes and potential drawbacks of various optimization methods and developed mitigation techniques, providing readers with the knowledge necessary to make informed choices based on their specific context. The intermittent nature of wind and solar energy presents a demanding constraint for power system operators, as they must continually balance supply and demand in real-time. In this review paper, strategies for addressing uncertainty in solar and wind generation encompass various mitigation methods. These methods involve fundamental optimization under uncertainty techniques and the developed market and operation frameworks including (1) microgrid strategies, (2) ESS operation strategies, and (3) D-FCAS framework. However, incentive the customers to rely on energy mix for power supply rather than individual resources is not the focus of this study.

To address the uncertainty parameter modelled in the previous subsection, stochastic, robust and chance-constrained optimization can be selected. As the stochastic optimization assumes the known probability distribution of $\delta$ which is difficult to obtain in reality and the robust optimization is too conservative which ensures the performance under the worst-case conditions, this paper focuses on the implementation of chance-constrained optimization denoted as equation (5) and (6) to address the uncertainty in power system operations.

$$\underset{x, y_{\delta }}{\min }f\left(x\right)+R\left[g\left(x,y_{\delta },\delta \right)\right]$$

(5)

$$s.t. {{\rm P}}_{\delta }\left\{a\left(x,y_{\delta },\delta \right)=0, b\left(x,y_{\delta },\delta \right)\le 0\right\} \ge 1-\epsilon$$

(6)

$x$ represents the variables that maintain the same as the uncertainty parameter varies, whereas $y_{\delta }$ indicates the variables that vary when $\delta$ changes. $R$ represents the cost of additional reserve required to balance the power system. $a$ and $b$ are the equality and inequality constraints, respectively, which should be guaranteed with at least a probability of $1-\epsilon$. According to [9, 71], probabilistic distribution and distributional robust formulation of $\delta$ can be addressed via distributionally robust chance constrained (DRCC) optimization:

$$\underset{{{\rm P}}_{\delta }\in B}{\inf }{{\rm P}}_{\delta }\left\{a\left(x,y_{\delta },\delta \right)=0, b\left(x,y_{\delta },\delta \right)\le 0\right\} \ge 1-\epsilon$$

(7)

The ambiguity set $B$ is assumed to contain all possible candidates of ${{\rm P}}_{\delta }$. The model aims to minimize the objective function under the worst-case distribution of ${{\rm P}}_{\delta }$ within $B$. The Conditional-Value-at-Risk (CVaR) can be applied to make tractable approximations for the ambiguous chance constraints in equation (7). For independent identically distributed samples and robust uncertainty set defined by a set of scenarios, scenario-based chance constrained (SCC) optimization is applied. It should be noted that at least a number of $N$ scenarios need to be used to define the uncertainty set $\Delta$, where $\delta \in \Delta$, to ensure the probability of constraint satisfaction [74].

$$N\ge \frac{1}{\epsilon } \frac{e}{e-1}\left(ln\frac{1}{\beta }+N_x-1\right)$$

(8)

$\beta$ is the confidence factor and $N_x$ represents the number of decision variables. Among the contemporary operational frameworks for addressing the challenges of integrating solar and wind power into the grid, microgrids have demonstrated their suitability and stability, as evidenced by pioneering projects.

4.2. Uncertainties Affecting the Network Assets

The potential risks caused by the uncertainties defined in this paper can be mitigated both in the planning phase and in real-time emergency management. For real-time emergency response, several strategies can be considered:

If the local network is connected to the grid, the power capacity shortage can be compensated by emergency power imports from the grid given the fact that the transmission feeder has available capacity beyond the scheduled import power amount [89].

Minimization of potential load shedding during the power capacity shortage can be achieved by the emergency discharge and rescheduling of ESSs for both grid-connected and off-grid local networks [90].

Implementing various scheduling lead times, e.g., 5 minutes and 30 minutes ahead, to determine the charging and discharging profiles of ESSs based on generation and demand forecasts.

Implementing the real-time network reconfiguration which dynamically adjusts the configuration of the electrical network to optimize its performance and respond to the uncertainty events. This process typically occurs in real-time to enhance the overall reliability and efficiency of the power system.

Monte-Carlo simulation engine is widely used in the planning phase, which samples different outage scenarios of the system components including the outage of generators due to weather-related uncertainties, substations, and transmission feeder during high bushfire risk periods. Based on the controllable devices of the system and the proposed emergency response strategy, the Monte-Carlo simulation engine can be coordinated with the system dispatch engine in various ways to examine the reliability of the system under uncertainties [53].

For aging infrastructure, wildfire and rainfall that can directly impact the transmission system, thus, affecting the electricity supply, the following common mitigation approaches can be implemented [91]:

Annual line and easement inspections: qualified technicians will conduct thorough inspections of all lines, assets, and easements following the outlined process and evaluation criteria in "Lines Practices and Procedures."

Tower climbing inspections: scheduled at intervals of three, six, or nine years, these inspections are tailored to the probability of asset failure in specific areas, considering factors like bushfires.

Tower corrosion monitoring: tower legs and lattice members which might be influenced by previous weather-related events are monitored. Corrosion assessment and grading adhere to established standards, with results recorded for subsequent maintenance scheduling and replacement activities.

Targeted asset replacement: combining data from condition monitoring and line inspections, the identification of necessary targeted asset replacements, such as insulators, conductors, and ground wires, is carried out.

As the potential risk of aging assets is modelled, this subsection focuses on identifying the optimal preventive maintenance plan for a system to mitigate the associated risk. A MINLP problem is proposed in [92] to determine the optimal preventive maintenance scheduling for enhancing the resilience of power distribution systems. The objective of the presented MINLP problem is to minimize the total expected number of power outages throughout the entire planning time horizon, while adhering to a total budget limit and considering various levels of periodic budget constraints. This optimization directly enhances the resilience of the power system in the face of various uncertainties by reducing the overall likelihood of power outages. In [83, 93-95], reliability ranking is achieved based on the determined POF. Predictive maintenance identifies assets with the greatest impact on network reliability and the poorest health condition, posing operational risks that warrant immediate replacement. In contrast, assets with minimal impact on network reliability and a satisfactory relative condition do not need to take immediate action. It’s noteworthy that assets with minimal contribution to system reliability may still be in poor technical condition, while assets in good technical condition may significantly influence the relative contribution to system reliability. The key advantage of utilizing this type of approach lies in its capacity to concurrently visualize the relative condition of a given unit and its contribution to system reliability. Establishing a connection between asset HI, POF and COF are most crucial step, providing a means to estimate future condition and performance. According to [96], individual asset risks are determined using criticality factors corresponding to consequence categories, adjusting consequences based on each asset’s operating context. Criticality, expressed as a multiplication factor, ranges from $c$ for an average asset to $0$ for less crucial and $c^{max}$ for more pivotal assets. Factors consider network performance, safety, and financial considerations.

4.3. Uncertainties Affecting the Communication Link

As illustrated in Figure 8, the cyber-attack on power grid and industrial control systems can be mitigated by implementing cyber and physical security measures, e.g., control centres from unauthorized access, improving the privacy of customers, and deploying the defense system that monitors network and system activities for abnormal behaviour and potential cyber-attacks. The privacy of customers in the smart grid and industrial control system must be protected and cannot be shared with other parties without permission as sensitive information leakage is increasingly important in modern power systems. After knowing the vulnerabilities of CPS, several defense techniques must be deployed to prevent cyber-attacks. Detailed mitigation approaches can be found in the following literature [97-101]. This subsection focuses on the prevention strategies whereas the following subsection focuses on designing a fault tolerance network with resilient preventive and protective measures.

4.4. Emerging Technologies

This subsection offers insights into emerging technologies that could serve as ultimate solutions for addressing the uncertainties mentioned in this paper. It is important to note that this paper provides a review of these techniques rather than focusing on assessing their effectiveness. Microgrids, often described as localized energy ecosystems, offer an ingenious approach to managing the variability of solar and wind generation. These self-contained networks enable intelligent distribution, generation, and storage, providing flexibility in times of fluctuation [102]. They have shown remarkable effectiveness in enhancing energy resilience and sustainability [103]. The hierarchical control structure of a distribution management system (DMS) with MGs is depicted in Figure 9. The microgrid includes a microgrid central controller (MCC) and multiple load controllers (LCs) and micro source controllers (MSCs) which coordinate the RESs with the MG [104].

Energy storage, particularly in the form of batteries, is anticipated to revolutionize future power systems, significantly enhancing their resilience [54]. Depending on the energy capacity and charge/discharge capabilities, batteries can contribute to resilience through long-duration applications, such as reducing network flow congestions or supporting security of supply in isolated areas. Additionally, batteries play a crucial role in short-duration applications like regulation control, responding quickly to weather events’ impacts within seconds to minutes [60]. This distinction categorizes energy storage into energy applications (long-duration) and power applications (short-duration). Furthermore, energy storage includes bulk storage, employing large-scale units like pumped hydro storage (PHS) and compressed air energy storage (CAES), as well as distributed energy storage within load centers utilizing smaller units [33].

As energy storage system (ESS) technology advances, marked by increased energy density and reduced costs, these systems have become pivotal components in the quest for a stable and efficient energy landscape. Effective ESS operation strategies optimize the storage and release of energy, harmonizing supply and demand while cushioning the impact of intermittent generation [105]. Figure 10 demonstrates the expanding estimated range of the energy capacity, i.e., charging and discharging of the ESSs, and SOC as the control time horizon increases [106]. It can be seen that the uncertainty of the ESS state of charge (SOC) and required capacity propagates through the control horizon. In other words, the variance of uncertainties increases over time as the distance from the current time step increases. The estimation of the State of Charge (SOC) becomes more difficult as the prediction horizon increases. Therefore, it is crucial to implement a tractable approach to manage the SOC of ESSs under solar and wind generation uncertainties.

Figure 11 demonstrates an example of the change of SOC within an operation hour, where the blue curve is the reference trajectory of SOC for the scenario without any uncertainty and the green curve is the SOC for a scenario when the secondary reserves (SR) and redispatch (RD) mechanism are both activated. At the start of the hour, secondary reserves are deployed to compensate for the uncertainty within the first 15 minutes, and the ESS is required to store the excess generation in the system. The redispatch mechanism is not activated in the first interval as the actual SOC matches the reference value but is activated in the second interval, i.e., 15 to 30 minutes when there is a SOC mismatch. At 45 minutes, SOC drops due to energy deficiency caused by uncertainty, resulting in the activation of redispatch to cover the SOC drop compared to the forecast trajectory. In this way, the SOC can be maintained the same as the forecast in real-time operation and prevent the propagation of uncertainty through the time horizon.

The last technique included in this review to tackle the uncertainty is the D-FCAS framework which invites consumers to play an active role in grid management. This approach engages demand-side response to balance grid frequency and delivers essential ancillary services, further fortifying the grid against the vagaries of renewable generation. A detailed survey of the existing literature using the methods is given in Table 2.

5. Limitations and Possible Future Directions

5.1. Limitation for MCMC

The limitations of the aforementioned mitigation approaches can be reviewed from the technical perspective considering the computational burden of the operating system and from the point of market and operation framework. The scenarios generated by the MCMC model for robust uncertainty set and the input scenarios for the Monte-Carlo engine can add computational burden to the operation system. Therefore, researchers have either investigated the scenario reduction methods or adjusted their decision-making models to accommodate a wider range of input scenarios. For instance, a simultaneous backward model from [40] is adopted to reduce the number of scenarios and maintain as much information as possible. Considering a set of scenarios $\delta$ with a number of $Ns$, the Euclidean distance between any scenario pair is defined as $D_{i,j}=\sqrt{ {{\displaystyle \sum }}_{t=1}^T{\left({\delta }_t^i-{\delta }_t^j\right)}^2}$ in Figure 12.

Another way to improve the computational speed is the boundary tightening approach documented in [112].

In [113] the decisions are decoupled at various stages of the timeline. For day-ahead optimization DC power flow is selected instead of the non-convex AC power flow method to enhance computational efficiency while considering network constraints. During operation hours, where the number of decision variables is lower, more accurate SOCP relaxed power flow model is employed for network constraints. Future research directions involve finding a balance between the performance of operating systems and the required computational time.

The limitation in the current market and operation framework include:

Regulatory Framework: Regulatory frameworks and market designs may not fully account for the unique characteristics of RESs, such as their variability. Existing market structures and mechanisms may not incentivize the desired response to manage uncertainty.

Transmission and Grid Constraints: Limited transmission capacity and grid constraints can exacerbate the challenges of integrating RESs, leading to uncertainty in power flows and frequency control. Future improvements require the information exchange between distributed network service providers (DNSPs) and RESs operators.

Artificial intelligence: Many AI models, especially deep learning models, operate as black boxes, making it challenging to understand how they arrive at specific decisions. This lack of transparency can hinder trust and make it difficult to explain data-driven outcomes.

5.2. Limitations for defining the weather regions

The assumption underlying the methods mentioned for the modelling of wildfire and rainfall is that a weather condition concludes at the boundary separating two geographical weather regions. The network’s segmentation into weather regions relies on this assumption. However, in reality, there are no distinct borders between weather regions. Instead, weather events cross the network with different densities, speeds, and durations, which will affect the accuracy of the estimated line failure rate under wildfire or heavy rainfall, thus, impacting the reliability of the power system.

5.3. Limitations for Asset management

CBRM offers valuable insights into asset condition and risk, but it also comes with certain limitations. It is recommended to combine CBRM with other risk management methods to mitigate these limitations. Some of the key limitations are listed below.

Data accuracy and availability: CBRM heavily relies on accurate and reliable data for assessing asset conditions. Inaccuracies or insufficient data can compromise the effectiveness of the risk management process.

Complexity of degradation processes: predicting future conditions and performance is challenging due to the complexity of degradation processes. Some degradation mechanisms may not follow predictable patterns, making it difficult to precisely estimate asset deterioration.

Assumption sensitivity: CBRM involves making assumptions about the relationships between HI, POF, and COF. The accuracy of results is sensitive to the validity of these assumptions, and deviations can impact the reliability of risk assessments.

Limited predictive ability for catastrophic events: CBRM may not be well-suited for predicting rare but catastrophic events. Extreme events, such as natural disasters, may have consequences that are challenging to quantify accurately, leading to potential underestimation of risks.

6. Conclusions

The challenges faced by the electric grid, stemming from various sources, carry profound implications for reliability, stability, and economic efficiency. Notably, Australia’s vulnerability to sever weather events amplifies the significance of addressing power network vulnerabilities, particularly in remote areas. Despite these challenges, the existing literature lacks a thorough examination of uncertainties in contemporary power systems, including weather-related events, cyber threats, and asset management, along with an exploration of mitigation approaches and their limitations. To address this gap, the review delves into both conventional robust control methods and modern probabilistic, data-driven approaches for modeling uncertainty events and their correlation to the state of grid, facilitating optimal decision-making. The exploration extends to the development of robust and scenario-based operations, control technologies for MGs and ESSs, and D-FCAS and reserve provision for frequency regulation. These advancements aim to design a power system robust against uncertainties.

Additionally, the conclusion underscores the current prevalence of deep learning models in weather prediction within the power system. The trend of utilizing deep learning in power system decision-making processes, such as fault restoration and predictive maintenance for aging assets, is evident in existing literature. However, a primary challenge lies in the extensive data requirements for data-driven methods and the computational challenges for Monte Carlo engines. A potential future direction involves the reduction of input data through pre-processing and the implementation of realistic approximations to obtain accurate results promptly. Furthermore, the increasing intensity and frequency of severe weather conditions caused by global climate change necessitate the development of emerging technologies to enhance the reliability and stability of the current smart grid. As the penetration of AMI for the smart grid is anticipated to reach 100% in the future, cybersecurity measures need to be improved to prevent cyber-attacks. Besides, redundant generation and communication links must be applied to make the smart grid more robust against potential cyber threats. This exploration contributes insights to guide the understanding and management of uncertainties in evolving power systems, paving the way for informed decision-making and resilient grid architectures.