Random Convolutional Kernel Transform with Empirical Mode Decomposition for Medium Voltage Insulator Classification based on Ultrasound Sensor

1. Introduction

Electrical power grids form the backbone of modern society, and their components’ effective management and maintenance are of paramount importance [1]. Insulators play a critical role in ensuring the stability and reliability of these grids, as they serve as both mechanical supports for the wires and electrical potential insulation [2]. A degradation in an insulator’s characteristics can have severe consequences, leading to disruptive discharges, system failures, and compromised network dependability [3]. Therefore, it is crucial to develop robust and accurate methods for monitoring and assessing the performance of insulators [4].

Inspections of the electrical system using radio frequency-based techniques are increasingly being applied by power utilities since before a fault occurs, partial discharges can emit light or noise that humans have difficulty identifying [5]. For this purpose, specific equipment such as ultraviolet cameras, infrared cameras, and ultrasound detectors are used [6]. Therefore, when there is a higher probability of failure, the maintenance team can take action in advance, improving the reliability of the electrical network [7].

Based on the ultrasound, analyzing time series data captured from the insulators during the inspections becomes a promising avenue for fault detection [8]. Ultrasound-based techniques have proven effective in identifying various insulator faults, providing valuable insights into their condition, and allowing for timely maintenance and replacement [9]. However, the accurate classification of these time series data remains a challenge. The major advantage of using ultrasound compared to leakage current detection [10], for example, is that the ultrasound does not need to be in direct contact with the network; thus, inspections can be performed with greater speed and less risk for the technical team [11].

Time series classification is a task that involves predicting a categorical label for a given time series dataset. This data is a sequence of observations collected over time. Time series classification aims to learn a model that can classify new time series based on past observations. Recent advances in time series classification methods have opened new possibilities for addressing this issue [12]. The accuracy and effectiveness of time series classification methods have recently undergone several types of substantial developments based on statistical models [13], machine learning [14], and deep learning [15] approaches. In particular, the random convolutional kernel transform (Rocket) algorithms [16], including MiniRocket [17] and MultiRocket [18], has gained considerable attention from researchers due to their ability to efficiently and accurately process time series data.

Rocket is a kernel-based approach that uses random Fourier features to map data from time series into a feature space of high dimensionality. MiniRocket is a lightweight version of Rocket. MiniRocket is a faster and more memory-efficient method than Rocket, given that it only takes a small portion of the random Fourier features toward account. Despite each time point being represented by many variables, MultiRocket is a variant of MiniRocket capable of handling multivariate time series classification challenges. Intending to generate a shared feature representation for the multivariate time series, MultiRocket implements an innovative multivariate feature mapping technique that integrates the outputs from several univariate MiniRocket classifiers [19].

This paper proposes a novel approach combining Rocket algorithms with machine learning classifiers to enhance insulator time series classification accuracy and efficacy based on ultrasound data. The contributions of this research are summarized below:

(i) An efficient classification framework that combines the advantages of Rocket approaches and machine learning models for time series classification of medium voltage insulators is proposed, increasing classification accuracy and generalization capabilities.

(ii) The impact of integrating empirical mode decomposition methods with the proposed framework demonstrates significant improvements in classification accuracy.

(iii) Several classification algorithms are comprehensively compared to provide a benchmark for performance evaluation. This comparison will help engineers select the most appropriate method for their specific insulator classification task, considering classification accuracy versus model complexity.

The remainder of the work proceeds as follows: The related works are briefly approached in Section 2. The description of the classification problem is detailed in Section 3. The fundamentals of the evaluated methods and the proposed approach are explained in Section 4. Section 5 discusses the classifier designs regarding performance evaluation and result analysis, while Section 6 provides the main conclusions and suggestions for further research.

2. Related works

Several authors have explored the application of time series for identifying and predicting insulator failures [20]. Klaar et al. [21] used the empirical wavelet transform for denoising in a hypertuned long short-term memory (LSTM) framework to fault prediction in insulators considering a sequence-to-sequence problem. In this case, the prediction was regarding leakage current, similar to the method presented by Sopelsa Neto et al. [22] and Medeiros et al. [23], where several models were explored for this task.

Concerning evaluating the time series for predicting the increase of faults in the power supply system, Branco et al. [24] presented a study of the number of faults over the year. The failures can be related to climatic variations. Depending on the season, more failures can happen, especially in this study, where there is a rainy season, increasing the probability of failure [25]. A highlight that was presented in this research was the use of the wavelet transform to mitigate the impact of unrepresentative variations. This technique can be used in chaotic time series, such as ultrasound, which is studied to detect failures by power utilities [26].

Especially using ultrasound to classify the condition of the insulators of medium voltage power grids, Stefenon et al. [27] proposed using the echo state network. They showed that identifying a specific condition, such as drilling, is easier than performing multiclassification. They have highlighted that the broken and drilling insulators have more partial discharges than contaminated or clean insulators, been possible to have results of classification with over 99% accuracy when these conditions are evaluated. In this evaluation, the echo state network was more promising than the support-vector machine (SVM) or multilayer perceptron.

Ferreira et al. [28] proposed a method for calculating electrical insulator pollution using ultrasonic noise. The audio was reduced using the spectral subband centroid energy vectors algorithm before input into an artificial neural network that can distinguish between different degrees of pollution. Their method was applied to process ultrasonic sounds from different electrical equipment given to multiple forms of pollution. In [29] the contamination of insulators is evaluated using deep learning.

The problem of interpreting leakage current measurements for overhead insulator condition monitoring due to the intermittent harmonic content of the supply voltage was dealt by Ghosh et al. [30]. To monitor leakage current in the presence of voltage harmonics, the study suggested applying the instant value of the time integral of the leakage current as a low sensitivity parameter. The study demonstrated that changes in system voltage’s harmonic content significantly impact the harmonic properties of leakage current. The suggested measuring method has been tested and validated using experimental data that was captured in the lab and integrated into an online measurement tool that has been evaluated in the lab.

The problem of appropriately simulating the flashover phenomena in contaminated insulators was approached by Belhouchet et al. [31], which is made more challenging by the complexity of determining the arc constants generated in dry bands when the electrical voltage goes beyond critical levels. Using data from artificially contaminated insulator experiments, the authors suggest a strategy for optimization based on genetic algorithms and artificial neural networks (ANN) to identify the arc constants and dielectric properties of the surface. The research, which used a generalized pollution flashover model, observed that the inverse connection of flashover voltage and leakage current was validated by the optimized mathematical model’s realistic simulation of the experimental data.

A method for predicting line trip defects in power systems that combine support vector machine and LSTM networks was suggested by Zhang et al. [32]. The suggested approach addresses the shortcomings of existing approaches based on activities taken to preserve relays and electrical components. In order to acquire the final prediction results, the support-vector machine is used for classification and the LSTM networks are employed to capture the temporal aspects of multi-sourced data. The LSTM is well-indicated for time series with high nonlinearities [33] and it can be further improved by using the attention mechanism [34].

You only look once (YOLO) deep learning neural network model using the unmanned aerial vehicle has been presented in the work of Sadykova et al. [35] as an effective technique for detecting high voltage insulators. The purpose is to provide a real-time classification of insulator conditions while avoiding expensive manual inspections involving traveling across a wide area in adverse weather. The technique uses a training set size of 56,000 image samples and data augmentation to prevent overfitting. The experimental findings show how well the proposed method works for accurately determining insulators and assessing their surface conditions for the presence of ice, snow, and water through different classifiers. Also, in [36] and in [37] a hybrid version of YOLO is proposed for inspections of the power grids.

To automate visual inspection tasks, Prates et al. [38] suggested applying convolutional neural networks to recognize flaws and different insulators in overhead power distribution lines (OPDLs). More than 2,500 photos obtained from a studio and a realistic OPDL are used to train the model. Multi-task learning was also employed to enhance fault detection performance by predicting the insulator class. Also based on images, in [39] a new hybrid method is proposed, which combines object detection to convolutional neural networks (CNNs) for classification.

Polisetty et al. [40] concentrated on the significance of keeping a close watch on outside insulation systems to preserve the integrity of substations and overhead transmission and distribution lines. The study used an ANN and a commercial acoustic sensor to classify electrical discharge patterns in external insulating systems. The ANN then expanded to incorporate three different types of flaws on outdoor ceramic insulators and distinguished between five frequent discharges of electricity produced under controlled settings. The investigation successfully identified approximately 85% of controlled samples.

Intending to monitor the condition of equipment for high-voltage power stations, Mitiche et al. [41] addressed using bispectrum representations as complex input features in complex-valued deep CNNs. This approach achieved excellent classification accuracy for discharge signals. An automated inspection system that uses computer vision to gauge erosion in silicone rubber samples was presented by Ibrahim et al. [42]. Using the IEC-60587 (International Electrotechnical Commission) standard to describe failure, the system was intended to classify samples into one of three groups based on the level of erosion. The suggested system compared the performance of ANNs, applying feature extraction methods and pre-processing approaches.

A new method for insulator condition monitoring based on meteorological and environmental information was suggested by De Santos and Sanz-Bobi [43]. The method combined the random under-sampling technique to estimate important condition indicators with an adaptive boosting algorithm (RUSBoost). The proposed method was compared with other algorithms at France’s 245 kV test station. The findings indicate that RUSBoost outperformed the competitors’ algorithms, rating high for the estimation of insulator conditions. Advanced hybrid methods are been applied by several researchers [44,45,46], and the idea of combining techniques helps the model by using the advantages of more than one approach.

A knowledge-based optimization approach to deal with the challenge of determining the optimal process settings for manufacturing medium voltage insulators was proposed by Kong et al. [47]. Their method utilized historical approximations produced in the optimization process to enhance the accuracy of the gradient estimates and to adjust the size of the iteration step. Their approach reduced the cost and improved quality control efficiency for insulators, which is crucial for their efficient production and confident operation. Models based on the ensemble approach are promising considering that usually the need less computational effort compared to deep learning [48].

A novel model based on feature pyramid neural networks and adaptive threshold algorithm with line detection, image rotation, and vertical projection data applied to insulators fault detection in transmission lines was proposed by Zhao et al. [49]. Singh et al. [50] presented an evaluation of infrared thermal images, their method computes several features from the segmented region of interest and utilizes a Gaussian kernel SVM to classify the insulator faults. A robust methodology based on deep learning and uncertainty detection for automatic insulator fault inspection using aerial images was approached by Dai [51]. The bounding box prediction was improved, and the detection robustness was enhanced using the predicted uncertainty scores.

The adoption of deep learning algorithms for condition monitoring of high voltage equipment in electrical power systems was reviewed in Mantach et al. [52]. Contrasting conventional machine learning approaches, deep learning combines feature extraction with the learning stage and uses raw data as input. This paper included contemporary research on deep learning approaches for monitoring high-voltage equipment, including gas-insulated switchgear, transformers, cables, rotating machines, and outside insulators.

A novel approach for monitoring pollutant insulator discharge mode in high voltage lines by combining auditory emission signals with a one-dimensional convolutional neural network structure (1D-CNN) was provided by Hao et al. [53]. The procedure includes data collection in a lab accompanied by using 1D-CNN to reduce the dimensionality of the signal samples and extract features. With a recognition rate of over 99.84%, the model successfully replaces the need for human data preparation in conventional monitoring approaches and may be used to carry out monitoring tasks for pollution insulator discharge mode.

A CNN bidirectional LSTM named CNN-Bi-LSTM neural network design with hyperparameters optimization to classify leakage current levels according to sequential weather factors and insulator data was evaluated in Nguyen et al. [54]. The CNN-Bi-LSTM was employed in real-time monitoring services to improve the operations of the TaiPower electric utility in Taiwan. On the other hand, a CNN-LSTM neural network with hyperparameter tuning for categorizing leakage/discharge current on a web-based service was evaluated by Tham and Cho [55]. Leakage current surge and weather data are input parameters in four different models to predict leakage current classification.

The study of ultrasound has been explored by several authors [56,57,58,59], and can be applied for classification, as presented in this paper. Considering an experiment in medium voltage, the ultrasound equipment is employed to define insulator patterns under different conditions, as will be explained in detail in the next section.

3. Insulators Ultrasound Measurement

This section provides a detailed account of the classification problem and explains the experiment performed in the high-voltage laboratory presented in Figure 1. The experiment involves applying a voltage of 7.95 kV phase-to-ground to the insulators, equivalent to 13.8 kV phase-to-phase in the power system, the electrical potential used in the considered distribution branch located in southern Brazil.

This paper considered three conditions: an insulator in good condition, an artificially contaminated insulator, and a drilled insulator. The insulators are pin-type profiles, class 15 kV, from the Germer manufacturer. These insulator profiles are commonly installed in conventional distribution power grids in rural southern Brazil, exposed to organic contamination from unpaved roads, and saline contamination when close to the coast [60].

To simulate drilling caused by lightning, a perforation was performed on the top of insulators using a bench drill. Figure 1A presents the top view of the drilling, and Figure 1B shows the bottom view, where the fixation pin is attached. This problem is hardly found in the distribution grid because the perforations can be underneath the mooring. Perforation is more common in polymeric insulators, where the temperature required for carbonization is lower than in glass insulators [61].

The contamination over the insulator surface is an issue that increases the conductivity of the surface, leading to a higher leakage current and possible flashovers [62]. The flashovers occur mainly in bad weather conditions, making it challenging to identify the exact location of the fault during inspections. When lightning strikes the electrical power grid, perforations or boundary discharges might occur, resulting in a higher risk of irreversible failures, in which corrective maintenance is required [63].

To simulate the contamination on the insulators, the solid layer procedure given by NBR 10621/2017 (Brazilian Standard: High-voltage insulators to be used in alternating current systems - artificial pollution tests) was followed, based on IEC/507 (Artificial pollution tests on high-voltage insulators to be used in alternating current systems). The contaminants considered were kaolin and sodium chloride. The NBR 10621/2017 standard determines the tolerable performance of porcelain or glass insulators for outdoor applications [64].

The experiment was conducted inside an acrylic chamber since the ultrasound detector is sensitive to capturing noise from external sources. The ultrasound detector was set at a distance of 0.4 m from the insulators (see Figure 2), and it recorded noise signals with a maximum frequency of 500 kHz; this distance was fixed for comparative purposes. In the distribution grids, the operator may face scenarios with varying relief that may result in a greater need for measurement. The difficulty in reaching the grid is one of the significant challenges in inspecting power distribution networks by the electric utility company [65].

To mitigate the interference from partial discharges coming from the mooring of the insulator, the fixing was done with non-conductive materials [66]. The chamber held two insulators affixed, and voltage was applied to these insulators while the ground had an equal reference. The ground was attached only to the insulator under evaluation to prevent one insulator from affecting the other. An M500 model from Petterson recorded the ultrasonic signal. The conductivity of the water used to spray the samples during the experiment was 56 kg/m3, which corresponds to a medium-high contamination level according to IEC-507.

Considering partial discharge typically occurs over the frequency range from 10 kHz to 210 kHz, a comprehensive assessment of more than 50 times the base frequency ensures that all frequencies beyond 10 kHz are captured under a single wave cycle. Besides the 500 kHz frequency rate, to ensure that the signal has been properly recorded for a sufficient time length, the data log was held for 50 seconds. After the signal was saved, a total of 1$\times {10}^5$ records were considered for a comprehensive assessment. Figure 3 presents an example of the signal recorded by the ultrasound equipment.

4. Methodology

The model proposed in this paper combines empirical mode decomposition methods with random convolutional kernel transform models and state-of-the-art classifiers to obtain a hybrid architecture, presented in Figure 4, which is explained in this section. The input signal is based on a time series measured by ultrasound during high-voltage experiments considering insulators under different conditions, as explained in the previous section.

A time series is a sequence of points of information collected over time, typically at fixed intervals. The classification of time series is related to the development of models to classify time series data into predetermined categories based on their patterns and characteristics over time. The Rocket [16], MiniRocket [17], and MultiRocket [18] are algorithms that have been widely evaluated recently for time series classification tasks.

The fundamental concept of Rocket methods is obtaining features from time series data and employing those features to train a classifier. These models use convolutional kernels to transform the time series data into features, which are then used for classification [67]. Given a time series $x=x_1,x_2,\dots ,x_T$, these algorithms compute features such as the maximum value (Max) and the proportion of positive values (PPV) for each of the k convolutional kernels. The convolutional operation for a kernel $\varphi ={\varphi }_1,{\varphi }_2,\dots ,{\varphi }_m$ can be expressed as:

$$z_i=(x\ast \varphi )i=\sum {j=1}^m{x}_{i+j-1}{\varphi }_j,$$

(1)

where ∗ denotes the convolutional operation, $z_i$ is the output of the convolution, and m is the kernel length. The Max and PPV features are computed as follows:

$$\mathrm{Max}=\underset{1\le i\le T-m+1}{max}{z}_i,$$

(2)

$$\mathrm{PPV}=\frac{1}{T-m+1}\sum _{i=1}^{T-m+1}\mathbb{1}(z_i>0),$$

(3)

where $\mathbb{1}(·)$ is the indicator function. The extracted features are then used to train a linear classifier for time series classification.

MiniRocket distinguishes itself from Rocket by computing features using a fixed set of k convolutional kernels with a shorter kernel length, resulting in greater computational efficiency and refining the convolutional process by introducing alterations to the kernels [68]. The MiniRocket transform calculates the Max and PPV features for each k fixed convolutional kernel. By leveraging a fixed set of kernels with shorter kernel lengths, MiniRocket significantly diminishes the computational effort while retaining competitive performance in time series classification tasks [69].

The MultiRocket algorithm extends the Rocket framework by incorporating multiple pooling operators and transformations to enhance the diversity of the generated features. MultiRocket employs first-order differences to transform the primary input series, thereby creating an alternative representation and processing the raw input series. Both representations undergo the application of convolutions, and the processing of convolution outputs is executed using four distinct pooling operators. The integration of additional procedures expands the range of features and improves the algorithm’s overall performance [18].

4.1. Empirical Mode Decomposition

For time series decomposition, feature extraction, and noise reduction the empirical mode decomposition (EMD) methods are applied [70]. The variations of the EMD include the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) [71], empirical wavelet transform (EWT) [72], and variational mode decomposition (VMD) [73]. These methods are advanced signal processing techniques that aim to decompose a given time series into a finite set of components, each representing an intrinsic mode function (IMF) [74] or oscillatory mode.

EMD is a data-driven method that decomposes non-linear time series in a set of IMFs [75]. The main idea behind EMD is the so-called sifting process, which iteratively extracts IMFs by identifying local extrema and fitting envelopes using cubic spline interpolation. Given a time series $x\left(t\right)$, the sifting process begins with the identification of all the local maxima and minima. Next, the upper and lower envelopes are created by the interpolation of the local maxima and minima employing cubic spline interpolation. The mean of the envelopes is then calculated as follows:

$$m\left(t\right)=\frac{1}{2}(e_{\mathrm{upper}}\left(t\right)+e_{\mathrm{lower}}\left(t\right)),$$

(4)

where $e_{\mathrm{upper}}\left(t\right)$ is the upper envelope and the $e_{\mathrm{lower}}\left(t\right)$ is the lower envelope [76].

The difference between the original signal and the mean is considered a candidate IMF:

$$c\left(t\right)=x\left(t\right)-m\left(t\right),$$

(5)

and this process is repeated on the IMF until it meets the predefined stopping criterion. Then, it is applied to the residual signal until all IMFs are extracted.

The EWT involves the decomposition of a given signal in oscillatory modes having varying scales and frequencies [77]. The EWT algorithm produces a collection of n non-linear functions known as IMFs from the signal $x\left(t\right)$ and a wavelet mother function $\psi \left(t\right)$. The process for generating these IMFs is outlined in Algorithm 1.

Algorithm 1:EWT

Once the set of IMFs is obtained, EWT employs a Fourier transform to each IMF to produce a set of n spectrograms, that are utilized to visualize the time-frequency information of the signal [78]. The EWT has the following expression:

$$\begin{array}{cc}\hfill x\left(t\right)& =\sum _{i=1}^n{\mathrm{IMF}}_i\left(t\right)+R_n\left(t\right)\hfill \\ \hfill {\mathrm{IMF}}_i\left(t\right)& =\int x\left(\tau \right)h_i(\tau -t)d\tau \hfill \end{array}$$

where $h_i\left(\tau \right)$ is the ith filter set as the convolution of the scaling function $\phi \left(t\right)$ and the $\psi \left(t\right)$ is scaled by a factor of $2^i$:

$$\begin{array}{cc}\hfill & h_i\left(\tau \right)=2^i\phi \left(2^i\tau \right)\psi \left(2^i\tau \right).\hfill \end{array}$$

(6)

The EWT combines the concepts of EMD and wavelet transform. The main idea of EWT is to decompose the signal in a set of oscillatory modes using an adaptive filter bank. The filter bank is designed based on the signal’s time-frequency content, estimated by the continuous wavelet transform [79]. The EWT decomposition is given by:

$$x\left(t\right)=\sum _{i=1}^N{w}_i\left(t\right)+r\left(t\right),$$

(7)

where $w_i\left(t\right)$ are the wavelet components, N is the number of modes, and $r\left(t\right)$ is the residual.

VMD is another decomposition technique that formulates the extraction of IMFs as a constrained variational problem. VMD decomposes the time series in a set of band-limited IMFs by minimizing the cost function that balances the compactness of the frequency spectrum and the smoothness of the time-domain signal [80]. The VMD optimization problem can be written as:

$$\begin{array}{cc}\hfill & \underset{u_k\left(t\right),{\omega }_k}{min}\sum _{k=1}^K\int {\left|{\partial }_t\left(\frac{u_k\left(t\right)}{1+j{\omega }_{k}t}\right)\right|}^2\mathrm{d}t\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.x\left(t\right)=\sum _{k=1}^K{u}_k\left(t\right)\hfill \end{array}$$

(9)

where $u_k\left(t\right)$ are the mode functions, K is the number of modes, and ${\omega }_k$ are the center frequencies of the modes.

4.2. Classification Methods

To evaluate the effectiveness of Rocket methods, including MiniRocket and MultiRocket, for classifying faults in insulators, a comprehensive analysis is conducted by combining these algorithms with various classifiers mentioned above. This experimental design aims to determine the best-suited combination of Rocket techniques and classification methods, ultimately enhancing insulator fault detection accuracy.

Logistic Regression: Logistic regression, a prevalent linear technique employed for classification, utilizes a logistic function to model the probability of a specific class or event [81]. The following equation represents the logistic function:

$$\begin{array}{c}\hfill P(y=1|x)=\frac{1}{1+e^{-({\beta }_0+{\beta }_{1}x)}}\end{array}$$

(10)

Ridge Regression: Ridge regression, also known as Tikhonov regularization, is a linear regression technique incorporating an $L2$ regularization term to address the issue of multicollinearity and improve the generalization of the model [82]. It is particularly useful when there are highly correlated features. The objective function for ridge regression can be written as follows:

$$\begin{array}{c}\hfill \mathcal{L}(\mathit{w},b)=\sum _{i=1}^n{(y_i-({\mathit{w}}^T{\mathbf{x}}_i+b))}^2+\lambda {\left|\mathit{w}\right|}^2\end{array}$$

(11)

where $\mathit{w}$ is the weight vector, b is the bias term, $y_i$ and ${\mathbf{x}}_i$ are the true label and the feature vector for the i-th instance, respectively, and $\lambda$ is the regularization parameter that controls the trade-off between model complexity and the goodness of fit. The regularization term, ${\lambda \left|\mathit{w}\right|}^2$, discourages the model from assigning large weights to the features, leading to a more stable and robust solution.

Decision Tree: Decision tree classifier, a non-parametric, hierarchical model, recursively partitions the input space into discrete regions according to feature values. The decision rules are derived by minimizing the impurity of the resultant nodes, which can be quantified utilizing metrics such as Gini impurity or entropy [83].

The architecture of the classifier is built in the form of a tree structure where each internal node represents a feature or attribute, each branch represents a decision rule, and each leaf node represents a class label or a decision. According to Mishra et al. [84], the architecture can be further improved by using clustering techniques.

k-NN: The k-nearest neighbors (k-NN) classifier, a non-parametric, instance-based learning algorithm, classifies novel instances based on the majority class of their k nearest neighbors. The distance metric and the value of k are crucial to the algorithm’s performance. Since it is a classification problem, employing an odd k is more advantageous, thus avoiding draws [85]. For this task, the weighted mode is denoted by:

$$\begin{array}{c}\hfill {\gamma }_t=\arg \underset{c\in Y}{max}\sum _{i=1}^k{\omega }_{i}I(c,{\gamma }_i)\end{array}$$

(12)

where,

$${\omega }_i=\frac{1}{d(x_t,x_i)},$$

(13)

$I(a,b)$ returns 1 if $a=b$[86]. Given that ${\gamma }_i$ is the class of the $x_i$ example associated with the ${\omega }_i$ weight, and c is the class with the best-weighted mode. To calculate the neighbors the Euclidean, cosine, correlation, chebychev, city block, spearman, standardized Euclidean, Minkowski, and Mahalanobis distances methods can be applied [87].

LDA: Linear discriminant analysis (LDA), a technique utilized for dimensionality reduction and classification, identifies the linear combination of features that optimally separates distinct classes by maximizing the dispersion between classes and minimizing the dispersion within a class [88]:

$$\begin{array}{c}\hfill J\left(w\right)=\frac{w^T{S}_{B}w}{w^T{S}_{W}w}\end{array}$$

(14)

where $S_B$ and $S_W$ represent the between-class and within-class scatter matrix respectively.

Gaussian Naive Bayes: Gaussian Naive Bayes is a classification algorithm that is based on Bayes’ theorem [89], assuming the features are conditionally independent and follow a Gaussian distribution:

$$\begin{array}{c}\hfill P\left(A\right|B)=\frac{P\left(B\right|A\left)P\right(A)}{P\left(B\right)}\end{array}$$

(15)

where A and B are events or variables. The Gaussian Naive Bayes assumes that the features in the dataset are normally distributed and that they are independent of each other.

SVM: The support vector machine (SVM) classifier, endeavors to identify the optimal separating hyperplane between classes [90]. Its performance is governed by the kernel function and regularization parameter C:

$$\begin{array}{cc}\hfill & \underset{w,b,\xi }{min}\frac{1}{2}{w}^{T}w+C\sum _{i=1}^n{\xi }_i\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.y_i(w^T{x}_i+b)\ge 1-{\xi }_i\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill & {\xi }_i\ge 0.\hfill \end{array}$$

(18)

Random Forest: An ensemble learning methodology constructs multiple decision trees and amalgamates their outputs via majority voting [91]. The operator regulates the number of trees (T) and their maximum depth. Let $\mathcal{X}$ be the set of input features and $\mathcal{Y}$ be the set of output classes. The random forest classifier consists of T decision trees, ${h_t\left(x\right)}_{t=1}^T$, each grown to a maximum depth. Each tree is created by a randomly sampled subset of the train data, typically with replacement (i.e., bootstrapped samples), and a random subset of input features at each split. The random forest classifier is given in the following definition:

$$\begin{array}{c}\hfill H\left(x\right)=\arg \underset{c\in \mathcal{Y}}{max}\sum _{t=1}^{T}I(c,h_t\left(x\right)),\end{array}$$

(19)

where $H\left(x\right)$ represents the final classification, again $I(a,b)$ returns 1 if $a=b$, and 0 otherwise, and $h_t\left(x\right)$ is the output of the t-th decision tree for input x.

Gradient Boosting: The gradient boosting classifier, an ensemble learning technique, sequentially builds weak learners, with each learner rectifying the errors committed by the preceding one [92]:

$$\begin{array}{c}\hfill F_m\left(x\right)=F_{m-1}\left(x\right)+{\rho }_m{h}_m\left(x\right)\end{array}$$

(20)

where $F_m\left(x\right)$ denotes the boosted model at step m, $h_m\left(x\right)$ signifies the weak learner, and ${\rho }_m$ represents the step size. The gradient boosting method has also been utilized for prediction by various authors [93,94,95].

AdaBoost: Adaptive boosting (AdaBoost) classifier, an adaptive boosting technique, combines weak learners to form a robust classifier, with each learner weighted based on its accuracy [96]. The algorithm updates the weights of the training instances at each iteration, assigning greater importance to misclassified instances:

$$\begin{array}{c}\hfill D_{t+1}\left(i\right)=\frac{D_t\left(i\right)e^{-{\alpha }_t{y}_i{h}_t\left(x_i\right)}}{Z_t}\end{array}$$

(21)

where $D_t\left(i\right)$ is the weight of instance i at iteration t, $h_t\left(x_i\right)$ is the prediction, $y_i$ is the true label, ${\alpha }_t$ is the weight of the weak learner, and $Z_t$ is the normalization factor.

Gaussian Process: Gaussian process classifier, a Bayesian, non-parametric model, employs a Gaussian process prior over the function space and yields probabilistic classification results [97]. It is determined by a mean $m\left(x\right)$ and a covariance function $k(x,x^{\prime })$:

$$\begin{array}{c}\hfill f\left(x\right)\sim \mathcal{GP}(m\left(x\right),k(x,x^{\prime }))\end{array}$$

(22)

XGBoost: The extreme gradient boosting (XGBoost) algorithm is a highly efficient and scalable tree-based ensemble learning model, designed for both classification and forecasting problems [98]. It is an extension of the gradient boosting algorithm, employing advanced regularization techniques to improve the model’s generalization and control overfitting. XGBoost optimizes the following objective function:

$$\begin{array}{c}\hfill \mathcal{L}\left(\varphi \right)=\sum _{i=1}^{n}l(y_i,\widehat{y}i)+\sum _{j=1}^K\Omega \left(f_j\right)\end{array}$$

(23)

where $\varphi$ represents the model parameters, $l(y_i,{\widehat{y}}_i)$ denotes the loss function comparing the true label $y_i$ and the forecasted label ${\widehat{y}}_i$, and $\Omega \left(f_j\right)$ is the regularization term for the j-th tree. The regularization term comprises the tree complexity, measured by the number of leaves T, and the $L2$-norm of the leaf scores w:

$$\begin{array}{c}\hfill \Omega \left(f\right)=\gamma T+\frac{1}{2}\lambda {\left|w\right|}^2\end{array}$$

(24)

The algorithm employs second-order gradient information (Hessian) and the first-order gradient for updating the model, making the learning process more accurate and faster. Furthermore, it utilizes column block and sparsity-aware techniques to efficiently handle sparse data and parallelize the tree construction process, enabling it to tackle large-scale datasets efficiently [99].

LightGBM: The light gradient boosting method (LightGBM), a boosting framework, leverages tree-based learning algorithms and is designed to be efficient and scalable for large datasets [100]. It adopts gradient-based one-side sampling and exclusive feature bundling to expedite training and diminish memory usage.

In the next section, the results of the application of the proposed method are presented, initially the results of using different classifiers considering window sizes of 10, 50, and 100 records are presented. Then, the incorporation of Rocket, MiniRocket, and MultiRocket models with 10, 50, and 100-time steps are evaluated. And finally, the use of EMB methods to reduce noise that is not significant is evaluated.

5. Results

In the experiments presented in this section, it is utilized a k-fold cross-validation methodology to evaluate the performance of the models, in particular, the k is equal to five. Cross-validation is a widely used technique to estimate the predictive performance of a model; in particular, 5-fold cross-validation involves splitting the dataset into five equal-sized partitions. Four partitions are used for training the model for each fold, and the remaining partition is utilized for testing. This procedure is repeated five times such that each fold serves as the test set exactly once. The resulting accuracy scores from each fold are then averaged to estimate the model’s accuracy The default scikit-learn [101] parameters were employed in all the classification algorithms.

The performance of various algorithms for fault detection in insulators is evaluated using three different time window sizes, namely WS10, WS50, and WS100. Table 1 presents the accuracy results of 14 algorithms, including logistic regression, ridge regression, decision tree, k-NN, LDA, Gaussian Naive Bayes, SVM, random forest, gradient boosting, AdaBoost, Gaussian process, XGBoost, LightGBM, and CatBoost. The results indicate a clear trend regarding the time window size and the overall performance of the algorithms, with the critical difference diagram shown in Figure 5.

From Table 1 it is evident that tree-based methods, such as decision tree, random forest, gradient boosting, AdaBoost, XGBoost, LightGBM, and CatBoost, exhibit superior performance compared to other algorithms, as can be further seen in Figure 6. CatBoost, LightGBM, and gradient boosting show the highest accuracies in WS50 and WS100 time windows, particularly strong results. Moreover, the table demonstrates that the accuracy of the algorithms generally improves as the time window size increases from WS10 to WS100. This observation suggests that longer time windows provide more information for the algorithms to identify the patterns and relationships between the features, resulting in improved performance. For instance, CatBoost’s accuracy increases from 0.8842 ± 0.0658 in WS10 to 0.95 ± 0.0459 in WS100, highlighting the significance of using longer time windows for fault detection in insulators.

Table 2, Table 3, and Table 4 present the results of different machine learning algorithms applied to insulator fault detection when using Rocket, MiniRocket, and MultiRocket data transforms for three different time windows, WS10, WS50, and WS100, respectively. These transforms have been applied to enhance the time-series data and improve the performance of the algorithms. A notable outcome of these transformations is the improvement in accuracy across all algorithms, particularly in the case of linear algorithms.

Upon applying the Rocket, MiniRocket, and MultiRocket transforms, linear algorithms such as logistic regression, ridge regression, and LDA exhibit a substantial increase in their accuracy, as can be observed in the tables. These improvements can be attributed to the transforms’ ability to capture better the underlying patterns in the data, which allows linear algorithms to leverage this information and perform more effectively.

For instance, in Table 2, the accuracy of logistic regression increases from 0.7552 ± 0.0353 with Rocket to 0.8465 ± 0.06 with MultiRocket. In contrast, the accuracy of ridge regression increases from 0.6762 ± 0.0462 with Rocket to 0.8068 ± 0.0447 with MultiRocket. Similarly, in Table 3, logistic regression and ridge regression exhibit high accuracy of 0.955 ± 0.0395 and 0.9533 ± 0.036 with Rocket, respectively. These results suggest that the use of data transforms boosts the performance of linear algorithms, enabling them to compete with more complex models.

It is essential to note that the algorithm’s performance still improves as the time window size increases, consistent with the earlier observation in Table 1. This trend is evident across all three tables, reinforcing the importance of considering longer time windows when applying these algorithms to insulator fault detection.

5.1. Empirical Mode Decomposition

Table 5 presents the logistic regression results applied to fault detection in insulators with the MiniRocket transform for a window size of 100 (WS100), and further explores the impact of combining the MiniRocket transform with three different EMB methods: EWT, CEEMDAM, and VMD. The purpose of applying these EMB methods before the MiniRocket transform is to explore if the predictions can be further improved.

When these EMB methods are applied in conjunction with MiniRocket, the accuracy of logistic regression significantly improves compared to using the MiniRocket transform alone. This improvement can be attributed to the ability of EMB methods to decompose the time series into different components, thereby highlighting the underlying patterns and structures in the data that may not be easily captured by the MiniRocket transform.

For example, when using the EWT method, the accuracy of logistic regression increases from 0.9783 ± 0.0194 without EMB to 0.995 ± 0.0067 with EWT. Similarly, the accuracy of logistic regression improves to 0.9917 ± 0.0105 with CEENDAM and 0.98 ± 0.0187 with VMD. Figure 7 presents the critical difference diagram comparing the methods. These results indicate that applying EMB methods before the MiniRocket transform enhances the performance of logistic regression by providing a more refined representation of the data.

5.2. Discussion

The findings in the preceding subsections offer insightful information on how various machine learning techniques for defect detection perform. The study of these results can aid the selection of acceptable methodologies and techniques for fault detection in insulators, notably the effects of time window size and data transforms. The performance of the models was constantly enhanced by increasing the temporal window size. Longer time windows give the algorithms more information to find patterns and connections between features, improving insulator defect identification.

According to this study, adopting larger time windows should be favored in practical applications to improve the precision of defect detection models. The advantages of extended periods must be weighed against the additional computational expenses. The amount of data being processed and the complexity of the models grow as the time window size grows. More extended training periods and increased computational demands may result from this. Therefore, when choosing the time window size for insulator fault detection, practitioners should carefully consider the trade-offs between the improvements in accuracy and the extra computational resources needed.

The results show that tree-based approaches, considering random forests, decision trees, gradient boosting, AdaBoost, XGBoost, and LightGBM consistently outperform other algorithms in insulator fault detection. These techniques offer great accuracy when identifying insulator faults and are particularly good at managing non-linear connections between features. This shows that tree-based approaches should be the best options for insulator failure detection jobs. However, tree-based methods might be more prone to overfitting than other algorithms, mainly when working with small datasets. Pruning is one regularization approach that should be used to decrease overfitting risks and preserve model generalizability.

Logistic regression, ridge regression, and LDA perform much better when Rocket, MiniRocket, and MultiRocket data transforms are used. These modifications allow linear algorithms to take advantage of the information and perform better, even competing with more complicated models, by capturing more underlying patterns in the data. This result suggests that data transforms can be a useful preprocessing step in real-world applications, especially when using linear algorithms for insulator fault detection. Engineers may simplify their models by using these transforms while maintaining excellent fault detection accuracy.

Further, the results indicate that combining EMD methods and MiniRocket transform enhances the performance of logistic regression by providing a more refined representation of the data. This suggests that using EMD methods can improve fault detection capabilities when used in conjunction with rocket-like algorithms. The following guidelines it can be given for insulator failure detection using ultrasound signals in light of the study’s findings:

• Consider the trade-offs with computational resources and training timeframes carefully when using longer time windows to increase the fault detection models’ accuracy.
• Consider tree-based algorithms for insulator failure detection, such as CatBoost, LightGBM, and gradient boosting, while being cautious of overfitting concerns and using regularization techniques as necessary. To improve the efficiency of linear algorithms and potentially reduce model complexity while retaining high accuracy, use data transforms like Rocket, MiniRocket, or MultiRocket.
• Employ EMD methods to enhance the performance of less complex regression methods by providing a more refined representation of the data and improving fault detection capabilities.

By using these suggestions, engineers can create more successful insulator failure detection models that improve the dependability and safety of electrical power systems. The next section presents the final remarks and suggestions for future work. The conclusions will be related to the applicability of ultrasound, the best structure to classify the time series, and what can be accomplished with the ultrasound equipment in future projects.

6. Conclusion and future directions of research

Using ultrasound as a decision-making support tool during inspections of the electrical power grid combined with deep learning proves to be promising since the proposed model achieves acceptable classification results. A significant advantage of using this equipment is that it is not necessary to measure contact with the electrical potential, reducing the risk to the operator and improving flexibility in inspecting the distribution power grid.

The findings indicate that tree-based methods, such as decision tree, random forest, gradient boosting, AdaBoost, XGBoost, and LightGBM generally outperform other algorithms in terms of accuracy. Longer time windows (e.g., WS100) resulted in improved performance across all algorithms, suggesting that larger windows provide more information for pattern identification. Additionally, the application of Rocket, MiniRocket, and MultiRocket data transforms led to a significant increase in accuracy for linear algorithms such as logistic regression, ridge regression, and linear discriminant analysis. This improvement can be attributed to the transforms’ ability to capture the underlying patterns in the data better, enabling linear algorithms to perform more effectively.

In future work, it is promising to evaluate the development of an embedded system to perform the inspections based on the model presented in this paper. The evaluated model has a low computational effort in the test phase, making its application in an embedded system an interesting solution to be explored, besides the test performed with the aid of a personal computer.