Global and Local Knowledge Distillation Method for Few-Shot Classification of Electrical Equipment

1. Introduction

As an essential component of the power system, daily inspection of electrical equipment is imperative to ensure the secure and stable operation of the power system [1]. The conventional manual screening and analysis approach can no longer meet the escalating demand for image analysis of electrical equipment. With the advent of the smart grid, an increasing number of unmanned aerial vehicles (UAVs) are being deployed for online inspection. The application of artificial intelligence techniques for condition monitoring of electrical equipment can significantly enhance the efficiency of detection and maintenance. Image classification is a crucial prerequisite for equipment condition monitoring based on image information. For instance, to monitor the normalcy of electrical equipment such as transformers or insulators, their images must be initially distinguished.

With the advancement of deep learning in image recognition applications, various classification and recognition methods for electrical equipment images based on deep convolutional neural network (CNN) have been proposed. However, deep CNN training often relies on large-scale labeled data, which is challenging to obtain for all categories of electrical equipment due to their safety and sensitivity. Therefore, this paper adopts few-shot learning (FSL) method to the task of electrical equipment image classification. This method randomly divides the power image dataset into a base class set and a new class set, trains the model backbone on the base class set, and finally combines the model backbone with the classifier to complete the recognition training of the new class with a small number of samples.

Furthermore, power inspection heavily relies on intelligent mobile devices, such as inspection robots and UAVs. However, due to the limited storage capacity of these devices, the classification model's capacity must not be excessively large. Otherwise, it cannot be deployed on such mobile devices. To address these challenges, this paper presents a novel few-shot electrical image classification algorithm based on knowledge distillation.

Knowledge distillation [2,3] is an efficient model compression method that compresses the knowledge of teacher networks into very small student networks. Early knowledge distillation methods that minimize the KL divergence of predicted class probability distributions between student and teacher networks rely on the output of the last layer of the model and learn a limited amount of information. Recent work has begun to study the features of the middle layer of the distillation network, focusing on the learning of local features of the image. However, there are both differences in the global appearance and local details between electrical equipment, and the algorithm model must fully mine this information in the learning process in order to comprehensively represent the electrical equipment images and achieve higher classification precision. Therefore, this paper proposes a global and local knowledge distillation method for few-shot classification of electrical equipment.

In this paper, we propose three main contributions:

• We present a novel distillation approach that compresses the knowledge of teacher networks into a compact student network, enabling efficient few-shot classification. The incorporation of global and local relationship strategies during the distillation process effectively directs the student network towards achieving performance levels akin to those of the teacher network.
• We contribute a new dataset that contains 100 classes of electric equipment with 4000 images. The dataset contains a wide range of various electrical equipment, including power generation equipment, distribution equipment, industrial electrical equipment, and household electrical equipment.
• We demonstrate the effectiveness of our proposed method by validating it on three public datasets and comparing it with the SOTA methods on the electric image dataset we introduced. Our proposed method outperforms all other methods and achieves the best performance.

2. Related Work

2.1. Electrical Images Classification

Image-based equipment condition monitoring has been proven to be effective in enhancing the working life of equipment and providing early failure warning. In recent years, machine learning has made significant progress in the field of image classification for electrical equipment. Bogdann presented a machine learning method for determining the state of each switch by analyzing images of the switches in power distribution substations [4].Zhang implemented FINet based on improved YOLOv5 to inspect the insulators and their defects for ensuring the safety and stability of power system [5].To address few fault cases and deficient monitoring information in transformer diagnostic tasks, Xu provides an improved few-shot learning method based on approximation space and belief functions [6]. Yi proposed a label distribution CNNs classifier to estimate the aging time of the conductor morphology of high-voltage transmission line [7].

It is noteworthy that the majority of the aforementioned investigations have concentrated on a restricted range of electrical apparatus. These models necessitate a substantial quantity of training data to guarantee optimal performance. Nevertheless, acquiring adequate electrical equipment images in a practical setting may prove to be challenging, and the proportion of labeled samples is minimal. To a certain extent, the classification of electrical equipment images does not truly belong to a big data problem. Rather, it pertains to FSL domains.

2.2. Few-shot Classification

In recent years, FSL has attracted researcher’s widespread attention in the field of computer vision and machine learning, and a large number of few-shot image classification algorithms have been proposed. Depending on the learning paradigm used, these methods can be broadly divided into two categories: meta-learning-based methods and transfer-based learning methods.

Meta-learning is a promising approach that leverages episodic training to simulate the real test environment by randomly selecting several subtasks. This enables the acquisition of meta-knowledge that facilitates the rapid identification of new categories. Based on the type of meta-knowledge learned, meta-learning methods can be classified into optimization-based and metric-based methods. Optimization-based meta-learning methods employ a two-tier optimization process to learn the optimizer for quickly processing new tasks. A well-known example of such methods is Model-Agnostic Meta-Learning (MAML) [8]. MAML obtains the optimal initialization parameters of the model through meta-training, enabling the model to adapt to new tasks after a few gradient updates. In addition, the learning rate and gradient direction are also important factors for the optimizer [9,10]. However, these methods require storage and computation of higher-order derivatives, resulting in high memory and computational costs. On the other hand, metric-based methods use nonparametric classifiers as the basic learner, avoiding the aforementioned issues. The key factors of these methods are feature extraction and similarity measurement, which offer ample room for improvement. PARN [11] proposed a feature extractor which is learning an offset for each cell in the convolution kernel to extract more efficient features, building by deformable convolutional layers. CC+rot [12] improved the transfer ability of feature extractors by adopting auxiliary self-supervised tasks. Zhang et al. [13] used the pre-trained visual saliency detection model to segment the foreground and background of the image, and then extract the foreground and background features respectively. With the proven effectiveness of attention mechanisms in extracting discriminating features, several few-shot classification (FSC) tasks have adopted this method, including CAN [14], AWGIM [15], and CTM [16]. Additionally, in metric-based meta-learning methods, the measurement of similarity is also crucial. SEN [17] combines Euclidean distance and norm distance to improve the effectiveness of Euclidean distance measurement in high-dimensional spaces. FPN [18] calculated the reconstruction error between the support sample and the query sample as the similarity score. DN4 [19] and Deep EMD [20] obtain rich similarity measures directly on local features.

Recent studies have indicated that FSC of transfer-learning method can attain comparable performance to that of meta-learning method with complex episodic training. Such methods typically combine pre-trained feature extractors on all base class datasets with arbitrary traditional classifiers to make classification decisions for query samples of unknown classes. Reference [21] showed that pre-training the entire base class dataset using the cross-entropy loss function, followed by fine-tuning the pre-trained model using support samples of the visible class, can provide a powerful baseline for FSC tasks. Since then, several works have been proposed to improve the representation performance of feature extractors. For example, Neg-Cosin [22] proposed to use the non-negative interval Cosine loss function to optimize the model, thereby increasing the distance between the training sample and its corresponding parametric prototype, which can effectively improve the generalization performance of the model. S2M2[23] used manifold mixing as an effective regularization method to improve the generalization performance of the model. Reference [24] and [25] used rotation prediction and mirror prediction as self-supervised tasks to add to the pre-training process, and experimental results show that self-supervised tasks are effective methods to improve feature representation performance.

In conclusion, many recent works have emphasized the importance of feature representation, both meta-learning-based and transfer-learning-based methods tended to employ highly complex networks to enhance feature representation. Therefore, deploying these methods to real-world applications usually occupies high computing resources (storage space, computing power, etc.) and introduces high time delays, which cannot meet the actual needs of the classification tasks of electrical equipment images. Hence, in this study, we employ the knowledge distillation-based model compression algorithm to accomplish the task of few-shot image classification to reduce the model parameters.

2.3. Knowledge Distillation

Knowledge distillation is one of the most effective model compression methods, which has garnered significant research interest in both industry and academia due to its simple training strategy and effective performance. It leverages the knowledge acquired by a teacher network with a large scale to guide the training of a small-scale student network, enabling the latter to achieve comparable performance despite having fewer parameters.

Two key elements in current knowledge distillation methods can be summarized as: (1) the definition of effective knowledge types, (2) Effective transfer of knowledge from teacher networks to student networks. The classical knowledge distillation method minimizes the KL divergence of the predicted class probability distribution between the student and teacher networks. In order to make better use of the knowledge information contained in the teacher network, the follow-up work focuses more on how to better mine the feature knowledge hidden in the middle layer of the network. For example, AT [26] proposed to take the spatial attention of the hidden layer features of the teacher network as knowledge, and instructs the student network to imitate its attention feature map. Recently, the relationship between samples features has been proposed as a more effective knowledge. RKD [27] proposed a relational knowledge distillation method, which used distance and angle to measure the relationship between samples features, as a valid type of knowledge during distillation. Peng et al. [28] used the kernel function to obtain higher-order relationships between samples features as effective distillating knowledge.

Although the model compression methods based on feature relation knowledge mentioned above can effectively improve the performance of small-capacity student networks, the current work only focuses on the local relationship between individual sample features, ignoring the global relationship between samples features. Therefore, this paper proposes a method based on global and local knowledge distillation and applies it to the task of FSC of electrical equipment images.

3. Methodology

3.1. Problem Definition

In few-shot image classification tasks, given a certain size of image dataset $I$ , it is randomly divided into three subsets represented as ${{\displaystyle I}}_{train}$, ${{\displaystyle I}}_{\mathrm{val}}$and ${{\displaystyle I}}_{\mathrm{test}}$ respectively. ${{\displaystyle I}}_{train}$, also known as the base dataset, is used for pre-training the classification model. Assuming that the training set has ${{\displaystyle \mathrm{C}}}_t$ categories, the m^th image sample is represented as ${{\displaystyle \mathrm{x}}}_m$, and its corresponding label is represented as ${{\displaystyle \mathrm{y}}}_m$. ${{\displaystyle I}}_{\mathrm{val}}$ is used for validation, while ${{\displaystyle I}}_{\mathrm{test}}$, also known as the new class dataset, is used for testing. For ${{\displaystyle I}}_{\mathrm{val}}$, and ${{\displaystyle I}}_{\mathrm{test}}$, many N-way-K-shot subtasks are randomly sampled, and each task consisting of a support sample set ${{\displaystyle I}}_S$ and a query sample set ${{\displaystyle I}}_Q$. ${{\displaystyle I}}_S$ is constructed by randomly selecting N categories from ${{\displaystyle I}}_{\mathrm{val}}$ or ${{\displaystyle I}}_{\mathrm{test}}$ , and then randomly selecting K samples from each category. The set of the n^th category is denoted as ${{\displaystyle I}}_n={{\displaystyle \{({{\displaystyle I}}_k,{{\displaystyle y}}_k)\}}}_{k=1}^K$ , and the k^th image in the n^th category is denoted as ${{\displaystyle I}}_k$. ${{\displaystyle I}}_Q$ is composed of Q samples randomly selected from each residual sample category, denoted as ${{\displaystyle I}}_Q={{\displaystyle \left\{{{\displaystyle I}}_q\right\}}}_{q=1}^Q$, where ${{\displaystyle I}}_q$ represents the q^th query sample. Therefore, the problem of few-shot image classification can be described as using the model trained on the base class dataset and the support sample set to make classification decisions for query samples

3.2. FSC Network based on Global and Local Knowledge Distillation

We propose a novel few-shot electrical image classification algorithm based on knowledge distillation. Figure 1 shows the overall architecture of our network. We first trained a high-performance teacher network through self-supervised learning, and then guided the training of the student network by the teacher network. To fully utilize the prior knowledge of the teacher network, we designed a knowledge distillation method based on global and local relationships. This method can transfer the global and local features of the images extracted by the teacher network to the student network, enabling the small-capacity student network to learn more effective features about the images and achieve better image classification.

3.2.1. Pre-train of Teacher Network

The teacher model consists of a backbone convolutional neural network and two linear classifiers. The backbone network ${{\displaystyle f}}_{\theta }(•)$ is used for feature extraction of images, one classifier ${{\displaystyle L}}_w(·)$ is used for predicting the base class of image samples, and the other classifier ${{\displaystyle L}}_r(·)$ is used for predicting the rotation category in self-supervised tasks. Additionally, each classifier is followed by a Softmax layer. M image samples are randomly selected from the base class dataset, and each image is rotated at 0°, 90°, 180°, and 270°, with the m^th image represented as ${{\displaystyle x}}_m$, its corresponding base class label as ${{\displaystyle y}}_m$, and its corresponding rotation label as ${{\displaystyle \widehat{y}}}_m$=[0,1,2,3].

The image is fed into the teacher network, and the d-dimensional feature representation ${{\displaystyle f}}_{\theta }({{\displaystyle x}}_m)$ is extracted by the backbone network. The classification scores of the base class prediction classifier and the rotation prediction classifier for the features are expressed as:

$$\{\begin{array}{c}{{\displaystyle {{\displaystyle S}}_{mb}=L}}_w({{\displaystyle f}}_{\theta }({{\displaystyle x}}_m))\\ {{\displaystyle {{\displaystyle S}}_{mr}=L}}_r({{\displaystyle f}}_{\theta }({{\displaystyle x}}_m))\end{array}$$

(1)

Furthermore, the aforementioned classification scores are transformed into base class and rotation class prediction probabilities through a Softmax layer, as shown follows.

$$\{\begin{array}{c}{{\displaystyle p}}_1({{\displaystyle y}}_m=c|{{\displaystyle x}}_m)=\frac{{{\displaystyle e}}^{{{\displaystyle S}}_{mbc}}}{{\displaystyle \sum _{c=1}^{{{\displaystyle C}}_b}{{\displaystyle e}}^{{{\displaystyle S}}_{mbc}}}}\\ {{\displaystyle p}}_1({{\displaystyle \widehat{y}}}_m=r|{{\displaystyle x}}_m)=\frac{{{\displaystyle e}}^{{{\displaystyle S}}_{mrr}}}{{\displaystyle \sum _{r=1}^4{{\displaystyle e}}^{{{\displaystyle S}}_{mrr}}}}\end{array}$$

(2)

where, ${{\displaystyle S}}_{mbc}$ represents the c^th component of the score vector for the base class, ${{\displaystyle S}}_{mrr}$ represents the r^th component of the score vector for the predicted class, ${{\displaystyle C}}_b$ represents the number of base class labels, and $p({{\displaystyle y}}_m=c|{{\displaystyle x}}_m)$ and $p({{\displaystyle \widehat{y}}}_m=r|{{\displaystyle x}}_m)$ represent the probability output values of the base classifier and the rotation classifier, respectively. The cross-entropy loss function and the self-supervised loss function are calculated to obtain the training loss function, as shown follows.

$$L(\theta ,w,r)=-{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{c=1}^{{{\displaystyle C}}_b}{{\displaystyle y}}_{mc}}}\log p({{\displaystyle y}}_m=c|{{\displaystyle x}}_m)-{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{r=1}^4{{\displaystyle \widehat{y}}}_{mr}}}\log p({{\displaystyle \widehat{y}}}_m=r|{{\displaystyle x}}_m)$$

(3)

Based on the loss function in Equation (3), the parameters of the teacher network are optimized to complete the pre-training process of the teacher network.

3.2.2. Global and Local Knowledge Distillation

Firstly, a student network is constructed, which consists of a backbone neural network ${{\displaystyle B}}_{\varphi }(\cdot )$ composed of a small number of convolutional layers and a linear classifier ${{\displaystyle C}}_H(\cdot )$. Next, a batch of M images randomly selected from the base dataset ${{\displaystyle I}}_{train}$ are inputted into both the teacher network and the student network. The m^th image is represented by feature maps ${{\displaystyle z}}_m^t{{\displaystyle =f}}_{\theta }({{\displaystyle I}}_m)$ and ${{\displaystyle {{\displaystyle z}}_m^s=B}}_{\varphi }({{\displaystyle I}}_m)$ obtained from the backbone of the teacher network and the student network, respectively. Finally, the features are fed into the linear classifier to obtain the output value of the student network, as shown follows.

$${{\displaystyle {{\displaystyle S}}_m=C}}_H({{\displaystyle z}}_m^t)$$

(4)

Furthermore, the above output classification scores are transformed into classification prediction probabilities through the Softmax layer, as shown follows.

$${{\displaystyle p}}_t({{\displaystyle y}}_m=c|{{\displaystyle x}}_m)=\frac{{{\displaystyle e}}^{{{\displaystyle S}}_{mc}}}{{\displaystyle \sum _{c=1}^{{{\displaystyle C}}_b}{{\displaystyle e}}^{{{\displaystyle S}}_{mc}}}}$$

(5)

The formula for calculating the cross-entropy loss function between the output values of a student network and the true labels is shown as follows.

$${{\displaystyle l}}_1(\varphi ,H)=-{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{c=1}^{{{\displaystyle C}}_b}{{\displaystyle y}}_{mc}}}\log {{\displaystyle p}}_t({{\displaystyle y}}_m=c|{{\displaystyle x}}_m)$$

(6)

In order to enable students to learn the representation of global features of images by the teacher network through online learning, we adopt the maximum mean discrepancy between the feature spaces of the two networks as the global loss function, which is calculated as follows.

$${{\displaystyle l}}_2(\varphi ,H)=\frac{1}{{{\displaystyle M}}^2}{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{m\text{'}=1}^M{z}_m^t{z}_{m\text{'}}^t{}^T}}+\frac{1}{{{\displaystyle \mathrm{M}}}^2}{\displaystyle \sum _{m=1}^{\mathrm{M}}{\displaystyle \sum _{m\text{'}=1}^{\mathrm{M}}{{\displaystyle z}}_m^s{z}_{m\text{'}}^s{}^T}}-\frac{1}{{{\displaystyle \mathrm{M}}}^2}{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{m\text{'}=1}^M{{\displaystyle z}}_m^t}}{z}_{m\text{'}}^s{}^T$$

(7)

In addition, we calculate the Euclidean distance between each sample feature in the two networks as the local loss function, and its calculation formula is as follows.

$${{\displaystyle l}}_3(\varphi ,H)=\frac{1}{M}{\displaystyle \sum _{m=1}^M{{\displaystyle \Vert {{\displaystyle z}}_m^t-{{\displaystyle z}}_m^s\Vert }}^2}$$

(8)

In summary, the total loss function for the student network is shown in equation (9). Based on equation (9), the student network is trained and the parameters in the network are updated until optimal, thereby completing the knowledge distillation process from the teacher network to the student network.

$$L(\varphi ,H)={{\displaystyle l}}_1(\varphi ,H)+{{\displaystyle \alpha }}_1{{\displaystyle l}}_2(\varphi ,H)+{{\displaystyle \alpha }}_2{{\displaystyle l}}_3(\varphi ,H)$$

(9)

3.2.3. Few-shot Evaluation

After completing the knowledge distillation task in Section 3.2.2, the base classifiers in the student network are first removed. Then, the parameters of the backbone neural network ${{\displaystyle \mathrm{B}}}_{\phi }(•)$ are fixed, and features are extracted from both the support and query samples. Finally, based on N-way-K-shot method, the query samples are classified using Equation (10), where the features of the k^th support sample and the q^th query sample are denoted as ${{\displaystyle \mathrm{B}}}_{\phi }({{\displaystyle I}}_k)$ and ${{\displaystyle \mathrm{B}}}_{\phi }({{\displaystyle I}}_q)$, respectively, and ${{\displaystyle g}}_{\phi }\{•\}$ represents a classifier with parameters $\phi$. Any traditional classifier can be used to complete the classification prediction task.

$${{\displaystyle \widehat{y}}}_q={{\displaystyle g}}_{\phi }\{{{\displaystyle B}}_{\varphi }({{\displaystyle I}}_q)|{\displaystyle \sum _{k=1}^{NK}{{\displaystyle B}}_{\varphi }({{\displaystyle I}}_k)}\}$$

(10)

4. Experiments

4.1. Experiments on Public Datasets

4.1.1. Experiment Setup

We evaluate our knowledge distillation method on three public datasets, namely MiniImageNet, CIFAR-FS, and CUB. The experiments are conducted on a workstation equipped with NVIDIA 3090Ti GPU and implemented using Pytorch software. To ensure a fair comparison with current small sample image classification methods, a commonly used 4-layer convolutional neural network and ResNet12 are adopted as the student network and teacher network, respectively. During the training phase, we use the SGD optimizer to optimize our models in all experiments, where the momentum is set to 0.9 and weight decay is set to 5×10-4. We train for 100 epochs, with an initial learning rate of 0.025, which is reduced by half after 60 epochs. In the testing phase, we conduct 5-way-1-shot and 5-way-5-shot tests. Specifically, we randomly perform 2000 classification subtasks in the testing dataset. In each subtask, 15 images are randomly selected from each class as query images for testing. The evaluation criterion for the algorithm's classification performance is the average accuracy of all subtasks, and the standard deviation of the accuracy under a 95% confidence interval should also be provided.

4.1.2. Parametric Analysis Experiment

It can be seen from formula (9) that ${{\displaystyle \alpha }}_1$ and ${{\displaystyle \alpha }}_2$ are important hyperparameters in the process of distilling the student network. Firstly, the parameter ${{\displaystyle \alpha }}_2$ is fixed at 1, and the value of parameter ${{\displaystyle \alpha }}_1$ is varied with a step size of 0.1 within the range of [0,1]. The test accuracy of the student network under different values of ${{\displaystyle \alpha }}_1$ is shown in Figure 2 (a) and (b). It can be seen from the results that the model performance is optimal when the value of ${{\displaystyle \alpha }}_1$ is 0.5. Therefore, the value of ${{\displaystyle \alpha }}_1$ is set to 0.5, and then the ${{\displaystyle \alpha }}_2$ is varied with a step size of 0.01 within the range of [0,0.1]. The test accuracy of the student network under different values of ${{\displaystyle \alpha }}_2$ is shown in Figure 2(c) and (d). It can be seen from the results that the optimal value for ${{\displaystyle \alpha }}_2$ is 0.1.

4.1.3. Ablation Experiment

The innovation of this work lies in proposing a knowledge distillation algorithm for global and local relationships. In order to verify the effectiveness of the proposed method, detailed ablation experiments are conducted on three public datasets. The knowledge distillation algorithms using only global and local relationships are denoted as Global and Local, respectively, and their fusion is denoted as Global-Local. The classification accuracies of these methods on 5-way 1-shot and 5-way 5-shot tasks are shown in Table 1. The results in the table indicate that for both 5-way 1-shot and 5-way 5-shot tasks on all datasets, the classification accuracy of Global-Local is consistently higher than that of Global and Local. The experiments demonstrate that global and local relationships are complementary, and their fusion can extract richer image features. Therefore, the knowledge distillation algorithm based on global and local relationships can further improve the performance of knowledge distillation.

4.1.4. Comparison Experiment with Existing Methods

This paper compares our method with the state-of-the-art (SOTA) methods in recent years, which are mainly divided into two categories: meta-learning-based methods and transfer learning-based methods. The comparison results with these methods are shown in Table 2.

According to the results in Table 2, it can be observed that: (1) On the MiniImageNet dataset, our proposed method achieves the best classification performance. Compared with the best performing method in the meta-learning-based category, HGNN, our method outperforms it by 2.23% and 0.9% on 1-shot and 5-shot classification tasks, respectively. In the transfer learning-based category, compared with the best performing method, CGCS, our method outperforms it by 2.33% and 1.26% on 1-shot and 5-shot classification tasks, respectively. (2) On the CIFAR-FS dataset, our proposed method also achieves the best performance. Compared with the best performing method, PSST, our method outperforms it by 2.67% and 0.42% on 1-shot and 5-shot classification tasks, respectively. (3) On the CUB-200-2011 dataset, our proposed method achieves the best classification performance. Compared with the best performing method, HGNN, our method outperforms it by 1.42% and 0.99% on 1-shot and 5-shot classification tasks, respectively.

4.2. Electrical images Dataset

4.2.1. EEI-100 Dataset

EEI-100(electric equipment image of 100 classes) is a few shot electric equipment classification dataset established by us, which contains 100 classes of electric equipment with 4000 images. The majority of the images were obtained through on-site collection, with a small number of images sourced from online platforms. To the best of our knowledge, this is one of the first datasets specifically designed for classification of electrical equipment. This dataset is an extension of our previous EEI-40 [29]. It includes substation equipment, distribution station equipment and common electrical equipment, ranging from large-scale equipment such as heavy-duty transformers to small-scale equipment such as circuit breakers. A few images from the proposed dataset illustrated in Figure 3. More images illustrated in Appendix A.

4.2.2. Parametric Analysis Experiment

By following the approach outlined in Section 4.1.4, the values of parameters ${{\displaystyle \alpha }}_1$ and ${{\displaystyle \alpha }}_2$ are determined to optimize the performance of the model on EEI-100. Experimental results show that ${{\displaystyle \alpha }}_1$ has the optimal value of 0.6 within the range of [0.1,1], as illustrated in Figure 4(a). Similarly, ${{\displaystyle \alpha }}_2$ has the optimal value of 0.1 within the range of [0.01,0.1], as illustrated in Figure 4(b).

4.2.3. Comparison Experiment with Existing Methods

To demonstrate the superiority of our proposed method in the classification of electric equipment images, this section presents a comparative experiment with three existing methods, namely CGCS, Neg-Cosine, and HGNN, on the EEI-100 dataset. These three methods have recently achieved good performance on public datasets. The classification accuracy of the test set is presented in Table 3. Specifically, our method achieves the highest classification accuracy (up to 94.12%) compared to the other methods.

5. Conclusions

In conclusion, this paper proposes a novel few-shot electrical image classification algorithm based on knowledge distillation. The proposed algorithm addresses the challenges of limited storage capacity of intelligent mobile devices and the difficulty in obtaining large-scale labeled data for all categories of electrical equipment. By adopting few-shot learning method and global and local knowledge distillation, the proposed algorithm achieves high classification accuracy with only a small number of samples. The results on EEI-100 demonstrate that the prediction accuracy of the proposed method reaches 94.12% by collecting only 5-shot images. This research provides a promising solution for the online inspection of electrical equipment, which can significantly enhance the efficiency of detection and maintenance in the power system. Future work can explore the application of the proposed algorithm in other domains and investigate the potential of combining it with other deep learning techniques to further improve the performance.