1. Introduction

2. Deep Belief Network classifier

3. Multi-DBN weighted voting algorithm

3.1. Method of voting

Voting, or Majority Voting[24], is a commonly employed method for combining classifiers. This method is based on the following principle: a set of trained base classifiers (Base Classifier) is used to make classification decisions on the feature data to be classified. Each base classifier casts a vote for the decision result that corresponds to the target class, and the class with the most votes is selected as the classification result for the entire set of classifiers. In this paper, three classifiers are utilized to determine the classification results through a voting process. Please refer to Figure 4 for an illustration of the classification process.

Assume that there are L base classifiers C_l (l=1, 2, ..., L) in the combined classifier, and the output of the each base classifier C_l is a class probability estimate P(f(w)=k|C_l ), which represents the magnitude of the probability that the feature data to be classified, w, is predicted as class k (k=1, 2, ...,) by the respective classifier C_l . The probability estimates of all base classifiers are combined to calculate the following equation's value:

$$P(f(w)=k)=\frac{1}{L}{\displaystyle \sum _{l=1}^{L}P}(f(x)=k|C_l)$$

(11)

The combined classifier determines the probability that the feature vector, w, belongs to class k, and assigns w to the class with the highest probability value. If all classifiers within the combined classifier classify the feature vector w to be classified as an independent event, the classification accuracy of the base classifiers after voting is raised to the P(w), where each base classifier C_l in the combined classifier has a classification accuracy of Pc.

$$P(w)=1-{\displaystyle \sum _{l=0}^{floor(L/2)}{C}_L^l}{(P_c)}^l{(1-P_c)}^{L-l}$$

(12)

The function floor(.) rounds downward, while floor(L/2) represents the greatest integer not exceeding L/2. Take L=3 as an example to illustrate the classification accuracy of the combination classifier, the classification accuracy of the combination classifier is:

$$\begin{array}{cc}\hfill P(w)& =1-C_3^0{(P_c)}^0{(1-P_c)}^3-C_3^1{(P_c)}^1{(1-P_c)}^2\hfill \\ & =1-{(1-P_c)}^3-3P_c·{(1-P_c)}^2\hfill \\ & =1-{(1-P_c)}^2·(2P_c+1)\hfill \end{array}$$

(13)

The classification accuracy P(w) is calculated with Pc as the variable, and figure 5.7 displays the trends of classification accuracy obtained using a base classifier with 1 and 3 classifiers, respectively.

Figure 5. Trend of Combined Classifier Classification Accuracy.

Figure 5. Trend of Combined Classifier Classification Accuracy.

When base classifiers accurately classify feature vectors over 50%, the accuracy of a combined classifier that utilizes three base classifiers to classify feature vectors through voting is significantly greater than that of a single classifier. The increased voting that takes place after classification by multiple classifiers allows for effective comparison of misclassified feature vectors of individual classifiers and subsequent correction of errors through the application of the "majority rule" principle.

The classification results using the voting algorithm in the best-case scenario without assigning weights to each classifier vote are displayed in Table 1

When processing the voting results, the category with the most votes is identified as A, and the corresponding target category for the feature data then was classified as A. However, when the classifier is not set with the voting weights, voting classification using the voting method may result in target categories receiving the same number of votes, leading to an inability to classify them. Table 2 displays an example of such voting results.

As shown in Table 2, the tallied votes reflect an equal number of votes for the three categories: A, B and D. Consequently, it is unfeasible to determine the category corresponds to the given feature data with the highest votes. To address this predicament, we proposing voting weight values after achieved the classification results from each classifier.If the classification result of a classifier is considered to be more credible, it is given a larger voting weight, whereas a less credible result will be given a smaller weight. Even when multiple classifiers have voting results as displayed in Table 5.2, the final vote counts for each class will differ once weights are implemented. The class of the data to be classified can still be successfully determined.

3.2. Calculation of classifier weights

The primary challenge regarding the incorporation of voting weights is to establish the level of confidence in the classifier's classification results. To classify the target data, the DBN network is first trained using sample dataset and then implemented to classify the feature data of the target. The accuracy of classification depends on the performance of the DBN classifier that has been trained and the features of the data being classified. The features of the data cannot be controlled, and the classifier's performance is significantly influenced by the training sample dataset that was used. The principle of classifying neural networks assumes a functional relationship between target feature data and corresponding categories. Sample dataset is used to train the network so that it fits this relationship, processing the feature data for classification. Therefore, the degree to which the neural network accurately fits the function relationship of the entire network with the genuine mapping relationship between the target feature data and the corresponding category is highly dependent on the sample dataset used for training. The accuracy of the mapping relationship between the feature data fitted by the network and the corresponding class is greater when there is a large distance between target feature data of different classes and no data mixing occurs in the sample dataset. The Lance-Williams method[25] is used to calculate the distance between sample data for training the classifier. The distance is calculated as follows:

A, B, C, and D represent the sample data of four different vehicle signals detected by the WSN monitoring system. The feature matrices, which can also be regarded as a set of observations of the feature vectors, are shown in matrices A and B in equation (14). These matrices are obtained through feature extraction method applied to the sample data of these targets.The vector w_ai denotes the ith column observation of the feature matrix corresponding to the A target signal, and w_bi denotes the ith column observation of the feature vector corresponding to the B target signal.

(14)

As demonstrated by the above equation, the characteristics A and B of the signal features for both kinds of targets are N × M matrices. N, which is the number of rows in the matrix, represents the number of segments into which the target signals that correspond to feature A are divided. M represents the number of variables in the feature vectors that are extracted from each signal sequence segment, which is twice the number of wavelet decomposition layers in thefeature extraction method. Columns in the matrix correspond to the feature vectors of the target signal in different dimensions. Hence, while computing the distance between two feature matrices, the first step is to calculate the distance between the eigenvectors of the same column of both feature matrices using equation (15).

$$d(w_{ai},w_{bi})=\sqrt{(w_{ai}-w_{bi}){(w_{ai}-w_{bi})}^T}=\sqrt{{\displaystyle \sum _{j=1}^N{(a_{ij}-b_{ij})}^2}}$$

(15)

Where a_ij represents the jth variable in the ith eigenvector of the feature matrix A and b_ij is the jth variable in the ith eigenvector of the feature matrix B. The feature matrix of the target signal contains M eigenvectors (w₁ , ... , w_M ), after calculating the distance between the feature vectors in the feature matrix, quantization of the distance between the feature matrices and the target signal sequences is necessary.Equation (16) is utilized to calculate the distance between the matrices.

$$d(A,B)=\frac{1}{M}{\displaystyle \sum _{i=1}^{M}d(w_{ai},w_{bi})}=\frac{1}{M}{\displaystyle \sum _{i=1}^M\sqrt{{\displaystyle \sum _{j=1}^N{(a_{ij}-b_{ij})}^2}}}$$

(16)

Where a_ij represents the jth variable in the ith eigenvector of the feature matrix A and b_ij is the jth variable in the ith eigenvector of the feature matrix B. The feature matrix of the target signal contains M eigenvectors (w₁ , ... , w_M ), after calculating the distance between the feature vectors in the feature matrix, quantization of the distance between the feature matrices and the target signal sequences is necessary.Equation (16) is utilized to calculate the distance between the matrices.

In the cross-voting classification method, merging multiple target sample data is required to train a classifier. Consequently, it is essential to compute the distance between merged matrices of multiple target signal feature matrices, utilizing the distances between single-target signal feature matrices. We apply the Lance-Williams formula for phase dissimilarity in a recursive manner to determine the distance between a solitary target dataset A and a combined set of multi-class target signal characteristics (B ∪ C):

$$d(A,(B\cup C))={\alpha }_{B}d(A,B)+{\alpha }_{C}d(A,C)+\beta d(B,C)+\gamma \left|d(A,B)-d(A,C)\right|$$

(17)

Equation (5.19) can be used for further derivation, yielding the distance between the merged dataset (A ∪ D) and the merged dataset (B ∪ C):

$$\begin{array}{l}d((A\cup D),(B\cup C))={\alpha }_{A,D}d(A,(B\cup C))+{\alpha }_{D,A}d(D,(B\cup C))\\ ={\alpha }_{A,D}({\alpha }_{B,C}d(A,B)+{\alpha }_{C,B}d(A,C))+{\alpha }_{D,A}({\alpha }_{B,C}d(D,B)+{\alpha }_{C,B}d(D,C))\end{array}$$

(18)

where α_A,D etc. is the adjustment factor, calculated through equation (5.20).

$${\alpha }_{B,C}=\frac{\left|N_B\right|}{\left|N_B\right|+\left|N_C\right|},{\alpha }_{C,B}=\frac{\left|N_C\right|}{\left|N_B\right|+\left|N_C\right|},\beta =0,\gamma =0$$

(19)

where N_B represents the number of rows in the feature matrix of class B signals and N_C represents the number of rows in the feature matrix of class C signals, which refers to the number of segments of the class C target signal during feature extraction.

The classification accuracy of a classifier is positively correlated with the spacing of the sample data used to train the classifier; the larger the spacing between the sample feature data of the target, the higher the average classification accuracy of the trained classifier; the smaller the distance, the lower the average classification accuracy. In view of the relationship between the training dataset and the performance of the classifiers, the multi-DBN voting classification method uses the spacing of the training sample data used by different classifiers to determine the voting weights of the classification results of the classifiers when using the voting method to integrate the results of different classifiers. When the training dataset spacing is larger, the trained binary classifiers have higher classification confidence and are assigned larger weights to their results; when the training dataset spacing is smaller, the corresponding classifiers are assigned smaller weights to their classification results.

Classifier weights are calculated from the sample feature data distance, which is used to map the distance between sample data to the voting weights domain of [0.7,1.3].This range was chosen due to its optimal performance in previous studies. The voting weights domain guarantees that the combined weights of the two classifier results are greater than the largest classifier result weight. The combined set spacing of the sample data for the training of the three classifiers in the weighted voting method is respectively Dist(1), Dist(2), Dist(3), we use the following formula to calculate the weights of each classifier result when voting:

$$P(i)=\frac{2}{\sqrt{2\pi }}{e}^{-2Nor{D}^2(i)}+0.5$$

(1)

where NorD(i) represents the normalized distance between the merged sets of sample data, P(i) is the voting weight of the result of the ith classifier. To ensure that the value of the classifier result voting weight P(i) is in the range of [0.7, 1.3], the absolute value of the distance between the normalized merged sets denoted as |NorD(i)| must provide a value within the range of [0.2, 0.8]. Since the value range of $\frac{Dist\left(i\right)-Max\left(Dist\right(i\left)\right)}{Max\left(Dist\right(i\left)\right)-Min\left(Dist\right(i\left)\right)}$ is [-1, 0], an adjustment coefficient, δ, is required to ensure that |NorD(i)| meets the condition. The acceptable range for δ is between 0.2 and 0.8, and we have chosen to set to 0.3.

4. Engineering implementation of the algorithm

Figure 6. Flow ot the Multi-DBN Vote Classification Algorithm.

Figure 6. Flow ot the Multi-DBN Vote Classification Algorithm.

4.1. Training of Neural Networks

Figure 7. Flow of "Upper Training" in Multi-DBN Vote Classification Algorithm.

Figure 7. Flow of "Upper Training" in Multi-DBN Vote Classification Algorithm.

Neural networks can be computationally demanding and time-consuming during training, particularly when used in lower-performance embedded processor units. To address this challenge, a common strategy is to train the network on the PC and then transplante it to the embedded processor unit for use.

The top-training phase of the multi-DBN weighted voting classification algorithm, which involves training and transplanting the DBN network, is illustrated in Figure 5.10. Here, the DBN network is initially trained utilizing sample feature data, and the parameters of the DBN classifiers are obtained following the completion of the training.

When training a DBN network, predetermined values should be set for the number of layers and number of neurons in each layer. The number of neurons in the input and output layers depends on the sample feature data and the number of target categories. The feature vectors extracted in this study are 16-dimensional, thus requiring 16 neurons in the input layer to correspond to each value. The output layer consists of four neurons corresponding to the four target categories (A, B, C, and D). For the four target categories, A, B, C, and D are assigned the values of (1 0 0 0), (0 1 0 0), (0 0 1 0), and (0 0 0 1) respectively. This allows for the output of 1 to correspond to a neuron in the output layer for each category. The network requires a pre-set number of intermediate layers and neurons in each layer. For this paper, we have set 2 intermediate layers with 100 and 60 neurons per layer, respectively.

For details on the training methods and specific processes of the DBN network, please refer to the literature[26]. The trained DBN network can be represented by a set of parameters, including the input layer network weight matrix iw, input layer neuron bias vector ib, intermediate layer network weight matrix hw, intermediate layer neuron bias vector hb, output layer network weight matrix ow, and the output layer bias vector ob.

4.2. Transplanting of DBN Classifier

After training the DBN, a set of network parameters are settled, which are also the parameters of the trained classifier. These network parameters are exported and deposited into the processor unit on the node of the WSN intelligent monitoring system to classify the data according to this set of parameters.

Figure 8. Calculation process using DBN classifier network parameters for classification.

Figure 8. Calculation process using DBN classifier network parameters for classification.

The computational flow for classifying the target data using the trained network parameters is illustrated in Figure 5.11. The feature vector utilized for classification is represented by the symbol w. The equation for determining the classification result, denoted as r, is displayed in Eq.(20).

$$r=sigm\{sigm\left[sigm(w·iw+ib)·hw+hb\right]·ow+ob\}$$

(20)

Where sigm(.) is the selected sigmoid activation function for training, the formula displayed below:

$$sigm(x)=\frac{1}{1-e^{-x}}$$

(21)

5. Performance of multi-DBN weighted voting algorithm

6. Conclusions

References

1. Sohraby, K.; Minoli, D.; Znati, T. Wireless sensor networks: technology, protocols, and applications, 1st ed.; John Wiley & Sons, INc.: Hoboken, New Jersey, 2007.
2. Mishra, D.P.; Dorale, S.S. An Application of Wireless Sensor Network in Intelligent Transportation System. 2013 6th International Conference on Emerging Trends in Engineering and Technology, Nagpur, India, 2013, pp.90-91.
3. Balid, W.; Tafish, H.; Refai, H.H. Development of Portable Wireless Sensor Network System for Real-time Traffic Surveillance. 2015 IEEE 18th International Conference on Intelligent Transportation System, Spain, 2015, pp.1630-1637.
4. Hsu, C.W.; Lin, C.J. A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 2002. 13, 415-425. [CrossRef]
5. Manikandan, J.; Venkataramani, B. Design of a Modified One-Against-All SVM Classifier, 2009 IEEE International Conference on Systems Man and Cybernetics, San Antonio, USA, 2009, 1869-1874. [CrossRef]
6. Fei, B.; Liu, J.B. Binary tree of SVM: A new fast multiclass training and classification algorithm. IEEE Transaction on Neural Networks 2006. 17, 696-704. [CrossRef]
7. Cheng, L.L.; Zhang, J.P.; Yang, J.; Ma, J. An Improved Hierarchical Multi-Class Support Vector Machine with Binary Tree Architecture. 2008 International Conference on Internet Computing in Science and Engineering, Proceedings, Harbin, China. 2008, 106-109.
8. Wang, Y ; Cheng, XL ; Li, X ; Li, BQ ; Yuan, XB. Powerset Fusion Network for Target Classification in Unattended Ground Sensors. IEEE SENSORS JOURNAL 2021. 21. 12. [CrossRef]
9. Xu, TH; Wang, N  ; Xu, X. Seismic Target Recognition Based on Parallel Recurrent Neural Network for Unattended Ground Sensor Systems. IEEE ACCESS 2019. 7,137823-137834. [CrossRef]
10. Belsare, K ; Singh, M; Gandam, A; Malik, PK; Agarwal, R; Gehlot, A. An integrated approach of IoT and WSN using wavelet transform and machine learning for the solid waste image classification in smart cities. TRANSACTIONS ON EMERGING TELECOMMUNICATIONS TECHNOLOGIES 2023.9. [CrossRef]
11. Kim, Y; Jeong, S; Kim, D. A GMM-Based Target Classification Scheme for a Node in Wireless Sensor Networks. IEICE TRANSACTIONS ON COMMUNICATIONS 2008.11.3544-3551. [CrossRef]
12. Zhao, Q ; Guo, F ; Zu, XS ; Chang, YC; Li, BQ ; Yuan, XB. An Acoustic Signal Enhancement Method Based on Independent Vector Analysis for Moving Target Classification in the Wild. SENSORS. 2017.12.17-10. [CrossRef]
13. Xu, T.A. New Sphere-Structured Multi-Class Classifier, Proceedings of the 2009 Pacific-Asia Conference on Circuits, Communications and Systems 2009,520-525,Chengdu, People’s Republic of China. [CrossRef]
14. Tomar, D.; Agarwal, S.A. Comparison on multi-class classification methods based on least squares twin support vector machine. Knowledge-based Systems 2015. 81, 131-147. [CrossRef]
15. Mohamed, AR ; Dahl, GE ; Hinton, G. Acoustic Modeling Using Deep Belief Networks. IEEE TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING. 2012.20-1. [CrossRef]
16. Zhang, H ; Pan, ZM; Zhang, WN. Acoustic-Seismic Mixed Feature Extraction Based on Wavelet Transform for Vehicle Classification in Wireless Sensor Networks.2018.18-6. [CrossRef]
17. Wang, GM; Qiao, JF ; Bi, J ; Jia, QS; Zhou, MC. An Adaptive Deep Belief Network With Sparse Restricted Boltzmann Machines. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS. 2020-31-10. 4217-4228. [CrossRef]
18. Sarikaya, R; Hinton, GE ; Deoras, A. Application of Deep Belief Networks for Natural Language Understanding. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING. 2014. 22-4. [CrossRef]
19. Ahmadian, S ; Khanteymoori, AR. Training back propagation neural networks using asexual reproduction optimization. 2015 7TH CONFERENCE ON INFORMATION AND KNOWLEDGE TECHNOLOGY (IKT). 2016.
20. Mourgias-Alexandris, G ; Tsakyridis, A; Passalis, N ; Tefas, A ; Vyrsokinos, K; Pleros, N. An all-optical neuron with sigmoid activation function. OPTICS EXPRESS. 27-7. 9620-9630. [CrossRef]
21. Ma, XS ; Wang, XJ. Average Contrastive Divergence for Training Restricted Boltzmann Machines. ENTROPY. 2016. 18-1. [CrossRef]
22. Roussel, C; Cocco, S ; Monasson, R. Barriers and dynamical paths in alternating Gibbs sampling of restricted Boltzmann machines. PHYSICAL REVIEW E. 104-3. [CrossRef]
23. Larochelle, H ; Mandel, M ; Pascanu, R ; Bengio, Y. Learning Algorithms for the Classification Restricted Boltzmann Machine. JOURNAL OF MACHINE LEARNING RESEARCH. 2012. 13.643-669.
24. Huang, CH ; Wang, JF. Multi-weighted Majority Voting Algorithm on Support Vector Machine and Its Application. TENCON IEEE Region 10 Conference Proceedings. Singapore. 2009.
25. Tang, Q; Xiao, JY ; Wu, KF. A novel evidence combination method based on stochastic approach for link- structure analysis algorithm and Lance- Williams distance. PEERJ COMPUTER SCIENCE. 2003. 9. [CrossRef]
26. Zhou, SS ; Chen, QC ; Wang, XL . Fuzzy deep belief networks for semi-supervised sentiment classification. NEUROCOMPUTING. 131. 312-322. [CrossRef]
27. Duarte, M.F.; Hu, Y.H. Vehicle classification in distributed sensor networks. J. Parallel Distrib. Comput. 2004, 64, 826–838. [CrossRef]
28. Asadi, R; Kareem, SA. Review of Feed Forward Neural Network classification preprocessing techniques. PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON MATHEMATICAL SCIENCES. 1602. 567-573. [CrossRef]
29. Zhang, ZH. Naive Bayes classification in R. ANNALS OF TRANSLATIONAL MEDICINE. 2016. 4-12. [CrossRef]
30. Chao, W ; Bo, L; Lei, W ; Pai, P . Improving boosting methods with a stable loss function handling outliers. INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS. 14-7. 2333-2352. [CrossRef]
31. Cao, JW ; Lin, ZP ; Huang, GB ; Liu, N. Voting based extreme learning machine. INFORMATION SCIENCES. 2012. 185-1. [CrossRef]