Robustness of Bayesian Random Forest in High-Dimensional Analysis with Missing Data

1. Introduction

Rolling bearings play a crucial role in the operation of rotating machinery, exerting a significant impact on equipment reliability and stability [1]. However, prolonged exposure to complex working environments renders rolling bearings susceptible to malfunction, which can adversely affect machine performance and pose safety hazards [2,3]. Hence, effective fault diagnosis techniques for rolling bearings are imperative [4].

Vibration signal analysis has long been pivotal in diagnosing rolling bearing faults due to the plethora of fault-related features inherent in such signals [5,6]. However, signals collected from rolling bearings in real-world industrial settings often suffer from interference, presenting significant challenges [7]. Over the years, considerable attention has been devoted to reducing noise interference and extracting meaningful fault features [8,9,10]. Various signal denoising techniques have been explored, including empirical mode decomposition (EMD) [11,12], variational mode decomposition (VMD) [13,14] wavelet analysis (WT) [15,16,17], and threshold denoising [18,19] Nevertheless, these methods often lack adequate adaptability, especially exhibiting suboptimal performance in the presence of strong noise backgrounds.

The emergence of deep learning techniques has opened promising avenues for fault diagnosis in rolling bearings, supplanting traditional manual feature extraction methods with direct feature extraction from raw data, thereby enhancing adaptability. Among these techniques, convolutional neural networks (CNNs) have garnered significant attention, leveraging convolutional operations to efficiently extract features and compress dimensions [10,20]. Numerous researchers [21,23] have proposed CNN-based models for diagnosing rolling bearing faults, significantly enhancing diagnostic accuracy.

However, increasing the depth of neural networks may lead to issues such as overfitting, error accumulation, and gradient vanishing, thereby diminishing model performance [24]. To address this challenge, deep residual networks (ResNets) [25] were introduced, leveraging identity shortcuts to alleviate these issues and reduce the difficulty of weight optimization. Nevertheless, in harsh industrial environments, noise in signals acquired by sensors limits the applicability of fault diagnosis based solely on ResNets, necessitating a dedicated module for signal denoising [26,28].

The emergence of deep learning has introduced a fresh perspective to traditional threshold denoising techniques by leveraging a network’s local attention mechanism to dynamically train thresholds and denoising functions. Zhao et al. proposed the deep residual shrinkage network (DRSN) [29] to augment the feature learning capability of noisy signals. By integrating a soft threshold function and automatically determining thresholds through training, DRSN offers a novel approach to signal denoising. However, manually setting denoising functions still poses subjective and experiential challenges, potentially rendering them unsuitable for different datasets [21].

In this research, the denoising module of existing methodologies is enhanced by eliminating the necessity to specify a predefined denoising function. Instead, the network is enabled to autonomously learn denoising functions during training, thereby effectively preserving signal features while eliminating noise. Expanding upon this improved denoising module, an adaptive denoising autoencoder (ADAE) tailored for signal denoising preprocessing is introduced. This approach not only significantly improves the signal-to-noise ratio but also reduces network depth and computational complexity.

2. Theoretical Foundations

2.1. Problem Statement

Vibration signals in rolling bearings, denoted as $s$, can be expressed as the sum of the original signal $u$ and noise signal $n$. In practical scenarios, these signals often contain noise from various sources, approximated using Additive White Gaussian Noise (AWGN). The objective is to remove noise $n$ from the observed vibration signal $s$ to recover the original fault pulse signal $u$.

For the training process, a training set $M={\left\{\right(s_i,u_i\left)\right\}}_{i=1}^N=S\times U$ is utilized, where $S$ and $U$ represent matrices of noise signals and clean signals, respectively. The network parameters are trained to minimize the loss error $L$ between the network input $s_i$ and output $\widehat{s_i}$.

2.2. Basic Components of CNN

Convolutional Neural Networks (CNNs) are comprised of various fundamental components that play essential roles in processing and extracting features from input data. These components include convolution (Conv), deconvolution (Deconv), nonlinear activation functions (ReLU), batch normalization (BN), and fully connected (FC) layers, among others [31,32].

2.3. Dropout: Addressing Overfitting in Deep Learning

Dropout [30] is a widely adopted technique in deep learning models, renowned for its efficacy in mitigating overfitting. By randomly dropping out neurons during training, Dropout encourages the network to learn more robust and diverse representations of the data. This prevents the model from relying too heavily on specific features or training samples, thereby improving its generalization performance on unseen data. Dropout is straightforward to implement and does not require significant modifications to the network architecture, making it a popular choice among researchers and practitioners in the field of deep learning. Its ability to enhance model performance while maintaining simplicity has contributed to its widespread adoption across various applications.

2.4. Theoretical Basis of Autoencoders

Autoencoders [33] represent a class of neural networks designed to replicate input data with precision, aiming to distill essential features while discarding redundant information. Comprising input, hidden, and output layers, autoencoders are structured to encode input data into a lower-dimensional latent space and subsequently decode it back to its original form [34]. Positioned between the input and output layers, the hidden layer in autoencoders acts as a bottleneck, deliberately restricting the network’s capacity to learn a compressed representation of the input data.

This component transforms the input data $x$ into a latent representation $h$ by means of nonlinear transformations, facilitated by parameters such as the activation function $\phi$, weight matrix $W$ and bias vector $b.$

$$h=f\left(x\right)=\phi (Wx+b),$$

(1)

Subsequently, the decoder network reconstructs the latent representation $h$ back into the original input $\tilde{x}.$ This process involves another set of nonlinear transformations using $\sigma$, $W^{\prime }$ and $b^{\prime }$ for the activation function, weight matrix, and bias vector of the decoder, respectively.

$$\tilde{x}=g\left(h\right)=\sigma (W^{\prime }h+b^{\prime }),$$

(2)

The performance of an autoencoder is evaluated based on its ability to reconstruct the input faithfully. The reconstruction error [35], denoted by $L(x,\tilde{x}$), measures the discrepancy between the original input $x$ and its reconstruction $\tilde{x}.$ The optimization process revolves around fine-tuning the network parameters, to minimize the reconstruction error observed across the training samples. This objective underscores the endeavor to glean insightful representations of the input data, capable of faithfully reconstructing the original inputs with precision and fidelity.

3. The Proposed Method

3.1. Shortcut Connection

Gradient-related challenges like vanishing and exploding gradients are common in deep neural network training, often causing convergence issues or slow training. To mitigate these problems, the Shortcut Connection [36] technique has been introduced, incorporating skip connections to alleviate gradient-related issues. Shortcut Connection allows the network to bypass one or more layers and directly add input from previous layers to subsequent layers’ outputs. This facilitates:

• Gradient Propagation: Providing a direct path for gradients to propagate back to earlier layers helps mitigate the vanishing gradient problem, thus facilitating smoother training.
• Information Flow: Faster information flow within the network is enabled, aiding in quicker and more efficient learning of effective feature representations.
• Feature Reuse: Accessing original inputs or feature maps from preceding layers directly prevents information loss, crucial for tasks like image super-resolution and segmentation, where preserving detailed information is vital.

In summary, the incorporation of Shortcut Connection enhances training efficiency, stability, and generalization ability, leading to improved performance across various deep learning tasks.

3.2. Adaptive Shrinkage Unit (ASU)

Traditional denoising techniques like filter design, wavelet thresholding, and sparse representation are effective in retaining useful information while removing noise from signals. However, these methods often demand substantial domain expertise. Deep learning approaches, such as DRSN [29], combine soft thresholding with deep learning to automatically learn channel thresholds. Nevertheless, manually defining threshold functions and their varying impact present challenges in adaptively selecting appropriate functions for specific problems.

In this research, we propose a method for denoising signals by training shrinkage coefficients through a local attention mechanism. These coefficients undergo training using convolutional and deconvolutional operations, with their values constrained within the [0,1] range via a Sigmoid function. Specifically, as depicted in Figure 1, for a noisy signal, our Adaptive Shrinkage Unit (ASU) trains a corresponding set of shrinkage coefficients. When multiplied with the noisy signal, these coefficients effectively remove interference components while retaining essential features related to faults.

Compared to traditional soft threshold denoising, ASU dynamically learns denoising coefficients during network training, enhancing the network’s generalization ability and overall performance. This adaptive approach reduces the reliance on manual threshold definitions and improves adaptability to different datasets and noise levels, thereby addressing key challenges in denoising tasks.

3.3. Architecture of the Proposed Method

Based on the encoder-decoder structure, a new signal denoising framework named Adaptive Denoising Autoencoder (ADAE) is constructed, as depicted in Figure 2. ADAE integrates the Adaptive Shrinkage Unit (ASU) twice into this framework, facilitating both data representation and reconstruction.

The encoder captures temporal signal features and compresses temporal dimension information using convolutional ASU to generate latent variables for learning signal fault features. Conversely, the decoder employs deconvolutional ASU to amplify compressed information and regenerate the original signal based on latent variables, thereby restoring the compressed dimensions. By minimizing the reconstruction error, background noise is effectively removed.

Strategically deploying the Dropout technique atop the first convolutional kernel introduces random disturbances to the input data, enhancing the network’s robustness and generalizability. These disruptions encourage the network to learn more diversified feature representations, akin to human adaptability to noise interference. Consequently, the model not only achieves high accuracy and responsiveness but also excels in handling signals rich in random or redundant information, significantly improving the recognition capability of faint fault feature signals.

In our denoising process, the presence of additive white Gaussian noise (AWGN) necessitates the assessment of the denoising algorithm’s effectiveness. We employ the signal-to-noise ratio (SNR) as a fundamental metric, defined as:

$$SNR=10\mathit{log}\frac{\left|\right|u|{|}_2^2}{\left|\right|s-u|{|}_2^2},$$

(3)

We aim to eliminate AWGN noise, mimicking real industrial scenarios. To gauge the algorithm’s performance comprehensively, we utilize three evaluation metrics [10]: SNR improvement, Root Mean Square Error (RMSE), and Percent Root Mean Square Difference (PRD), defined as:

$$SN{R}_{imp}=10\mathit{log}\frac{\left|\right|s-u|{|}_2^2}{\left|\right|\widehat{u}-u|{|}_2^2},$$

(4)

$$RMSE=\sqrt{\frac{1}{n}\left|\right|\widehat{u}-u|{|}_2^2},$$

(5)

$$PRD=\sqrt{\frac{\left|\right|\widehat{u}-u|{|}_2^2}{\left|\right|u|{|}_2^2}}\times 100,$$

(6)

Here $u$, $\widehat{u}$ and $s$ have the same meanings as before. A higher ${SNR}_{imp}$ and lower $RMSE$ and $PRD$ values indicate stronger denoising capabilities of the algorithm. The overall network structure is visually represented in Figure 3.

4. Experimental Validation

The denoising performance of ADAE is rigorously evaluated on the CWRU dataset across different noise levels to ascertain its effectiveness in real-world scenarios.

4.1. Experimental Setup and Data Description

The CWRU dataset is a commonly utilized third-party dataset that offers a reliable method for assessing the effectiveness of existing algorithms, as depicted in Figure 4. This dataset includes drive-end samples captured at a sampling frequency of 12 kHz, containing signals related to four different health conditions: normal, rolling element fault, inner race fault, and outer race fault. Each health condition consists of four different operating loads, and within each fault mode, three different fault sizes are examined. Therefore, there are a total of ten categories of bearing data, each corresponding to four operating loads.

To ensure data consistency and comparability, all collected data undergoes z-score standardization, mitigating the potential impact of sensor placement. Gaussian white noise (AWGN) is subsequently added to the data based on signal-to-noise ratio (SNR) for performance validation of ADAE and comparison with other mainstream algorithms. The dataset is split into training and testing samples at a ratio of 4:1.

4.2. Hyperparameter Optimization and Ablation Study

In this section, critical structures and parameters are optimized and tested using the CWRU dataset. All experiments are conducted under a noise level of SNR = -6 dB. We employ a sliding window approach to construct the sample set, with each sample containing 2048 data points, and no overlap between adjacent samples. For each fault category and each load, 50 samples are generated, resulting in a total of 2000 samples. This experimental design ensures comprehensive evaluation and comparison of model performance under different conditions.

4.2.1. Impact of Dropout Probability on Denoising Performance

This section explores the influence of dropout probability ($p$) on the denoising efficacy of the Adversarial Domain Adaptation Encoder (ADAE) model, conducted under Signal-to-Noise Ratio (SNR) conditions of -6dB. The experimental setup involved varying $p$ values from 0.1 to 0.7, with results recorded and analyzed as presented in Table 1.

Results indicate that $p$ values ranging from 0.1 to 0.3 correspond to superior denoising performance across all metrics. Beyond $p=0.3$, denoising performance experiences a notable decline. This trend suggests that smaller $p$ values lead to less signal disruption by dropout, thereby enhancing the model’s recovery capability and overall performance. Conversely, larger $p$ values induce significant signal disruption, thereby impeding effective information extraction and consequently diminishing denoising performance. To address potential noise level variations, a decision was made to randomize $p$ values within the range of [0.1,0.3]. This randomized approach enhances the model’s adaptability to handle noise signals across different SNR scenarios. The utilization of randomized $p$values contributes to enhanced denoising capabilities, effectively establishing an optimal range for $p$ and ensuring robust model performance under diverse noise conditions.

In summary, the analysis underscores the critical role of dropout probability in optimizing the denoising performance of the ADAE model, with randomized $p$ values proving instrumental in enhancing adaptability and efficacy across varying noise levels.

4.2.2. Impact of Initial Convolutional Kernel Width

The initial convolutional layer serves as a fundamental component in the model architecture, with the width of its kernel directly influencing the model’s ability to extract features from the input data. Drawing an analogy to human vision, the convolutional kernel can be likened to a window sliding over the data, akin to our visual field when observing objects. In this analogy, the size of the kernel mirrors the breadth of perception when scrutinizing objects; larger kernel sizes expand the perceptual scope, thereby capturing a more comprehensive array of information.

In our experimentation, we adopted kernel widths as powers of 2 to ensure uniformity and facilitate comparative analysis. We explored kernel widths ranging from 2 to 10 powers of 2, while maintaining the dropout probability ($p$) at 0.2. The denoising performance of the Adversarial Domain Adaptation Encoder (ADAE) model under these varying kernel widths is summarized in Table 2.

The optimal denoising effect of ADAE was observed with a kernel width of 128, indicating that wider kernels capture a broader spectrum of information from the input samples. Conversely, smaller kernel widths resulted in diminished denoising performance, underscoring the significance of wider kernels in extracting critical information effectively.

The analysis highlights the pivotal role of the initial convolutional kernel width in determining the denoising efficacy of the ADAE model, emphasizing the importance of selecting an optimal kernel size to facilitate robust feature extraction and enhance overall model performance.

4.2.3. Ablation Study of Shortcut Connections

In this research, we explore the impact of incorporating shortcut connections at different positions within the ADAE architecture. Three positions are considered: Position I (from input to output of the encoder), Position II (from input of the encoder to output of the decoder), and Position III (from input to output of decoder). We analyze the performance of eight combinations of these connections, as detailed in Table 3.

Notably, models with only a Position III shortcut demonstrate the best denoising performance, significantly outperforming other connection combinations. Any combination containing a Position III shortcut yields relatively good performance, while combinations with Position I and Position II shortcuts offer limited improvement. Therefore, the final optimized ADAE model structure incorporates only Position III shortcuts, as shown in Figure 5. The specific network parameters are listed in Table 4.

4.2.4. Impact of Leaky ReLU on Denoising Performance

Convolution and deconvolution operations are inherently linear processes. However, the relationship between signals and noise is complex, requiring nonlinear transformations for effective denoising. While Rectified Linear Unit (ReLU) activation functions are commonly used in computer vision tasks, they tend to disregard negative values during signal denoising. Negative values are pivotal in denoising tasks, especially as signal means are normalized to zero, rendering negative values significant. ReLU simply nullifies negative values, resulting in considerable information loss.

To address this limitation, Leaky ReLU activation functions are introduced, which scale negative values by a leakage factor ($\alpha$). In this section, we replace all ReLU activations with Leaky ReLU and explore the impact of different $\alpha$ values on the denoising performance of the Adversarial Domain Adaptation Encoder (ADAE).

Figure 6. Denoising performance of ADAE with different leakiness under SNR=-6dB.

Figure 6. Denoising performance of ADAE with different leakiness under SNR=-6dB.

As depicted in Figure, denoising performance initially increases with $\alpha$, reaching a peak before diminishing returns are observed. This trend suggests that increasing the leakage factor accentuates the importance of negative values, thereby enhancing denoising effectiveness. ADAE achieves maximal denoising efficacy when $\alpha$ is set to 0.3. Beyond this value, denoising capability starts to deteriorate, indicating a delicate balance between capturing negative values and nonlinear learning capabilities.

In summary, a judicious selection of the leakage factor ($\alpha$) is crucial to achieving optimal denoising performance in ADAE. The findings suggest that $\alpha =0.3$ offers the best balance between capturing negative values and nonlinear learning capabilities, thereby maximizing denoising effectiveness. This optimal configuration is adopted for subsequent experiments to ensure robust denoising performance.

4.3. Comparative Analysis

4.3.1. Known Noise Intensity

To underscore the superiority of the proposed method, we conducted a comparative evaluation with several state-of-the-art and traditional denoising techniques, including WT [17], EMD [12], JL-CNN [4], SEAEFD [37], and NL-FCNN [10]. The denoising performance of ADAE was assessed under varying SNR of -6 dB, -3 dB, and 0 dB, with the results summarized in Table 5.

ADAE demonstrates significantly superior denoising performance compared to other methods across all noise intensities, with ${\mathrm{S}\mathrm{N}\mathrm{R}}_{\mathrm{i}\mathrm{m}\mathrm{f}}$ nearly doubling or more in comparison. Even when noise intensity is low, ADAE achieves an ${\mathrm{S}\mathrm{N}\mathrm{R}}_{\mathrm{i}\mathrm{m}\mathrm{f}}$ greater than 15, showcasing its robustness and ability to learn noise characteristics while preserving original fault information.

To provide a more intuitive understanding of ADAE’s denoising efficacy, we visualize both temporal and frequency waveforms of signals under the three noise intensities. Figure depicts denoising results of ADAE under SNR= -6dB, when noise intensity is known.

These visualizations underscore ADAE’s remarkable denoising capabilities, effectively removing noise while preserving fault information across various fault modes and noise intensities. The distinct denoising effects observed in the frequency spectra demonstrate ADAE’s ability to remove irrelevant frequency components, further validating its effectiveness even under challenging noise conditions.

Figure 7. Denoising results of ADAE with SNR=-6dB when noise intensity is known.

Figure 7. Denoising results of ADAE with SNR=-6dB when noise intensity is known.

4.3.2. Unknown Noise Intensity

To further evaluate the denoising performance of ADAE in real-world scenarios where noise levels are unpredictable, we conducted experiments with randomly varied SNR ranging from 0 dB to -6 dB. The results, compared with other denoising methods, are presented in Table 6.

In all tested scenarios, ADAE consistently showcases superior denoising capabilities. When trained on a dataset with randomly varied SNR, ADAE exhibits even better performance compared to the previous scenario with constant noise intensity. Particularly notable is the significant enhancement in the ${\mathrm{S}\mathrm{N}\mathrm{R}}_{\mathrm{i}\mathrm{m}\mathrm{f}}$ metric under 0 dB and -3 dB conditions, where ADAE achieves values surpassing 22, representing an improvement of 7 compared to the previous results. This substantial improvement underscores ADAE’s adaptability and effectiveness in handling varying noise levels.

Furthermore, relative to other denoising methods, ADAE consistently outperforms them across all tested scenarios, highlighting its unparalleled advantages in signal denoising. This reaffirms ADAE’s robustness and generality, making it a highly reliable solution for denoising tasks in diverse environments.

To provide further insights into ADAE’s denoising efficacy, temporal and frequency waveforms of test samples under an SNR of -6 dB are presented in Figure 8. These samples encompass signals from ten bearing health states under 1hp load. As observed in the figures, ADAE successfully preserves temporal fault features while effectively removing irrelevant frequency components. This capability underscores ADAE’s role as an efficient preprocessing method for rolling bearing fault diagnosis, particularly in environments characterized by strong noise backgrounds.

5. Conclusions

In this paper, the Adaptive Denoising Autoencoder (ADAE) was introduced as a novel approach designed for reducing noise in vibration signals. ADAE integrates an adaptive shrinkage unit local attention mechanism within its encoder and decoder modules to selectively attenuate noise while preserving fault features by eliminating interfering information. Additionally, ADAE incorporates a dropout structure to enhance its adaptability dynamically. The research also explored the impact of Leaky-ReLU size and convolutional kernel size on ADAE’s denoising performance, along with investigating the effectiveness of shortcut connections.

Experimental findings showcase ADAE’s superiority over other noise reduction methods across scenarios with both known and unknown noise. ADAE effectively removes noise components in both the time and frequency domains, demonstrating exceptional performance, especially in low signal-to-noise ratio scenarios. This robust performance positions ADAE as a valuable preprocessing tool for fault diagnosis applications, particularly in the challenging context of strongly noisy rolling bearings.

In summary, the proposed ADAE method offers a promising solution for noise reduction in vibration signals, exhibiting superior performance across diverse noise scenarios and showcasing potential for enhancing fault diagnosis in the presence of significant noise.