Explainable Machine Learning for Bubble Leakage Detection at Tube Array Surfaces in Pool

1. Introduction

The detection of bubbles within complex fluidic systems is a critical challenge in the maintenance and integrity management of industrial plants. The generation of bubbles is often a primary indicator of an anomalous event, such as a piping rupture, necessitating robust detection capabilities within intricate piping structures.

This challenge is particularly acute in fast-reactor steam generators, which facilitate heat transfer between water flowing through heat-transfer tubes and liquid sodium on the outside. A heat transfer tube rupture can trigger a rapid Sodium-Water Reaction (SWR), requiring immediate detection to ensure plant safety. While various methods, such as acoustic emission and electrochemical hydrogen sensors, have been proposed, detecting bubbles, especially at low flow rates and small diameters, in a high-noise environment and in the presence of multiple scattering, remains difficult [1,2,3]. Passive acoustic methods, though suited for continuous online monitoring, are prone to false positives and identification challenges due to environmental fluctuations [4,5,6,7]. Similarly, while electrochemical hydrogen sensors have demonstrated ppb-level sensitivity, they face the significant challenge of discriminating between reaction-generated hydrogen (bubbles) and background hydrogen (e.g., dissolved H or NaH) [8,9]. These limitations suggest the need to combine multiple techniques for bubble detection, rather than relying on a single method [5]. The ultrasonic method is a promising candidate, as it can acquire high-resolution spatial and geometric information [2,10]. Direct imaging in liquid metals, as a visual alternative, is also theoretically possible [11]. However, the current application of ultrasonic testing (UT) in this field is predominantly focused on internal flaw inspection within structures [12,13]. Consequently, research on the direct imaging of bubbles in sodium remains limited.

Although core boiling is suppressed in a Pressurized Water Reactor (PWR) during regular operation, bubble generation can occur during abnormal transients and accidents. This phenomenon is a significant concern, as it can compromise core safety by reducing cooling capacity and altering reactivity. Therefore, early detection is critical, particularly for characterizing changes in the boiling state before the onset of Critical Heat Flux (CHF). At present, neutron noise analysis is the primary method utilized for anomaly detection. Neutron detectors typically reflect the global behavior of the core, making it challenging to precisely localize boiling that occurs within a specific fuel assembly [14].

An underwater LNG leak from a piping rupture in a regasification unit at an LNG terminal is highly hazardous, as it triggers explosive boiling known as Rapid Phase Transition (RPT). For complex process equipment such as the Open Rack Vaporizer (ORV), which contains numerous heat transfer tubes, identifying the specific tube where the anomaly is occurring is crucial. However, the piping array structure, with its densely packed heat transfer tubes, obstructs and reflects acoustic propagation, making precise localization difficult [15]. Furthermore, sensor directional properties can also impede accurate location identification [16].

UT can be leveraged across all the aforementioned bubbles detection scenarios. It offers three primary advantages. First, it can separate and manipulate frequency ranges. Background noise is typically concentrated in the low-frequency spectrum. It can potentially be filtered out because the ultrasonic method operates at high frequencies [17]. Furthermore, applying an incident wave tuned to the bubble generation resonant frequency amplifies the bubble signal, significantly enhancing detection [18]. This might enable the extraction of clear signals even in high-noise environments. The second advantage is its high spatial resolution [19]. The third is its rapid response time, which enables instantaneous anomaly detection [18,20].

UT can often make anomaly detection difficult in complex structures. Signal attenuation during propagation through the media and reflections from multiple interfaces between different media can result in only weak signals being received from the target object [21]. Furthermore, distinguishing whether a received signal originates from an anomaly or is simply a valid reflection from the structure itself can be problematic, especially when their characteristics are similar [22]. An additional challenge arises specifically in detecting bubbles within a fluid; unlike the sharp, distinct echoes produced by a single crack, bubble signals often manifest as a diffuse collection of small reflection sources, which can be difficult to discern [23]. To address these limitations, combining the UT with Machine Learning (ML) offers a promising approach. By learning subtle differences in signal shape and patterns associated with anomalies, as well as characterizing the complex patterns inherent in normal data, ML models may enable the detection of anomalies that are difficult for human operators to identify through visual inspection alone [24,25]. While several effective methods for anomaly detection exist, this study proposes a complementary approach to enhance detection accuracy further. Our method combines UT and ML, aiming to overcome the challenge of precisely localizing bubble generation points within complex structures, where detection is hindered by signal complexity and attenuation. This approach, demonstrated using intentionally generated bubbles in a tube array submerged in water to simulate events such as gas production, is designed for applications in systems such as fast reactor steam generators, PWRs, and LNG regasification units.

2. Materials and Methods

2.1. Ultrasonic Testing

The proposed methodology, combining UT and ML, is intended for broad application in detecting bubbles within a complex tube array. The ability to precisely locate such anomalies is critical across various industrial applications, including PWR cores, LNG regasification units, and the steam generators of fast reactors, where internal tube arrays often hinder UT detection by attenuation. Among these applications, the fast reactor steam generator presents a special case because it is filled with high-temperature liquid sodium. This unique acoustic medium necessitates careful consideration regarding the validity of using water-based experiments to simulate its behavior, forming a key methodological foundation for this study.

The vessel and heat transfer tubes in the Monju, prototype reactor of fast reactor, are both made of 2.25Cr-1Mo steel; the medium is liquid sodium, and the bubbles generated by the SWR are hydrogen in the Monju reactor’s steam generator [26,27]. The amplitude of an ultrasonic wave reflected from an interface is determined by the difference in acoustic impedance between the two media. Acoustic impedance can be calculated using the following formula [28]:

Z =ρ x υ

(1)

Under the current experimental conditions, the acoustic impedances of stainless steel, copper, water, and air were calculated according to equation (1) using their respective values for density and sound velocity [28]. For the vessel and heat transfer tubes under actual plant conditions, the acoustic impedance was calculated using the following equation, which accounts for the temperature-dependent rate of change in steel’s physical properties relative to a 25 °C baseline.

ρ(t) =ρ₂₅ x {1 – 0.0001 x (t - 25)}

(2)

υ(t) =υ₂₅ x {1 – 0.0001 x (t - 25)}

(3)

The density of liquid sodium under actual plant conditions is calculated from its density at the melting point and a corresponding coefficient of change, as shown in the equation below:

ρ(t) =ρ_m x 0.23 x (t -t_m)

(4)

The sound velocity is calculated using linear interpolation, and the acoustic impedance is determined with equation (1). For hydrogen under actual plant conditions, the density was calculated using the ideal gas law, and the sound velocity was determined from the temperature-dependent equation for sound velocity [28]. The calculation was performed using the following parameters: a pressure of 12 kg/cm²G in the secondary system [26], a molar mass of 2.016 g/mol for hydrogen, a gas constant of 8.314 J/(mol·K), and a reference sound velocity for hydrogen of 1,270 m/s [28]. The resulting values are presented in Table 1. To determine the maximum and minimum temperatures in the region containing liquid sodium within the Monju steam generator, calculations were performed for two conditions: the sodium inlet temperature (469 °C) and the outlet temperature (325 °C) [26].

Table 1 shows that the acoustic impedances of the respective structural materials are similar. Consequently, their acoustic reflectivity is also expected to be nearly identical. This suggests that the data obtained under our experimental conditions can effectively simulate data from an actual plant, thereby supporting the validity of our approach. By contrast, the current experiment simulates this configuration using a Type-304 stainless steel vessel, copper piping to simulate the heat transfer tubes, and water at room temperature as the solvent.

Figure 1 illustrates the lateral view and overhead view of the experimental setup. To simulate the attenuation of ultrasonic waves caused by the numerous structures in actual plant, seven copper pipes were arranged in water. The piping has an outer diameter of 10 mm and an inner diameter of 9 mm. They were fixed using a stage fabricated with a 3D printer (Bambu Lab A1) and PolyTerra PLA filament. This stage was designed with seven 10 mm holes spaced 30 mm apart to hold the copper piping in place. This experimental setup, which effectively replicates the ultrasonic attenuation encountered in tube array, is applicable to bubble detection in various systems like PWRs and LNG plants.

For the normal pattern, representing conditions without a heat transfer tube rupture, data were acquired from the apparatus configured as shown in Figure 1 using an ultrasonic flaw detector (EPOCH 1000i). To create the anomalous pattern, a tube rupture was simulated by drilling a 3.7 mm hole in one of the copper pipes. Bubbles were generated from within the copper piping’s interior using a bubble generator (BL12PP-12-SC4), connected to a self-priming pump (HP-100) and installed at the top of the cooper piping. Figure 2 illustrates four patterns of bubble generation from the copper piping. To simulate the various modes of tube rupture that could occur in an actual plant, these four patterns were tested by varying the location of the hole on a single pipe. This process was repeated for each of the seven pipes, yielding 28 distinct anomalous patterns in total.

2.2. Machine Learning

In this study, ML was performed using ultrasonic image data acquired using the S-scan technique of the phased-array method. The acquired S-scan images contain strong artifacts inherent to the imaging equipment, which appear at the boundary between the signal and background regions. Furthermore, the background itself may contain brightness gradients that are unrelated to the actual ultrasonic signals from bubbles. These irrelevant features pose a risk of overfitting for the ML model. To mitigate these issues, a morphological operation was applied to the images. As a specific morphological operation step, the color image is first converted to grayscale. Subsequently, two distinct masks are generated based on luminance: a signal mask for pixels with luminance greater than 10 and a background mask for those with luminance less than 10. To eliminate any remaining black pixels within the signal mask, the entire signal mask was converted to solid white. Finally, the original experimental image is modified by dilating the background mask by one pixel into the signal region and setting this entire expanded area to black [0, 0, 0].

2.1.1. Transfer Learning for Convolutional Neural Network

In this study, transfer learning is implemented by using a pre-trained EfficientNet-b0 as the backbone for our Convolutional Neural Network (CNN). Since this approach leverages an existing network architecture rather than building one from scratch, it significantly reduces both training time and computational cost, enabling high accuracy [29]. Among various models, EfficientNet-b0 is particularly well-suited for this, as it achieves high accuracy despite having a relatively small number of parameters [30]. To adapt the model to 29 classifications, the Fully Connected (FC) and classification layers of the EfficientNet-b0 model were modified to match the number of labels. Input images are resized to 224x224x3 to meet the model’s input requirements, and the output is the predicted probability for each class. During each training iteration, cross-entropy is used as the objective function, and optimization is performed using the Adam optimizer. The image data for each label is partitioned into 80% for training, 10% for validation, and 10% for testing. To improve learning efficiency, the learning rates for the modified FC and classification layers are set to 10 times those of the other layers. The CNN’s key hyperparameters were determined using 50 Bayesian optimization trials. The search space included the learning rate (from 1x10⁻⁶ to 1x10⁻⁴), the number of epochs (an integer from 5 to 40), and the mini-batch size (8, 16, or 32) [29]. The optimal values were selected using classification error rate as the objective function in the optimization process.

2.1.2. Autoencoder

In a real plant, the number of possible anomalous data patterns is effectively infinite, making it unfeasible to collect a comprehensive dataset of all failure modes in an experiment. Therefore, it is crucial to develop a model that can generalize to detect previously unseen anomalies. This is achieved by training the model exclusively on normal data and verifying its ability to identify anomalous patterns. For this reason, an unsupervised learning approach is employed. The autoencoder was adopted for its capacity to perform effective dimensionality reduction. Its neural network architecture captures not only linear but also complex correlations within the data, compressing them into a low-dimensional latent space [31,32].

Figure 3 shows the architecture of the autoencoder used in this study. The number of convolutional blocks is set to five, which is a relatively shallow configuration. This design choice was made to encourage the model to capture global spatial features rather than extracting local features, as the characteristics of a normal pattern can vary due to factors such as internal convection conditions [33]. Moreover, prior studies have demonstrated that applying k-means clustering to the latent space generated by the encoder is an effective method for unsupervised classification [34,35]. Accordingly, this established methodology was adopted in this study. The objective function for the autoencoder is to minimize the Root-Mean-Square Error (RMSE) on the validation data. To achieve this objective, hyperparameter optimization was performed using 50 Bayesian optimization trials with the Adam optimizer. The optimization focused on key parameters known to influence performance significantly: the dimensionality of the latent space, the number of nodes in the FC layer following the convolutional blocks, the learning rate, the mini-batch size, and the number of epochs [36,37,38]. The threshold for anomaly detection was defined based on the reconstruction error of the normal data. Specifically, it was set at 1.2 times the maximum RMSE value observed on the regular validation set. For the autoencoder, only normal data was utilized for training (80%) and validation (10%). Its performance was subsequently evaluated on a test set comprising the remaining 10% of normal data and the entire anomalous data set. In contrast, for k-means classification, both standard and anomalous datasets were partitioned into training (80%), validation (10%), and testing (10%) sets.

3. Results and Discussion

3.1. Phased-Array Ultrasonic Testing Image Results

The dataset used in this study consisted of 2,800 images representing normal patterns and 200 images for each of the 28 distinct anomalous patterns. A representative example of an acquired phased array ultrasonic image is shown in Figure 4.

Under all tested conditions, some strong signals are observed near the first copper piping proximal to the probe. However, due to attenuation of the ultrasonic wave, this signal diminishes significantly in the subsequent copper piping, becoming almost indiscernible to human eyes near the seventh piping. In the “No Hole” case, faint signals are visible to the right of the center and in the left area of the image. These are hypothesized to be caused by side lobes. Several observations support this hypothesis. First, these signals are eliminated when a 100 mm wide stainless steel plate is placed between the copper piping. Given its width, this plate not only obstructs direct signal transmission between the piping but also attenuates ultrasonic waves propagating through the surrounding water. This suggests that the side lobe path is not a direct reflection between adjacent piping; instead, the ultrasonic waves appear to travel through the wider surrounding area, curve along the way, and then reflect off a piece of piping before returning to the probe. Second, the position of the signals does not change when the probe’s height is adjusted, making it unlikely that they are reflections from the bottom or wall surfaces. These findings collectively suggest that the observed faint signals are attributable to side lobes. In the “West” image, the bubble signal is readily identifiable in the lower left section. Similarly, in the “East” image, the bubble signal is clearly visible in the lower right. For the “North” condition, bubbles are emitted from the seventh copper piping and flow toward the top of the image. Although the emission point from the hole is not directly visible due to the attenuation of the ultrasonic wave, it is evident that the bubbles collide with the sixth copper piping and disperse across the upper region of the image. In the “South” case, a flow of bubbles is present toward the lower part of the image from the seventh copper piping. However, the ultrasonic wave is significantly attenuated, making direct signal identification difficult. These results indicate that there are conditions in which it is difficult for human eyes to distinguish between normal and anomalous data. Therefore, it is necessary to apply ML for effective anomaly detection.

Figure 4. Phased-Array Images for Underwater Visualization. The “No Hole” label denotes the baseline condition, in which only the copper piping is present. The cardinal directions indicate the emission direction of the bubbles, all of which originate from the seventh copper pipe.

Figure 4. Phased-Array Images for Underwater Visualization. The “No Hole” label denotes the baseline condition, in which only the copper piping is present. The cardinal directions indicate the emission direction of the bubbles, all of which originate from the seventh copper pipe.

3.2. Bubble Detection Using a CNN with EfficientNet-b0

3.2.1. CNN Training Results

Based on Bayesian optimization, the model was trained with the following hyperparameters: a mini-batch size of 16, an epoch of 31, and a learning rate of 8.9 × 10^-5. The learning curves in Figure 5 confirm that the CNN model was effectively trained. This is evidenced by the increase in accuracy and decrease in loss as the number of iterations increases. The model achieved a test accuracy of 100%, indicating a perfect classification with no errors. This CNN model not only detects the presence or absence of bubbles but also identifies the specific copper piping from which they originated and the direction from which they are emitted. Such precise information would enable a more rapid and effective response to operational issues in an actual plant.

3.2.2. CNN Model Explainability

Significant progress has been made in detecting sodium-water reactions in fast reactor steam generators using ML. For instance, Mikami et al. demonstrated that deep learning can classify normal piping acoustics and bubble-jet noise under simulated anomalous conditions with an extremely high accuracy of 99.76% [1]. Furthermore, Marklund et al. applied Hidden Markov Models (HMMs) to demonstrate that leak signals can be detected even in low signal-to-noise ratio (SNR) environments using actual plant data [6]. The CNN developed in this study also demonstrated high detection performance, consistent with prior work. However, these high-performance models share a common challenge —across the nuclear field, AI, and other sectors —namely, their internal decision-making processes lack transparency, functioning as “black boxes” [39,40]. This lack of transparency in the prediction process is a significant barrier to operators’ trust in and use of AI outputs in practical operations, especially in the nuclear field, where safety is paramount. This lack of “Explainability” is recognized as a cross-disciplinary challenge in the application of ML to critical societal infrastructure. Therefore, this study aims not only to improve detection performance but also to enhance system reliability fundamentally. Specifically, to visualize the basis for the CNN’s judgments, the Gradient-weighted Class Activation Mapping (Grad-CAM) method is applied to the CNN model. Furthermore, using t-distributed Stochastic Neighbor Embedding (t-SNE), the model’s classification features were visualized. This visualization will verify whether the model successfully captures distinct acoustic features corresponding to each anomalous pattern [41,42]. In this study, the Grad-CAM method was applied to data for which the predicted label matched the actual label. As shown in the following equation, the importance weights for each feature map are calculated by first differentiating the raw output score for the target class $y^c$ (before the softmax function) with respect to the feature maps of the final convolutional layer ${\mathit{A}}_{\mathit{i}\mathit{j}}^{\mathit{k}}$, and applying global average pooling to these gradients [40].

$${\alpha }_k^c={\displaystyle \frac{1}{Z}}\sum _i\sum _j{\displaystyle \frac{\partial y^c}{\partial {\mathit{A}}_{\mathit{i}\mathit{j}}^{\mathit{k}}}}$$

(5)

A heatmap $L_{Grad-CAM}^c$ is generated by computing a weighted linear combination with feature maps ${\mathit{A}}_{\mathit{i}\mathit{j}}^{\mathit{k}}$ using these calculated weights ${\alpha }_k^c$.

$$L_{Grad-CAM}^c=ReLU\sum _k{\alpha }_k^c{\mathit{A}}^{\mathit{k}}$$

(5)

This process is performed for each data point. To visualize the areas of importance for each class, the resulting heatmaps are averaged pixel-wise for each respective label.

Figure 6 shows the average Grad-CAM results for the one normal pattern and four anomalous patterns in the seventh copper piping. In the “No Hole” condition, the central region of the sector-shaped ultrasonic image is identified as an important area. This observation suggests two hypotheses. The first is that the model is monitoring the region where bubbles would typically appear, thereby confirming their absence. The second hypothesis is that the model is tracking for the presence of a side lobe. If bubbles were to exist in an area superior to the side lobe’s propagation path, they would cause attenuation of the ultrasonic signal and eliminate the side lobe. Consequently, the model may be using the presence or attenuation of these side lobes as an indirect feature. The “West” condition accurately localizes the bubble region in the lower-left of the ultrasonic image, and the “East” condition correctly identifies the bubbles in the lower-right. In the “7-North” condition, excessive ultrasonic attenuation prevents direct observation of the bubble emission point; however, the model clearly captures the faint signals from bubbles that have collided with the sixth copper piping and dispersed into the upper region of the image. Similarly, for the “South” condition, the bubbles emitted from the seventh copper piping are not captured directly. Instead, it appears to identify the fluid flow toward the lower part of the image caused by the bubble emission, which explains the localization of the critical region in the upper area. These results demonstrate that the CNN model successfully determines the presence and location of bubble generation. It accomplishes this by using a diverse set of indicators, including direct bubble signals, signals from migrated bubbles, and even subtle signals associated with internal fluid flow.

Figure 6. Visualization of the basis for classification using the Grad-CAM method. The results are averaged pixel-wise for individual heatmaps, calculated only from test data where the predicted label matched the actual label. The heatmap pixels align with those of the phased-array image, and the red regions correspond to the area most important to the CNN classification decision.

Figure 6. Visualization of the basis for classification using the Grad-CAM method. The results are averaged pixel-wise for individual heatmaps, calculated only from test data where the predicted label matched the actual label. The heatmap pixels align with those of the phased-array image, and the red regions correspond to the area most important to the CNN classification decision.

t-SNE is used to visualize whether the model successfully extracted distinct features for each class during training [41]. t-SNE method computes the similarity between images in a high-dimensional space and attempts to find a low-dimensional space embedding that preserves these similarities. Specifically, to evaluate similarities in the high-dimensional space, a degree of similarity is defined as follows [41].

$$p_{j|i}={\displaystyle \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(-\frac{{‖x_i-x_j‖}^2}{{2\sigma }_i^2}\right)}{{\sum }_{k\ne i}\mathrm{e}\mathrm{x}\mathrm{p}\left(-\frac{{‖x_i-x_k‖}^2}{{2\sigma }_i^2}\right)}}$$

(5)

The bandwidth of the Gaussian kernel is determined by finding a value that makes the perplexity of the distribution equal to a user-specified value. This process creates a similarity matrix of high-dimensional space. Subsequently, to reproduce these similarities for visualization, the data features for each image are embedded at random positions in a low-dimensional space. The similarity between these points is then evaluated using a Student’s t-distribution, as shown in the equation below [41].

$$q_{ij}={\displaystyle \frac{{\left(1+{‖y_i-y_j‖}^2\right)}^{-1}}{{\sum }_{k\ne i}{\left(1+{‖y_k-y_l‖}^2\right)}^{-1}}}$$

(6)

The final t-SNE embedding is found by minimizing the Kullback-Leibler divergence between these two distributions using gradient descent. While this exact method is computationally intensive, for larger datasets the Barnes-Hut algorithm is effective, which uses a tree-based structure to accelerate computation [42].

A comparison of the visualizations in Figure 7 reveals a stark contrast. In the raw experimental data, the feature-space distinctions are ambiguous, with the normal and anomalous patterns, as well as the differences among the various anomalous classes, essentially indistinguishable. Conversely, it is evident that during feature extraction, the CNN successfully isolates and extracts salient class-specific features. This results in distinct, well-separated clusters in the t-SNE plot. This result confirms not only that the model’s classification basis is effective but also that it can identify and extract distinct, meaningful features for each class.

3.3. Bubble Detection Using Autoencoder

3.3.1. Detection of Unseen Anomalies

While the CNN, as mentioned above, can classify not only the presence of bubbles but also their specific emission locations and directions, it would struggle to identify anomalous conditions that were not simulated in this experiment. Therefore, to develop a system that can adapt to unknown anomalous patterns, an autoencoder is employed. This model is trained exclusively on normal data, aiming to classify any deviation from this learned normality as an anomaly.

After 50 Bayesian optimization trials, with the objective function set to minimize reconstruction error, the optimal hyperparameters were determined: a Latent dimension of 15, 59 epochs, a batch size of 8, a learning rate of 8.8 × 10^-4, and 131 units in the FC. The reconstruction error threshold for distinguishing between normal and anomalous data was calculated to be 3.0 × 10^-4, using the three-sigma method on the normal validation set, as shown in Figure 8. The final anomaly detection accuracy was 99.1%. The model’s successful detection of 28 unseen anomalous patterns suggests its capacity for generalization to anomalous conditions not reproduced in the experiment. Furthermore, since the image pixel values were normalized to the [0, 1] range, the validation RMSE of 0.01191 indicates that the model reconstructed the normal validation data with an error of approximately 1.2%. The overall test RMSE, which included both normal and anomalous data, was 0.02454. This higher value is attributable to significantly larger reconstruction errors for the anomalous data, thereby validating the model’s ability to differentiate between the two classes effectively.

3.3.2. Autoencoder Explainability and Feature Extraction

The Grad-CAM method can be applied by calculating gradients with respect to a specific class score [40]. An autoencoder, however, does not output class-specific scores; instead, it outputs the reconstruction error, a global metric rather than a class-specific one. Therefore, to visualize the model’s decision-making basis, the Local Interpretable Model-agnostic Explanations (LIME) method is employed in this study. LIME is a model-agnostic technique that can be applied regardless of the model’s internal architecture, as it fits a linear approximation to explain the model’s predictions [39]. When determining the linear approximation, fidelity loss is calculated as shown in the equation below. This metric quantifies the extent to which the simple linear model fails to replicate the original, complex autoencoder.

$$\mathcal{L}(f, g, {\pi }_x)=\sum _{z\in \mathit{Z}}{\pi }_x\left(z\right) {(f\left(z\right)-g\left(z^{\prime }\right))}^2$$

(7)

The LIME method is sensitive to several hyperparameters, necessitating careful selection [43]. In this study, the optimal values were determined for the kernel width (0.25, 0.50, or 0.75), segmentation method, and its associated parameters. When using Simple Linear Iterative Clustering (SLIC), the number of superpixels was tested at 25, 50, and 100. When using the Watershed Algorithm, the gradient minima suppression level was attempted at 0.1, 0.08, and 0.05. The number of perturbation samples generated was held at 1000. To determine the optimal parameters, an exhaustive search was performed over the defined hyperparameter range. This process was applied to correctly classified images for five labels to output an averaged LIME importance map ultimately. The evaluation flow proceeded in three stages. First, a stability check was used as a cut-off. LIME was executed seven times on the same image, and the average of the superpixels importance standard deviations was calculated [43,44]. If this average exceeded 0.05, the set was discarded. Second, the remaining sets were ranked by fidelity, which was evaluated by calculating the infidelity (the discrepancy between the original model’s predictions and the surrogate model’s predictions by LIME on the perturbation samples) [43,44]. Finally, the top-ranked, high-fidelity combinations were qualitatively reviewed by a human eye to select the set that provided the most plausible and interpretable explanations. Following this evaluation, the optimal hyperparameter was determined to be: SLIC as the segmentation method, a kernel width of 0.75, and a superpixel count of 50.

Figure 9 reveals that the region identified as necessary for anomaly detection is consistently a narrow area at the top of the image across all classes, including the normal pattern and the anomalous ones. This is a critical finding, as each label corresponds to a unique experimental condition with a different anomaly location and direction. Consequently, the crucial regions were expected to differ across classes, but they did not. This discrepancy suggests that the autoencoder may be capturing a subtle fluid flow near the probe rather than identifying the bubbles.

3.3.3. Identifying Anomaly Location and Direction Using an Autoencoder and k-means

While an autoencoder is effective for anomaly detection, it struggles with multi-class classification. However, by applying k-means to the latent space generated by the encoder, which captures the most salient features of the data, accurate classification can be achieved [34,45]. The k-means assigns data to clusters and aims to minimize the sum of squared distances between each data point and its assigned cluster centroid, as defined by the objective function below:

$$I= \sum _{l=1}^K\sum _{\mathit{h}\in C_l}{‖\mathit{h}- {\mathit{\mu }}_{\mathit{l}}‖}^2$$

(8)

In this study, the k-means was trained on 80% of the normal data and 80% of the anomalous data. Its performance was confirmed using both test sets of 10%.

Table 2 shows the performance of k-means applied to the features extracted from the encoder. The overall test accuracy was 46.3%, and the macro F1-score was 0.164, indicating that this model struggled with multi-class classification. In contrast, the normal (“No Hole”) class achieved high precision and recall, resulting in a high F1-score. However, for specific anomalous patterns, such as “1-East,” zero correct predictions were made. These results suggest that while the model can effectively detect the presence or absence of bubbles, it struggles to identify the specific mechanism of bubble generation.

3.4. Anomaly Detection and Multi-Class Classification by Autoencoder-based Feature Extraction

3.4.1. Autoencoder Limitations in Rationale and Classification and Proposed Method

Figure 10 shows examples of input images to the autoencoder and their corresponding reconstructed output images. As the model was trained to reconstruct normal data, the reconstructed normal images are nearly identical to the experimental ones by visual inspection. However, focusing on the anomalous data reveals that the signals corresponding to bubbles have been removed or suppressed during reconstruction. This implies that the latent features —the encoder’s output— also lack the critical features associated with bubble signals. This would explain the previously observed issues: the poor performance of k-means and the inappropriate decision rationale identified by LIME. This failure is likely because the autoencoder, which optimizes for whole reconstruction error, failed to adequately learn these small anomalous features, as the bubble region is extremely small relative to the entire image. Prior research indicates that clustering using latent features extracted by an autoencoder can yield superior performance compared to methods that rely solely on reconstruction error [46,47,48,49,50,51].

Therefore, the objective of this study is to enhance the classification accuracy of anomaly location and direction, while simultaneously ensuring that anomaly detection is grounded in a sound, interpretable decision-making basis [50]. To achieve our objective, an autoencoder was leveraged as a feature extractor, and its encoder latent features will be used for both anomaly detection (One Class Support Vector Machine (SVM)) and classification of anomaly types (k-means). Furthermore, in this approach, both normal and anomalous data are intentionally input into the autoencoder without labels. The goal is to leverage the learning process by allowing the model to extract common, underlying features from both data types. The novelty of this proposed method lies in its simultaneous integration of three specific components. First, this architecture employs Bayesian optimization to tune the autoencoder’s hyperparameters by maximizing the validation Area Under the Curve (AUC) of a subsequent One-Class SVM trained on the autoencoder’s latent features. Second, the latent features extracted from this encoder are subsequently input into k-means to classify the specific types of anomalies. Third, both normal and anomalous data are intentionally fed into the autoencoder without labels, allowing the model to learn the underlying standard features. The autoencoder architecture itself remains the same as previously described.

Figure 11 shows the anomaly score distribution for the test data and the resulting confusion matrix for the One Class SVM anomaly detection. Within the One-Class SVM, the boundary for the normal class is determined using a Gaussian kernel. This model’s hyperparameters were determined via 50 Bayesian optimization trials, with the validation set AUC maximized as the objective function. The optimal values were found to be: a latent dimension of 5 (from a range of 4 to 128), 222 units in the FC (from 128 to 384), a learn rate of 5.29 x 10^-4 (from 1.0 x 10^-5 to 1.0 x 10^-2), a batch size of 8 (4, 8, 16, 32, 64), 40 epochs (from 5 to 60). The score distribution clearly illustrates that nearly all data points are correctly classified using a threshold of 0 as the decision boundary. The model achieved a test accuracy of 99.8%, representing a significant improvement over the previous autoencoder-only method.

3.4.2. Identifying Anomaly Location and Direction by Autoencoder and One Class SVM

To visualize the decision-making basis, the LIME method was employed. The hyperparameters for LIME were selected using the same methodology as for the autoencoder, with identical hyperparameter types, search ranges, and selection criteria based on stability and fidelity. In this trial, the LIME objective function was set to the One Class SVM score. This optimization process resulted in the following optimal parameters: SLIC as the segmentation method, a kernel width of 0.75, and a superpixel count of 50.

Figure 12 illustrates the criteria for the anomaly detection performed by the One-Class SVM, which used features extracted from the autoencoder. The visualization shows the average classification accuracy for correctly classified data (i.e., when the prediction matched the true label) for each class. In the normal (“No Hole”) condition, the critical regions are localized around the probe and the periphery of the copper piping array, suggesting the model is actively monitoring these areas to confirm the absence of bubbles. For the West pattern, a high degree of importance is correctly placed on the bubble signal. Similarly, East also shows the bubble location as the critical region. In the North condition, the model identifies the upper area of the image where bubbles have dispersed after colliding with the sixth pipe. For South, consistent with the previous CNN results (Figure 6), the model appears to capture the fluid flow associated with bubble emission rather than the bubbles themselves. Considering these results comprehensively, while some challenges with the decision criteria persist, this represents a dramatic improvement compared to the decision criteria of the autoencoder-only method shown in Figure 9.

3.4.3. K-Means Multi-class Classification by Autoencoder-based Feature Extraction

Table 3 shows the results of the K-means classification applied to the latent features extracted by the autoencoder to classify bubble generation location and direction. The analysis conditions were consistent with the previous autoencoder, specifying 29 classes for K-means. The model was trained on 80% of the data per label, and its performance was verified on a 10% test set. The overall test accuracy was 74.5%, and the macro F1-score improved dramatically from 0.164 to 0.559. Furthermore, recall was generally higher than precision. This is a favorable outcome for adaptation to an actual plant, as it indicates a lower probability of false negatives. Additionally, classes that previously had an F1-score of 0.00, such as “1-East,” “3-East,” and “6-West,” showed significant improvement, with their scores increasing to 0.884, 1.00, and 1.00, respectively. Nevertheless, challenges persist, as the F1-scores for “3-West” and “4-East” remained at 0.00.

3.4.4. Comparison with the Autoencoder Using Per-Class Feature Visualization

Figure 13 shows an example comparison between the input experimental images and the corresponding reconstructed images from this model’s autoencoder decoder. In sharp contrast to the previous autoencoder trained only on normal data (Figure 10), it is evident that the signals corresponding to the bubbles are not removed; instead, they are adequately reconstructed. This indicates that the crucial features of the bubbles are preserved within the latent features at the encoder’s output. This preservation of features enables subsequent One Class SVM and k-means models to effectively leverage this information for classification, which explains the dramatic increase in accuracy and the improved classification performance.

Figure 14 contrasts the feature separation capabilities of two different models. (a) and (c) show the results for anomaly detection and anomaly type identification, using the autoencoder-only method. In contrast, (b) and (d) show the results for the proposed method, which utilizes the autoencoder as a feature extractor. The t-SNE results were computed using the encoder’s latent features as input. A comparison of (a) and (b) reveals that while the autoencoder-only method shows significant overlap between normal and anomalous data, the proposed method reduces this overlap. This indicates that by intentionally inputting both normal and anomalous data into the encoder without labels, the model successfully learns common, essential features that must be preserved. The latter approach is more effective at extracting features unique to each class. A similar comparison between (c) and (d) confirms that the proposed method is also superior at extracting specific features for each distinct anomaly type. These results validate the proposed methodology, which combines input from both normal and anomalous data using a hybrid objective function (minimizing reconstruction error while maximizing the One Class SVM’s AUC). Furthermore, this approach demonstrates strong generalization, comparable to or exceeding that of the CNN. This is because the feature extraction during autoencoder training is trained on unlabeled data. Moreover, the One Class SVM anomaly detector is trained only on the latent features of normal data. Consequently, the proposed method achieves both high test accuracy and strong generalization.

5. Conclusions

In this study, we successfully confirmed the presence of bubbles. We accurately identified their generation location and direction using a CNN, with the bubbles themselves as the basis for decision-making. Furthermore, we demonstrated successful anomaly detection for previously unseen anomalies by training an autoencoder solely on normal data. However, this autoencoder method revealed issues with a flawed decision basis and the use of inappropriate features. To address this, we proposed a novel technique in which the autoencoder was leveraged as a feature extractor, trained on both normal and anomalous data to learn their common underlying features. By combining this with a One-Class SVM and k-means clustering, we identified the location and direction of bubble generation without requiring labeled anomaly data. This methodology shows significant potential for application in the anomaly-detection systems of fast-reactor steam generators, PWR cores, and LNG regasification units. To further assess its viability for practical implementation in an actual plant, future work must validate the model’s performance under various noise conditions.