Real-Time Analysis of Industrial Data Using the Unsupervised HDBSCAN Method in Monitoring the Welding Process in a Robotic Cell

1. Introduction

The growing popularity of artificial intelligence (AI) methods, especially the methods of machine learning, found application in many branches of science and technology [1,2,3,4,5,6,7]. However, the application of machine learning methods in real industrial processes is not so widely disseminated. Many commercial devices have been equipped with controllers, like e.g. PLC (Programmable Logic Controllers) for many years, however sensorics of physical quantities in the industrial environment are underdeveloped. The main aim of this paper is to report the implementation of HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise) algorithm [8] to analyze data with the goal to anticipate undesirable situations, including those that may lead to a complete breakdown, stopping the production process, here the process based on welding technology.

The paper is organized as follows. Firstly, the real data in the time domain is presented. Next, the k-means unsupervised machine learning approach [9,10], as well as the Density Based Spatial Clustering of Application with Noise (DBSCAN) algorithm [11,12], are reviewed in order to finally justify the practical advantages of the HDBSCAN implementation [13,14]. The algorithms were tested for their effectiveness in determining the two distinct situations, namely; the well working system and, on the contrary, the system revealing some instabilities during welding. In addition, the visual information was evaluated from a practical perspective, specifically in terms of what type of graphics is better perceived by the operator who oversees the production process. Numerical analysis was performed with the use of Scikit-learn Python package [15].

Major industry players are capable of measuring many parameters in real-time, generating large amounts of data (the so-called big data), however their processing to solve practical problems is not so frequently met. Among dominating applications, we can mention geoscience [16,17,18], monitoring of transport in urban areas [19,20,21,22,23], and numerical data evaluation in data-base applications [24,25,26]. From the machine industry standpoint of view, we can meet some process optimization approaches with the use of k-means algorithm [27], including anomaly detection in automotive industry [28]. Also, the HDBSCAN method was implemented in hydropower unit monitoring [29]. Fault detection was reported in few works, for example in [30,31], while the good review of the Industry 4.0 approach, which relies on big data collection, can be found in [32].

2. Collecting Industrial Data and Samples

The subject of the presented analysis is related to data obtained during the real welding process, in a robotic cell, using a Fronius® welding system. The data were recorded over a period of 78 minutes, during which the robot cyclically repeated several types of welds, which differed in length and the shape of its welding path. Data storing has been realized at the constant rate of 4.2 seconds. During operation, the following parameters were recorded: electric current intensity (A), electric power (W), electric voltage of the welding system power supply (V), welding time (s), wire feed speed (WFS) (m/min), and the system energy consumption (kJ). The applied welding method was the MIG/MAG (the gas metal arc welding). As samples for experiments the heat exchangers suitable for domestic applications, made of ordinary carbon steel, were used (Figure 1). The typical length of the weld paths ranged from about 60 mm to 80 mm, namely there were six welds mounting the tubes around their circumferences and the eight straight welds mounting the front squared surfaces. A typical weld process, for these paths, were splitted into single events of different time duration, while a single event was no longer than 6 s.

3. Analysis of Results

The collected industrial signals are presented below in time domain (Figure 2). As can be seen, the recorded data are grouped into dominant areas, while some abnormal data, that can potentially be treated as undesirable ones, appear outside the dominant regions. For example, in Figure 1a there are five dominating areas (islands) of data, shown in detail in zoomed Figure 1b. Also, in the figure, there are two significant regions; around initial moment of registration (Time=0) – there are non-zero values of current, and zero-valued currents below the islands, registered for Time >70000. Hence, the other points reveal random behavior (abnormal data).

A more in-depth look at the behavior of the measured parameters can give a study with the omission of the time domain, which, as is widely known, is a typical preliminary step for data clustering analysis. These modified results are given in Figure 3. It is obvious that other combinations of data sets are also possible, however the relations provided are sufficiently representative for further analysis. Nevertheless, the data presented in this way does not provide the full picture, that is, the possibility to deeply recognize faults and realize the eventual corrections of the technological process.

In the following analysis, we will discuss in detail the relationship seen in Figure 3a, that is, the dependence of energy on current.

3.1. K-Means Algorithm

The classical k-means data clustering method consists of dividing data into sub-areas, for which a specific number of clusters, along with attraction centers, are determined, so that the sum of distances of individual points, measured from the cluster centers, are at minimum. In the method, the number of centers and the method of counting distances should be specified by a user. There are several possibilities. For the presented data analysis, the Euclidean quadratic distance $d_{ki}$ between the actual (i-th) measured-point position ${\overrightarrow{r}}_{ki}$ and the center of the k-th cluster ${\overrightarrow{c}}_k$ was used. It can be defined as

$$d_{ki}\left({\overrightarrow{r}}_{ki},{\overrightarrow{c}}_k\right)={\left({\overrightarrow{r}}_{ki}-{\overrightarrow{c}}_k\right)}^2,$$

(1)

and the sum of distances, within a given cluster, and for all k-clusters, falls to a minimum

$$\underset{{\overrightarrow{c}}_k}{\min }\left(\sum _k\sum _i{d}_{ki}\left({\overrightarrow{r}}_{ki},{\overrightarrow{c}}_k\right)\right).$$

(2)

An equivalent example of the k-means clustered relationship between normalized data of energy and current, is shown in Figure 4 (on the left). Normalization is not necessary, although it is a common approach, especially met in the analysis of data belonging to completely unrelated subject areas. In the right panel of Figure 4 we justify why the existence of 4 clusters was assumed. In order to do so, we applied the so-called overall Silhouette value (SV) parameter, which states how well the data were separated to form clusters. Thus, by looking for a minimum for each i-th data point, taken from an overall N points data set, we have the following individual Silhouette values

$$SV(i)={\displaystyle \frac{{\overline{D}}_i-{\overline{d}}_{ki}}{max\left\{{\overline{D}}_i,{\overline{d}}_{ki}\right\}}},$$

(2)

where ${\overline{D}}_i$ is the averaged distance of a given data point (i) to all N points, that is including other cluster members, and ${\overline{d}}_{ki}$ is the averaged distance between i-th point and the other points within the k-th cluster. Thus, for the N data points the overall SV parameter equals

$$SV={\displaystyle \frac{1}{N}}\sum _{i=1}^{i=N}SV\left(i\right).$$

(2)

In the presented analysis we usually looked after the first minimum in the dependence between the overall SV and the assumed number of clusters.

The optimized number of clusters has straightforward interpretation, in the case of our data, if to analyze results from the perspective of different welding times (Figure 5). Obtained clusters clearly indicate a clear relationship between the consumed energy, applied currents, and the duration of the welding process.

While the k-means analysis provides sharp boundaries between obtained areas (Figure 4) - which can be seen as an advantage of the method - then this type of graphical presentation does not deliver sufficient and fast information about undesirable situations to the operator. From the practical point of view, however, observation of the cluster-centers positions can provide some hints to the production process quality. On the other hand, such observations might not be fully objective, relying on human recognition and decisions.

3.2. DBSCAN Algorithm

In the next step forward, testing first of all, the effectiveness of monitoring the production process, we used algorithm which tried to exclude some outstanding results from clusters, the results treated as undesirable noise indicating onto non-demanding situations. The DBSCAN approach is an example of such a solution. The algorithm gives an opportunity to declare a minimum number of data points - for the assumed distance of attraction (impact range - epsylon) - in order to classify its coordinates as belonging to a given clustered region. Figure 6 shows four cases, for the assumed minimum number cluster members equal to: 5, 10, 25, and 50, again based on the same exemplary energy-current data series, in order to determine the optimum value of the impact range (epsylon range). In this way we are able to predict the expected number of clusters as well as possible number of noise points.

Comparing the previous k-means analysis, we can conclude that epsylon=10 along with minimum number of samples creating a cluster between 25 and 50 (lower panels in Figure 6) are good choices. Too small value of this parameter generates numerical instabilities, while too large one limits the number of clusters, i.e. loses information. Proper selection of these two parameters (epsylon, minimum number of points) determines the unambiguous distinction between desired data and noise, and consequently the possibility to control the stability of production process. In Figure 7 the clusterization of data is shown for different epsylon values (eps=5, 10, 15, 25) and for the same minimum number of cluster-members equal to 50.

Obtained results (comp. Figure 4 with Figure 7), especially for eps=10, identify very well, reasonably, the relevant areas of the device's operation from the point of view of the welding process. While DBSCAN algorithm has this basic advantage, the possibility to recognize outstanding data from the dominating ones, the very similar method, the HDBSCAM approach, is capable of identifying the variable density of local clusters. The variable density means the possibility to inform operator about overlapping, repeated points. This improved method will be discussed below.

3.3. HDBSCAN Algorithm

Identification the variable density of cluster members can be graphically presented as e.g. circles of variable diameter being proportional to number of samples possessing the same coordinates. The algorithm, similarly, to DBSCAN, assumes the measure of separation from one cluster to another. This measure is named cut distance, equivalent to the epsylon parameter of the DBSCAN approach. Also, in the HDBSCAN method the minimum number of data points, forming the nucleus of cluster, is the adjustable parameter. In Figure 8 several cases of the minimum data points, 15, 20, 25, 30, 40, and 80, are given (from left to right, and up to down). The cut distance has been fixed to 0.1 for further analysis.

The choice of 30 minimum samples gives an opportunity to recognize the two clusters at zero-valued current, what might be significant information about the resting mode of operation. Hence, the other relationships, between different data series, are shown in Figure 9. The data points of enhanced density are represented by larger circles, while there are no significant differences between their sizes, what means density of overlapping data points is, more or less, uniform within the process of welding.

While the HDBSCAN information is more accurate, in comparison to previously described methods, the practical observation of variable production process, thus, the observation of the above graphics is not unambiguous until the end. This is why other characteristics have been proposed. There are – registered in time – the variable number of clusters and the variable number of noised points (Figure 10). The figures given were obtained for the minimum number of samples to create a cluster equal to 30. In the figure it is easy to recognize the initial phase, after which the first two clusters have been created and, at the same time, the number of noise points have dropped down. After that moment, the production process evolved into 4-clusters state, while the number of noised point grown linearly possessing a relatively small slope (the growth rate) equal to about 0.015 points/time_step – excluding the almost final step of production, in where sudden increase in the level of noise was registered.

In order to simulate a hypothetically more inappropriate production situation, using the same kind of graphical representation, we reduced the number of samples needed to form a cluster to the value of 15. The results are shown in Figure 11.

We can therefore conclude, the number of clusters and the number of noise points reveal instabilities, e.g. for time moment equal to about 600. Another instability is seen for 660 value, when the system rapidly evolved from 4 to 6 clusters via intermediate 5-clusters state. This case is more clearly seen in the noise data figure (right panel of Figure 11), and the noise signal changes in a non-linear manner in comparison to results given in Figure 10. However, it can be said that from a practical point of view, the welded parts had no shortcomings. Consideration of the case shown in Figure 11 is only a simulation of a hypothetical, undesirable situation. Thus, returning to the main assumptions of the analyzed results, that is, the case when the minimum number of points forming a cluster is 30, in the next figure (Figure 12) we show the real-time recorded noise dependencies for other data series. The conclusion drawn from these observations is that one should not be limited to observing only one set of data. For example, for the energy-power relationship, some instabilities in the welding process are observed around Time=600, which are not visible for other sets of data series.

In order to provide even more detailed insight into the observed effect, is the usage of the 1^st derivate from noise signal. These results are presented in Figure 13.

From that perspective, the two dominating, significant instabilities were detected; the first one around time-moment of 620 (seen clearly at Figure 13b) and the second situation nearly close to the production process, for time-moments above 1000 (most visible at Figure 13a,b). These two uncertain situations can be recognized in direct dependencies analysis, especially in the energy vs. power dependencies (Figure 14 and Figure 15). Both figures show the states before the abrupt change in the noise level (on the left) and during the increased level of noise (on the right).

In both cases, the registered noise level jumps reflect the system's scattered energy consumption values (0-15 kJ), for the power falls in a fuzzy range around the 3500 W value. Thus, the data has not been classified as members of clusters.

4. Discussion and Conclusions

In the paper we showed the practical implementation of unsupervised machine learning methods in the welding process of heat-exchanger samples. We provided the adequate graphical representation of production process for HDBSCAN algorithm along with comparison of its usefulness, from the perspective of the classical k-means and DBSCAN approaches, in order to classify uncertain production moments. By the use of the proposed first derivative of the registered in-time noise signal, it was possible to recognize instability in energy and power supply. Hence, the visual information, in the form of number clusters and noise level, was evaluated from a practical point of view, namely from the perspective of the technological operator who oversees the production process. We can indicate that such graphical characteristics should be observed for a few selected combinations of data series. In future work we will focus on the implementation of supervised methods, in where labeling of data is the next challenging task, especially for multidimensional results.