Small Defects Detection of Galvanized Strip Steel via Schatten-p Norm-Based Low-Rank Tensor Decomposition

1. Introduction

The galvanized strip steel is widely used in automobile manufacturing, household electrical appliances and other daily-use products, surface defects might lead to some product quality issues for both in-progress and downstream products. Among various types of surface defects, white-spot defects that as shown in Figure 1, mainly caused by random zinc dross and ash in the hot dip or electro galvanizing process, are considered as the most serious threat to the steel surface quality due to their high concurrence and typical periodicity. This defect can be detected and recorded by the surface defect inspection based on machine vision, and then the quality issues with strip steel will be controlled at the early stage [1,2].

For galvanized steel sheet, the white-spot defects are always ultra-tiny in size, occupying less than 5% of the whole raw image, the shape and size of each defect are greatly different, and the defect does not have a clear semantic relationship with the complex background. Not only are these objects tiny in size, they also lack feature information of color or texture. In fact, industrial production images frequently contain some prior information, for example, defect rarely appears, the vast majority of images is no defect. The inherent issues of tiny size and low-contrast of white-spot defect under complex clutters and heavy noise. poses a challenge on the localization and segmentation of this defect: (a) they are sensitive to tiny defects because of their extreme small sizes, and (b) they cannot accurately locate the random appeared defects whose semantic relationship with the background context is weak.

In the field of machine vision, small or tiny object detection has always been a hot study issue, especially in industrial applications [3,4,5,6,7]. These researches can be classified into three categories: traditional filtering-based methods, sparse and low-rank representation-based methods, and data-driven methods [8]. Filtering-based methods typically design relevant filtering operators in either grayscale or derivative spaces to suppress background noise and enhance targets. While these methods offer a certain level of real-time performance, they often require manual design of operators tailored to specific scenes when facing complex backgrounds, which limits their generalization ability and adaptability. Unlike the methods above that rely on manually designed features based on data priors, data-driven approaches utilize convolutional neural networks (CNNs) for deep feature extraction automatically [9,10]. Although these methods perform well and can extract deeper information, they depend heavily on image samples, which affects their real-time applicability. Additionally, the cost of obtaining a large amount of labeled datasets is often prohibitively high. Furthermore, most deep learning methods exhibit poor generalization capabilities in test datasets that differ from the training datasets. Moreover, due to the size limitations of small targets, which typically consist of only a few pixels, there is a risk of losing critical information during target detection, which may be ineffective or even degrade detection performance. In contrast, some traditional methods offer strong interpretability and maintain a certain level of robustness when faced with complex scenarios, maintaining certain advantages.

In recent years, tensor decomposition-based sparse and low-rank representation methods have achieved remarkable success, especially in infrared or remote sensing small target detection [11,12,13,14,15]. These methods leverage the low-rank nature of the infrared background and the sparse characteristics of small infrared targets, the background is modeled as a low-rank component, while the infrared targets are typically modeled as sparse components. The tensor model effectively preserves spatial structures and utilizes temporal information across multiple frames, which helps achieve more accurate target detection. These methods can combine spatial and temporal local information and iteratively optimize to approach the optimal detection results, significantly improving robustness to detect small targets in complex scenes. Due to distinct domains and different concerns, the aforementioned methods are less widely applied for defect detection in real-world industrial field, such as surface defect detection for the printed circuit board [16], tiny target detection for steel sheet. They may fail without temporal information across frames.

As the defect is random in the production line of strip steel, different image frames are irrelated, and the spatial and temporal information between different image frame is lost. As single-frame information is limited, these methods often miss targets or retain excessive false alarms when dealing with edge overlapping or complex backgrounds, thus failing to provide optimal decomposition guidance. As shown in Figure 1, the white-spot defect of galvanized steel sheet is random and keeps changing all the time, small or tiny white-spot among 4096×2048 raw images, are still easily ignored due to insufficient appearance information. At the same time, the highly varied nature of the background image and small target characteristics make the detection process extremely difficult.

Motivated by the above discussions, there is a need to develop an advanced object detection framework for tiny defect detection for steel sheet. This article thoroughly takes into account the production intrinsic prior to emphasizes the importance of establishing adaptive tensor model to accurately detect small defects on galvanized steel sheet.

The main contributions of this article are outlined as follows:

• We propose a SLRTD method by digging out inter-patch correlation-ships of surface defect images of galvanized strip steel. The separated defect foreground target information with sparse outliers is embedded in the background of low-rank representation.
• To achieve an accurate estimation of non-defect background rank, we incorporate weighted Schatten p-norm regularization for the background component, allowing for better noise removal while preserving edges, ultimately leading to improved detection results. Concurrently, a nonlinear reweighting strategy and tensor singular value decomposition (t-SVD) are adopted to help the model more delicately balance the low-rank and sparse components throughout the iterative process, which elevates the separation accuracy between the defect target and non-defect background.
• On the basis of the alternating direction method of multipliers (ADMM), an effective approach is introduced to solve the sparse and low-rank component decomposition problem. Experiments validate the feasibility and effectiveness of the proposed SLRTD method.

The remainder of this paper is structured as follows: Section 2 provides a concise overview of related work. Section 3 presents a comprehensive introduction to the architecture and components of the proposed SLRTD approach. Section 4 demonstrates numerical comparisons and module analysis. Finally, Section 5 concludes with a discussion of future research prospects. In our paper, tensors, matrices, and vectors are denoted using calligraphic letters, uppercase, and lowercase italics.

2. Related Works

In this section, we provide a brief overview of existing defect detection methods based on filtering, data-driven and tensor decomposition methods.

2.1. Filtering-Based Methods

Generally, filter-based methods design relevant filtering operators in grayscale or derivative spaces to suppress background clutter and enhance the target. These methods can be further classified into threshold method (e.g., Otsu, cross-entropy), morphological method (e.g., morphological operations, template matching), and spectral method (e.g., Fourier transform, Gabor transform, wavelet transform). However, threshold and morphological methods often have difficulties to determine the optimal threshold and need manual adjustments to find the defect regions, especially when the distributions of scene intensity and texture are complicated. The spectral method transform digital images from time-domain into frequency-domain are not sensitive to small objects in the image, so that often miss some small defects in metal surface inspection. Machine learning methods usually include the two-stage of feature extraction and pattern classification [17]. Man-made features are extracted and used by the classifier to predict the defect types. These features include the local binary pattern features, histogram of oriented gradient features, and some other grayscale statistical features. However, these man-made features are still sensitive to defect shape variances, different lighting conditions and background conditions, lacking generalization ability, and showing poor adaptability and robustness.

2.2. Data-Driven-Based Methods

During the past few years, tremendous efforts have been devoted to small defect target detection based on deep learning [18]. Zhang et al. [19] constructed a novel feature enhancement network to improve the performance of small object detection. Hou et al. [20] proposed a contextual information and spatial attention that is based network for detecting small defects in manufacturing industry. Yang et al. [21] introduced RSADUnet model for metal surface tiny defect inspection. However, the satisfactory performance of these methods relies heavily on expensive labeled images, big numbers of parameters and computational complexity to obtain strong feature awareness. Additionally, they are mainly used for natural scene or objects and medical images detection that the shape of these objects does not change much, but the shapes of the metal surface defects are quite different. Thus, these methods may not suitable for metal surface defect inspection in the industry, especially for small defects with varied shapes. At the same time, small defect targets occupy little pixels and lack obvious texture and structure characteristics. These small defects with fewer pixels coverage are not easy to draw attention and provide reliable features.

2.3. Tensor Decomposition-Based Methods

In contrast, tensor decomposition-based sparse and low-rank model always transform the detection problem into an iterative optimization task by exploring new representations of the target and background features. Each patch is directly used as the obverse side slice to construct tensor data, which can guarantee the local features of defect image can be reserved and is conductive to withdrawing the prior information. Many methods proposed different regularization to constrain low-rank and sparse attributes. The reweighted infrared patch-tensor model (RIPT) model proposed by Dai et al. [22] utilizes the target sparse prior and background nonlocal self-correlation prior, and the local structure weight is designed for the target tensor term and served as an edge indicator in the weighted model. However, RIPT still has limitations because the nuclear norm cannot accurately estimate the background. To alleviate this issue, inspired by t-SVD and non-convex approximation of rank based on the tensor nuclear norm, Lu et al. [23] exploited tensor robust principal component (TRPCA) method, which regularizes all singular values of tensor data equally and shrinks all singular values with the same parameter. Zhang et al. [24] introduced a novel nonconvex low-rank constraint, partial sum of the tensor nuclear norm (PSTNN), combining with a weighted l1 norm to effectively suppress the background while that effectively preventing target over-shrinkage. Gao et al. [25] developed an enhanced TRPCA (ETRPCA) model by the weighted tensor Schatten p-norm minimization, which makes the large singular values shrink less in tensor nuclear norm minimization. The Schatten p-norm can better approximate the NP-Hard problem, optimize the recovery of its convex relaxation and will not over-shrink the low-rank components of the data. Chen et al. [26] introduced logarithmic norm regularization as the nonconvex surrogate of matrix and tensor rank, which achieves more accurate low-rank approximation and high computational efficiency.

As it does not consider the influence from noise and clutter, the target detection effect will be reduced greatly in the face of defect image in more complex background. Luo et al. [27] utilized the sparse components of small targets along with a background dictionary, employing the fully connected tensor nuclear norm for low-rank estimation. Wang et al. [28] built a unique regularization term as tensor correlated total variation, which essentially encodes both low-rank and sparse priors of a tensor simultaneously. Geng et al. [29] developed a nonconvex and nonlocal TRPCA (NN-TRPCA) model based on the tensor adjustable logarithmic norm, which adaptively shrink small singular values more and shrink large singular values less. The weighted parameters are manually set as per different scene demands, and they cannot achieve self-adaptive adjustment to avoid influencing the detection effect. Huang et al. [30] introduced a two-stage feature complementary improved tensor low-rank sparse decomposition method, which is divided into two stages: tensor initialization and tensor decomposition, effectively integrating local and nonlocal features. Based on above analysis, the detection capability of these methods is dependent on the contrast between the target and the background. When there is strong contrast, these methods yield good detection results, otherwise, they tend to have a high false positive rate. Moreover, these methods depend on the measurement of background rank in terms of accuracy, and obviously reduces performance and is time-consuming in the face of dim target with complex background.

3. Methodology

The overall framework of the proposed SLRTD method is shown in Figure 2, which consists of construction of tensor model for defect image, model solution, and model analysis.

3.1. Construction of Tensor Model for Defect Image

Tensor can be expanded into matrix along n-modes and its i-th frontal slice are denote as ${\mathcal{X}}^{\left(i\right)}$. We denote $\overline{\mathcal{X}}\in {\mathbb{R}}^{n_1\times n_2\times n_3}$ as the result of Discrete Fourier Transformation (DFT) along its third dimension by using $\overline{\mathcal{X}}$, that is, $\overline{\mathcal{X}}=\mathrm{f}\mathrm{f}\mathrm{t}\left(\mathcal{X},\left[\right],3\right)$. The inverse operator computes $\mathcal{X}$ from $\overline{\mathcal{X}}$, that is, $\mathcal{X}=\mathrm{f}\mathrm{f}\mathrm{t}\left(\overline{\mathcal{X}},\left[\right],3\right)$. The tensor nuclear norm is defined as ${‖\mathcal{X}‖}_{\mathrm{*}}=\sum _{i=1}^r\mathcal{S}\left(i,i,j\right)={\displaystyle \frac{1}{n_3}}\sum _{i=1}^r\sum _{j=1}^{n_3}\overline{\mathcal{S}}\left(i,i,j\right)$, where, $r={\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}}_t\left(\mathcal{X}\right)$, and $\overline{\mathcal{S}}\left(i,i,1\right)$ are the entries on the diagonal of the first slice of $\overline{\mathcal{S}}$, which has a decreasing order property. And we denote the l0 norm, the l1 norm and the Frobenius norm of as ${‖\mathcal{X}‖}_0$, ${‖\mathcal{X}‖}_1$, ${‖\mathcal{X}‖}_{\mathrm{F}}$, respectively.

In defect image, the small defect targets will often be submerged in the background clutter, with low SNR. The target image carries very little information, causing difficulty of target detection. Such defect image data are mainly composed of defect target, the background, and noise, which can be expressed as

$$f_D=f_T+f_B+f_N$$

(1)

where, $f_D$, $f_T$, $f_B$, and $f_N$ represent the input of original defect image, defect image, background image, and noise defect, respectively.

The original gray image of small defect target belongs to two-dimensional data, and the two-dimensional gray image will be converted to tensor data structure. Assuming the gray image is D and the tensor structure obtained after data reconstruction is $\mathcal{D}$. The whole image shall be browsed by sliding window, the small patch obtained each time as the obverse side slice of $\mathcal{D}$ is shown in Figure 2.

Specific reconstruction steps are as follows:

(i) The sliding window’s size and step length are $m\times n$ and $k$, respectively. The image $I$ is browsed from left to right and from top to bottom. The image patch with size of $m\times n$ is an obverse side slice in tensor data.

(ii) After ergodicity, assume the total sliding times of window is $l$. The obverse side slice obtained by image small patch constitutes the patch tensor $\mathcal{D}\in {\mathbb{R}}^{m\times n\times l}$.

The tensor model of defect image can be constructed as follow:

$$\mathcal{D}=\mathcal{T}+\mathcal{B}+\mathcal{N}$$

(2)

where, $\mathcal{D}$, $\mathcal{T}$, $\mathcal{B}$, and $\mathcal{N}$ represent the surface defect patch tensor, defect target patch tensor, background patch tensor, and random noise, respectively.

Target patch-tensor T: For defect image from the strip steel production line, defects with fewer pixels with respect to the whole image, it’s sparse compared to the large background regions. Thus, the surface defect image can be considered as a sparse matrix, which makes the corresponding defect target patch-tensor is an extremely sparse tensor, which can be depicted as:

$${‖\mathcal{T}‖}_0\le d$$

(3)

where, $d$ is an integer that is related with defect target characteristics of number and size.

Background patch-tensor B: As illustrated in Figure 1, the gray-value of normal area in defect image is considered to be uniform, which means that the local and non-local patches are highly correlated with each other even though the pixel distance between two patches may be large in an image.

Actually, the mode-1, mode-2 and mode-3 unfolding matrices of the background patch-tensor are also low-rank. In Figure 3 (b)-(d), the singular values convergence characteristics of all the unfolding matrices decline into approximate zero rapidly, which demonstrates that every unfolding mode of the background patch-tensor is intrinsically low-rank. Based on this property, we can consider the background patch-tensor B as a low-rank tensor, and their unfolding matrices are also all low-rank defined as:

$$\left\{\begin{array}{l}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left({\mathcal{B}}_{\left(1\right)}\right)\le r_1\\ \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left({\mathcal{B}}_{\left(2\right)}\right)\le r_2\\ \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left({\mathcal{B}}_{\left(3\right)}\right)\le r_3\end{array}\right.$$

(4)

where, ${\mathcal{B}}_{\left(1\right)}$, ${\mathcal{B}}_{\left(2\right)}$, and ${\mathcal{B}}_{\left(3\right)}$ represent the background patch tensor’s model-1, model-2, and model-3 unfolding matrices, respectively; $r_1$, $r_2$, and $r_3$ are positive numbers that constrains the complexity of the background image, the larger the value is, the more complex the background is

Noise patch-tensor $\mathcal{N}$: The noise is usually modeled as additive white Gaussian noise, and it satisfies ${‖\mathcal{N}‖}_F^2\le \delta$, where, $\delta >0$ denotes the Gaussian noise level.

Based on above analysis, the small white-spot defects detection on galvanized strip steel based on tensor decomposition can be formulated as the following optimization problem:

$$\begin{array}{l}\underset{\mathcal{T},\mathcal{B},\mathcal{N}}{\min }\left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left(\mathcal{B}\right)+\lambda {‖\mathcal{T}‖}_0+\eta {‖\mathcal{N}‖}_F^2\right)\\ \mathrm{s}.\mathrm{t}.\mathcal{D}=\mathcal{B}+\mathcal{T}+\mathcal{N}\end{array}$$

(5)

where, $\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\left(\bullet \right)$ represents the matrix rank and ${‖\bullet ‖}_0$ represents the 0 norm that indicates the number of non-zero elements in matrix, ${‖\bullet ‖}_{\mathrm{F}}^2$ indicates F normal which is used to depict the random noise.; $\lambda$, $\eta$ represent the regularization parameter that indicates the sparse error and random noise, which are used to adjust the low-rank characteristics and sparse characteristics.

In general, weights are inversely correlated to singular values, since the larger values help retain the edges in the background area and should be less regularized, weights are inversely proportional to singular values. A reweighted strategy is always added to recover the low-rank and sparse tensor of existing noise and clutter. The empirical results show that the weighted nuclear norm minimization (WNNM) based methods outperform the nuclear norm minimization (NNM) to some extent. Nevertheless, it still suffers from the over-shrinkage problem as the same as NNM. Both NNM and WNNM are actually special cases of weighted Schatten p-norm minimization (WSNM) [31,32]. The principle of WSNM is to assign different weights to different singular values. WSNM has been validated in theory and experiments that it can recover the low-rank component more accurately than WNNM and NNM by adjusting power p to a more suitable value, which can also be applied to separate target from background. Therefore, we incorporate the WSNM as low-rank regularization for improving the accuracy of low-rank and sparse tensor decomposition framework. Thus, we need to generalize the definition of WSNM for tensor, which can be defined as below:

$${‖\mathcal{B}‖}_{\mathcal{W}}^p={\displaystyle \frac{1}{n_3}}\sum _{i=1}^r\sum _{j=1}^{n_3}{\left(\mathcal{W}\left(i,i,j\right){\left(\overline{\mathcal{S}}\left(i,i,j\right)\right)}^p\right)}^{{\displaystyle \frac{1}{p}}}$$

(6)

where, $n_3$ is the number of patches, $\mathcal{W}\left(i,i,j\right)={\displaystyle \frac{\xi \sqrt{n_1{n}_2}}{\overline{\mathcal{S}}\left(i,i,j\right)+{\mathcal{E}}_{\mathcal{B}}}}$ denotes a weight tensor, which is defined in Algorithm 1; $\overline{\mathcal{S}}\left(:,:,j\right)$ represents the singular values of $\mathcal{B}\left(:,:,j\right)$, $\xi$ is a tunning parameter, $\mathcal{E}$ is a positive constant.

In order to solve the model, l₂,₁-norm replaces l₀ -norm for relaxation processing. Hence, the convex optimization model is finally solved as follows:

$$\begin{array}{l}\underset{\mathcal{T},\mathcal{B},\mathcal{N}}{\min }\left({‖\mathcal{B}‖}_{\mathcal{W}}^p+\lambda {‖\mathcal{T}‖}_{\mathrm{2,1}}+\eta {‖\mathcal{N}‖}_{\mathrm{F}}^2\right)\\ \mathrm{s}.\mathrm{t}.\mathcal{D}=\mathcal{B}+\mathcal{T}+\mathcal{N}\end{array}$$

(7)

3.2. Model Solution

In order to solve the Equation (7), the ADMM are selected to solve the separable convex optimization problem. The Lagrange function can be constructed as shown in Equation (8), which can be converted to Equation (9), where $\mathcal{Y}\in {\mathbb{R}}^{r\times r\times n_3}$, $\mu$, and $〈\bullet 〉$ represent the Lagrangian multiplier tensor, penalty factor and inner product among tensors, penalty factor and inner product among tensors.

$$\mathcal{O}\left(\mathcal{B},\mathcal{T},\mathcal{N},\mathcal{Y},\mu \right)={‖\mathcal{B}‖}_{\mathcal{W}}^p+\lambda {‖\mathcal{T}‖}_{\mathrm{2,1}}+\eta {‖\mathcal{N}‖}_{\mathrm{F}}^2+{\displaystyle \frac{\mu }{2}}{‖\mathcal{D}-\mathcal{B}-\mathcal{T}-\mathcal{N}‖}_{\mathrm{F}}^2+〈\mathcal{Y},\mathcal{D}-\mathcal{B}-\mathcal{T}-\mathcal{N}〉$$

(8)

$$\mathcal{O}\left(\mathcal{B},\mathcal{T},\mathcal{N},\mathcal{Y},\mu \right)={‖\mathcal{B}‖}_{\mathcal{W}}^p+\lambda {‖\mathcal{T}‖}_{\mathrm{2,1}}+\eta {‖\mathcal{N}‖}_{\mathrm{F}}^2+{\displaystyle \frac{\mu }{2}}{‖\mathcal{D}-\mathcal{B}-\mathcal{T}-\mathcal{N}+{\displaystyle \frac{\mathcal{Y}}{\mu }}‖}_{\mathrm{F}}^2$$

(9)

Based on ADMM, $\mathcal{O}$ can be broken into several sub-problems for iterative update.

(i) Fixation of the rest of the variables and update of $\mathcal{B}$, which is defined by Equation (10).

$${\mathcal{B}}^{k+1}\begin{array}{c}=\arg \underset{\mathcal{B}}{\min }\left({\displaystyle \frac{1}{2}}{‖\mathcal{D}-{\mathcal{T}}^k-{\mathcal{N}}^k+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}-{\mathcal{B}}^k‖}_{\mathrm{F}}^2+{\displaystyle \frac{1}{{\mu }^k}}{‖\mathcal{B}‖}_{\mathcal{W}}^p\right)\end{array}$$

(10)

The Generalized Soft-Thresholding (GST) [33] is incorporated into tensor singular value thresholding (t-SVT) as follows:

$${\mathcal{B}}^{k+1}\begin{array}{c}={\mathcal{F}}_{{\displaystyle \frac{1}{{\mu }^k}}}\left(\mathcal{D}-{\mathcal{T}}^k-{\mathcal{N}}^k+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}\right)\end{array}$$

(11)

where, $\mathcal{F}\left(\bullet \right)$ denotes solving WSNM-tensor problem, and the detailed process is given in Algorithm 1. ${\left(\bullet \right)}^k$ represents the value of variables in k-th iteration.

Algorithm 1: Solving Equation (11)

Input: $\mathcal{X}\in {\mathbb{R}}^{n_1\times n_2\times n_3}$, power p Output: ${\mathcal{F}}_{\lambda }\left(\mathcal{X}\right)$

step 1: Conduct FFT operation: $\overline{\mathcal{X}}=\mathrm{f}\mathrm{f}\mathrm{t}\left(\mathcal{X},\left[\right],3\right)$

step 2: Conduct SVD operation on each frontal slice ${\overline{\mathcal{X}}}^{\left(i\right)}$ of $\overline{\mathcal{X}}$:

for $i=1,2,\dots ,⌈{\displaystyle \frac{n_3+1}{2}}⌉$ do

$\left[U,\Sigma ,V\right]=\mathrm{s}\mathrm{v}\mathrm{d}\left({\overline{\mathcal{X}}}^{\left(i\right)}\right)$, $\Sigma =\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({\sigma }_1,\dots ,{\sigma }_r\right)$

Compute $\mathit{w}=\left[w_1,w_2,\dots ,w_r\right]$

for $j=\mathrm{1,2},\dots ,r$ do

${\delta }_i=\mathrm{G}\mathrm{S}\mathrm{T}\left({\sigma }_i,\lambda w_i,p\right)$;

end for

$\Sigma =\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({\delta }_1,\dots ,{\delta }_r\right)$, ${\overline{\mathcal{W}}}^{\left(i\right)}=U\Sigma V^T$;

end for

for $i=⌈{\displaystyle \frac{n_3+1}{2}}⌉+1,\dots ,n_3$ do

${\overline{\mathcal{W}}}^{\left(i\right)}=\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{j}\left({\overline{\mathcal{W}}}^{\left(n_3-i+2\right)}\right)$;

end for

step 3: Compute $\mathcal{F}\left(\mathcal{X}\right)=\mathrm{i}\mathrm{f}\mathrm{f}\mathrm{t}\left(\overline{\mathcal{W}},\left[\right],3\right)$

(ii) Fixation of the rest of the variables and update of $\mathcal{T}$, which is confirmed by Equation (12).

$${\mathcal{T}}^{k+1}\begin{array}{c}=\arg \underset{\mathcal{T}}{\min }\left(\lambda {‖\mathcal{T}‖}_{\mathrm{2,1}}+{\displaystyle \frac{{\mu }^k}{2}}{‖\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{N}}^k+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}-\mathcal{T}‖}_{\mathrm{F}}^2\right)\end{array}$$

(12)

As $l_{\mathrm{2,1}}$-norm of $\mathcal{T}$ is defined as the sum of $l_2$-norm of each mode-2 fiber, we matricize each tensor along the 2nd mode, so ${‖{\mathcal{T}}^{k+1}‖}_{\mathrm{2,1}}={‖{\mathcal{T}}_{\left(2\right)}^{k+1}‖}_{\mathrm{2,1}}$. It can be transformed into the matrix form with Equation (13).

$${\mathcal{T}}_{\left(2\right)}^{k+1}\begin{array}{c}=\arg \underset{{\mathcal{T}}_{\left(2\right)}}{\min }\left(\lambda {‖{\mathcal{T}}_{\left(2\right)}‖}_{\mathrm{2,1}}+{\displaystyle \frac{{\mu }^k}{2}}{‖{\mathcal{D}}_{\left(2\right)}-{\mathcal{B}}_{\left(2\right)}^{k+1}-{\mathcal{N}}_{\left(2\right)}^k+{\displaystyle \frac{{\mathcal{Y}}_{\left(2\right)}^k}{{\mu }^k}}-{\mathcal{T}}_{\left(2\right)}‖}_{\mathrm{F}}^2\right)\end{array}$$

(13)

Let $X={\mathcal{D}}_{\left(2\right)}-{\mathcal{B}}_{\left(2\right)}^{k+1}-{\mathcal{N}}_{\left(2\right)}^k+{\displaystyle \frac{{\mathcal{Y}}_{\left(2\right)}^k}{{\mu }^k}}$, so

$${\mathcal{T}}_{\left(2\right)}^{k+1}\begin{array}{c}=\arg \underset{{\mathcal{T}}_{\left(2\right)}}{\min }\left(\lambda {‖{\mathcal{T}}_{\left(2\right)}‖}_{\mathrm{2,1}}+{\displaystyle \frac{{\mu }^k}{2}}{‖X-{\mathcal{T}}_{\left(2\right)}‖}_{\mathrm{F}}^2\right)\end{array}$$

(14)

According to [34], it has the following close-form solution with Equation (15), where $X\left(:,j\right)$ represents the j-th column of the matrix $X$.

$${\mathcal{T}}_{\left(2\right)}^{k+1}\left(:,j\right)\begin{array}{c}=\left\{\begin{array}{l}{\displaystyle \frac{{‖X\left(:,j\right)‖}_2-\frac{\lambda }{{\mu }^{\left(k\right)}}}{{‖X\left(:,j\right)‖}_2}}X\left(:,j\right), \mathrm{i}\mathrm{f} {‖X\left(:,j\right)‖}_2>{\displaystyle \frac{\lambda }{{\mu }^{\left(k\right)}}}\\ 0, \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\right.\end{array}$$

(15)

After ${\mathcal{T}}_{\left(2\right)}^{k+1}$ is solved, it can be transformed into tensor form ${\mathcal{T}}^{k+1}$.

(iii) Fixation of the rest of the variables and update of $\mathcal{N}$ is obtained by Equation (16).

$${\mathcal{N}}^{k+1}\begin{array}{c}=\arg \underset{\mathcal{N}}{\min }\left(\eta {‖\mathcal{N}‖}_{\mathrm{F}}^2+{\displaystyle \frac{{\mu }^k}{2}}{‖\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{T}}^{k+1}+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}-\mathcal{N}‖}_{\mathrm{F}}^2\right)\end{array}$$

(16)

Differentiating it with respect to $\mathcal{N}$, and let it to be zero:

$$2\eta \mathcal{N}-{\mu }^k\left(\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{T}}^{k+1}+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}-\mathcal{N}\right)=0$$

(17)

Then, we have

$${\mathcal{N}}^{k+1}\begin{array}{c}={\displaystyle \frac{{\mu }^k}{2\eta +{\mu }^k}}\left(\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{T}}^{k+1}+{\displaystyle \frac{{\mathcal{Y}}^k}{{\mu }^k}}\right)\end{array}$$

(18)

(iv) Fixation of the rest of the variables and update of $\mathcal{Y}$

$${\mathcal{Y}}^{k+1}\begin{array}{c}={\mathcal{Y}}^k+{\mu }^k\left(\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{T}}^{k+1}-{\mathcal{N}}^{k+1}\right)\end{array}$$

(19)

(v) Update of $\mu$

$${\mu }^{\left(k+1\right)}=\min \left(\rho {\mu }^{\left(k\right)},{\mu }_{\mathrm{m}\mathrm{a}\mathrm{x}}\right)$$

(20)

where $0<\rho <1$, ${\mu }_{\mathrm{m}\mathrm{a}\mathrm{x}}={10}^5$.

(vi) Inspection of termination condition

$${\displaystyle \frac{{‖\mathcal{D}-{\mathcal{B}}^{k+1}-{\mathcal{T}}^{k+1}-{\mathcal{N}}^{k+1}‖}_{\mathrm{F}}}{{‖\mathcal{D}‖}_{\mathrm{F}}}}<\epsilon$$

(21)

where ${10}^{-3}<\epsilon <{10}^{-5}$.

The empirical condition ${\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}}_t\left({\mathcal{T}}^{k+1}\right)={\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}}_t\left({\mathcal{T}}^k\right)$ can help reduce the computational time, and its effectiveness has been validated in the experiments. Finally, we summarize the algorithmic process of the proposed SLRTD method in Algorithm 2.

From target-background separation, the tensor $\mathcal{D}$ is decomposed into the defect target $\mathcal{T}$ and background $\mathcal{B}$. As the sliding step is less than the window size, and then the small block will overlap in replacement. To eliminate the possible artefacts at the patch borders, we use the mean filter to handle with the overlapping area. The defect target image $f_T$ and background image $f_B$ are reconstructed from $\mathcal{T}$ and $\mathcal{B}$, respectively. Finally, the defect targets can be extracted and segmented by the adaptive thresholding segmentation method.

Algorithm 2: Solving Equation (7) by ADMM

Input: Original defect image sequence tensor $D\in R^(m\times n\times l)$, power p, $\lambda$,$\eta$

Output: $\mathcal{B}$, $\mathcal{T}$, $\mathcal{N}$

Initialize: $\mathcal{D}$, ${\mathcal{T}}^{\left(0\right)}={\mathcal{B}}^{\left(0\right)}={\mathcal{N}}^{\left(0\right)}=0$, $\mathcal{W}=\mathcal{I}$, $\mathcal{Y}=0$, ${\mu }_{\mathrm{m}\mathrm{a}\mathrm{x}}={10}^5$, $\rho =1.1$, $k=0$

While: not converged, do

step 1: Update ${\mathcal{B}}^{k+1}$ by Equation (11)

step 2: Update ${\mathcal{T}}^{k+1}$ by Equation (15)

step 3: Update ${\mathcal{N}}^{k+1}$ by Equation (16)

step 4: Update ${\mathcal{Y}}^{k+1}$ by Equation (19)

step 5: Update ${\mu }^{\left(k+1\right)}$ by Equation (20)

step 6: Check the convergence condition

step 7: Update $k=k+1$

end while

3.3. Model Analysis

3.3.1. Computational Complexity

For surface defect image D, it could be constructed into the tensor $\mathcal{D}\in {\mathbb{R}}^{m\times n\times l}$ with sliding window’s size $m\times n$ and times l. The complexity of computing is mainly depending on the optimization of $\mathcal{B}$, which includes three parts: FFT operator, SVD and GST. The computation complexity of FFT operation is $\mathcal{O}\left(Amnl\mathrm{l}\mathrm{o}\mathrm{g}\left(l\right)\right)$, and $A$ represents number of iterations in Algorithm 1. For SVD operators, t-SVT can reduce its number of calculations in half, i.e., only $⌈{\displaystyle \frac{l+1}{2}}⌉$ times. Therefore, the cost of this part is $\mathcal{O}\left(Am{n}^2⌈{\displaystyle \frac{l+1}{2}}⌉\right)$. The GST algorithm costs $\mathcal{O}\left(Bmn\right)$, and $\mathcal{B}$ denotes the number of iteration for GST. In conclusion, the whole complexity of updating $\mathcal{B}$ is $\mathcal{O}\left(Amnl\mathrm{l}\mathrm{o}\mathrm{g}\left(l\right)+Am{n}^2⌈{\displaystyle \frac{l+1}{2}}⌉+Bmn\right)$. For updating $\mathcal{T}$, the computational cost is $\mathcal{O}\left(mn\right)$. Therefore, the whole computational cost of optimizing variables is $\mathcal{O}\left(C\left(Amnl\mathrm{l}\mathrm{o}\mathrm{g}\left(l\right)+Am{n}^2⌈{\displaystyle \frac{l+1}{2}}⌉+Bmn\right)+mn\right)$, where $C$ is the number of iterations in ADMM.

3.3.2. Convergence of Algorithm

The convergence of model Equation (9) has been proven in [25]. We evaluate the convergence of the proposed SLRTD method to empirically show the convergence through experiments in different iterations. We set a convergence error value of ${‖\mathcal{D}-\mathcal{B}-\mathcal{T}-\mathcal{N}‖}_{\mathrm{F}}/{‖\mathcal{D}‖}_{\mathrm{F}}$ and found that the algorithm takes approximately 0.5 seconds per frame for small target detection. Figure 4 shows the variation in convergence error as the number of iterations progresses from 0 to 30. It can be observed that after 10 iterations, the error of our algorithm gradually decreases, and the convergence curve stabilizes, indicating that our method meets the convergence requirements and successfully achieves optimal or near-optimal solutions

4. Experiment

To fully verify the effectiveness of the proposed SLRTD method, we conduct a series of qualitative and quantitative experiments. The details of data collection and preprocessing, evaluation metrics are firstly introduced. Secondly, the influence of the key parameters used in our model is analyzed. Finally, four baseline methods are included for performance comparison.

4.1. Experimental Setup

4.1.1. Data Collection and Preprocessing

The proposed SLRTD method is validated on a dataset from real-world galvanized strip steel production line, and the image acquisition platform can be observed in Figure 5. Considering the calculation speed and to ensure the generalization of the model, we crop the raw images 4096×1024 pixels into 200×200 pixels to make the dataset. In total, we get 100 defect images and 100 non-defect images. For each image, the pixel-level ground truth is manually marked by using “1”to denote defective pixels and “0” to denote defect-free pixels. The size of the white-spot defect in the production line is about 1 mm to 3 mm, which is about 5 to 8 pixels in the image with the 200×200 resolution. Defect samples and the corresponding masks of the constructed datasets are shown in Figure 6.

4.1.2. Evaluation Metrics

In order to evaluate the performance of the proposed SLRTD model on defect inspection. The Precision, Recall, precision-recall (P-R) curve, receiver operating characteristic (ROC) curve, area under ROC curve (AUC) and mean square error (MAE) are adopted as evaluation metrics.

Precision is used to evaluate how many pixels, which are predicted positive, are correctly classified, and is calculated by:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{TP}{TP+FP}}$$

(22)

where, true positive ($TP$) means the number of defect pixels which are correctly classified into defect categories, false positive ($FP$) indicates the number of background pixel mistakenly identified as defect.

Recall is used to evaluate how many pixels of the same class are correctly classified, and is calculated by:

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{TP}{TP+FN}}$$

(23)

where, false negative ($FN$) means the number of defect pixels which are not classified to their actual defect class.

Accuracy is used to show the percentage of all the correctly classified pixels, and is calculated by:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{TP+TN}{P+N}}$$

(24)

where, true negative ($TN$) means the number of non-defect pixels which are correctly classified into background, and $P+N$ (Positive+Negative) is the total number of pixels of the defect image.

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{\sum _{i=1}^H\sum _{j=1}^W\left|BW\left(i，j\right)-G\left(i，j\right)\right|}{H\times W}}$$

(25)

where, $H$ and $W$ denotes height and width of surface defect image.

Using Otsu’s threshold, we determine the Precision and Recall by changing thresholds from 0 to 255 to obtain pairs of Precision and Recall.

4.2. Validation of the Proposed Method

4.2.1. Parameter Analysis

The parameters in our model have a key influence on defect target detection performance. The regularized parameter represents the influence of defect target patch-tensor, and it is set to $\lambda ={\displaystyle \frac{l}{\sqrt{\max \left(m,n\right)}}}$, which is adaptive to the patch size of the input image, and number of patches. For the proposed SLRTD method, the patch size, sliding step size and power p are very important parameters which usually affect its robustness. In this subsection, we evaluate the effects of the three parameters by the AUC and MAE on defect images. It should be noted that the performances obtained by changing one of the parameters with the others fixed may not be globally optimal.

a. Patch size

From the perspective of detection performance, we want the patch size to be as large as possible to improve the sparsity and low-rank of small target. However, the larger patch size would also increase the computational complexity simultaneously. To balance the detection performance and computational complexity, we vary the patch size from 20 to 50 with 10 intervals and give the corresponding AUC and MAE in Table 1. As the patch size increases, the overall detection probability of proposed SLRTD method tends to decrease. The degradation is more serious especially when the patch size is 50×50. It is because that the larger patch size would destroy the correlation between the non-local patches, affecting the separation of targets and backgrounds

b. Step size

The choice of sliding step also impacts the patch image size. In practice, we hope to take a larger step in exchange for the reduction of computational complexity. In this experiment, we fix the patch size and vary the step size from 10 to 40 with 10 intervals. The evaluation results are shown in Table 1. It could be observed that the performances of larger steps are always better than that of smaller ones, but when the step size is similar with the patch size, the performance becomes worse.

c. Value of Schatten-p

The WSNM is adopted in our model to solve the over-shrink problem of the low rank estimation, and power p is a decisive factor. Table 2 shows the AUC and MAE on defect datasets with p varying from 0.4 to 1 with 0.3 intervals. The performance of $p=1$ is worse than the results of $p=0.7$ and $p=0.4$, since the over-shrinkage problem is serious in this case, which validates the effectiveness of WSNM. With p decreasing, the low-rank components can be closer to true rank, while more high rank components would become zeros. However, the performance is not good when $p=0.7$, since too small p would also make some low-rank components become zeros, which also degrade the accuracy of estimation. Therefore, we set $p=0.7$ in the following experiments.

.

4.2.2. Robustness to Noise

The noise conditions of real factories are complex and random, thus the robustness to noise is very vital in evaluating small defect target detection method. Here we evaluate the proposed SLRTD method’s performance in the noise case with different levels. Additive Gaussian noise with different signal to noise ratio (SNR) is introduced to the original defect image, including 36dB, 32dB, and 28dB. The experimental results are shown in Figure 7 and Table. 3. It could be observed that the SLRTD is robust to noise in most cases, binarization defect images obtained by SLRTD are clear and accurate. When SNR decreases gradually, the AUC and MAE metrics can remain a relative high level, for example, AUC still remain around 0.85 under the condition of $\mathrm{S}\mathrm{N}\mathrm{R}=28\mathrm{d}\mathrm{B}$, which is considered as less sensitive to noise. However, there are still a few cases that the small targets could not be detected accurately, for example, some pixels of background are misclassified for defect. We think this failure is acceptable, since the small target is totally overwhelmed by the noise. Therefore, it is difficult for SLRTD to discriminate the noise from target, and the detection of small defect target under heavy noise is still a great challenge.

4.3. Comparison with the State-of the-Art Methods

The proposed SLRTD method is compared with four state-of-the-art algorithms, including TRPCA [23], PSTNN [24], ETRPCA [25], and NN-TRPCA [29]

4.3.1. Qualitative Comparison

To facilitate a more intuitive comparison of the performance of each algorithm, we selected representative defect image from the dataset. The false detected targets are highlighted with green circle, which are shown in Figure 8. The qualitative comparison results between the proposed SLRTD method and other four methods are shown in Figure 8. These images are various illumination and gray levels. NN-TRPCA and PSTNN could enhance defect targets, but it also enhances many clutters and noises of background and the defect object could not be uniformly highlighted, which would cause high false alarm rate. These methods may lose the targets with heavy noise. NN-TRPCA exhibits poor robustness in detecting small targets against complex backgrounds, resulting in considerable background residue. TRPCA and ETRPCA method can suppress most of the clutters and detect the targets in 2nd, 4th row, but it still remains some background residuals. There are many pixels that belonging to the background are misjudged by defect, which leads to low accuracy. By contrast, SLRTD separates the defect objects from the image background successfully and locates various defects precisely. It more efficiently highlights the whole defect object with well-defined boundaries than the other methods. It can be concluded from the difference of the results between ETRPCA and SLRTD that WSNM regularization could improve the performance. Besides, WSNM regularization of SLRTD contributes to the good performance, and thus, is the important factor to obtain more precise segmentation results than other methods. What’s more, it’s reasonable to conclude that treat the matrix singular values differently by which the most important characteristics of defects or background can be preserved

The superiority of the proposed SLRTD method is shown in comparison with other competitive approaches, which could suppress the clutters and noise clearly and detect the targets, which can generate high-quality binary segmentation results by simple threshold method.

4.3.2. Quantitative Comparison

To further demonstrate the superiority of the proposed SLRTD method, the P-R and ROC curves are displayed in Figure 9. Here, we take four competitive methods TRPCA, ETRPCA, NN-TRPCA, PSTNN for comparison. It could be observed that for the same false alarm ratio, our method can achieve the highest detection probability compared with other methods in most cases, meaning that it has the best performance. By imposing p-norm to the patch-image, SLRTD can suppress backgrounds effectively for all defect image and lay a good foundation for the subsequent target segmentation. Table 4 summarizes the quantitative results of five methods, and the best results are marked in bold. It demonstrates that SLRTD also has better performance than other four methods. Most of AUC results are higher than 85%, and SLRTD achieves 0.9560. MAE of SLRTD is typically the lowest among all the methods. Compared with TRPCA, it’s increased by 2.08% and 1.55% in AUC and MAE, respectively. Based on the above qualitative and quantitative analyses, it confirms that our proposed SLRTD method consistently outperforms some state-of-the-art methods and verifies the effectiveness of the proposed SLRTD method.

5. Conclusion

In order to accurately and rapidly and detect small white-spot defect of surface of galvanized strip steel, the SLRTD method is proposed in the paper. The nonlinear reweighting strategy based on Schatten p-norm are adopted to separate the defect image into smooth non-defect background and random noise. A solution framework is proposed by ADMM algorithm to obtain the defect target image. Based on the self-constructed defect dataset, experiments are conducted by the qualitative and quantitative method, which achieves the best performance among the state-of-the-art defect detection methods. In the future, we will focus on improving the ability of the proposed SLRTD method to detect the other tiny defect under weak illumination conditions or irregular defect-like textures surfaces. In addition, more three-dimensional information about the surface defect through binocular and structured-light stereo vision, which can provide more feedback and reference for industrial production.