Crack Detection of Concrete Based on Improved CenterNet Model

1. Introduction

With the rapid development of China's economy, civil engineering construction projects are increasing, and as one of the pillar industries of the economy, the construction industry has played an irreplaceable role in national construction. With the increasing number of buildings, roads, bridges, tunnels, and other infrastructures, maintaining them in good working conditions is extremely important for public safety. Concrete cracks are usually caused by internal stress and environmental action, leading to the internal fatigue of the material and resulting in cracks and fractures on the surface of the concrete [1]. The occurrence of cracks often represents a change in the structure where the cracks occur. Over time, further cracking and falling off often occur, and water seepage occurs. Therefore, crack detection is of great significance for the healthy operation of construction projects [2]. Based on the location of cracks in the material, they can be divided into surface and internal cracks. The main research object of this study was the surface cracks in construction engineering concrete.

Surface crack detection methods include eye observation, ultrasonic detection [3], eddy current detection [4], speckle interference [5], penetration detection [6], laser holography [7], X-ray detection [8], computer vision detection [9]. Most of the aforementioned methods have formed a relatively complete detection system that can perform surface crack detection well; however, they also have their adaptation scenarios and shortcomings. For example, although ultrasonic detection is sensitive to planar defects, it is difficult to detect nonplanar structures owing to acoustic coupling, and the surface crack detection effect of arch structures facing some projects could be better. Although the detection accuracy is high, optical detection is significantly affected by ambient light interference and vibrations during actual operation. The infrared detection method has a fast detection speed; however, the detection environment is limited due to the equipment's large size. Current computer vision detection technology often obtains the surface image or video of the research object through a camera and other sensing equipment; then, the obtained image or video is pre-processed and feature extracted, and different algorithm models are trained and tested to finally achieve the purpose of target recognition or positioning [10].

Wang Fan [11] studied the problem of crack detection using mathematical morphology and image fusion. Chambon [12] conducted a study on road crack detection and evaluation using computer vision. Tongji University studied an MTI-100 tunnel-detection system and achieved crack detection and location [13]. Soukup [14] used convolutional neural networks to detect the surface cracks. Yaping et al. applied the SegNet network to surface crack detection and achieved satisfactory results. Using an adaptive iterative method, Peng [15] used an improved Otsu threshold segmentation algorithm to study crack images. Yang [16] proposed a new image analysis method for concrete crack detection and conducted a detailed study on a detection method based on edge cracks. Fernandez [17] studied a decision-tree heuristic algorithm for crack detection and achieved satisfactory simulation results. Li [18] studied a concrete surface crack detection method by combining the improved C-V model with the Canny iterative operator; however, the operation time was relatively long, and there were certain limitations.

In summary, current methods for solving the surface crack detection problem based on computer vision technology can be divided into three categories: image classification, object detection, and pixel segmentation. With the continuous development of computer technology and machine learning algorithms, concrete crack detection based on computer vision technology is expected to be increasingly applied in various scenarios. Crack pictures on the surface of concrete were collected by UAV shooting technology. Then the cracks on the surface of construction concrete were detected using computer vision detection technology, which has the characteristics of simple principle, convenient operation, strong flexibility, high precision, low cost, and no contact.

2. CenterNet

The CenterNet algorithm is a single-stage model without an anchor frame and was first proposed in 2019 [19]. CenterNet algorithm has the characteristics of high precision, fast training speed, and simple network structure. The principle of the CenterNet model is as follows: the center point of the target is used to replace the anchor frame, the peak value of the thermal map is used as the center point of the detection object, and then the threshold is set for screening and comparison of the target center point, and finally the category information is obtained by regression using the image features. The training process of CenterNet does not need to consider the anchor mechanism, nor does it need to set or postprocess hyperparameters in advance, significantly reducing the computational load on the entire network.

The original CenterNet uses ResNet18, DLA-34, and Hourglass convolutional networks for feature extraction and then transfers the feature map to the detection module for processing. Finally, the target centre point and category, target length and width prediction, and centre point bias are transferred through the convolution operation [20]. A schematic of the CenterNet algorithm is shown in Figure 1.

The CenterNet algorithm model makes predictions through three convolution blocks: target centre point and category, target length and width prediction, and centre point bias. The loss function of the CenterNet algorithm consists of the loss function of the centre point and classification, loss function of the target frame size, and loss function of the centre point bias [21].

The loss function L_k of the centre point and classification is the focal loss function, and the calculation formula is shown in Equation (1) [22].

$$Lk=-\frac{1}{N}{\displaystyle \sum _{xyc}\left\{\begin{array}{c}{{\displaystyle (1-\widehat{Y}xyc)}}^{\alpha }lg\left(\widehat{Y}xyc\right),Yxyc=1\\ {{\displaystyle (1-\widehat{Y}xyc)}}^{\beta }{{\displaystyle (\widehat{Y}xyc)}}^{\alpha }\times lg(\mathrm{g}-\widehat{Y}xyc),\\ others\end{array}\right.}$$

(1)

In the above formula, the subscript k in the centre point and classification loss function L_k represent the kth input image, N represents the number of keypoints in the image, subscript xyc represents the positive and negative samples of the image, and Yxyc is the label of the true value.

The centre point bias loss function L_offset adopts the L_loss function, and the calculation formula is shown in Equation (2) [23]:

$$L_{offset}=\frac{1}{N}{\displaystyle \sum _p\left|{\widehat{O}}_{\tilde{p}}-(\frac{p}{R}-\tilde{P})\right|}$$

(2)

In the above formula, P is the coordinate of the true value of the original image target, and R represents the subsampling multiple.

The L_loss function is used for the target frame size loss function L_size. The calculation formulas are shown in Equations (3) [24], where S_k represents the size of the original target frame.

$$L_{size}=\frac{1}{N}{\displaystyle \sum _{k=1}^N\left|{\widehat{S}}_{pk}-S_k\right|}$$

(3)

The final loss function was obtained by multiplying the loss function of the centre point by the classification, the loss function of the target frame size, and the loss function of the centre point bias by the corresponding coefficients, as shown in Equation (4).

$$L=L_k+{\lambda }_{size}{L}_{size}+{\lambda }_{off}{L}_{off}$$

(4)

3. CenterNet Optimisation

The improvement of the original CenterNet model includes three aspects: adding a new channel space attention mechanism, adding a feature selection module, and optimising the loss function.

3.1. Addition of Channel Space Attention Mechanism

In the convolutional block attention module (CBAM), the channel attention uses global average pooling and global maximum pooling to obtain the global statistics of each channel, and learns the weight of the channel through two fully connected layers. Each channel was scaled using a sigmoid function to normalise the weights between 0 and 1. Finally, the scaled channel features are multiplied by the original features to produce features with enhanced channel importance [25,26].

The function of the channel attention mechanism is to continuously enhance the importance of the channel during the training process to improve the training effect on the network. The attention mechanism diagram of the CenterNet channel used in this study is shown in Figure 2.

The spatial attention module in the CBAM uses maximum and average pooling to obtain the maximum and average values for each spatial position. Because more channels are generated after convolution, the function of the spatial attention mechanism is to perform maximum pooling and average pooling operations on the channels of each feature point, obtain two different results, concatenate them, and then learn the weight of each spatial position through a convolution layer and sigmoid function. Finally, weights were applied to each spatial position on the feature map to produce features with enhanced spatial importance.

By introducing an attention module, the spatial attention mechanism enables the model to learn the attention weights of different regions adaptively so that it can pay more attention to important image regions while ignoring unimportant ones [27]. The spatial attention mechanism of CenterNet added in this study is shown in Figure 3.

In this study, a channel space attention mechanism was added, and a model combining channel and space attention was constructed to enhance the focus of convolutional neural networks on images and improve the algorithm's performance. The original network joining the CBAM mechanism is illustrated in Figure 4.

3.2. Addition of Feature Selection Module

After adding the feature selection module, the feature map in the downsampling stage was scaled to a unified size and combined with the channel dimensions. In the upsampling stage, the feature selection module adaptively selects the combined features and then adds them to the output features. The structure of the feature selection module is illustrated in Figure 5.

When the CenterNet model was proposed, the original network only extracted the most profound feature map for detection, which led to poor retention of deep and shallow semantic information in the entire network during training, ultimately leading to a decline in the accuracy of the entire network. The feature selection module added in this study can effectively enhance the network extraction of target features and has a stronger ability to capture effective features. The details of the feature selection module are shown in Figure 6.

3.3. Optimization of the Loss Function

The target size loss changes from Smooth L1 Loss to IoU Loss because Smooth L1 Loss cannot adapt to targets of different sizes. The calculation formula is shown in Eq (5). When calculating the IoU Loss, it is assumed that the centre point is the same, and the calculation formula is shown in Equation (6).

$$L_{Iou}=1-\frac{\left|A\cap B\right|}{\left|A\cup B\right|}$$

(5)

$$L_{IOU}=\ln (IOU({box}_1,{box}_2))$$

(6)

The IoU Loss is an indicator used to evaluate the distance between two rectangular boxes. This indicator has all the distance characteristics, including symmetry, nonnegativity, identity, and triangular inequality. The advantages of using the IOU Loss include the following:

1. It can more accurately measure the matching degree between the prediction box and the real box;

2. It has scale invariance, which means that regardless of the size of the prediction box and the actual box, as long as they are located near each other, their IoU values will be similar. This helps the model to have a better generalisation ability when dealing with objects of different scales and sizes.

4. Experiment and Result Analysis

4.1. Experimental Environment

The experimental environment used in this paper was an Ubuntu18.04 64-bit operating system, 754GB running memory, a Tesla V100S graphics card, 32GB graphics memory, and an Intel(R) Xeon(R) Gold 6240 CPU. The PyTorch deep learning framework was used to build the model with CUDA version 10.1 and cudnn version 7.6.0.

4.2. Evaluation Index

Generally, current mainstream computer vision algorithm model evaluation indicators include accuracy and performance [28]. The index used to measure the accuracy of the target detection algorithm is generally the AP, and the performance index includes the FLOPs, FPS, and video memory occupations.

Table 1. Evaluation index and meaning interpretation.

Index	Implication
FLOPs	The number of floating-point operations used to measure the computational complexity of the model
FPS	The number of images the algorithm processes per second, the higher the value, the faster the algorithm processes
p	The size of the video memory occupied by the algorithm in the inference stage. The smaller the video memory occupation, the less resources are required

Table 1. Evaluation index and meaning interpretation.

Index	Implication
FLOPs	The number of floating-point operations used to measure the computational complexity of the model
FPS	The number of images the algorithm processes per second, the higher the value, the faster the algorithm processes
p	The size of the video memory occupied by the algorithm in the inference stage. The smaller the video memory occupation, the less resources are required

Average Precision (AP) was obtained by calculating the area of the PR curve. The calculation formula is shown in Equation (7) [29]:

$$AP={\displaystyle {\int }_0^1\mathrm{p}(\tau )d(\tau )}$$

(7)

4.3. Data

The experimental dataset in this study consisted of 3000 crack pictures captured by the UAV, which were divided into training and test sets in a 9:1 ratio.

In the preprocessing stage, part of the training set was augmented to improve the generalisation ability of the algorithm model. The image transformation method adopted in the dataset enhancement was still close to the tunnel crack image collected after image processing, including random brightness transformation, random horizontal flipping, and random vertical flipping. The transformation results after processing are shown in Figure 7.

The image was scaled and standardised before being input into the network. The widths and heights of the scaled images were 512. The mean values of the standardized RGB three-channel are (123.675,116.28,103.53), and the standard deviation is (58.395,57.12,57.375).

CenterNet determines the target's location by predicting the target centre point, target centre point bias, and target size. Therefore, the corresponding labels of the image include the target centre-point Gaussian heat map, target centre-point bias, and target size, which are represented by a tensor of the same size as the network output.

4.4. Training Process and Experimental Results

To ensure the real and effective results of the comparative experiments, the training parameters used in all the experiments involved in this study were completely consistent. The initial learning rate of the training was 0.0001. The cosine annealing learning rate adjustment method was adopted, and the minimum learning rate was 0.00001. The batch size was set to 8 during the training process. A total of 300 epochs were trained using the SGD optimisation algorithm.

The training experiments were conducted in five groups: original CenterNet with the backbone network of ResNet18, CenterNet with the channel space attention mechanism, CenterNet with the feature selection module, CenterNet with target size loss improvement, and CenterNet with the above three improvements. Table 2 compares the performance of CenterNet with the addition of CBAM and feature-selection modules, including FLOPS, FPS, and video memory.

In the data training process, owing to the different difficulties in data feature extraction, there are overlaps and omissions in some data, as shown in Figure 8. Given this situation, the optimised model in this study adopts the method of strengthening the feature extraction. This situation changed significantly after adding the feature extraction module, and the data processing accuracy was effectively improved.

When the test environment of the controlled experiment was the same as that of the training environment, the batch size of the experiment was set to one. The ablation experiments are summarised in Table 3. From the ablation experiment, the following results were obtained:

After the CBAM module was added, the model size increased by 1.4MB, FPS decreased by 106.6, video memory increased by 2MB, FLOPs remained unchanged, and AP increased by 0.072.

After adding the feature selection module, the model size increased by 0.8MB, FPS decreased by 46.3, video memory increased by 58MB, FLOPs increased by 3.29, and AP increased by 0.101 compared with the original model.

After IOU optimization in the original model, the size increased by 0.5MB, FPS decreased by 123.7, video memory increased by 31MB, FLOPs increased by 2.2, and AP increased by 0.021 compared with the original model.

After the addition of the feature selection module, the optimised model decreased the target size loss faster than the original CenterNet because the feature selection module can adaptively select the underlying features (such as the target texture and edge information) in the downsampling process and add them to the feature map in the upsampling process. Thus, the target size can be learned more quickly.

The change curve of the CenterNet target size loss after the original CenterNet and the addition of the feature selection module are shown in Figure 9.

The feature selection module can adapt to underlying features, which is also evident in the actual detection effect. As shown in Figure 10, after adding the feature selection module, the optimised model can predict the crack size more accurately owing to the inclusion of information such as the crack edge.

The CBAM and feature selection modules, particularly the CBAM module, significantly impact the reasoning speed of the network. This is because, after the CBAM module is added to each ResBlock, the FPS of the network decreases overall, whereas the feature selection module reduces the FPS. Regarding the video memory usage, the impact of the two additional modules was relatively small.

The feature information of the entire network is compressed by the subsampling module, which reduces the workload of subsequent network training and increases the reasoning speed of the entire network. The input information in the upper layer of the network is enhanced after the feature extraction module, and the upsampling stage uses fewer convolutional layers to improve the running speed of the network. The information on each input and output layer of the overall network optimised in this study is shown in Table 4.

To demonstrate the performance improvement of the model before and after optimisation more intuitively, five groups of training processes were randomly selected for comparison, as shown in Figure 11. Dark blue represents the data processing accuracy of the original CenterNet model, and yellow represents the improvement in accuracy brought about by the optimised CenterNet-CBAM-FS-IOU model.

After optimisation, the overall processing accuracy of CenterNet improved to a certain extent, and it could effectively identify cracks in construction concrete with less training time. The actual detection effect is shown in Figure 12, where the red box represents the detection crack prompts, and the number represents the detection number.

As a classic anchor-free model in the field of computer vision, the CenterNet model has a wide range of applications and optimisations in various disciplines. Table 5 lists the AP values of CenterNet and the improved model structure for the dataset. The improvement in the AP values also verifies the effectiveness of the proposed algorithm model optimisation scheme.

5. Conclusions

Based on the original network of CenterNet, a detailed algorithm model optimisation experiment was carried out for the problem of concrete crack detection in construction engineering using pictures of concrete cracks taken by drones, including the addition of a double-attention mechanism, introduction of a feature selection module, and optimisation of the loss function.

The experimental results show that the FPS of the improved CenterNet model is reduced by 123.7, the memory is increased by 62MB, FLOPs are increased by 3.81, and AP is increased by 0.154. The proposed method for detecting cracks in construction projects based on the improved CenterNet network has good robustness and accuracy for the processed datasets and has the potential to be applied to target detection and recognition methods in relevant practical scenarios.