Low-Pass Image Filtering to Achieve Adversarial Robustness

1. Introduction

Сonvolutional neural networks have gained a wide range of applications in modern computing since they allow to automate the wide class of tasks, such as image classification and segmentation [2], object detection and tracking in video streams [3], and image generation [4], [5]. Also, convolutional neural networks are the most effective machine learning tool for some audio processing tasks [6] [7]. Recently, an increasing part of computational processing power is involved in multimedia processing. The growth of overall computing power allows to use increasingly complex and demanding machine learning algorithms. The convolutional neural networks also allow to extract features from multimedia efficiently and process big data, so they are used to solve the difficult or fuzzy tasks.

However, a significant unsolved problem for convolutional neural networks is their sensitivity to distortions and noise. Neural networks trained using clean data do not provide sufficient generalizability to recognize distorted or noisy images. So far, the precise noise/distortion robustness characteristics of convolutional neural networks are unknown yet, only few studies in this field are available up to date [8] [9] [10]. The adversarial distortions severely reduce the image recognition accuracy since they are targeted to the exact neural network model. One of the first mentions of this problem is the study [11], which demonstrated, among other limitations, the weaknesses in the neural network’s generalization ability. The authors have also found out that adversarial distortions are relatively effective for variety of neural networks with a diverse number of layers, architectures, or trained using different datasets. Adversarial images are transferable to other neural networks, even if these networks are trained with different hyperparameters or dataset. Later, a range of techniques for generating adversarial examples were proposed, including FGSM [12], Deepfool [13], One-pixel attack [14] [15], and many others. The maxout network [16], initially achieving an error probability of 0.45%, after application of Fast Gradient Sign Method (FGSM) misclassified 89.4% of adversarial examples, with an average confidence rate of 97.6%. Moreover, with higher image resolution, the recognition error of adversarial examples increases. Currently, the “arms race” of adversarial attacks and countermeasures is relevant [17] [18] [19].

Numerous digitally presented natural images also have distortions induced during the imaging process. Such distortions appear in the images without the attacker’s involvement (unusual camera angles and perspectives, camera matrix thermal noise and lens specifics, atmospheric distortions, image digitization and compression artefacts). Natural adversarial examples are unpredictable, so the corresponding mitigation methods are often not obvious.

These distortions are referred as domain shifts [20] and can be exploited by attackers [21]. One of the first works on natural adversarial examples is [22]. Basing on the ImageNet dataset, which includes tens of millions of images, the authors created datasets (ImageNet-A and ImageNet-O) containing images that are the worst recognized by the state-of-the-art machine learning models. At the same time, the images presented by the authors contain a limited number of false features (Figure 1).

State-of-the-art convolutional network models such as AlexNet, DenseNet-121, ResNet-50, SqueezeNet, and VGG-19 achieve a recognition accuracy no higher than 2.2% on the ImageNet-A dataset (which is approximately 90% lower than the recognition accuracy of the ImageNet dataset by the same networks). The work [22] shows that existing data augmentation methods do not improve performance significantly. Training on other public datasets provides limited improvement. However, [22] doesn’t propose efficient ways to overcome the effect of adversarial distortion.

The above-mentioned problems must be addressed in developing modern convolutional neural network-based image recognition systems.

Some works focused on mitigation methods to cope with distortions and noise in images, are known up to date [23] [24] [25] [26] [27] [28]. Some of these works propose various denoising filters, i.e., image preprocessing, generative adversarial networks and training with noisy data. Most image preprocessing systems are specific to certain types of distortions and adversarial attack designs, so they are being quickly overcome by new adversarial algorithms [29] [30]. Important requirements for denoisers, such as boundaries and textures preservation, do not give an advantage in resisting adversarial attacks.

Another known technique to provide adversarial robustness is to use two or more opposing networks. Here, a competing adversarial network generates distorted images to provide misclassification by the classifier. The classifier is trained to resist these attacks [31], [32]. Accordingly, adversarial examples can be a good source of augmentation. This augmentation method is effective for increasing the convolutional neural network robustness to unobvious and unobservable distortions. However, this approach significantly complicates the development process, the neural network training, and also requires training process monitoring, and is still not always reliable [33]. A crucial way to counteract noise and distortion in test data is to train a neural network using augmented data [34], [35]. Various methods, specific to the task, are used for data augmentation. However, a significant amount of research related to convolutional neural networks application still does not address this problem.

In this paper, we propose a technique which reduces the high-frequency noise influence on the CNNs. We adopt radio engineering principles - filtering noisy images using a low-pass Gaussian filter [36] [37]. Image filtering allows to suppress high-frequency noise, at the same time it blurs the image, reducing its sharpness. This leads to a recognition accuracy decrease, as CNN is initially trained to recognize sharp images. Thus, filtering images with a Gaussian filter allows us to reduce the problem of overcoming high-frequency adversarial attacks to the problem of blurred image recognition, considered in our previous work [38]. The essence of the proposed technique is shown in Figure 2.

To improve the recognition accuracy, it is essential to train the recognition system to recognize blurred images efficiently. It can be done by training the recognition system using augmentation with blurred images [1]. In this paper, we prove that this technique for image pre-processing effectively improves noisy image recognition accuracy without significant reduction of clean image recognition accuracy. We show the existence of an optimum for the Gaussian filter size and propose a technique for finding this optimum. We analyze and compare the behavior of two neural networks: a simple convolutional neural network and the state-of-the-art EfficientNetB3 network [39]. Our simple CNN is tested on datasets with a small number of classes on ImageNet. The EfficientNetB3 is tested on CIFAR-10, Natural Images and Rock-Paper-Scissors datasets.

2. Materials and Methods

2.1. Datasets

We used 4 datasets to train the networks and analyze the results, including CIFAR-10, ImageNet, Rock-Paper-Scissors and the Natural Images dataset.

CIFAR-10 is one of the most widely used image sets for CNN training and testing. The dataset includes 60000 images in 10 classes, the image resolution is 32×32×3 [40]. This resolution is relatively low, which, on the one hand, allows to spend much less time and computational resources for training. On the contrary, it significantly reduces the recognition accuracy of distorted or noisy images, even with low noise intensity (Figure 3).

Natural Images is a comparatively small dataset of natural images [41] consisting of 6899 images of 8 different classes (aircraft, car, cat, dog, flower, fruit, motorcycle, human) (Figure 4). Since training neural networks using large datasets such as ImageNet-1k is challenging, we used the Natural Images set to run a broad class of tests in order to reduce the time and computational cost.

ImageNet-1k [42], a subset of the ImageNet dataset, is a large dataset containing ~1.4 million images labelled into 1000 classes. The image resolution is not standardized. Images are represented in 3 channels. ImageNet-1k is widely used for testing automated image localization and classification systems, as it is rather complex in terms of feature sets and class diversity. We used the ImageNet-1k dataset to extend and validate the results of this research on a complex dataset.

The Rock-Paper-Scissors (RPS) Images dataset [43] contains images of hand gestures from the Rock-Paper-Scissors game. Images are obtained as part of a project [42] to implement a Rock-Paper-Scissors game using computer vision and machine learning. The dataset contains 2188 images corresponding to the gestures “Stone” (726 images), “Paper” (710 images) and “Scissors” (752 images). All images are made on a green background with relatively equal illumination and white balance. All images are RGB with 300×200 pixels resolution.

3.2. Convolutional Nets

In this study, we used two architectures of convolutional neural networks:

• Simplified high-speed CNN called SimConvNet; defined below;
• The commonly used EfficientNetB3 [39].

We obtained the results of the first experiments using a simplified high-performance network. The network contains 914,960 parameters, which allows us to conduct short tests at the expense of overall classification accuracy (Figure 5).

To extend the research and validate results, we used EfficientNet [39]. The research [39] highlighted that insufficient attention is paid to balancing resolution, width and depth in the new CNN architectures, and pointed out the importance of such balancing. An efficient method for the combined CNN scaling to any size is proposed in [39]. With orders of magnitude fewer parameters and training time compared to many state-of-the-art network architectures, the EfficientNetB3 architecture achieves higher Top-1 classification accuracy results on various datasets. Since we provide a broad test set in this research, we use EfficientNetB3 to limit the time and computational resources spent on experiment. It allows us to analyze complex image sets with an acceptable accuracy.

2.3. Adversarial Attacks

FGSM (Fast Gradient Sign Method) is currently one of the most popular adversarial attack methods [12]. The core idea of the method is to add some non-random vector to the original image. The direction of this vector matches the loss function gradient. FGSM additive vector can be represented as:

η=ε·sgn(∇_xJ(θ,x,y)),

where θ—the neural network model parameters, x is the input vector (image), y is the true class of vector x (if available), J(θ, x, y) is the loss function, ε is the empirically chosen gain factor, ∇x is the gradient in image space.

This adversarial vector looks, to human perception, as a high-frequency, low-intensity noise that does not affect object recognition ability. However, this noise is extremely efficient in reducing object recognition accuracy by neural networks. The intensity of the attack is chosen in order to minimize the visible changes in the image and at the same time to achieve sufficient attack success rate. It is possible to perform the attack on some state-of-the-art CNN models preserving non-visibility of changes to a human (Figure 6).

Although FGSM is one of the first adversarial attack algorithms, it is considered one of the most efficient, simple to implement and fast. A more complex variant of FGSM is the PGD (projected gradient descent) algorithm. The essence of the PGD algorithm is to iterate the FGSM algorithm to improve the attack efficiency [44].

Many other adversarial attack algorithms are also based on FGSM [45]. We can presume that proposed high-frequency noise countermeasure technique can be rather efficient against high-frequency distortions such as PGD [44], C&W attack [48], ZOO [49], HSJA [50], DeepFool [13]. At the same time, we should note that the proposed technique will not work well against different adversarial attacks such as physical space attacks [46] and Square attack [47].

2.4. The theoretical approach to the problem solution

An important feature of image recognition CNNs is the low receptivity to the object’s size. It makes the influence of both low-frequency and high-frequency image components nearly equal. It is the fundamental difference between the functioning of modern CNNs and human perception. The research [51] investigated the impact of various image frequency spectrum components on the CNN. High-frequency image components cause CNNs’ vulnerability to adversarial attacks [51]. Despite that, human vision is immune to high-frequency image components [52]. Some commonly used filters can exacerbate CNNs’ high frequency distortion vulnerability [51]. Additionally, adversarially robust neural networks tend to use smoother gradients in the convolutional kernels (filters) [51].

Most adversarial attack algorithms exploit CNNs’ high frequency distortion vulnerability of convolutional neural networks [53]. Some research aimed at detecting the adversarial attacks is based on image spectrum analysis [54] [55]. Low-pass filters, such as the Gaussian filter, protect the recognition system from high-frequency distortions, thus being efficient in counteracting adversarial attacks. After applying low-pass filtering, the high-frequency components of the image will be lost, but the overall structure of the image, the position of the objects of interest and their shapes remain distinguishable (Figure 7).

Figure 7 shows the Cartesian Fourier power spectrum of the image. FGSM attack erodes the image spectrum. The low-pass filter limits the spectrum, bringing it closer to the original.

To confirm the hypothesis about the efficiency of low-pass filtering to overcome the adversarial attack, we analyze the Gaussian blurring effect on the image and attack matrix structure. Red curve in Figure 8 shows the dependence of mutual energy (scalar product values) of the blurred and original image on the Gaussian filter size. Blue curve in Figure 8 shows the dependence of mutual energy (scalar product values) of the blurred and original attack matrix on the Gaussian filter size.

As one can see in the Figure 8, with the Gaussian filter size growth, the mutual energy of the original and blurred attack matrix decreases faster than the mutual energy of the original and blurred image. With filter size (standard deviation) exceeding 10 pixels, the blurred and initial attack matrices are nearly uncorrelated. Since the attack matrix is a target function (each pixel is not random), the attack performance will decrease with increasing Gaussian filter size growth more rapidly than the quality of image recognition.

2.5. The proposed technique

The block diagram of the proposed image processing algorithm is shown in Figure 9.

As one can see from Figure 9, the adversarial images are filtered and fed to a neural network. CNN is pre-trained considering the necessity of recognizing the blurred data [1] [56]. This approach is efficient since the implementation of a Gaussian filter is computationally cheap. Augmentation procedure also uses only this simple filter. We train the neural network in one shot. The training also doesn’t require computationally complex adversarial attack algorithms for data augmentation. The image blurring intensity is depending on the neural network and data characteristics. Gaussian filter significantly reduces the effect of the high-frequency image component. High-frequency image component includes the adversarial attack, other high-frequency noise (e.g., impulse or thermal noise for natural images) and small image patterns. The overall image structure degrades much less significantly.

This technique is a trade-off of the overall recognition accuracy for the adversarial image recognition accuracy. The first one decreases just slightly, and the second one rises significantly.

3. Results

We obtained the results of testing dataset recognition for various neural networks using the algorithm presented in the Figure 9. The following graphs (Figure 10) show the dependence of image recognition accuracy on FGSM attack intensity and Gaussian filter size. We further evaluate the FGSM attack intensity as a percentage of the image dynamic range (DR). We further evaluate Gaussian filter size as a percentage of the image size.

To obtain these graphs, we performed 400 (1600 for some SimConvNet tests) independent experiments on testing dataset recognition with adversarial distortions injection. We varied distortion intensities and subsequently processed images with a Gaussian filter. As one can see from the Figure 10, the image recognition accuracy decreases rapidly with increasing adversarial distortion intensity. At the adversarial distortion intensity equal to 4-5% of the image dynamic range (Figure 10(a)), the recognition accuracy drops to the random level. However, the accuracy increases with Gaussian-filtered adversarial test images. As we further increase the filter size, important image features are lost, and the recognition accuracy drops. The Figure 10 shows that as the intensity of adversarial distortion increases, a wider Gaussian filter size is required. Image recognition accuracy does not reach the initial values (as for clean images) but approaches it. With a further increase in the adversarial distortion intensity, the Gaussian filtering becomes inefficient. The optimal value of the Gaussian filter size depends on the adversarial distortion intensity as well as on the parameters of the data and the neural network, as shown in Figure 10 and Figure 11. For example, CNN with the Rock-Paper-Scissors dataset using augmentation (blurred images) showed high performance at low values of the adversarial distortion intensity (less than 3% of the dynamic range). With further adversarial distortion intensity increase, a greater gain was obtained by the network trained without augmentation (Figure 11).

Since, in practically important cases, the intensity of the adversarial attack does not exceed 10-15% of the dynamic range of the original images, the use of image augmentation in training neural networks gives an advantage in recognition accuracy with a wider spread of Gaussian filter size values.

The results obtained are transferable to complex CNN architectures. In this paper, we conducted experiments using the proposed algorithm (Figure 9) for the EfficientNet using the Natural and ImageNet datasets (Figure 12). We used the augmented ImageNet dataset (augmentation was made using Gaussian filter). We trained the model without Transfer Learning.

The following table (Table 1) shows classification accuracy at various adversarial distortion intensities and possible accuracy gain by applying the filter.

The optimal filter size was chosen due to the maximization of the recognition accuracy for various values of the adversarial attack intensity.

$${\sigma }_{opt}=\mathrm{a}\mathrm{r}\mathrm{g}(\max \left(\sum _{I_{FGSM}=0}^{I_{FGSM}^{max}}{A}_{LPF}\left(\sigma ,I_{FGSM}\right)\right))$$

where ${\sigma }_{opt}$ – optimal filter size, $A_{LPF}$ – accuracy achieved using low-pass filtering, $I_{FGSM}$ – adversarial attack intensity, $I_{FGSM}^{max}$ – maximal adversarial attack intensity.

Accuracy gain G is calculated using the following formula:

$$G=\frac{(1-A_{no LPF})}{(1-A_{LPF})}$$

where G – accuracy gain, $A_{no LPF}$ – accuracy achieved without use of low-pass filtering, $A_{LPF}$ – accuracy achieved using low-pass filtering with optimal filter size. The gain G shows the relative drop of recognition error rate in case of using low-pass filtering compared to the bare CNN usage.

4. Discussion

We proposed a technique to increase the adversarial robustness of deep convolutional neural networks. The method is based on low-frequency image filtering and usage of a network trained to recognize blurred images. We show that increasing Gaussian filter size decreases adversarial attack efficiency faster than the original image feature quality. Thus, the adversarial attack efficiency exchange on the image blurring is found to be efficient. Training the neural network to recognize blurred images is an important part of the proposed technique. This training reduces the impact of image blurring on image recognition accuracy.

The accuracy gain G achieved using the proposed technique is in any case not less than 1.4 times, the average accuracy gain (excluding EfficientNet evaluated on Natural Dataset and FGSM intensity $I_{FGSM}$=5, where the gain is infinite due to the absence of recognition errors with the use of low-pass filtering) is G=8.8 times.

The proposed approach is computationally efficient as it requires only a simple training dataset augmentation performed once before training, and simple image filtering before recognition.

The filtering time depends on the resolution of the image. With a simple CNNs like SimConvNet, the time spent on filtering takes less than 0.4% of the overall image recognition time. With complex networks like EfficientNetB3, the relative time consumption for image filtering is 0.25%.

Several parameters, such as image resolution and neural network type, should be considered when choosing the Gaussian filter size. Excessively high filter size may distort the object features important for classification, thus reducing the overall quality of the neural network algorithm. We show how to choose the optimal filter size.

The proposed method, due to its high efficiency and low complexity, can be used in various image recognition and vision systems implemented on a variety of hardware platforms, including those with extremely limited computational resources.