Probability Distribution-Guided Adversarial Sample Attacks

1. Introduction

With the advent of the era of big data and the development of deep learning theory, AI technology is like a sword of Damocles, which brings social progress but also brings serious security risks [1]. For example, the adversarial sample attacks [2], which adds subtle perturbations to the source sample to generate new sample objects. Detection, classification, and other AI algorithms are very sensitive to these subtle perturbations and thus yield false results. In driverless scenarios in the civilian field, attackers can transform road signs into corresponding adversarial samples, causing the driverless system to misjudge the road signs and thus causing traffic accidents [3]. In the UAV strike scenario in the military field, adversarial sample generation can be used to enforce camouflage stealth on the target for protection, which blinds the UAV from completing the attacks [4]. Thus, it can be seen that research on adversarial samples is an important way to improve the AI security.

Adversarial sample attack can be divided into targeted and non-targeted attacks [1]. Taking the targeted attacks as an example, the traditional adversarial sample generation process is $x^{\prime }=x-{\nabla }_{x}L\left(y^p,y^t\right)=x+{\nabla }_{x}log{p}_{\theta }\left(y^t\mid x\right)$, where x is sample, $x^{\prime }$ is adversarial sample, $L\left(y^p,y^t\right)$ is the loss function of predicted labels $y^p$ and target labels $y^t$, and $p_{\theta }\left(y^t\mid x\right)$ is the probability that the classifier predicts the sample x as target label $y^t$. The above method is guided by the decision boundary of the classifier, so that the adversarial sample moves in the direction of the gradient that reduces the classification loss or increases the probability of the target class label, which takes the boundary crossing as the only judgment metric, has no consideration for the probability distribution of the sample itself, and results in an irregular way of adding perturbations to the adversarial sample, an unclear path of generation, and a lack of interpretability. For security reasons, most deep learning models in realistic scenarios are black boxes, and the different structures of classifiers lead to more or less differences in decision boundaries, which also seriously affect transferability of the adversarial samples and will fail when attacking realistic black box models. As shown in the blue box in the left part of Figure 1, The adversarial sample generated by the classifier decision boundary-guided approach cannot break the decision boundary of classifier B even if it breaks the decision boundary of classifier A, thus the transferable attack fails.

From the perspective of probability, any type of sample has its own probability distribution, which reflects the unique semantic characteristics. Classifiers with different structures will generally give similar classification results for a batch of independently and identically distributed samples, in other words, the probability distribution of samples plays a more critical role in the classification process than the structure of the classifier. If we manipulate the probability distribution of samples to guide the generation and attack of adversarial samples, we can get rid of the limitation of classifier structure to generate adversarial samples with high aggressiveness and transferability, and explain the process of generation from the perspective of mathematical theory. As shown in the red box in the right part of Figure 1, the adversarial sample generated by the probability distribution-based approach not only breaks through the decision boundary of classifiers A and B, but also reaches the probability distribution space of the target class, and the generation path is clear, thus the transferable attack is successful.

How to obtain the probability distribution of samples? Let’s solve this problem from the perspective of a probabilistic generative model. The generation of adversarial samples is essentially a special probabilistic generative model, except that the data generation process is less random and more directional. For the probabilistic generative model, if $p_{\theta }\left(x\right)$ learned by the neural network can estimate the true probability density $p_{\mathrm{data}}\left(x\right)$ of the sample, according to the Stochastic Gradient Langevin Dynamics(SGLD) sampling method [5], we iteratively move the initial random noise in the direction of the logarithmic gradient of the sample probability density, then a new independent and identically distributed sample $x_k$ can be sampled according to Eq.(1):

$$x_k=x_{k-1}+\frac{\alpha }{2}·{\nabla }_{x_{k-1}}log{p}_{\mathrm{data}}\left(x_{k-1}\right)+\sqrt{\alpha }·\epsilon =x_{k-1}+\frac{\alpha }{2}·{\nabla }_{x_{k-1}}log{p}_{\theta }\left(x_{k-1}\right)+\sqrt{\alpha }·\epsilon$$

(1)

where $\epsilon$ is the random noise used to promote diversity in the generation process, k is the number of iterations, and $\alpha$ is the sampling coefficient. Inspired by the above idea, if the randomness due to noise is reduced by removing the tail term $\sqrt{\alpha }·\epsilon$, the adversarial sample generation can be regarded as static SGLD sampling according to Eq.(2):

$$x_k=x_{k-1}+\alpha ·{\nabla }_{x_{k-1}}log{p}_{\theta }\left(y^t\mid x_{k-1}\right)=x_{k-1}+\alpha ·{\nabla }_{k-1}log{p}_{\mathrm{data}}\left(x_{k-1}\mid y^t\right)$$

(2)

At this point, it is only necessary to use the classifier to approximate the logarithmic gradient $\nabla log{p}_{\mathrm{data}}\left(x\mid y^t\right)$ of the sample true conditional probability density, then the adversarial sample can be moved out of the original probability distribution space and approached toward the probability distribution space of the target class, which naturally can obtain a higher transferability and a more reasonable explanation.

Therefore, in order to solve the problems of insufficient transferability and poor interpretability of traditional adversarial sample attack methods, we propose a Score Matching-Based Attack(SMBA) method to guide the adversarial sample generation and attack by manipulating the probability distribution of samples. The main contributions of this paper are shown as follows:

• The limitations of traditional adversarial sample generation methods based on the decision boundary guidance of classifiers are broken through, and the generation mechanism of adversarial samples can be interpreted from the perspective of sample probability distribution.
• The transformation of the classification model into the energy-based model is achieved, making it possible to estimate the probability density of samples using the classification model.
• The classifier learns the probability distribution of the samples by aligning the gradient of the classifier with the logarithmic gradient of the probability density of the samples, which can guides the adversarial sample generation directionally and improves the transferability and interpretability in the face of different structural models.

2. Related Works

In this section, the literature related to adversarial sample attack is reviewed first, then the research related to probability density estimation in probabilistic generative models is introduced, and finally the advantages and application scenarios of the Score Matching(SM) method are presented, explaining the reasons why it can be used to adversarial sample attacks.

2.1. Adversarial Sample Attack Methods

Adversarial sample attack methods can be divided into targeted and non-targeted attacks according to the presence or absence of a specific target to be attacked, white-box and black-box attacks according to whether the attacker knows the target model, and gradient-based attacks, optimization-based attacks, transfer-based attacks, decision-based attacks, and other attacks according to the attack methods [1].

Gradient-based attacks: Goodfellow et al. [6] are the first to propose the gradient-based attack method FGSM, which adds perturbations along the reverse direction of the gradient to make the loss function change and eventually lead to model misclassification. However, the attack success rate is low due to the single-step attack with large computational perturbation error. To address this problem, Kurakin et al. [7] propose the iteration-based method I-FGSM, which subdivides the single-step perturbation into multiple steps and restricts the image pixels to the effective region by clipping, thus improving the attack success rate. I-FGSM tends to overfit to the local extremes, which affects the transferability of the adversarial samples, thus Dong et al. [8] propose MI-FGSM, which introduces the momentum idea to stabilize the gradient update direction while crossing the local extrema. PGD [9] is also an improvement based on the above method, which greatly improves the attack effect by adding a layer of random initialization processing and increasing the number of iterations. The alternative method such as ${\mathrm{DI}}^2$-FGSM [10] is designed on the preprocessing of the image, which enhances the transferability and stability of the method. Besides FGSM and its variants, Papernot et al. [11], inspired by the saliency map concept [12], propose the JSMA method, which uses gradient information to calculate the pixel positions that have the greatest impact on the classification results and adds perturbations to them.

Optimization-based attacks: The essence of the adversarial sample attack algorithm is to find relatively small perturbations to generate an effective adversarial sample to implement the attack, so the process of adversarial sample generation can be defined as an optimization problem to be solved. The Box-constrained L-BFGS method propose by Szegedy et al. [2] is a prototype of an optimization-based attack method, which uses quasi Newton method in numerical optimization to solve. $\mathrm{C}\&\mathrm{W}$ [13] is the most classical optimization method, which defines different objective functions and can increase the space size of the optimal solution by changing the variables in the objective function, thus significantly improving the success rate of the attack. Unlike the above attack methods, Su et al. [14] propose the One-pixel method that requires only one pixel point to be modified for a successful attack, which uses differential evolutionary optimization algorithm to generate the perturbation by determining the location of the single pixel point to be modified.

Transfer-based attacks: An attacker uses a white-box attack approach to perform an attack on the surrogate model of the target model to generate a transferable adversarial sample and successfully attack the target model. Querying the target model to obtain a similar training dataset to generate a surrogate model is the main idea of obtaining a surrogate model [15]. Li et al. [16] select the most informative sample for querying through an active learning strategy, which further reduces the query cost while improving the model training quality. Inspired by data enhancement strategy, Xie et al. [10] quickly and effectively expand the dataset by transformations such as cropping and rotating the training dataset, thus the overfitting phenomenon of surrogate models has been solved. Wu et al. [17] introduce the concept of model attention and use the attention-weighted combination of feature mappings as a regularization term to further address the overfitting phenomenon of surrogate models. Li et al. [18] find that multiple integrated surrogate models do not need to have large variability, thus they use existing surrogate models to generate several different virtual models for integration, which significantly enhances the transferability of adversarial samples and reduces the training cost of alternative models.

Decision-based attacks: In transfer-based attack methods, querying the target model is an essential step, and the attack will fail when the query to the target model is restricted. In contrast, the decision-based black-box attack methods successfully get rid of the reliance on querying the target model by random wandering, which are more in line with the actual attack scenario. In simple terms, the attacker first obtains the initial adversarial sample with a large perturbation value and uses it as a basis to search for smaller perturbation values near the model decision boundary to obtain the final adversarial sample. Hence, how to determine the search direction for smaller perturbation values and how to improve its search efficiency are the two aspects that need to be focused on. Dong et al. [19] use CMA-ES [20] to model the search direction on the decision boundary with local geometry, thus reducing the search dimension and improving the search efficiency. Brunner et al. [21] propose a biased decision boundary search framework to find better search directions by restricting the decision boundary of search to perturbations with higher attack success rate. Shi et al. [22] propose CAB approach by exploring the relationship between initial perturbations and search-improved perturbations, which is able to obtain smaller values of adversarial perturbations. Rahmati et al. [23] observe that the decision boundaries of deep neural networks usually have a small mean curvature near the data samples, and accordingly propose GeoDA with high query efficiency.

Other attacks: Attack methods based on generative adversarial network(GAN) [24] can generate adversarial samples without knowing information about the target model. GAN uses a generator to generating adversarial samples, and then feeds the adversarial samples to a discriminator to ensure that the differences between the adversarial samples and the source images are small enough. An external target model determines the difference between the predicted label and the true label of the adversarial sample. The attack is successful if the final adversarial sample obtained is true and natural and misclassifies the target model. The above method is named as AdvGAN [25]. Subsequently, Jandial et al. [26] change the input of the generator from the source image to its potential feature vector, which reduces the GAN training time and significantly improves the attack success rate. Based on the idea of hyperplane classification, Moosavi et al. [27] proposes the DeepFool method by calculating the shortest distance between the source sample and the classification boundary of the target model. Later, Moosavi et al. [28] go on to propose the UAP method, which generates adversarial perturbations with strong generalization capability by calculating the shortest distance between the source sample and multiple classification decision boundaries of the target model. Similar to the idea of JSMA method to find salient graphs, Zhou et al. [29] use CAM to filter out important features of images to generate adversarial samples by content perception means to achieve low-cost and high transferability of adversarial attack.

2.2. Probability Density Estimation Methods

Probability density distribution estimation is originally applied to probabilistic generative models and can be used to model different raw data. We can estimate the true distribution by observing finite samples and sample a new independent and identically distributed sample. Its working principle is mainly based on maximum likelihood estimation. On the one hand, for the estimation of explicit probability distributions with parameters $\theta$, the model gives a likelihood of $L={\prod }_{i=1}^m{p}_{\mathrm{model}}\left(x^{\left(i\right)},\theta \right)$ to the m training samples, and the maximum likelihood principle is to choose the parameter ${\theta }^{*}=\underset{\theta }{argmax}{\prod }_{i=1}^m{p}_{\mathrm{model}}\left(x^{\left(i\right)},\theta \right)$ that maximizes that probability; on the other hand, for the estimation of implicit probability distributions, the maximum likelihood can be approximated as the solution of the parameter ${\theta }^{*}=\underset{\theta }{argmax}{D}_{KL}\left(p_{\mathrm{data}}\left(x\right)\parallel p_{\mathrm{model}}(x;\theta )\right)$ that minimizes the Kullback-Leibler Divergence[30] between the model distribution and the data distribution. Therefore, likelihood-based generative models can be divided into implicit models and explicit models [31].

Implicit models: Typical representatives are GAN and GSN [32], the core purpose of which is to make the distribution $p_g\left(x\right)$ of data generated by the model approximate the true distribution $p_{\mathrm{data}}\left(x\right)$ of the original data. GAN does not explicitly model the probability density function $p_g\left(x\right)$ and cannot be solved by the maximum likelihood method, however, it can directly use the generator to sample from the noise and output samples, and force the minimum distance between $p_g\left(x\right)$ and $p_{\mathrm{data}}\left(x\right)$ to be learned with the help of a discriminator. The GSN differs from GAN in that it needs to use Markov chains to sample after reaching a smooth distribution, and the huge computational effort makes it difficult to extend to high-dimensional spatial data.

Explicit models: For models with explicitly defined probability density distribution $p_{\mathrm{model}}(x;\theta )$, the process of maximum likelihood is relatively straightforward by substituting the probability density distribution into the expression of the likelihood and updating the model in the direction of the increasing gradient, but the challenge is to define the model in such a way that it can express the complexity of the data and facilitate the calculation. Tractable explicit models define a probability density distribution that is easy to compute, and the main model is FVBN [33], which uses the chain rule of probability and transforms it into the form of the joint product of conditional probabilities, but the disadvantage is that the generation of element values depends on the previous element values, which is inefficient. MADE [34] and PixelRNN [35] all belong to this class of models. Approximate explicit models avoid the above limitation of needing to set a probability density function that is easy to solve, and use some approximate methods to solve the maximum likelihood instead. VAE [36] transforms solving the maximum likelihood into solving the extreme value problem with ELBO(Evidence Lower Bound) by variational approximate inference. MCMC [37] is a method that uses Markov chains to simplify the computational process of Monte Carlo random sampling to obtain approximate results. EBM [38] represents the maximum likelihood by constructing an energy function to estimate the minimum energy of the samples.

2.3. Score Matching Methods

Considering a dataset of $Set=\left\{x_1,x_2,\cdots ,x_n\right\}$ from the true distribution $p_{\mathrm{data}}\left(x\right)$ of source data, probabilistic generative model based on the maximum likelihood estimation must find $p_{\theta }\left(x\right)$ to approximate $p_{\mathrm{data}}\left(x\right)$. take EBM as an example, the probability density function is modeled as $p_{\theta }\left(x\right)=\frac{exp\left(-E_{\theta }\left(x\right)\right)}{Z\left(\theta \right)}$. where $E_{\theta }\left(x\right)$ denotes the energy of the sample x, the lower the energy the higher the probability, which is a non-normalized probability and can be trained by a deep neural network. $Z_{\theta }$ is a normalization constant depending on $\theta$ so as to guarantee $\int p_{\theta }\left(x\right)dx=1$. However, since $Z_{\theta }$ involves all the data in the probability distribution and is difficult to solve, in order to make maximum likelihood training feasible, the likelihood-based generative model must restrict its model structure or approximate $Z_{\theta }$, which is more computationally expensive. The good thing is that this problem is cleverly circumvented by the score function $s\left(x\right)={\nabla }_{x}logp\left(x\right)$[39], which is the logarithmic gradient of the probability density and points to the gradient field in the direction of the fastest growth of the probability density function. According to ${\nabla }_{x}log{p}_{\theta }\left(x\right)=-{\nabla }_x{E}_{\theta }\left(x\right)-{\nabla }_{x}log{Z}_{\theta }=-{\nabla }_x{E}_{\theta }\left(x\right)$, the score function eliminates $Z_{\theta }$ by taking the derivative, which makes the solution easier.

Hyvarinen et al. [39] first propose the Score Matching(SM) method to solve the unstandardized statistical model by estimating the difference between ${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$ and $s_{\theta }\left(x\right)={\nabla }_{x}log{p}_{\theta }\left(x\right)$. Since the solution of the SM method involves the calculation of ${\nabla }_x{s}_{\theta }\left(x\right)$, i.e., the Hessian matrix about $log{p}_{\theta }\left(x\right)$, which involves multiple backpropagation and is computationally intensive, Song et al. [40] propose the Sliced Score Matching(SSM) method on this basis, which projects the high-dimensional vector field of $log{p}_{\theta }\left(x\right)$ onto the low-dimensional randomly sliced vector v of a simple distribution (e.g., multivariate standard Gaussian distribution, uniform distribution, etc.) for solution, and the vector problem is scalarized, requiring only one backpropagation, which greatly reduces the computational effort.

Since the SM method estimates ${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$ by solving for $s_{\theta }\left(x\right)={\nabla }_{x}log{p}_{\theta }\left(x\right)$, and the adversarial sample generation process precisely requires gradient information, the SM method can be applied. Compared with traditional adversarial sample attack methods that rely only on the decision boundary guidance of classifiers, our method considers the probability distribution of samples, breaks through the limitations of different structural classifiers, effectively improves the transferability, and gives a reasonable explanation from the mathematical theory and visualization perspectives.

3. Methodology

In this paper, we focus on exploring the probability distribution of samples, and estimate the gradient of the true probability density of samples by the SM method, so as to guide the source samples to move towards the probability distribution space of the target class to generate adversarial samples with higher transferability. The overview of our proposed SMBA framework is shown in Figure 2.

3.1. Estimation Of The Logarithmic Gradient Of The True Conditional Probability Density

Given the input sample x and the corresponding label y, the classifier can be represented as $y=f(x,\theta )$. Let the total number of classifications be n and $f_{\theta }\left(x\right)\left[k\right]$ denote the Kth output of the logits layer of the classifier. The conditional probability density formula for predicting the label as y is:

$$p_{\theta }(y\mid x)=\frac{exp\left(f_{\theta }\left(x\right)\left[y\right]\right)}{{\sum }_{k=1}^{n}exp\left(f_{\theta }\left(x\right)\left[k\right]\right)}$$

(3)

Targeted attacks are committed to move in the direction of the gradient that reduces the classification loss or increases the probability of the target class label:

$$x^{\prime }=x-{\nabla }_{x}L\left(f_{\theta }\left(x\right),y^t\right)=x+{\nabla }_{x}log{p}_{\theta }\left(y^t\mid x\right)$$

(4)

The opposite is true for non-targeted attack:

$$x^{\prime }=x+{\nabla }_{x}L\left(f_{\theta }\left(x\right),y\right)=x-{\nabla }_{x}log{p}_{\theta }(y\mid x)$$

(5)

The subsequent derivation of the formula is based on the targeted attack, and the non-targeted attack is obtained in the same way. According to the idea that adversarial sample generation can be regarded as static SGLD sampling in probabilistic generative models, if the logarithmic gradient $\nabla log{p}_{\mathrm{data}}\left(x\mid y^t\right)$ of the sample true conditional probability density can be approximated using the classifier, then the ideal adversarial sample can be obtained according to Eq.(2).

Next derive the estimation method for $\nabla log{p}_{\mathrm{data}}\left(x\mid y^t\right)$. According to Bayes Theorem:

$$p_{\mathrm{data}}\left(x\mid y^t\right)=\frac{p_{\mathrm{data}}\left(x,y^t\right)}{p_{\mathrm{data}}\left(y^t\right)}=\frac{p_{\mathrm{data}}\left(y^t\mid x\right)·p_{\mathrm{data}}\left(x\right)}{p_{\mathrm{data}}\left(y^t\right)}$$

(6)

After taking the logarithm of both sides, we have:

$$log{p}_{\mathrm{data}}\left(x\mid y^t\right)=log{p}_{\mathrm{data}}\left(y^t\mid x\right)+log{p}_{\mathrm{data}}\left(x\right)-log{p}_{\mathrm{data}}\left(y^t\right)$$

(7)

By eliminating the tail term by derivation we get:

$${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\mid y^t\right)={\nabla }_{x}log{p}_{\mathrm{data}}\left(y^t\mid x\right)+{\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$$

(8)

The first term on the right-hand side of which can be approximated by the regular gradient term ${\nabla }_{x}log{p}_{\theta }\left(y^t\mid x\right)$ of the adversarial sample generated by the classifier. The second term is the logarithmic gradient of the input sample probability density. From the generative model point of view, we need to reconstruct a generative network (generally EBM) to solve the second term by the SM method.

The SM method requires the closer the score function $s_{\theta }\left(x\right)$ learned by the generative network to the logarithmic gradient of the sample probability density ${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$, and the mean squared error loss is used as a measure:

$$S{M}_{-}Loss=\frac{1}{2}{E}_{p_{\mathrm{data}\left(x\right)}}\left[{∥s_{\theta }\left(x\right)-{\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)∥}_2^2\right]$$

(9)

Due to the difficulty in solving $p_{\mathrm{data}}\left(x\right)$, after a theoretical derivation [39], Eq.(9) can be simplified to:

$$S{M}_{-}Loss=E_{p_{\mathrm{dada}\left(x\right)}}\left[tr\left({\nabla }_x{s}_{\theta }\left(x\right)\right)+\frac{1}{2}{∥s_{\theta }\left(x\right)∥}_2^2\right]$$

(10)

where the unknown $p_{\mathrm{data}}\left(x\right)$ is eliminated and the solution process only requires the score function $S_{\theta }\left(X\right)$ learned by the generative network.

Considering that $tr\left({\nabla }_x{s}_{\theta }\left(x\right)\right)$ involves a complex calculation of the Hessian matrix, it can be further simplified by the SSM method as follows:

$$SS{M}_{-}Loss=E_{p_v}{E}_{p_{\mathrm{data}\left(x\right)}}\left[v^T{\nabla }_x\left(v^T·s_{\theta }\left(x\right)\right)+\frac{1}{2}{∥s_{\theta }\left(x\right)∥}_2^2\right]$$

(11)

The simplified loss function simply needs to be supplied with a randomly sliced vector v of a simple distribution to approximate ${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$ with $S_{\theta }\left(x\right)$.

The problem now is that the classification model and EBM have different structures, and the SM and classification processes are independent of each other, so how to combine the classification model and EBM to construct a unified loss function to establish the constraint relationship is what we need to solve.

3.2. Transformation Of Classification Model To EBM

Inspired by the literature [41], we can transform the classification model into the EBM. The probability density function of the EBM is:

$$p_{\theta }\left(x\right)=\frac{exp\left(-E_{\theta }\left(x\right)\right)}{Z\left(\theta \right)}$$

(12)

Consider the conditional probability density function for the universal form of the $\mathrm{n}$ classification problem as:

$$p_{\theta }(y\mid x)=\frac{exp\left(f_{\theta }\left(x\right)\left[y\right]\right)}{{\sum }_{k=1}^{n}exp\left(f_{\theta }\left(x\right)\left[k\right]\right)}$$

(13)

Now use the values $f_{\theta }\left(x\right)\left[y\right]$ of the logits layer of the classification model to define a Joint EBM :

$$p_{\theta }(x,y)=\frac{exp\left(f_{\theta }\left(x\right)\left[y\right]\right)}{Z\left(\theta \right)}$$

(14)

According to the definition of EBM, it can be seen that $E_{\theta }(x,y)=-f_{\theta }\left(x\right)\left[y\right]$. By integrating over y, we can obtain:

$$p_{\theta }\left(x\right)=\sum _y{p}_{\theta }(x,y)=\frac{{\sum }_{y}exp\left(f_{\theta }\left(x\right)\left[y\right]\right)}{Z\left(\theta \right)}$$

(15)

Taking the logarithm of both sides and deriving, we get:

$${\nabla }_{x}log{p}_{\theta }\left(x\right)={\nabla }_x\left(-E_{\theta }\left(x\right)\right)={\nabla }_{x}log\sum _y{p}_{\theta }(x,y)={\nabla }_{x}log\sum _{y}exp\left(f_{\theta }\left(x\right)\left[y\right]\right)$$

(16)

At this point, as long as the values $f_{\theta }\left(x\right)\left[y\right]$ of the logits layer of the classification model are obtained, the classification model can be directly used to replace the EBM for SM estimation of ${\nabla }_{x}log{p}_{\mathrm{data}}\left(x\right)$.

3.3. Generation Of Adversarial Samples On Gradient-Aligned Classifiers

The classifier can classify the source samples with high accuracy after CE loss training, and the gradient direction of the classifier can be aligned with the gradient direction of the true probability density logarithm of the source sample after SM estimation, so as to guide the adversarial sample generation process in a more directional way. By effectively combining the loss functions of both, the impact of parameter adjustment on the classification accuracy during gradient alignment can be reduced, and the final joint training objective is:

$${\theta }^{*}=\underset{\theta }{argmin}\left[\mathrm{Loss}=C{E}_{-}Loss+\lambda ·SS{M}_{-}Loss\right]$$

(17)

where $\lambda$ is the constraint coefficient. The gradient-aligned classifier can now be used to generate adversarial samples with high transferability according to Eq.(2) and (8).

4. Experiments And Results

In this section, the experimental setup is first described, followed by a comprehensive comparative analysis of the proposed method with existing adversarial sample attack methods, and finally the effectiveness of the model is demonstrated from a visualization perspective.

4.1. Experimental Settings

The dataset we take is ImageNet-1K [42], whose training set has a total of 1.3 million images in 1000 classes, and after preprocessing such as random cropping and flipping, the input size is transformed from uniformly $3\times 224\times 224$ pixels to the $[0,1]$ interval with the batch size set to 16. In the joint training process, we choose ResNet-18 [43] as the surrogate model, which is trained according to Eq.(17), and optimized by SGD [44] optimizer with a learning rate of 0.001, and the constraint coefficient $\lambda =5$. In the attack scenario, we select five advanced adversarial attack methods (PGD, MI-FGSM, DI2-FGSM, C&W, SGM [45]) to compare with our method. Five normal models with different structures (VGG-19 [46], ResNet-50 [43], DenseNet-121 [47], Inception-V3 [48], ViT-B/16 [49]) and three robust models (Adversarial Training [50], SIN [51], Augmix [52]) processed by adversarial training and data enhancement methods are selected as target models. The attacks are divided into non-targeted and targeted attacks. The maximum adversarial perturbation allowed by default is $\epsilon =16/255$, step size is $\eta =2/155$, and iteration number is $N=10$. Random 10,000 images correctly classified by the surrogate model on the validation set of ImageNet-1K are selected to generate adversarial samples for evaluation, and the evaluation metric is the attack success rate (1 - correct classification rate). To demonstrate the generality of our approach, experiments are also performed on the CIFAR-100 dataset [53]. Finally, in the visualization part, the perturbation strength is appropriately increased to generate more observable adversarial samples.In order to facilitate the description of the generation path and the attack process, and give a reasonable explanation, the Principal Component Analysis(PCA) [54] method is adopted to reduce the embedding of the adversarial samples from high-dimensional features to two-dimensional feature space for observation.

4.2. Metrics Comparison

In this section, we firstly compare the performance of different methods for targeted and non-targeted attacks on different target models, Subsequently compare the performance of the different methods on different datasets.

4.2.1. Attack Target Model Experiments

As shown in Table 1 and Table 2, in attacking the normal model, the transferability of traditional methods is relatively low when either non-targeted or targeted attacks are performed. Especially from CNN network structures (VGG-19, ResNet-50, DenseNet-121, Inception-V3) to Vision Transformer network structure (ViT-B/16), in the case of PGD non-targeted attacks, for example, the transferability drops from a maximum of 45.05% to 3.25%. Fortunately, our SBMA method achieves the highest attack success rate compared to other methods and exhibits good transferability.

From Figure 3, we can see more intuitively that the transferability of the traditional methods decrease significantly when transforming from non-targeted attacks(subplot (a)) to targeted attacks(subplot (b)), while our SBMA method can still maintain a high transferability. This indicates that network models with different structures have a significant impact on the methods guided by the decision boundaries of the classifiers, but have less impact on the methods guided by the sample probability distributions.

Table 3 and Figure 4 represent the comparison of the attacks on the normal model of DenseNet-121 and its robust models. From the results, we can see that the success rate of the PGD method and the FGSM series methods almost fail when transfer to attack the robust models, and only the SGM method maintains a relatively high success rate. In contrast, our SBMA method still maintains the highest success rate, although it also has some indicator drop. This shows that our approach, guided by the probability distribution of the samples, is able to achieve high transferability by ignoring the negative effects of classifier robustness improvement, while the rest of the methods clearly fail in the face of the robustness model.

4.2.2. Experiments On The CIFAR-100 Dataset

As shown in Table 4, when the dataset is replaced with CIFAR100 and the target attacks are implemented, we still get similar conclusions as in Table 2, indicating that our method is still effective in the face of different datasets. However, we found that all metrics decreased in varying degrees when compared to the results under the ImageNet-1K dataset, as shown in subplot (a) of Figure 5. The CIFAR-100 dataset is divided into 100 classes and the training set contains only 500 images of 3×32×32 for each class, which is fewer in number and smaller in size compared to ImageNet-1K. Considering that the estimation of the sample probability distribution requires more high quality of samples, we input ImageNet-1K images of different sizes and different numbers to validate our suspicion. The results are shown in Table 5, subplots (b) and (c) of Figure 5, where the more the number of input images and the larger the size, the higher the attack success rate is obtained.

4.3. Visualization And Interpretability

In this section, the traditional PGD method is chosen for a visual comparison with our SMBA method, which is illustrated from both the perspective of generating adversarial samples and the corresponding feature space.

4.3.1. Generating Adversarial Samples

We appropriately increased the perturbation strength and observed the adversarial samples images and their perturbations generated by different methods from the perspective of targeted attacks. As shown in Figure 6 and Figure 7, the perturbations generated by the PGD method are irregular, while the SMBA method clearly transfers the tree frog (source sample) toward the semantic features of vine snake and corn (target classes), and the generated perturbation has the semantic features form of the target class. It shows that our method is indeed able to learn the semantic features of the target class.

To further demonstrate the reliability of our SMBA method, we set the source samples as pure gray images and the target classes as multi-classes, as shown in Figure 8. Obviously, the adversarial samples generated by the PGD method are irregularly noisy, while the adversarial samples generated by our SMBA method can still learn the semantic features of the target classes clearly.

4.3.2. Corresponding Feature Space

In the non-targeted attacks, we visualize the two-dimensional feature space distribution patterns of the source sample and the adversarial sample. As shown in Figure 9, when the adversarial samples generated by the non-targeted attacks on the surrogate model(ResNet-18) transfer to attack the target model(DenseNet-121), the black dashed circle area of subplot (a) shows a mixed state , which means that the adversarial samples generated by the PGD method are not completely removed from the feature space of the source samples. While the adversarial samples generated by the SMBA method have been completely removed from the feature space of the source samples along the black arrow of subplot (b). It indicates that our method can make the adversarial samples completely out of the original probability distribution space to enhance the transferability when performing non-targeted attacks.

In the targeted attacks, we visualize the wandering paths of the individual adversarial sample generated by different methods in the feature space. As shown in Figure 10, when the perturbation strength increases, for the surrogate model in subplot (a), the adversarial samples generated by different methods gradually move out of the feature space of the source class and wander toward and into the feature space of the target class, and finally the attacks are successful. For the target model in subplot (b), the adversarial samples generated by the PGD method cannot move out of the feature space of the source class (blue ➀ to ➆) and cannot cross the decision boundary of the target model, thus the transferable attacks fail. Fortunately, the adversarial samples generated by the SMBA method can still move out of the feature space of the source class and move towards and into the feature space of the target class. The pink dashed circle in subplot (b) indicates the successful transferable attacks (red ➄ to ➆), which corresponds to the images with ’ ✓ ’ mark in (c) row, and we can see that the tree frog (source sample) has gradually acquired the semantic features form of the corn (target class). It indicates that our method can make the adversarial sample completely out of the original probability distribution space and wander toward and into the probability distribution space of the target class when conducting the targeted attacks, thus the success rate of transferability is higher.

5. Conclusions

In this paper, we break through the limitation of traditional adversarial sample generation methods based on the decision boundary guidance of classifiers and reinterpret the generation mechanism of adversarial samples from the perspective of sample probability distribution. We find that if the adversarial samples are directed to move from the space of original probability distributions to the space of target probability distributions, the adversarial samples can learn the semantic features of the target samples, which can significantly improve the transferability and interpretability when faced with classifiers of different structures. Therefore, we propose a probability distribution-guided SMBA method, which uses the SM method to align the gradient of the classifier with the gradient of the sample probability distribution after transforming the classification model into an EBM model, so that the gradient of the classifier can be used to move the adversarial samples out of the original probability distribution and wander toward and into the target probability distribution. Extensive experiments demonstrate that our method shows good performance of transferability when faced with different datasets and models, and can give a reasonable explanation from the perspective of mathematical theory and feature space. Meanwhile, our findings also establish a bridge between probabilistic generative models and adversarial samples, providing a new entry angle for the study of adversarial samples and bringing new thinking to AI security. For future work, we will explore how to apply sample probability distribution estimation methods with non-gradient approximation to adversarial sample generation and attacks.