A GAN-based Augmentation Scheme for SAR Deceptive Jamming Templates with Shadows

1. Introduction

Synthetic aperture radar (SAR) deception jamming technology is effective in concealing important military facilities and operational equipment [1,2], enabling covert military operations[3,4]. The SAR deception jamming technology has the advantage of low power requirement, making it a popular research topic in SAR jamming technology[5,6,7,8]. At present, the methods for SAR deception jamming at the software level include using a SAR perception Jamming template library and electrical deception models. Between the existing techniques, using a SAR deception jamming template library costs less time and manpower than using electromagnetic scattering models for deception jamming, which further facilitates the rapid implementation of deception jamming in practical scenarios. The effect of using a SAR deception jamming template library in deception jamming depends on the refinement level of the deception jamming templates[9,10,11]. However, deception jamming templates with lower authenticity can be easily detected by an enemy, which reduces the effectiveness of deception jamming. In the SAR imaging of side-looking radar, real targets exhibit shadow features. Therefore, using SAR deception jamming templates with shadows in deception jamming is more deceptive than using templates without shadows. By augmenting the existing templates, an efficient library of SAR deception jamming templates with shadows can be established.

Currently, there are two types of sample augmentation schemes for SAR deception jamming templates with shadows, traditional schemes and deep learning-based schemes. The first type involves traditional techniques, such as translation, rotation, and scaling, to obtain augmented SAR deception jamming template libraries with shadows[12]. However, these schemes do not fundamentally alter the internal information on the images but only change the shape of the image targets at a geometric level. The processed shadow parts often lose their authentic correspondence with the targets, which limits the utility of deception jamming. The second type involves deep learning-based approaches for image sample augmentation, where deep learning models effectively capture the complex data distribution and features in SAR deception jamming templates with shadows, thus enabling the generation of more realistic and diverse templates[13]. In deep learning schemes, there are two types: generating using a single template and generating using a dataset. Due to the easier availability of individual templates, using a single template has more advantages in terms of usability [14,15].

However, both types of the above-mentioned schemes do not consider the influence of SAR’s inherent speckle noise, leading to a lower similarity with the input template and a significant decrease in the authenticity of the deception jamming templates. Therefore, it is necessary to consider the characteristics of SAR’s speckle noise and design a fast and accurate sample augmentation network specifically for SAR deception jamming templates with shadows. This will enable the acquisition of a library of shadowed SAR deception jamming templates that are highly adaptable, diverse, and authentic within a short period of time.

The remainder of this paper is organized as follows: Section 2 describes the features of the input SAR spoofing interference template with shadows and the training process and structure of the proposed generative adversarial network model. Section 3 evaluates the generated images and compares the proposed method with the SinGAN method. Finally, Section 4 concludes this paper.

2. Materials and Methods

2.1. Characteristics of the input template

During the training of the network proposed in this scheme, this network requires an input of a SAR target deceptive jamming template with shadows. Based on the speckle noise characteristics and shadow features of the input template, the network performs sample augmentation on the SAR target deceptive jamming template with shadows.

The speckle noise refers to the granular speckle patterns that appear in SAR images due to the interaction of different echo phases during the SAR imaging process[16,17,18]. This noise is an inherent characteristic of SAR images. Coherent speckle noise represents a multiplicative noise in SAR images, and a random distribution of SAR images can be mathematically modeled as follows[19]:

$$I(x,y)=A(x,y)\times Z(x,y),$$

(1)

where $I(x,y)$ represents the observed SAR image, $A(x,y)$ is the ideal image without speckle noise, and $Z(x,y)$ denotes the speckle noise generated during the SAR system imaging.The amplitude of speckle noise in a SAR image follows a Rayleigh distribution, which is expressed by[20]:

$$P(Z(x,y))=\frac{Z(x,y)}{{\sigma }^2}{e}^{-\frac{Z{(x,y)}^2}{2{\sigma }^2}},$$

(2)

where ${\sigma }^2$ represents the variance.

Given the imaging characteristics of SAR side-looking, certain areas of a target may be occluded and not illuminated by radar, resulting in no echo being generated. As a result, in the image domain, unilluminated areas appear as dark regions, known as shadows[21,22,23]. Since the shadow regions are not illuminated by radar, a receiver does not receive any echo signals from these areas, and thus, there is no interaction between different echo phases, leading to the absence of speckle noise in the shadow regions of a SAR deceptive jamming template. As the pixel intensity values of shadow regions should be minimal in the entire image, it is possible to identify shadow points by searching for subregions with the minimum average intensity in an image. In a SAR deceptive jamming template with shadows, a sliding window with a size of ${(2C+1)}^2$ is moved within an image, and the average intensity value within each window is computed. The shadow point $(x_s,y_s)$ corresponds to the center point of the subregion with the minimum average intensity, which can be expressed as follows:

$$(x_s,y_s)=\arg \min C<x\le B-C,C<y\le B-C({\displaystyle \sum _{i=x-C}^{x+C}{\displaystyle \sum _{j=y-C}^{y+C}I(i,j)/{(2C+1)}^2}}),$$

(3)

where $B$ represents the side image length; $C<x\le B-C,C<y\le B-C$, and $C<(B-1)/2$.

The shadow region can be expressed by:

$$I_s=\{I(x,y)|x=x_s,y=y_s\},$$

(4)

2.2. Proposed Scheme

2.2.1. Scheme Overview

To realize rapid and realistic sample augmentation of SAR deception jamming templates with shadows, it is necessary to design a network that considers the speckle noise and shadow features of templates. This design aims to generate SAR deception jamming templates with shadows that have higher similarity and accuracy than the input template with shadows. Since speckle noise is an inherent noise in SAR images, simulating the speckle noise and using it as one of the noise inputs is essential to enhance the deceptive nature of the generated SAR deception jamming templates with shadows. Through processing by a GAN, the speckle noise can be preserved in generated templates with shadows. Moreover, in SAR deception jamming templates with shadows, the background brightness is usually slightly darker in the shadowed areas than in other image areas, resulting in minimal brightness differences between the shadow regions and the surrounding areas. This makes it challenging for a GAN to learn the shadow features effectively. Therefore, it is necessary to enhance a network’s ability to extract shadow features. The specific workflow of proposed scheme is as follows.

The network proposed in this scheme employs a pyramid-like multiscale structure as an overall framework to capture the internal information on SAR deception jamming templates with shadows. Each level of the pyramidal structure has a GAN responsible for generating and discriminating samples of SAR deception jamming templates with shadows at the current scale[14]. The specific network architecture is shown in Figure 1. This approach requires capturing only structural data of a single SAR deception jamming template with shadows at different scales and using it as a training set. Namely, this approach focuses on capturing both the global information and detailed local information on SAR deception jamming templates with shadows.

In Figure 1, $N$ represents the scale of the pyramid, which is defined by the size of the input SAR deception jamming template with shadows; $\{u_N,u_{N-1},\cdot \cdot \cdot u_0\}$ represents the downsampled result set of the original input SAR deception jamming template with shadows at different scales, with a downsampling factor of $r^n$, where $n\in \{0,1,2,\cdot \cdot \cdot ,N\}$ and $r>1$. The pyramid model starts training from the coarsest scale, and the first generator $G_N$ is capable of generating augmented samples ${\tilde{u}}_N$ based on the input mixed noise $z_N$, which is a combination of Gaussian white noise $z_g$ and speckle noise $z_s$. The specific process can be expressed as follows:

$${\tilde{x}}_N=G_N(z_N),$$

(5)

After generating sample ${\tilde{u}}_N$ by mixing noise $z_N$, the upsampled result of ${\tilde{u}}_N$ and Gaussian white noise are both inputted to the next scale’s generator $G_{N-1}$; then, generator $G_{N-1}$ generates a new sample ${\tilde{u}}_{N-1}$. This process performs iteratively, and the output of each scale’s generator can be represented as follows:

$${\tilde{u}}_{N-1}=G_{n-1}[z_{n-1},{({\tilde{u}}_n)}^{\uparrow r}],n<N+1,$$

(6)

where ${\tilde{u}}_{n-1}$ represents the generator output at the ($n-1$)th scale, where $G_{n-1}$ is the generator at that scale; $z_{n-1}$ refers to the mixed noise input specific to the ($n-1$)th scale, and ${({\tilde{u}}_n)}^{\uparrow r}$ represents the upsampled output of ${({\tilde{u}}_n)}^{\uparrow r}$ from the $n$th scale generator.

Noise $z_n$ from the $n$th scale and upsampled output ${\tilde{u}}_n$ of the generator at the ($n+1$)th scale are simultaneously inputted into the generator. The main function of the generator is to generate the missing data in ${\tilde{u}}_{n+1}$ and incorporate them into ${\tilde{u}}_{n+1}$. This process generates a new sample of SAR deception jamming template with shadows, which is denoted by ${\tilde{u}}_{n+1}$ and can be expressed as follows:

$${\tilde{u}}_n={({\tilde{u}}_{n+1})}^{\uparrow r}+f_n[z_n+{({\tilde{u}}_{n+1})}^{\uparrow r}],$$

(7)

where $f_n$ represents the mapping function from the upsampled output ${\tilde{u}}_{n+1}$ and noise $z_n$, which is used to generate the details for generator at the ($n+1$)th scale.

Generators at the same scale share a similar structure. The entire training process progresses from bottom to top, starting from coarse to fine scales. At each scale, the output ${\tilde{u}}_n$ of a generator $G_N$, in addition to being passed to the generator at the next scale, is fed to the discriminator $D_n$. The discriminator $D_n$ compares the generated output ${\tilde{u}}_n$ from the $n$th scale generator with the data obtained by downsampling the input SAR deception jamming template with shadows $u_n$ at the current scale. This process continues until discriminator $D_n$ is unable to distinguish between real and fake samples. A SAR deception jamming template with shadows, which is denoted by $u_n$, is composed of three regions with different features: the target region $I_t$, the shadow region $I_s$, and the background region $I_b$, as given in Eq. (8). The target region $I_t$ contains complex and bright detailed information with regular shapes; a shadow region $I_s$ is characterized by darker areas and relatively clean content; the background region $I_b$ usually exhibits distinct texture details, and lacks clear geometric shapes, making it easier for the network to learn.

$$u_n=I_t+I_s+I_b,$$

(8)

2.2.1. Specific Description of Scheme

• Generator Structure;

A generator $G_N$ uses the Gaussian white noise $z_g$ as its original input and incorporates the speckle noise $z_s$ into a mixed noise input. Between the five fully convolutional networks composed of convolutional layers (Conv)[24], batch normalization layers (BN), and leaky rectified linear units (Leaky ReLU), a spatial attention mechanism (SAM) block is introduced[25,26,27,28]. This mechanism aims to enhance the learning ability of a network toward the target and shadow regions. At the image region level, the SAM block helps the network capture the high-response areas in a feature map, particularly focusing on the regions corresponding to shadows $I_s$ in the feature map. It facilitates the processing of the shadow features in the SAR deception jamming template UN. Since the target and its shadow are crucial during the learning process, and the extraction of shadow features is challenging, an attention mechanism is adopted to improve the network’s capability of extracting shape and contour features of shadow region $I_s$ in a SAR deception jamming template $u_n$. The inception block[29], which is placed in front of the generator $G_N$, consists of multiple scales, thus enabling a more detailed extraction of shape contours and internal details of a SAR target and its shadow, thereby enhancing the authenticity of the generated samples. This block also reduces redundant information and accelerates the convergence speed. The residual dense block with the attention mechanism uses the features extracted by convolutional layers at various levels fully[30], thus further improving the feature extraction capability of the shadow. Additionally, it prevents the problem of gradient vanishing that may occur in deep networks and enhances network stability. The structure of generator $G_N$ is shown in Figure 2.

2.

Discriminator Structure;

Discriminator $D_N$ adopts the patch-GAN approach[31,32,33], which is inspired by the Markovian discriminator concept. Discriminator $D_N$ consists of five fully convolutional layers that use a downsampling scheme to capture data distribution at the current scale, as shown in Figure 3. The discriminator uses both the input SAR deception jamming template with shadows $u_n$ and the generated SAR deception jamming template with shadows ${\tilde{u}}_n$ as input data. The fully convolutional network is responsible for learning the internal distribution information on the two input SAR deception jamming templates at the same scale. By calculating the loss function, the fully convolutional network discriminates between the real and generated SAR deception jamming templates with shadows ${\tilde{u}}_n$ at the current scale. In the adversarial game between generator $G_N$ and discriminator $D_N$, the generated SAR deception jamming templates with shadows become increasingly realistic. The comparison and loss function calculation are performed by contrasting the SAR deception jamming template with shadows generated at the current scale with the downsampled SAR deception jamming template with shadows.

3.

Loss Function;

The model training starts from the coarsest scale and follows the multiscale structure shown in Figure 1. After training a scale, the corresponding GAN for that scale is fixed. The training loss of the $n$th GAN includes the adversarial loss $S_{\mathrm{loss}\_\mathrm{adv}}$ and the reconstruction loss $S_{\mathrm{loss}\_\mathrm{rec}}$, which can be expressed as follows:

$$S_{{}_{\mathrm{loss}-\mathrm{train}}}=\underset{G_n}{\min }\underset{D_n}{\max }{S}_{\mathrm{loss}\_\mathrm{adv}}(G_n,D_n)+\alpha S_{\mathrm{loss}\_\mathrm{rec}}(G_n),$$

(9)

where $\alpha$ represents the weight of the reconstruction loss in the training loss.

Adversarial loss: Each scale’s generator $G_N$ is accompanied by a Markovian discriminator $D_N$, which discriminates the authenticity of the generated shadowed SAR deception jamming templates at that scale. The discriminator’s discrimination of the mean value of the spectrum can be expressed by:

$$S_{\mathrm{loss}\_\mathrm{adv}}=\mathrm{adv}\{E_{x_n~p_{\mathrm{data}}}[D(u_n)]+E_{{\tilde{u}}_n~p_g}[D({\tilde{u}}_n)]\}+\lambda E_{\widehat{u}~p_{\widehat{u}}}{[\parallel \nabla D(\widehat{u})\parallel -1]}^2,$$

(10)

where $\mathrm{adv}$ is the average value function; $p_{\mathrm{data}}$ is the distribution of a real image; $p_g$ is the distribution of the generated image; $u_n~p_{\mathrm{data}}$ is the distribution of $u_n$ obeying $p_{\mathrm{data}}$; ${\tilde{u}}_n~p_g$ is the distribution of ${\tilde{u}}_n$ obeying $p_g$; $u_{\mathrm{data}}$ is the concentration area of the real sample; $u_g$ is the concentration area of the generated sample; $\widehat{u}$ is randomly interpolated between $u_{\mathrm{data}}$ and $u_g$, and $\widehat{u}=\epsilon u_{\mathrm{data}}+(1-\epsilon )u_g,\epsilon \in [0,1]$; $\widehat{u}~p_{\widehat{u}}$ is the distribution of $\widehat{u}$ obeying $p_{\widehat{u}}$; $D(u_n)$ represents the discriminator output when discriminating the input shadowed SAR deception jamming template; $D({\tilde{u}}_n)$ represents the output of the discriminator when discriminating the generated shadowed SAR deception jamming template; $E$ represents the expectation; $\nabla$ is the gradient operator; $\parallel$$\parallel$ represents the $\mathrm{L}1$ norm; $\lambda$ represents the weight of the gradient loss function.

Reconstruction loss: To generate a specific noise map of the original image $x$, assume that $\{z_N^{\mathrm{rec}},z_{N-1}^{\mathrm{rec}},\cdot \cdot \cdot ,z_0^{\mathrm{rec}}\}=\{z^{\ast },0,\cdot \cdot \cdot ,0\}$, where $z_n^{\mathrm{rec}}$ is the noise reconstructed at the $n$th scale, and $n\in \{0,1,2,\cdot \cdot \cdot ,N\}$; $z^{\ast }$ is the fixed noise map; ${\tilde{x}}_n^{\mathrm{rec}}$ is the image generated at the $n$th scale using the noise map. When $n<N$, the reconstruction loss can be expressed by:

$$S_{\mathrm{loss}\_\mathrm{rec}}={\Vert G_n(0,[{\tilde{u}}_n^{\mathrm{rec}}{\uparrow }^r)-u_n\Vert }^2,$$

(11)

When $n=N$, the reconstruction loss can be expressed by:

$$S_{\mathrm{loss}\_\mathrm{rec}}={\Vert G_N(z_g^{\ast }+z_s^{*})-u_N\Vert }^2,$$

(12)

3. Results

3.1. Data Description

The experimental data included images from the MSTAR dataset[34,35,36]. The MSTAR data were collected using the Sandia National Laboratories SAR sensor platform and the Sentinel-1A multimode SAR sensor platform with X-band SAR sensors, having a resolution of 0.3 m in the spotlight mode. The publicly available MSTAR dataset consists of ten different categories of ground targets, including armored vehicles (BMP-2, BRDM-2, BTR-60, and BTR-70), tanks (T-62, T-72), rocket launchers (2S1), anti-aircraft units (ZSU-234), trucks (ZIL-131), and bulldozers (D7). Furthermore, the MSTAR dataset covers various depression angles and orientations and has been widely used for testing and performance comparison of SAR automatic target recognition algorithms.

In the experiment, the MSTAR data on a T72 tank with shadows were used as input data for training the network proposed in this scheme. The goal was to generate a library of SAR deception jamming templates with shadows.

3.2. Experimental Result

The main objective of the proposed network is to extract multilevel features of a target and generate SAR deception jamming templates with shadows that have a high similarity to the input SAR deception jamming templates with shadows of the T72 tank. The 50 SAR deception jamming template with shadows of the T72 tank generated by the proposed network is presented in Figure 4. The comparison between the real SAR deception jamming template with shadows of the T72 tank and the SAR deception jamming template with shadows of the T72 tank generated by the proposed network is presented in Figure 5.

The SAR deception jamming template with shadows of the T72 tank generated by the proposed network showed that the image exhibited prominent features of speckle noise, indicating a high level of authenticity. Moreover, the shadow contour of the T72 tank was realistic and well defined, and the edges and internal details of the tank were accurately represented. The visual comparison of the real and generated SAR deception jamming template with shadows demonstrated that the generated template had a high level of authenticity.

3.3. Effectiveness Analysis of Scheme

3.3.1. Quantitative Analysis of Image Quality

To evaluate the quality of the generated SAR deception jamming templates with shadows, its targets, shadows, and speckle noise were assessed.

The equivalent number of looks (ENL) was used to measure the relative strength of speckle noise in SAR images[37,38,39]. A lower ENL value indicated a greater presence of speckle noise. The ENL was calculated by:

$$ENL={\left(\frac{\mu }{\sigma }\right)}^2,$$

(13)

where $\mu$ represents the mean value of a SAR image, and $\sigma$ represents the standard deviation of the SAR image. The ENL was calculated for the background regions in each image.

The structural similarity (SSIM) index[40,41,42], which reflects the similarity between two images in terms of luminance, contrast, and structure, was also used. A higher SSIM value indicated a higher similarity of images. The SSIM was obtained by:

$$SSIM(u_n,{\tilde{u}}_n)=\frac{(2{\mu }_1{\mu }_2+k_1{l}^2)(2{\sigma }_1{\sigma }_2+k_2{l}^2)}{({\mu }_1^2+{\mu }_2^2+k_1{l}^2)({\sigma }_1^2+{\sigma }_2^2+k_2{l}^2)},$$

(14)

where ${\mu }_1$ represents the average value of an image $u_n$; ${\mu }_2$ is the average value of an image ${\tilde{u}}_n$; ${\sigma }_1$ and ${\sigma }_2$ represent the standard deviations of images $u_n$ and ${\tilde{u}}_n$, respectively; $l\in L$, where $L$ is the dynamic range of pixel values; $k_1$ and $k_2$ are constants set to default values of 0.0001 and 0.0009, respectively. The SSIM was calculated for the red rectangular regions in each image.

The evaluation results of the first three images in Figure 4 regarding the ENL and SSIM metrics are presented in Table 1.

Table 1 shows that the original image in Figure 5(a) had an ENL value of 2.5604. The generated SAR deception jamming templates with shadows have ENL values of 1.9764, 1.9090 and 1.9092, with an average value of 1.9315. Therefore, the difference between the ENL values of the generated templates and the original image was small, indicating a high similarity between the images of 75.44%. This result suggested that the generated SAR deception templates has high authenticity.

For the original image and the generated SAR deception templates, the SSIM values of the target and shadow regions were calculated. The SSIM values of the target and shadow regions of the original image and the generated templates were 0.9617, 0.9620, and 0.9643, with an average value of 0.9627, which indicated a high similarity of 96.27% between the generated and original images. Therefore, the proposed method could generate SAR deception jamming templates with shadows that exhibit high similarity to the original image and have high authenticity.

3.3.2. Comparison with SinGAN Method

This experiment was performed on shadowed T72 tank samples from the MSTAR dataset. Since SinGAN was currently one of the methods that can perform sample augmentation on the shadowed SAR deception jamming template and obtain good results, so the SinGAN method was employed for image sample augmentation of SAR deception templates with shadows, resulting in a dataset of 50 T72 tank SAR deception templates, as shown in Figure 6. The comparison between the real SAR deception jamming template with shadows of the T72 tank and the SAR deception jamming template with shadows of the T72 tank generated by the proposed network is presented in Figure 7.

Table 2 presents the evaluation results of the first three images using two metrics, ENL (Equivalent Number of Looks) and SSIM (Structural similarity Measure).

The results in Table 2 show that the ENL value of the original image in Figure 7(a) was 2.5604, and the ENL values of the generated SAR deception jamming templates with shadows were 1.6677, 1.7150, and 0.5637, with an average value of 1.3155. The average ENL of the generated templates showed a significant difference from the original image’s ENL value, indicating a low similarity of only 51.38%. This result suggested that the generated templates exhibited weaker speckle noise than the proposed approach, resulting in lower authenticity.

For the original image and the generated SAR deception templates, the SSIM values of the target and shadow regions were calculated. The SSIM values of the target and shadow regions were 0.3846, 0.37794, and 0.3925, having an average value of 0.3855 and indicating a 38.55% similarity between the generated templates and the original image. These values were lower than those of the proposed approach for target and shadow regions. Therefore, the proposed approach could generate SAR deception jamming templates with shadows that exhibit high similarity to the original image and high authenticity.

4. Discussion

To achieve fast and effective SAR deception jamming, it is necessary to perform sample augmentation on SAR deception jamming templates to generate a high-quality template dataset. Currently, the existing sample augmentation methods for SAR deception jamming templates face two main problems: low authenticity due to the absence of speckle noise and low similarity between the shadow regions in the generated templates and the original image. Therefore, this paper proposes a sample augmentation method based on GANs that can generate high-quality SAR deception jamming templates with shadows.

The proposed method adopts a pyramid-style multiscale structure as an overall framework to capture the internal information on SAR deception jamming templates with shadows. Each level of the pyramidal structure has a GAN responsible for generating and discriminating SAR deception jamming samples at that level. The generator uses residual dense blocks with attention mechanisms, multiscale modules, and region attention modules to enhance the network’s learning capability of shadow features. In addition, the speckle noise is introduced as input data to the generator, ensuring that the generated images contain the characteristic features of speckle noise. The discriminator adopts a patch-GAN approach with five fully convolutional layers used to assess the quality of generated images and compute the corresponding loss function, which improves the ability of both the generator and the discriminator iteratively to produce increasingly realistic images.

The effectiveness of the proposed method is demonstrated by comparing its results with those of the SinGAN method regarding two evaluation metrics, the ENL and SSIM values. The comparison results show that the proposed approach can achieve a 75.44% similarity between the generated and real images in speckle noise, which is significantly higher than the result of 51.38% achieved by the SinGAN method. Moreover, the SSIM values of the proposed method of the generated and real images for targets and shadows reach 96.27%, surpassing those of the SinGAN approach of 38.55%. This validates the effectiveness of the proposed approach in generating SAR deception jamming templates with shadows.

In future research, more complex inception modules, such as Inception V3, could be considered to improve the computational efficiency of the generator further.