Globally Conditioned Conditional FLOW (GCC-FLOW) for Sea Clutter Data Augmentation

1. Introduction

Sea clutter, backscatter from the ocean surface, is a challenge for radars to detect and classify objects at sea, and therefore it is important for radars to deal with sea clutter in various sea conditions for accurate detection and performance evaluation [1]. However, due to the expensive and time-consuming nature of acquiring the desired sea clutter data, intensive studies have been focused on mathematical modeling of sea clutter data based on the random process model to generate abundant sea clutter data in a short time [2,3,4,5,6].

Several distributions have been considered for modeling sea clutter amplitude. Early models adopted the Rayleigh distribution to characterize sea clutter [7], but measurements have shown deviations from this model and other alternatives have emerged such as log-normal [8], K-distribution [9], Weibull [10]. Among them, K-distribution is most widely used due to its accuracy [11,12] and its modified versions, KA [13] and KK [14] distributions, have been proposed to improve the accuracy on tail distribution at low grazing angles [15], To reduce the model complexity of KA, KK distributions, relatively simple Pareto distribution [16,17] is considered to fit both high and low grazing angle. In addition, the Memoryless Nonlinear Transform (MNLT) method [18], which allows the generation of samples with a specific correlation function, was used to model the spatiotemporal correlation. Although these parametric methods based on mathematical modeling have shown success in finding an appropriate distribution to fit the sea clutter data under ideal assumptions, they are still insufficient to model the full range of geometry and sea conditions in practice, especially for the time correlation [19].

Recently, the use of deep learning for data augmentation has become widespread in many fields. The advantages of deep learning approaches in data augmentation are that data with approximately the same statistical properties as sample data, such as distribution and correlation, can be generated without explicit mathematical models, and several generative techniques can be used to generate unlearned data with interpolated features [20,21]. There are a few attempts to generate sea clutter data using deep learning [22,23]. In [22], a Generative Adversarial Network (GAN) with two generators that independently generate the real and imaginary components of sea clutter data is proposed, and in [23], a so-called Auxiliary Classifier GAN (AC-VAEGAN) is proposed to generate one-dimensional (1-D) sea-land clutter amplitude data to deal with imbalanced data sets in classification tasks.

These existing methods are based on one-dimensional data generation, and therefore multi-dimensional structures such as real-imaginary dependency are difficult to implement. More importantly, since GAN independently generates data block by block, the length of the coherently generated data is limited to the length of the training data. However, the non-blockwise generative algorithms based on RNN and its variants tend to be inaccurate as the length of the sample sequence increases [24,25]. Therefore, it is desirable to design a deep generative algorithm capable of generating arbitrarily long multidimensional data from relatively short length of training data.

In this paper, it is proposed a novel sea clutter data augmentation scheme based on conditional FLOW [26], a variant of FLOW [27] which is one of the most robust deep generative methods. The proposed algorithm, named globally conditioned conditional flow (GCC-FLOW ), is a modified version of conditional flow that introduces a global condition variable to reflect the sea state condition. The autoregressive use of GCC-FLOW is able to generate arbitrarily long length of sea clutter sequences with the approximately the same statistical characteristics of the given sea clutter data. Furthermore, by using the global condition variable of the GCC-FLOW for different sea states, the proposed method is capable of generating diverse sea clutter data with the interpolated sea state features.

The rest of this paper is organized as follows: In Section 2, the architecture of the proposed GCC-FLOW is described in detail. In Section 3, the training sea clutter data, hyperparameter settings for the GCC-FLOW to generate sea clutter data are described. In Section 4, the experimental results are presented in terms of ensemble distribution and time correlations of the generated sea clutter compared to those of the training data. Finally, the conclusion is drawn in Section 5.

2. Methods

2.1. Autoregressive Data Generation Using GCC-FLOW

We assume that the sea clutter data is a stationary random process as assumed in most of the literature [2,3,4,5,28]. Existing generative methods based on the GAN sample sea clutter sequence $\mathbf{x}=[x_1,\cdots ,x_L]$ following the joint distribution $p^{*}\left(\mathbf{x}\right)$ learned from the train data. In order to increase L, the data size should be increased. The concatenation of two data $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$ by generated GAN cannot be used as a sequence of the sampled random process of length $2L$, since $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$ are independent due to the block-by-block data generation of GAN.

Figure 1 illustrates the autoregressive use of the proposed Globally Conditioned Conditional FLOW (GCC-FLOW ) in order to generate long data sequences. GCC-FLOW generates a sequence ${\mathbf{x}}_o=[x_{o1},\cdots ,x_{oL}]$ given a conditional sequence ${\mathbf{x}}_c=[x_{c1},\cdots ,x_{cL}]$ following a conditional distribution $p^{*}\left({\mathbf{x}}_o\right|{\mathbf{x}}_c;\alpha )$ parameterized by a global conditional variable $\alpha \in [0,1]$. The conditional pdf $p^{*}\left({\mathbf{x}}_o\right|{\mathbf{x}}_c;\alpha )$ is learned by GCC-FLOW to approximate the conditional distribution of the successive data $[x_{L+1+k},\cdots x_{2L+k}]$ given $[x_{1+k},\cdots x_{L+k}]$ for $k=0\cdots$. Given the initial sequence ${\mathbf{x}}_c^{\left(0\right)}$ from the data, ${\mathbf{x}}_o^{\left(0\right)}$ is generated by the GCC-FLOW and ${\mathbf{x}}_o^{\left(k\right)}$ is used as the condition sequence for ${\mathbf{x}}_o^{(k+1)}$ for $k=0,1,\cdots$. The joint distribution of the concatenated sample sequence $\mathbf{x}=[{\mathbf{x}}_o^{\left(0\right)},{\mathbf{x}}_o^{\left(1\right)},\cdots ,{\mathbf{x}}_o^{\left(T\right)}]$ satisfies

$$p({\mathbf{x}}_o^{\left(0\right)},{\mathbf{x}}_o^{\left(1\right)},\cdots ,{\mathbf{x}}_o^{\left(T\right)})=\prod _{K=0}^{T}p\left({\mathbf{x}}_o^{\left(k\right)}\right|{\mathbf{x}}_o^{(k-1)},\cdots ,{\mathbf{x}}_o^{\left(1\right)})$$

(1)

$$\begin{array}{cc}\hfill & \approx \prod _{K=0}^{T}p\left({\mathbf{x}}_o^{\left(k\right)}\right|{\mathbf{x}}_o^{(k-1)}),\hfill \end{array}$$

(2)

assuming the block Markov property,

$$\begin{array}{c}\hfill p(x_{N+L+1},\cdots ,x_{N+2L}|x_0,\cdots ,x_{N+L})\approx p(x_{N+L+1},\cdots ,x_{N+2L}|x_{N+1},\cdots x_{N+L}).\end{array}$$

(3)

Since each conditional distribution $p\left({\mathbf{x}}_o^{\left(k\right)}\right|{\mathbf{x}}_o^{(k-1)})$ is approximated by $p^{*}\left({\mathbf{x}}_o^{\left(k\right)}\right|{\mathbf{x}}_o^{(k-1)})$ learned from GCC-FLOW , the joint distribution $\mathbf{x}=[{\mathbf{x}}_o^{\left(0\right)},{\mathbf{x}}_o^{\left(1\right)},\cdots ,{\mathbf{x}}_o^{\left(T\right)}]$ generated by the GCC-FLOW autoregressively approximates the true data distribution as desired.

2.2. Architecture of GCC-FLOW

The structure of GCC-FLOW is similar to conditional-FLOW [26] except that the global condition variable $\alpha$ is fed into the coupling layer in the FLOW step as shown in Figure 2. GCC-FLOW consists of a pair of FLOW networks, the condition part and the output part, which basically has two identically independent deep network structures that are used repeatedly in the multi-scale frame; $f_{{\mathit{\varphi }}_{a,n,m}}$ and $g_{{\mathit{\psi }}_{a,n,m}}$, parameterized by the network weights ${\mathit{\varphi }}_{a,n,m}$ and ${\mathit{\psi }}_{a,n,m},$ respectively, where $m=1,\cdots ,M$ is the index for multi-scale stages and $n=1,\cdots ,N$ is the index for flow steps in a multi-scale stage, and $a=c$ for the condition part and $a=o$ for the output part.

In each flow step, ${\mathbf{x}}_c$ and ${\mathbf{x}}_o$ are updated as the following

$$\begin{array}{cc}\hfill {\overline{\mathbf{x}}}_c& =f_{{\mathit{\varphi }}_{c,n,m}}\left({\mathbf{x}}_c^n\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\overline{\mathbf{x}}}_o& =f_{{\mathit{\varphi }}_{o,n,m}}\left({\mathbf{x}}_o^n\right)\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill \left({\mathbf{s}}_c,{\mathbf{b}}_c\right)& =g_{{\mathit{\varphi }}_{c,n,m}}\left({\overline{\mathbf{x}}}_c^{up},\alpha \right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill \left({\mathbf{s}}_o,{\mathbf{b}}_o\right)& =g_{{\mathit{\varphi }}_{o,n,n}}\left({\overline{\mathbf{x}}}_o^{up},{\overline{\mathbf{x}}}_c^{up},\alpha \right)\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\mathbf{x}}_c^{n+1}& =\left[{\overline{\mathbf{x}}}_c^{up},{\mathbf{s}}_c\odot {\overline{\mathbf{x}}}_c^{down}+{\mathbf{b}}_c\right]\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill {\mathbf{x}}_o^{n+1}& =\left[{\overline{\mathbf{x}}}_o^{up},{\mathbf{s}}_o\odot {\overline{\mathbf{x}}}_o^{down}+{\mathbf{b}}_o\right]\hfill \end{array}$$

(9)

where ⊙ is element-wise multiplication and $({\overline{\mathbf{x}}}_c^{up}$, ${\overline{\mathbf{x}}}_c^{down}),({\overline{\mathbf{x}}}_o^{up},{\overline{\mathbf{x}}}_o^{down})$ are the result of halving the channel of ${\overline{\mathbf{x}}}_c,{\overline{\mathbf{x}}}_o$, respectively.

The structure of the coupling layer function $g_c$ and the conditional coupling layer function $g_o$ is described in Figure 3. The input tensors ${\overline{\mathbf{x}}}_c^{up}$ and ${\overline{\mathbf{x}}}_o^{up}$, both of size $c\times \ell$, are converted to the $C_{coupling}\times \ell$ tensors, denoted by ${\mathbf{y}}_c$, ${\mathbf{y}}_{co}$, and ${\mathbf{y}}_o$, by convolution networks. The global variable $\alpha \in [0,1]$ is multiplied by a linear weight ${\mathbf{v}}_a$ of size $1\times \ell$ ($a\in \left\{c,o\right\}$). These two tensors, ${\mathbf{v}}_o$ and ${\mathbf{v}}_c$, are properly scaled and added channel-wise via actnorm. For example, in the coupling layer part,

$$\begin{array}{c}\hfill {{\mathbf{w}}_c}_i=a_i{{\mathbf{y}}_c}_i+b_i+a_0{\mathbf{v}}_c\alpha +b_0\mathrm{for}i=1,\cdots ,C_{coupling},\end{array}$$

(10)

where ${\mathbf{w}}_c$ denote the intermediate output tensor of the size $C_{coupling}\times \ell$ and ${\mathbf{w}}_{ci}$ is the i-th channel vector of ${\mathbf{w}}_c$ and $\left\{a_i,b_i|i=1,\cdots ,C_{coupling}\right\}$ are the actnorm parameters for ${\mathbf{v}}_c$ and $a_0,b_0$ are the actnorm parameters for ${\mathbf{v}}_a\alpha$. Finally, ${\mathbf{w}}_c$ is processed through a convolution network to restore the original size $c$ and split it in half channel-wise to produce ${\mathbf{s}}_c$ and ${\mathbf{b}}_c$.

At each multi-scale stage, the output ${\mathbf{x}}_a^{mN}$ is split in half channel-wise, and the first half becomes the m-th stage output, ${\mathbf{z}}_a^m$, and the remaining half is squeezed, i.e., the length is reduced by half and the channel size is doubled, and fed into the next flow step. Note that in this split-squeeze setting the length of ${\mathbf{z}}_c^k$ at each multi-scale stage becomes $2^{-k}L$, where L is the input length of GCC-FLOW .

In the training phase, networks are trained for ${\mathbf{z}}_c^k$ to be i.i.d. Gaussian, ${\mathbf{z}}_a^k\sim \mathcal{N}(0,I_{L{2}^{-k}})$, and in the generation phase the sampled ${\mathbf{z}}_a^k$ from i.i.d. Gaussian $\mathcal{N}(0,I_{L{2}^{-k}})$ are used as input to the inverse network of GCC-FLOW , which is canonically given by the structure of FLOW [27].

3. Experiment Settings

3.1. Data Set

The sea clutter data used in the experiments come from the Dartmouth and Grimbsby databases collected by the IPIX radar [29]. The following two data datasets are used:

1)

High sea state data: The 3rd range bin of the data file #269 at Dartmouth labeled as high. The average wave height is 1.8m (max 2.9m)

2)

Low sea state data: The 5th range bin of the data file #287 at Dartmouth labeled as low. The average wave height is 0.8m (max 1.3m)

All the data samples are complex-valued and the length of the high sea state data and the low sea state data is $131,072$. The detailed specification of the data, such as resolution and sampling rate, can be found in [29].

3.2. GCC-FLOW Settings

The length of ${\mathbf{x}}_o$ and ${\mathbf{x}}_c$ is set to 512, i.e. $L=512$, and the height of the clutter data and is set to 2, i.e., $C=2$, since the data represent single range bin complex data. The number of multiscale stages of GCC-FLOW is set to 4, i.e., $M=4$ and in a multiscale stage the number of FLOW iterations is set to $N=4$. The channel size in the coupling layer is $C_{coup}=32$. The model is trained on NVIDIA RTX 3090 with a learning rate of $3\times {10}^{-4}$, batch size of 4, and trained on 512 randomly selected sequences with length of 1024 by randomly sampled starting points.

4. Experiment Results

4.1. Sea Clutter Augmentation Using GCC-FLOW

Figure 4 compares the magnitude histogram of the generated sea clutter data by the GCC-FLOW trained on the high sea state data and on the low sea state data, respectively, with the magnitude histograms of the original data. Figure 5 compares the magnitude autocorrelation of the generated sea clutter data by the GCC-FLOW trained on the high sea state data and on the low sea state data, respectively, with that of the original high sea state and the low sea sate data, where the magnitude autocorrelation is defined as

$$\begin{array}{c}\hfill \rho \left(s\right)=\frac{{\sum }_{l=1}^{L-s}(|x_l|-\overline{\mathbf{x}}\left)\right(|x_{l+s}|-\overline{\mathbf{x}})}{{\sum }_{l=1}^L{\left(|x_l|-\overline{\mathbf{x}}\right)}^2},\end{array}$$

(11)

where $\overline{\mathbf{x}}=\frac{1}{L}{\sum }_{l=1}^L\left|x_l\right|$, $s=0,\cdots ,L$, and L is set to 512.

The GCC-FLOW is trained using 256 blocks of 512 consecutive samples (131,072 training samples in total) with the global condition variable $\alpha$ set to 0. GCC-FLOW generates 262,144 consecutive samples and the first 131,072 samples are used to plot the histogram and the autocorrelation function.

To confirm that the generated samples have the same statistical properties at each time point, consecutive $10,000$ samples are taken from the beginning ($1\sim 10,000$), from the middle ($13,0001\sim 14,000$), and from the end ($25,2145\sim 26,2144$). Figure 6 and Figure 7 show the magnitude histogram and the magnitude autocorrelation of the generated sea clutter data using GCC-FLOW trained from the high sea state data and the low sea state data, respectively, at the three time points. Note that the characteristic of the high sea data (concentrated distribution and high frequency fluctuation in correlation) and the characteristic of the low sea data (spread distribution and low frequency fluctuation in correlation) are preserved in all time points.

4.2. Sea Clutter generation using GCC-FLOW with global condition

Using the global condition variable $\alpha$, a single GCC-FLOW can be trained on two different datasets; high sea state data with $\alpha =1$ and low sea state data with $\alpha =0$. Figure 8 and Figure 9 plot the magnitude histogram and magnitude autocorrelation of the generated sea clutter data using GCC-FLOW with $\alpha =1$ and $\alpha =0$, respectively, at the three different time points (beginning, middle, and end) from the $262,144$ samples.

The benefit of this single mode training on two different datasets is the feature interpolation, i.e., by adjusting $\alpha$ between 0 and 1, the GCC-FLOW can generate data with interpolated statistical characteristics between the high and low sea state. Figure 10 shows the magnitude histogram of generated data for $\alpha =0.2,0.5,0.8$ at three different time points (beginning, middle, and end) of $262,144$ samples. Figure 11 presents the autocorrelation of generated data for $\alpha =0,2,0.5,0.8$ at three different time points (beginning, middle, and end) of $262,144$ samples. In both figures, as $\alpha$ increases from 0 to 1 the characteristics of magnitude distribution and autocorrelation transforms from the low sea state characteristics to the high sea state characteristics. This suggests that by using GCC-FLOW with a global condition variable tuned to sea state, various sea state sea clutter can be generated without the need for the dedicated training datasets for different sea states.

5. Conclusions

A novel work, Globally Conditioned Conditional FLOW (GCC-FLOW ), has been proposed for sea clutter data augmentation due to the expensive and time-consuming nature of acquiring the desired sea clutter data. Experimental results show that GCC-FLOW successfully generates sequences of arbitrary length with the same statistical properties of the given sea clutter data, in contrast to the existing block-by-block methods. Moreover, by introducing a global state variable, the proposed GCC-FLOW can generate sea clutter with different sea state by feature interpolation without requiring the training data with a target feature.