SDF2Net: Shallow to Deep Feature Fusion Network for PolSAR Image Classification

1. Introduction

Polarimetric Synthetic Aperture Radar (PolSAR) images offer a specialized perspective in microwave remote sensing by capturing the polarization properties of radar waves, which provides detailed insights into Earth’s surface features, such as vegetation [1], water bodies [2], and man-made structures [3]. PolSAR images tackle the limitations of optical remote sensing images, which are susceptible to changes in illumination and weather conditions. Unlike optical systems, PolSAR is capable of functioning under all weather conditions and possesses a robust penetrating capability [4]. Moreover, PolSAR outperforms conventional SAR systems by comprehensively capturing extensive scattering information through four different modes. This enables the extraction of a wide range of target details, including scattering echo amplitude, phase, frequency characteristics, and polarization attributes [5]. Nowadays, PolSAR imagery has numerous applications in environmental monitoring [6], disaster management [7], military monitoring [8,9], crop prediction [10,11], and land cover classification [12].

PolSAR image classification is the task of classifying pixels into specific terrain categories. It involves analyzing the polarization properties of radar waves reflected from the Earth’s surface. This classification helps automate the identification and mapping of land cover for many applications. Conventional approaches to PolSAR image classification primarily rely on extracting distinctive features through the application of target decomposition theory [13]. The Krogager decomposition model, for instance, segregates the scattering matrix into three components, corresponding to helix, diplane, and sphere scattering mechanisms [14]. Another widely used method, the Freeman decomposition [15], dissects the polarimetric covariance matrix into double-bounce, surface, and canopy scattering components. Building upon the Freeman decomposition, [16] introduced a fourth scattering component, helix scattering power, which proves more advantageous in the classification of PolSAR images. Additionally, the Cloude decomposition [17] stands as a common algorithm for PolSAR image analysis. Despite the popularity of traditional classifiers like Support Vector Machine (SVM) [18] and decision trees [19] for PolSAR classification, challenges arise when dealing with PolSAR targets characterized by complex imaging mechanisms. This complexity often leads to inadequacies in representing these targets using conventional features, resulting in diminished classification accuracy.

Recently, Deep Learning (DL) technology has shown remarkable effectiveness in PolSAR image classification [20,21]. Specifically, Convolutional Neural Networks (CNNs) have exhibited impressive performance in this domain [4,22]. Chen et al. [23] successfully harnessed the roll-invariant features of PolSAR targets and the concealed attributes within the rotation domain to train a deep CNN model. This approach contributes to an enhanced classification performance. Zhou et al. [22] derived high-level features from the coherency matrix using a deep neural network comprising two convolutional and two fully connected layers, specifically tailored for the analysis of PolSAR images. Radman et al. [24] discussed the fusion of mini Graph Convolutional Network (miniGCN) and CNN for PolSAR image analysis. Spatial features from Pauli RGB and Yamaguchi were fed into CNN, and polarimetric features were utilized in miniGCN. The study aims to address the limitations of traditional PolSAR image classification methods and presents a dual-branch architecture using miniGCN and CNN models. Dong et al. [25] introduced two lightweight 3D-CNN architectures for fast PolSAR interpretation during testing. It applies two lightweight 3D convolution operations and global average pooling to reduce redundancy and computational complexity. The focus is on improving the fully connected layer, which occupies over 90% of model parameters in CNNs. The proposed architectures use spatial global average pooling to address the computational difficulties and over-fitting risks associated with a large number of parameters.

Phase information, a unique trait to SAR imagery, plays a vital role in various applications like object classification and recognition. Several research studies have explored the application of Complex-valued CNNs (CV-CNNs) for PolSAR data classification Owing to the difficulties encountered in this domain [26,27,28]. Unlike traditional CNNs, CV-CNNs handle complex-valued input data using complex-valued filters and activation functions, enabling effective capture of both phase and amplitude information. This capability is crucial for accurate PolSAR data classification.

Training CNNs usually demands a substantial volume of data, and acquiring an ample amount of high-quality ground reference data can be both costly and intricate. The scarcity of training samples often results in the generation of unreliable parameters and the risk of overfitting. Hence, it becomes crucial to attain high classification performance with a limited set of training data.

Shallow networks excel at capturing simple features but struggle when confronted with more intricate ones. On the contrary, deep neural network structures excel at extracting complex features. The integration of information from various depths in a network facilitates efficient learning, even when working with a small number of training samples. This combination greatly improves the network’s ability to unravel the complex details of the dataset.

In recent years, attention-based techniques have also been widely employed in PolSAR image classification to enhance the model’s ability to emphasize informative features and suppress less relevant ones by allocating more attention to the most important features rather than treating the entire input uniformly, which in turn improve the classification performance [29].

The main contributions of this paper are demonstrated as follows:

• A proposed novel model called Shallow to Deep Feature Fusion Network (SDF2Net), which incorporates feature extraction at various depths to enhance the classification performance of PolSAR images effectively.
• A feature-learning network with multiple depths and varying layers in each stream is developed. This design enables filters to simultaneously capture shallow, medium, and deep properties, enhancing the utilization of complex information in PolSAR. Experimental results indicate that the proposed model exhibits superior feature-learning capabilities compared to existing models in use.
• The model we propose surpasses current methods not only when dealing with a limited number of samples but also attains higher accuracy with an ample training dataset. This conclusion is drawn from statistical outcomes obtained through thorough trials on three PolSAR datasets, which will be further elaborated and discussed in the subsequent sections.

The article’s structure is outlined as follows: Section 2 provides an introduction to the related work, Section 3 provides a detailed presentation of the proposed network SDF2Net, Section 4 reports the experimental results and analysis on three PolSAR datasets. The article concludes with a summary of conclusions and an outlook on future work in Section 5.

2. Related Work

This section offers a concise examination of the relevant literature on CNNs and Attention mechanisms, with a particular focus on the squeeze and excitation module.

2.1. Overview CNNs

A standard CNN consists of an input layer, convolutional layers, activation layer, and an output layer. The initial input layer receives features from the image. Subsequently, convolutional layers employ convolutional kernels (illustrated in Figure 1) to extract the input features. These kernels operate by taking into account neighboring pixels, considering spatially correlated pixels within a close range (as depicted by the $3\times 3$ grid in Figure 1(a)). This approach enhances the network’s capacity to capture spatially related features. However, 2D-CNNs process each band individually and fail to extract the polarimetric information provided by PolSAR images. To better process PolSAR images, researchers often turn to more advanced architectures, such as Three-dimensional CNNs (3D-CNNs).

The application of convolution operations can be expanded into three dimensions, wherein calculations are conducted across all channels concurrently instead of handling each channel separately. Figure 1(b) visually depicts the concept of 3D convolution. In 3D convolution, the process encompasses height, width, and channels, making it a suitable approach for incorporating channel context. For an image represented as X with dimensions $N\times N\times B$ and a kernel denoted as K with dimensions $M\times M\times C$, the expression for 3D convolution at position $(x,y,z)$ is given by Equation 1.

$$F_{(x,y,z)}=\sum _{i=1}^M\sum _{j=1}^M\sum _{k=1}^C{K}_{(i,j,k)}{X}_{(x+i,y+j,z+k)}+b$$

(1)

Although 3D-CNNs can capture features in a better fashion when compared to 2D-CNNs [30], the complex nature of PolSAR images, represented by complex-valued data, adds an extra layer of complexity when considering the use of 3D-CNNs for their processing. Compared to traditional CNNs, CV-CNNs have proven their superiority in PolSAR classification tasks. To fully explore the complex values in PolSAR data, we utilized CV-3D-CNN. For an image $X=\Re \left(X\right)+i\Im \left(X\right)$ and a kernel $k=\Re \left(K\right)+i\Im \left(K\right)$, the result of the complex convolution Y can be expressed in Equation (2).

$$Y=X*K=\Re \left(X\right)*\Re \left(K\right)-\Im \left(X\right)*\Im \left(K\right)+i.(\Re \left(X\right)*\Im \left(K\right))+i.(\Im \left(X\right)*\Re \left(K\right))$$

(2)

where $\Re (.)$ and $\Im (.)$ represent the real and the imaginary parts of a CV number, respectively, i is the imaginary number $\sqrt{-1}$, and Y can be expressed as $\Re \left(Y\right)+i.\Im \left(Y\right)$. This is also illustrated in Figure 1(c).

2.2. Attention Mechanism

While CNNs have demonstrated promising outcomes in the classification of PolSAR images, there is limited research work regarding the enhancement of input identifiability when developing CNNs. To address this issue, the authors in [31] proposed the use of Squeeze-and-Excitation module (SE) [32] to enhance the performance of CNNs by selectively emphasizing crucial features in the input data. Also, SE improves channel interdependencies with almost no additional computational cost. Figure 2 shows the block diagram of SE.

In the SE process [34], given any transformation (e.g., convolution) $F_{tr}$ that maps the input X to feature maps $u_c$, where $u_c\in {\mathbb{R}}^{H\times W\times C}$, and $u_c$ represents the c-th $H\times W$ matrix in u, with the subscript c denoting the number of channels. In the Squeeze procedure $F_{sq}(.)$, global average pooling is employed to transform the input of $H\times W\times C$ into an output of $1\times 1\times C$. The squeeze function is defined in Equation (3).

$$z_c=F_{sq\left(u_c\right)}=\frac{1}{H\times W}\sum _{i=1}^H\sum _{j=1}^W{u}_c(i,j)$$

(3)

The equation reveals that the input data is transformed into a column vector during the squeeze process. The length of this vector corresponds to the number of channels. Subsequently, the excitation operation is conducted to autonomously discern the significance of each feature. This process amplifies features that exert a substantial influence on classification results while suppressing irrelevant features. The excitation function is expressed in Equation (4).

$$s=F_{ex(z,W)}=\sigma \left(g(z,W)\right)=\sigma \left(W_{2}ReLU\left(W_{1}z\right)\right),$$

(4)

where $\sigma$ represents the Sigmoid activation function, and $W_1\in {\mathbb{R}}^{\frac{C}{r}\times C}$ and $W_2\in {\mathbb{R}}^{C\times \frac{C}{r}}$ denote the two fully connected layers. Here, $W_1$ functions as the dimensionality reduction layer with a reduction ratio of r, while $W_2$ serves as the proportionally identical data-dimensionality increase layer. The variable z corresponds to the output from the preceding squeeze layer, and $W_{1}z$ signifies a fully connected layer operation. The Sigmoid function processes the final output of the excitation process, resulting in a value between zero and one. Following the squeeze and excitation steps, an attention vector s is obtained, which can be employed to adjust the channels of u.

3. Methodology

Within this section, a comprehensive description of the SDF2Net architecture is provided. Initially, the processing of polarimetric data from PolSAR images is showcased, followed by an exposition of the SDF2Net network architecture.

3.1. PolSAR Data Preprocessing

The construction of a polarimetric feature vector serves as a fundamental step in PolSAR imagery classification process. In PolSAR imagery, the description of each pixel is defined by a $2\times 2$ complex scattering matrix, denoted as S, as given in Equation (5) [35].

$$S=\left[\begin{array}{cc}{S}_{HH}& S_{HV}\\ S_{VH}& S_{VV}\end{array}\right],$$

(5)

where $S_{AB}(A,B\in H,V)$ represents the backscattering coefficient of the polarized electromagnetic wave in emitting A direction and receiving B direction. H and V represent the horizontal and vertical polarization channels, respectively. In the context of the data acquired by a monostatic PolSAR radar system, the assumption $S_{VH}$=$S_{HV}$ holds, indicating that the scattering matrix S is symmetric. This enables the simplification and reduction of the matrix to the polarization scattering vector $\overrightarrow{k}$. Employing the Pauli decomposition method, the expression for t$\overrightarrow{k}$ can be represented as [5].

$$\overrightarrow{k}=\frac{1}{\sqrt{2}}{[S_{HH}+S_{VV},S_{HH}-S_{VV},2S_{HV}]}^T.$$

(6)

In general, multi-look processing is essential for PolSAR data. Following the processing step, the obtained coherency matrix serves as the most commonly used representation for PolSAR data, as given in Equation (7) [5].

$$T=\frac{1}{n}\sum _{j=1}^n\overrightarrow{k_j}{\overrightarrow{k_j}}^H=\left[\begin{array}{ccc}{T}_{11}& T_{12}& T_{13}\\ T_{21}& T_{22}& T_{23}\\ T_{31}& T_{32}& T_{33}\end{array}\right],$$

(7)

where the operator ${(.)}^H$ stands for complex conjugate operation and n is the number of looks. It is worth mentioning that T is a Hermitian matrix with real-valued elements on the diagonal and complex-valued elements off-diagonal. As a result, the three real-valued and three complex-valued elements of the upper triangle of the coherency matrix (i.e. $T11$, $T12$, $T13$, $T22$, $T23$, $T33$) are used as the input features of the models.

Data pre-processing is an essential step for achieving higher classification accuracy. Initially, the mean $\overline{T}$ and standard deviation $T_{std}$ of each channel are computed. Subsequently, normalization is applied to each channel, ensuring optimal data preparation for classification. By taking $T_{11}$ as an example, each channel will be normalized as in Equation (8).

$$T_{11}=\frac{T_{11}-{\overline{T}}_{11}}{T_{{11}_{std}}},$$

(8)

3.2. Feature Extraction Using CV-3D-CNN

As explained in Section 2.1, CV-3D-CNNs are more suitable to process PolSAR data due to their complex nature. Each layer of CV-3D-CNN is followed by a ReLU activation function, and since the output features are complex, we propose the use of Complex Valued ReLu ($\mathbb{C}$ReLU(.)). It is obtained by applying the well-known ReLU(.) to both the real and imaginary parts separately, such that $\mathbb{C}$ReLU(x) = ReLU(ℜ(x)) + i.ReLU(ℑ(x)).

3.3. Architecture of the Proposed SDF2Net

Currently, the predominant approach in PolSAR classification tasks involves the utilization of 2D-CNN architectures. While 2D-CNNs effectively capture spatial information, they fall short in exploiting the intricate interchannel dependencies inherent in PolSAR images. On the other hand, a 3D-CNN exhibits superior feature extraction capabilities when compared to 2D-CNN architecture [30].

Figure 3 shows the comprehensive framework of the complete process of the proposed methodology. The model processes the data through a three-branch network, extracting features at different levels (shallow, medium, and deep), which are later concatenated. The concatenated features then pass through the Attention block to enhance the channel dependencies. The flattening layer is employed to transform the concatenated features into a one-dimensional vector. For the final classification, two fully-connected layers are employed, incorporating dropout to mitigate overfitting and a softmax layer for generating the final prediction. The detailed distribution of parameters of each layer is shown in Figure 3. The first branch incorporates a single layer of complex CV-3D-CNN with 16 filters, employing a 3×3×3 kernel size to capture shallow features. In contrast, the second branch focuses on medium features with two CV-3D-CNN layers, each maintaining a 3×3×3 kernel size. Meanwhile, the third branch is tailored to extract deep features through the utilization of three layers of CV-3D-CNN, employing 16 filters at each layer, and configuring the kernel size of each filter as 3×3×3. Due to the information loss caused by the pooling layer, it has not been utilized in this network architecture.

3.4. Loss Function

The training of the proposed SDF2Net involves the computation of Cross-entropy (CE) loss on the training samples. A softmax classifier is utilized to produce the predicted probability matrix for the samples, as shown in Equation (9).

$${\widehat{y}}_l^m=Softmax\left(\right|{\mathbf{x}}_{out}\left|\right),$$

(9)

where ${\mathbf{x}}_{out}$ is the output of the last fully connected layer, and $|.|$ is the magnitude operator. It is worth noting that ${\mathbf{x}}_{out}$ is complex, while the value of $|{\mathbf{x}}_{out}|=\sqrt{{(\Re \left({\mathbf{x}}_{out}\right))}^2+{(\Im \left({\mathbf{x}}_{out}\right))}^2}$ is real, and hence the value of ${\widehat{y}}_l^m$ will also be real. Subsequently, the presentation of the loss $Los{s}_{CE}$ is depicted as given in Equation (10).

$$Los{s}_{CE}=-\sum _{m=1}^M\sum _{l=1}^L{y}_l^{m}log\left({\widehat{y}}_l^m\right),$$

(10)

where $y_l^m$ and ${\widehat{y}}_l^m$ are the reference and predicted labels, respectively, and L and M are the land cover categories and the overall number of small batch samples, respectively.

4. Experiments and Results

In this section, three prevalent datasets in PolSAR classification, namely Flevoland, San Francisco, and Oberpfaffenhofen have been used in our experiments to validate the effectiveness of the proposed approach under various experimental configurations of the suggested framework. For a comprehensive demonstration, both visualized classification results and quantitative performance metrics are reported, in addition to comparisons with state-of-the-art approaches.

4.1. Polarimetric SAR Datasets

• Flevoland Dataset: The dataset consists of L-band four-look PolSAR data with dimensions 750 × 1024 pixels with 12 meters. It was acquired by the NASA/JPL AIRSAR system on August 16, 1989 for Flevoland area in the Netherland. It has 15 distinct classes: stem beans, peas, forest, lucerne, wheat, beet, potatoes, bare soil, grass, rapeseed, barley, wheat2, wheat3, water, and buildings [36]. Figure 4 shows the Pauli pseudo-color image (Left) and ground truth map (right). Table 1 shows the number of pixels per each class in the dataset.
• San Francisco Dataset: The San Francisco dataset, obtained from the L-band AIRSAR, covers the San Francisco area in 1989. The image size is 900 × 1024 pixels and has a spatial resolution of 10 meters. It comprises of five categorized terrain classes: mountain, water, urban, vegetation and bare soil [37]. Figure 5 shows a colored image formed by PauliRGB decomposition (left), and the reference class map (right). Table 2 shows the number of pixels per each class in the data set.
• Oberpfaffenhofen Dataset: The Oberpfaffenhofen dataset is captured by L-band ESAR sensor in 2002, encompassing the area of Oberpfaffenhofen in Germany. It includes a PolSAR image with dimensions of 1300 × 1200 pixels and a spatial resolution of 3 meters, annotated with three land cover classes: Built-up Areas, Wood Land, and Open Areas [38]. Figure 6 shows the PauliRGB composite (left) and the reference class map (right). Table 3 shows the number of pixels per each class in the data set.

4.2. Evaluation Metrics

Assessing the effectiveness of classification results involves comparing predicted class maps with the provided reference or ground truth data. Depending solely on visual inspection for confirming pixel accuracy in the image is subjective and may lack comprehensiveness. Therefore, adopting a quantitative evaluation approach is more reliable. In this context, three metrics are employed: Overall Accuracy (OA), Average Accuracy (AA), and the Kappa score (k).

OA calculates the ratio of correctly assigned pixels to the total number of samples. AA computes the mean classification accuracy across all categories or classes. The Kappa score assesses the agreement between the predicted classified map and the ground truth, with values ranging from 0 to 1. A value of 1 indicates perfect agreement, while 0 suggests complete disagreement. Typically, a Kappa value equal to or greater than 0.80 signifies substantial agreement, whereas a value below 0.4 indicates poor model’s performance [39].

4.3. Experimental Configuration

All the tests were conducted utilizing Python 3.9 compiler and TensorFlow 2.10.0 framework. Adam optimizer is adopted with a learning rate of $1\times {10}^{-3}$, the batch size is 64, and the training epoch is set to 250. In the course of model training, an early stopping strategy is implemented. Specifically, if there is no improvement in the model’s performance over a consecutive span of 10 epochs, the training process halts, and the model reverts to its optimal weights. The network configuration of the proposed model using Flevoland dataset is shown in Table 4. The number of samples used for training the model is set to $1\%$ for all three datasets to guarantee a fair comparison.

4.4. Experimental Results

In this section, we assess the classification performance of the suggested model both quantitatively and qualitatively, utilizing the three previously mentioned datasets: Flevoland, San Francisco, and Oberpfaffenhofen. To mitigate the impact of sample selection randomness on classification outcomes, the experiments were iterated 10 times, and the final result is presented as the average value of these repetitions. Furthermore, detailed classification outcomes for each category are provided.

4.4.1. Determining Optimal Window Size

In this part, we investigate the effect of the spatial characteristics of the aforementioned diverse datasets on the proposed model’s capability to categorize PolSAR data and identify the optimal window size for each dataset. The window size signifies the extent of the retrieved spatial information from the 3D patch that is utilized for assigning a label to the extracted patch. A larger window may encompass a significant amount of neighborhood data, potentially containing information from other classes, thereby impeding the feature extraction process. Conversely, if the chosen window is too small, the model’s capacity to extract features will be compromised by a notable loss of spatial information. This study validates the influence of window size on model performance across the three aforementioned datasets. In the experiment, the spatial sizes were configured as {$5\times 5$, $7\times 7$, $9\times 9$, $11\times 11$, $13\times 13$, $15\times 15$, and $17\times 17$}. It is evident from Figure 7 that for the Flevoland, San Francisco, and Oberpfaffenhofen datasets, the suitable window size for the proposed model is $13\times 13$, $15\times 15$ and $13\times 13$, respectively.

4.4.2. Ablation Study

This section is divided into two parts for the ablation study. The initial part focuses on assessing the influence of diverse combinations of network components. We conducted comprehensive experiments on the Flevoland dataset, as outlined in Table 5. The results demonstrate that our proposed fusion technique outperforms other combinations or fusion methods, such as Shallow (S), Medium (M), Deep (D), Shallow and Medium (S+M), Shallow and Deep (S+D), Medium and Deep (M+D), in terms of AA, OA, and Kappa metrics.

The second part examines the impact of incorporating the attention mechanism. Initially, we experimented with the model without any attention mechanism, followed by placing attention at each stream before feature fusion, and finally applying attention after feature fusion. Table 6 presents the outcomes for various attention locations on the same dataset. It is evident that the optimal location is immediately after the feature fusion step.

4.4.3. Comparison with Other Methods

Several techniques have been chosen for comparison with the SDF2Net model, such as SVM [40], 2D-CVNN [41], 3D-CVNN [42], Wavelet CNN [43], and our previously proposed method in [44], namely CV-CNN-SE. The specific experimental configurations for these methods are outlined below.

• SVM: The SVM employs the Radial Basis Function (RBF) kernel, with the parameter $\gamma$ set to 0.001 to regulate the local scope of the RBF kernel.
• 2D-CVNN: The model consists of two Complex-Valued CNN layers, with 6 and 12 kernels of size $3\times 3$ in each layer, and two fully connected layers. The input patch size is specified as $12\times 12$ in [41].
• Wavelet CNN: The proposed model utilizes the Haar wavelet transform for feature extraction to improve the classification accuracy of PolSAR imagery. It consists of three branches, each utilizing different concepts and advantages of CNNs. The model parameters were configured based on the values given in [43]
• CV-CNN-SE: This model utilizes the use of 2D-CVNNs at different scales to extract features from PolSAR data. Extracted features are then fused and passed to SE block to enhance classification performance.
• 3D-CVNN: In this model, four Complex-Valued convolutional layers, with 16, 16, 32 and 32 kernels of size $3\times 3\times 3$ in each layer, and one fully connected layer. The input patch size is specified as $12\times 12$ in [42].

As mentioned earlier, the experiments were conducted through 10 iterations, and the outcome is presented as the mean value of these repetitions. The quantitative assessments of these compared methods are presented in Table 7, Table 8 and Table 9, with the best results in each table highlighted in bold.

Based on the findings presented in Table 7, Table 8 and Table 9, it is evident that the proposed SDF2Net model surpasses other methods in performance. Examining the datasets utilized in this study, the Flevoland dataset is composed of 207,832 labeled samples, with 1% (2,078 samples) reserved for training, spanning across 15 classes. In contrast, the San Francisco dataset boasts a larger pool of labeled samples, with 8,023 allocated for training, explaining the higher classification performance observed in San Francisco.

The results on Flevoland dataset are reported in Table 7, SVM demonstrated the least OA, primarily because it heavily depends on 1-dimensional information. Additionally, classes with a low number of training samples, such as Grass, Bare Soil, Wheat 2, Barley, and Buildings, were hardly detected by SVM. This suggests that SVM is an unsuitable option for datasets with a limited number of training samples, a limitation evident in the quantitative metrics associated with each class. The metrics of the 2D-CVNN exhibited an enhancement, approximately 10% greater than that of SVM. Unlike SVM, the 2D-CVNN utilizes 2D filters for spatial information extraction, resulting in a moderate increase in accuracy. Nevertheless, certain classes with a limited number of training samples, like Grass and Bare Soil, recorded lower accuracy metrics, influencing the overall performance of the model. The Wavelet CNN achieved substantial improvement to the accuracy with 91.73%, utilizing Wavelet decomposition for feature extraction, thereby enhancing classification accuracy. CV-CNN-SE, employing three parallel branches of two-dimensional kernels, demonstrated an improvement over Wavelet CNN. The 3D-CVNN, using 3D filters for three-dimensional information extraction, did not surpass CV-CNN-SE in accuracy, but did outperform other methods used in this research. Our proposed SDF2Net method outperformed CV-CNN-SE by approximately 1.5% in OA and outperformed other methods in seven categories, as detailed in the Table 7. For a visual representation, Figure 8 illustrates the classification results for the six approaches alongside the reference map. It is clearly shown from Figure 8(c) that SVM had numerous incorrectly assigned pixels, while the classification map generated by SDF2Net closely aligned with the reference map, showcasing superior performance.

Regarding the San Francisco dataset, Table 8 displays the classification outcomes obtained from various models. In contrast to the Flevoland dataset, this dataset comprises only five target categories. SVM yields relatively poor classification results, suggesting that the original polarimetric features lack effective discrimination. The models 2D-CVNN and CV-CNN-SE show unsatisfactory performance on this dataset, particularly in the Bare Soil category. Conversely, the Wavelet CNN and 3D-CVNN methods prove more adept at handling the intricate polarimetric features, especially in the Bare Soil category. SDF2Net achieves the highest OA at 97.13%, Average Accuracy (AA) at 91.31%, and Kappa coefficient at 95.50%. In Figure 9, the classification maps generated by the methods employed in this research are presented alongside the reference map. The classification maps produced by our method closely align with the ground truth map, especially when observing the Urban class (Blue).

The classification results of Oberpfaffenhofen dataset are shown in Table 9. SDF2Net once again achieves the best performance in terms of OA, AA and Kappa metrics when compared to the other methods. The model also achieves the highest accuracy for all classes. Figure 10 shows the classification results of the methods used in this research, which asserts that our proposed method produces the closest classification map to the ground truth one.

According to the experimental findings, the proposed SDF2Net model, outperforms other classification methods. SVM exhibits the poorest classification performance due to its reliance on 1-dimensional features, leading to a loss of spatial information. In contrast, 2D-CVNN, Wavelet CNN, and CV-CNN-SE consider spatial information, resulting in enhanced OA compared to SVM. The 3D-CVNN method extracts hierarchical features in both spatial and scattering dimensions through 3-D Complex-Valued convolutions, effectively capturing physical properties from polarimetric adjacent resolution cells. Our proposed SDF2Net demonstrates superior results across the three datasets utilized in this research, leveraging the strengths of each branch to extract features at various levels and thereby enhancing the overall classification performance.

4.5. Post-Processing with Median Filtering

To further boost the classification accuracy, an additional spatial post-processing strategy employs a $3\times 3$ median filter to eliminate isolated misclassified pixels within the assigned class. The underlying assumption is that information classes tend to occupy spatial regions of relatively uniform characteristics, typically larger than a few pixels [45]. The application of median filtering yields a more refined version of the class map with smoother transitions. To showcase the effectiveness of this process, the smoothed classification map is compared with the reference map through the confusion matrix. For illustrative purposes, the Oberpfaffenhofen dataset will be utilized as an exemplar.

Figure 11 displays classification maps for Oberpfaffenhofen. In Figure 11(a), the map is generated using the proposed model. Figure 11(b) depicts the same map after smoothing with a median filter. Figure 11(c) represents the reference data classification map. Table 10 exhibits the confusion matrix comparing the generated classification map with the reference, while Table 11 shows the confusion matrix for the filtered generated classification map in comparison to the reference one.

The diagonal of the confusion matrix demonstrates the agreement between the predicted class map and the reference class map, while discrepancies are indicated by the non-diagonal elements. Table 10 and Table 11 display a decline in the values of these non-diagonal components, signaling a decrease in the number of misclassified pixels. As a result, there is an enhancement in OA. This improvement is noticeable, particularly in the Build-Up Areas class (depicted in red). Figure 11(a) highlights numerous pixels initially classified as Open Areas. However, a substantial correction is observed after applying the median filter. This correction is corroborated by Table 10 and Table 11, revealing an initial misclassification of 22,640 Build-Up Areas pixels as Open Areas, which reduces to 15,780 after the application of the median filter. In fact, the reduction appears on all non-diagonal values. Table 12 provides a summary of the enhancement achieved by applying the median filter to the resulting class map of each dataset.

4.6. Performance of Different Models at Different Percentages of Training Data

The model’s performance can be effectively assessed by examining the classification accuracy across different percentages of training data. We randomly chose 1%, 2%, 3%, 4%, and 5% of labeled samples for training, using the remaining samples for testing the model’s performance. The classification results for each dataset are depicted in Figure 12. It is evident that, across all methods employed in this study, the classification accuracy shows improvement with an increase in the number of training samples. Notably, the proposed model consistently outperforms others across all proportions of training samples on the three datasets.

5. Conclusion

This paper introduces a novel model called SDF2Net designed for PolSAR image classification. The model addresses existing challenges in PolSAR classification by incorporating a three-branch feature fusion structure and optimizing the creation of a complex-valued CNN-based model. The data is processed through three branches of CV-3D-CNN, and the generated features from each branch are combined. Then, the fused features undergo enhancement through an attention SE block to improve model performance, followed by the application of fully connected and dropout layers to yield the final classification result. Experimental results demonstrate the effectiveness of the proposed model in terms of OA, AA, and Kappa metrics. Notably, even with a limited number of training samples, the model produces classification results almost identical to the reference or ground truth data.

As part of future work, we plan to explore a lightweight architecture with fewer training parameters to reduce computational complexity without compromising model performance. Furthermore, our objective is to explore the potential of integrating Real-Valued CNNs and CV-CNNs to enhance classification accuracy across various datasets.