Phase Congruency-Guided Cross-Scale Contextual Fusion Network for Salient Object Detection in Optical Remote Sensing Images

1. Introduction

Salient object detection (SOD) is a fundamental task in computer vision that mimics human visual attention mechanisms to automatically identify and segment the most visually prominent regions in images [1]. In recent years, advances in deep learning have fueled substantial progress in SOD for natural scene images (NSI-SOD), with successful applications in image retrieval, object recognition, visual tracking and various other vision-related tasks [2,3,4,5]. At the same time, SOD for optical remote sensing images (ORSI-SOD) has garnered growing interest. It delivers essential prior information for downstream tasks like environmental change monitoring, aviation navigation, underwater detection, and urban planning [6,7,8]. Unlike natural scene images (NSI) captured by handheld cameras, optical remote sensing images (ORSI) refer to color imagery obtained by satellite or aerial sensors (with a wavelength range of 400–760 nm), leading to inherent distinctions between NSI and ORSI. Specifically, NSI typically exhibit fixed viewing angles, consistent object scales, and relatively simple backgrounds, which facilitate high foreground–background contrast and clear object boundaries. In contrast, ORSI often involve diverse imaging perspectives, large variations in object scale, and heterogeneous land cover compositions. These characteristics commonly lead to low contrast edges and severe background clutter. These notable differences make mature NSI–SOD methods perform subpar when directly applied to ORSI–SOD scenarios. This highlights an urgent need for specialized approaches to fully adapt to the unique characteristics of ORSI, ensuring reliable and accurate salient object detection.

In the field of ORSI–SOD, a key challenge lies in the low contrast of target objects, often caused by uneven illumination, atmospheric effects, or long imaging distances. Although CNNs excel at local feature extraction, limited receptive fields and reliance on intensity variations make them struggle to capture sufficient discriminative cues in low contrast regions [9,10]. Transformers capture global contextual dependencies well, but patch level tokenization often blurs fine edge details, especially in low contrast scenarios [11,12].

To better recover edge details in low contrast scenes, some recent works explore frequency domain representations to enhance structural cues. However, these methods mainly rely on magnitude spectra and pay limited attention to phase information [13,14,15]. Phase Congruency (PC) measures the agreement of phase angles across multiple frequencies, which makes it naturally insensitive to intensity contrast while remaining highly sensitive to edges and textures. Motivated by this property, we propose a novel Phase Congruency Enhanced Module (PCE). It injects PC cues into a Transformer backbone to strengthen target feature representations in low contrast scenes. To further amplify the effect of enhanced phase features on key regions [16], we also design a loss weighting mechanism (pc_weight) for subsequent loss supervision. It accurately strengthens the feature expression of low–contrast targets and significantly improves detection accuracy.

Another challenge arises from severe background interference. ORSI backgrounds often contain abundant target–irrelevant components such as clouds, vegetation, water bodies, and terrain shadows. These elements exhibit visual similarity to salient objects and are therefore easily confused with true targets, causing significant interference in accurate recognition and segmentation. Inspired by the idea of “residual attention guidance” in the field of image reconstruction and cross–attention dynamic fusion in the field of autonomous driving [17,18,19], we design a novel Dynamic Residual Fusion Module(DRF). It’s designed to address complex background interference in ORSI–SOD. Its core lies in inter–scale cross–scale feature integration. The module first fuses shallow and deep features, then uses dynamic spatial attention and channel attention to purify the fused features. This highlights task relevant regions and suppresses redundant background. In addition, residual connections are used during fusion to preserve fine grained structural details and prevent weak edges from being lost under complex interference. As a result, DRF performs both feature fusion and feature purification, improving robust foreground segmentation under severe background interference.

The main contributions are as follows:

• We propose a novel end–to–end network PCFNet. It integrates frequency domain phase enhancement with dynamic cross scale feature refinement to reduce blurred target boundaries under low contrast and improve robustness under complex background interference.
• We design a novel PC module based on Fourier decomposition theory. It fuses multi scale phase features with shallow Transformer features to solve blurred target perception caused by low contrast and compensate for local detail loss.
• We design a novel DRF module. It integrates dynamic spatial attention and residual connections to achieve complementary fusion and effective selection of multi scale features, suppressing complex background interference while preserving salient structures.
• We conduct extensive experiments on three benchmark datasets (ORSSD, EORSSD, ORSI4199). Through quantitative comparison with 23 SOTA methods, ablation experiments, and qualitative analysis of complex scenes, the effectiveness and robustness of the proposed network and core modules are fully demonstrated.

2. Related Work

2.1. Salient Object Detection for NSI

Early SOD methods for Natural Scene Images (NSI) relied on handcrafted features, which only utilized low-level visual cues such as color contrast [20]. Due to the lack of high-level semantic capture capability, these methods exhibited insufficient robustness in complex scenes. With the rise of deep learning, encoder-decoder architectures became the mainstream. However, early deep learning models had limitations in modeling long-range semantic relationships. Luo et al. [21] confirmed that this limitation would lead to semantic incoherence when processing large-scale targets, making it difficult to adapt to the requirements of complex scenes.

The introduction of self-attention mechanisms achieved a performance breakthrough. Such methods can directly model long-range semantic dependencies by calculating the correlation weights of all pixel pairs. For example, the DETR framework proposed by Carion et al. [22] transformed SOD into a set prediction task, abandoning anchor boxes to realize end-to-end segmentation and greatly simplifying the process. Liu et al.’s [23] Swin Transformer adopted a shifted window mechanism, which reduced computational complexity while retaining global modeling capability, adapting to high-resolution image scenarios. To fuse global semantics and local details, Xie et al. [24] designed a dual-branch Pyramid Grafting Network, achieving feature complementarity through cross-attention guidance.

Although numerous high-performance solutions have been developed for NSI-SOD, the particularities of ORSI in terms of target-motion blur, scale range, and imaging perspective determine that these methods cannot be directly adapted to ORSI-SOD tasks. Nevertheless, validated technical frameworks and design ideas still provide important references for research on ORSI-SOD.

2.2. Salient Object Detection for ORSI

In light of the distinctive properties of ORSI, many researchers have introduced innovative ideas and designed dedicated, effective algorithms for ORSI-SOD.

Li et al. [25] pointed out that the unique characteristics of ORSI put forward higher requirements for the adaptability of detection algorithms. Early methods combining handcrafted features and threshold segmentation were only applicable to small-scale simple scenes. With the rise of deep learning, U-Net and its variants [26] became the mainstream. However, the local modeling limitation of CNNs is more prominent in remote sensing scenarios, which easily leads to missed detection of dense targets or incomplete segmentation of targets with complex topological structures.

Deep learning promoted the development of efficiency and various optimization strategies have been targeted core problems. Zeng et al. [27] designed an adaptive edge-aware semantic interaction network, which improved the boundary accuracy of irregular targets such as rivers and edges. Some studies [28] also used boundary enhancement loss functions to optimize segmentation integrity. Gao et al. [29] proposed the Adaptive Spatialization Transformer, which enhances salient region representation by non-uniform token allocation and suppresses background redundancy. Li et al. [30] proposed a global context relation-guided network to capture spatial dependencies and aggregate multi-level features. Cheng et al. [31] proposed the Multimodal-Guided Transformer, which fuses RGB and depth information for heterogeneous feature collaboration network adopted the design of “global semantic modeling and local detail calibration” and achieved performance break-through through authoritative calibration. Despite the progress of the aforementioned studies, core challenges such as weak features of low-contrast targets and complex backgrounds in ORSI-SOD remain unsolved.

2.3. Frequency-Domain Analysis

Frequency-domain analysis provides a complementary perspective to spatial feature extraction by capturing target structure and texture information, and it has been widely applied in computer vision tasks. For example, feature enhancement networks based on discrete cosine transform and multi-scale analysis models using wavelet transform both improve target detail by enhancing medium-high frequency components [32].

As a classic frequency-domain analysis method, Phase Congruency (PC) is based on Fourier decomposition theory. It quantifies the phase synchronization of multi-scale frequency components and inherently offers advantages of illumination robustness and complete detail preservation. However, in ORSI-SOD, the application of frequency domain analysis and phase congruency still requires further exploration.

3. Proposed Method

In this section, we will introduce the proposed PCFNet in detail. Section 3.1 presents an overview of the general framework of PCFNet. Section 3.2 outlines the Transformer-based feature extractor. Section 3.3and Section 3.4 describe the core components PCE and DRF in detail, respectively. Section 3.5 introduces the decoder selected for the model. Finally, Section 3.6 explains the loss function used in the model.

3.1. Framework Overview

As shown in Figure 1, the proposed PCFNet adopts an encoder-decoder architecture. The input image passes sequentially through four Swin-B blocks to extract features at four scales. At the shallowest level, the PCE module compensates for target detail loss caused by low contrast through phase congruency computation. Subsequently, deeper-level features are refined by the DRF module, which fuses and optimizes multi-scale features to suppress background interference while preserving salient structures. Finally, the hierarchical features are aggregated by the SGAED decoder to produce a high-resolution saliency map.

3.2. Swin Transformer-based Feature Extractor

Following existing studies, we select the classical Swin-B as the basic feature extractor [23,25]. Its hierarchical representation and window based local self attention are well suited to the dense prediction requirements of ORSI-SOD. Different from the original Swin-B architecture designed for general vision tasks, this paper based adaptive optimization targeting the characteristics of remote sensing images. We retain the four-level feature extraction structure of the backbone network, and adjust the channel dimensions of deep layers to balance semantic representation capability and computational cost. The Swin-B in Figure 1 corresponds to the transformer-based encoder subnetwork, which consists of four feature extraction blocks. We take the output of each block as the side-output feature map, defined as ${\mathbf{f}}_t^i\in {\mathbb{R}}^{C_i\times H_i\times W_i}(i\in \{1,2,3,4\})$. For the input image, the channel numbers $C_{1,2,3,4}$ of each feature map are $\{128,256,512,1024\}$, and the resolutions $H_{1,2,3,4}=W_{1,2,3,4}$ correspond to $\{112,56,28,14\}$ respectively.

3.3. Phase Congruency Enhanced Module

ORSI often suffer from edge blurring and false target detection in low-contrast scenes, which impairs the discriminability of salient targets. To address this issue, we design a Phase Congruency Enhanced (PCE) Module (as shown in Figure 2), which is deployed after the first stage Swin-B feature extractor (for the feature map ${\mathbf{f}}_t^1$). Through the synergy of frequency-domain phase feature analysis and spatial-domain feature enhancement, the PCE module strengthens fine-grained local features of remote sensing targets, providing high-quality inputs for subsequent cross-scale feature fusion.

1)A: The Wave Filter Module serves as the frequency-domain response extraction unit of the PCE module, responsible for capturing multi-scale and multi-directional edge information of targets. First, the first-stage feature ${\mathbf{f}}_t^1$ output by Swin-B is compressed to 32 channels via a $1\times 1$ convolution to reduce computational complexity:

$${\mathbf{f}}_{in}^1={\mathrm{Conv}}_{1\times 1}\left({\mathbf{f}}_t^1\right)$$

(1)

Subsequently, ${\mathbf{f}}_{in}^1$ is fed into the Wave Filter Module. It is successively processed by the Difference of Gaussians (DoG) filtering and the multi-scale multi-orientation LogGabor filtering. The module applies the same filtering operation to all 32 channels. For each channel, the LogGabor filter bank is constructed with 2 scales (wavelengths of 8 pixels and 12 pixels) and 6 evenly distributed orientations($0^{\circ },{30}^{\circ },{60}^{\circ },{90}^{\circ },{120}^{\circ },{150}^{\circ }$). By this means, a total of 12 frequency-domain response maps(M) are generated, which cover edge texture features of targets at different scales and directions. They provide basic inputs for subsequent phase analysis.

2)B: The Phase Congruency Calculation module is the core functional unit of the PCE module. Based on the visual cognition law, image features concentrate at positions where the phase superposition of Fourier harmonics is maximum [34]. Therefore, this property can be exploited to achieve accurate localization of target features. After feeding M into this module, we first perform scale wise averaging. For each of the six orientations, we average the two scales with equal weights of 0.5 to obtain the refined ${\mathrm{M}}^{\prime }$. Here, each of the six waves corresponds to one of the 6 orientations (denoted by $\theta$, where $\theta =1,2,\dots ,6$). Then, we decompose each response map of the refined six waves into amplitude components ${\mathbf{A}}_{\theta ,c}$ (c denotes the number of channels, $c=1,2,\dots ,32$) and phase components ${\mathbf{P}}_{\theta ,c}$ via Fast Fourier Transform (FFT) for each channel. We calculate the amplitude-weighted mean phase ${\overline{P}}_c$ of the phase components:

$${\overline{P}}_c={\displaystyle \frac{{\sum }_{\theta =1}^6({\mathbf{A}}_{\theta ,c}\odot {\mathbf{P}}_{\theta ,c})}{{\sum }_{\theta =1}^6{\mathbf{A}}_{\theta ,c}+\epsilon }}$$

(2)

where $\epsilon$ is a small constant to avoid division by zero. Then, in the “calculate” step, we compute the phase deviation and phase congruency via Eq. (3) to (5), where Eq. (3) is dedicated to phase deviation calculation and Eqs. (4)(5) for phase congruency evaluation.

$$\Delta {\varphi }_{\theta ,c}=cos({\mathbf{P}}_{\theta ,c}-{\overline{P}}_c)-|sin({\mathbf{P}}_{\theta ,c}-{\overline{P}}_c)|$$

(3)

$${\mathrm{PC}}_c^{\prime }={\displaystyle \frac{{\sum }_{\theta =1}^{6}max\left({\mathbf{A}}_{\theta ,c}\odot (\Delta {\varphi }_{\theta ,c}-T_c),0\right)}{{\sum }_{\theta =1}^6{\mathbf{A}}_{\theta ,c}+\epsilon }}$$

(4)

$${\mathrm{PC}}^{\prime }=\mathrm{Concat}\left({\mathrm{PC}}_1^{\prime },{\mathrm{PC}}_2^{\prime },\dots ,{\mathrm{PC}}_{32}^{\prime }\right)$$

(5)

where $T_c=0.5·{\mathrm{Avg}}_{3\times 3}\left({\overline{\mathbf{A}}}_{\theta ,c}\right)$ denotes the local amplitude mean threshold. Here, ${\overline{\mathbf{A}}}_{\theta }$ is obtained by averaging the amplitude spectrum ${\mathbf{A}}_{\theta }$ over the orientation dimension (${\overline{\mathbf{A}}}_{\theta }={\displaystyle \frac{1}{6}}{\sum }_{\theta =1}^6{\mathbf{A}}_{\theta }$). ${\mathrm{Avg}}_{3\times 3}(·)$ represents $3\times 3$ average pooling. $max(·,0)$ filters out invalid phase contributions. Notably, the above operations are performed independently for each of the 32 channels in ${\mathbf{f}}_{in}^1$. Thus, the final dimension of ${\mathrm{PC}}^{\prime }$ is $32\times 112\times 112$, with a value range of $[0,1]$. A larger value indicates a higher probability that the position is a target feature.

3)C: Stripe Suppression Module has a dual function of eliminating stripe artifacts and recovering channel dimension. As shown in Figure 3, (a) and (c) are feature maps exhibiting stripe artifacts caused by phase congruency computation without the stripe suppression module, while (b) and (d) are the clean feature maps obtained after applying the module, with stripe interference clearly eliminated. After getting the clean feature maps, it restores the feature channel dimension to be consistent with that of the input feature. This lays a solid foundation for subsequent spatial attention enhancement. The input of this module is the preliminary phase congruency map ${\mathrm{PC}}^{\prime }$ and the specific processing process is as follows:

$$\left\{\begin{array}{c}\widetilde{\mathrm{PC}}={\mathrm{PC}}^{\prime }\odot \sigma \left({\mathrm{DepthwiseConv}}_{7\times 7}\left({\mathrm{PC}}^{\prime }\right)\right)\hfill \\ \mathrm{PC}=\sigma \left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{Clamp}\left(\mathrm{LayerNorm}\left(\widetilde{\mathrm{PC}}\right)\right)\right)\right)\hfill \end{array}\right.$$

(6)

$\sigma (·)$ represents sigmoid activation function. First, the $7\times 7$ depthwise separable convolution is performed on ${\mathrm{PC}}^{\prime }$ to capture the response of stripe artifact regions. And the suppression mask is generated by the Sigmoid function. The mask is multiplied with the original ${\mathrm{PC}}^{\prime }$ in an element-wise manner to suppress the response of stripe regions. Then, the layer normalization is used to standardize the feature distribution. And the value clipping is adopted to limit the dynamic range of feature values$[-3,3]$, completing the artifact elimination and feature stabilization. Subsequently, the $1\times 1$ convolution is applied to the artifact-removed features to restore the channel dimension from 32 to 128. Finally, the Sigmoid function is used to map the feature values to the range of $[0,1]$, generating the phase congruency weight map $\mathrm{PC}$.

4)D: The Variance-Based Spatial Attention is the output enhancement unit of the PCE module, used to improve the spatial discriminability of features. We use variance because in ORSI, boundaries or texture regions of salient objects usually exhibit noticeable gray-level jumps, leading to high local variance. High variance typically reflects the contours or locations of salient objects, while low variance corresponds to smooth regions. After feeding $\mathrm{PC}$ into this module, we first calculate the mean $\mu \left(\mathrm{PC}\right)$ and variance $\sigma \left(\mathrm{PC}\right)$ across all channels for each spatial position. And we concatenate them and compress the dimension via a $1\times 1$ convolution to generate spatial attention weights. Then we perform element-wise multiplication with $f_t^1$ and add a residual connection to avoid semantic loss. Finally the enhanced feature $f_p^1$ is outputted, as shown in Eq. (7):

$$f_p^1=f_t^1\odot \sigma \left({\mathrm{Conv}}_{1\times 1}\left(\left[\mu \right(\mathrm{PC}),\sigma (\mathrm{PC}\left)\right]\right)\right)+f_t^1$$

(7)

The enhanced feature $f_p^1$ with significantly strengthened target information is fed into the subsequent DRF module for cross-scale feature fusion.

3.4. Dynamic Residual Fusion (DRF) Module

The Dynamic Residual Fusion (DRF) Module is designed to alleviate the insufficient cross-scale feature fusion and suppress complex background interference. It leverages the residual attention mechanism to gradually achieve sufficient interaction between shallow and deep features in both channel and spatial dimensions. As shown in Figure 4, the DRF module consists of a feature concatenation unit, Residual Dual Attention Blocks (RABs), and a global residual connection. In the DRF, the feature concatenation unit is used to fuse cross-scale features. The dual attention mechanism is employed to filter out cross-scale features. And the residual connection ensures the preservation of detailed information and gradient stability during the feature enhancement process.

The input of DRF includes two cross-scale features $f_t^{i-1}$ and $f_t^i$. First, $f_t^{i-1}$ (after downsampling) and $f_t^i$ are concatenated along the channel dimension to obtain the feature $f_{cat}$. Then a $1\times 1$ convolution followed by BN and LeakyReLU is used to compress the channel dimension, yielding the initial feature $f_{init}$.

$$f_{init}=\mathrm{LeakyReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{1\times 1}\left(f_{cat}\right)\right)\right)$$

(8)

Subsequently, $f_{init}$ is fed into a residual attention group composed of 5 stacked RABs. Enlightened by the gradient residual attention network [17], which effectively aggregates fine-grained features via stacked residual dense blocks. We choose 5 RABs as intra-group units by balancing the feature enhancement performance, computational overhead and gradient stability. The validity of this configuration is demonstrated in subsequent ablation experiments. For each RAB, the input feature is first preprocessed by two $3\times 3$ convolutions. The first layer is followed by BN and LeakyReLU, and the second layer only retains BN, resulting in the feature $f_{conv2}$.

$$f_{conv1}=\mathrm{LeakyReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left(f_{R,in}\right)\right)\right)$$

(9)

$$f_{conv2}=\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left(f_{conv1}\right)\right)$$

(10)

Based on $f_{conv2}$, Channel Attention (CA) Module is first used to generate channel weights. Global max pooling (GMP) and global average pooling (GAP) along the channel dimension are performed on $f_{conv2}$ to obtain $f_{gmp}$ and $f_{gap}$. These two features are fed into a shared MLP for dimension reduction and restoration, then summed up and activated by sigmoid to generate the weight $w_{CA}$. The weight is element-wise multiplied with $f_{conv2}$.

$$f_{gmp}={\mathrm{GMP}}_c\left(f_{conv2}\right),f_{gap}={\mathrm{GAP}}_c\left(f_{conv2}\right)$$

(11)

$$w_{CA}=\sigma \left(\mathrm{MLP}\left(f_{gmp}\right)+\mathrm{MLP}\left(f_{gap}\right)\right)$$

(12)

$$f_{CA}=f_{conv2}\odot w_{CA}+f_{conv2}$$

(13)

Then, the Dynamic Spatial Attention (DSA) Module is applied to $f_{CA}$ to achieve spatial feature enhancement. First, global average pooling is performed on the input feature $f_{CA}$, followed by two successive $1\times 1$ convolutions and ReLU activation. Then, a $3\times 3$ convolution kernel is generated through the Reshape operation. The module is dynamic in that the kernel shape remains $3\times 3$, while its weight parameters are not fixed preset values. Instead, the weight parameters are adaptively calculated from the independent input features of each individual sample. Each sample in a batch is endowed with a unique set of $3\times 3$ convolution kernel weights, which can adapt to the spatial feature pattern of the sample itself in a targeted manner. This is the core embodiment of the “dynamic” property in our DSA module. Meanwhile, the channel-wise mean operation is conducted on $f_{CA}$ to obtain a single-channel spatial feature map. The sample-specific $3\times 3$ dynamic convolution kernels are used to perform dynamic convolution on this single-channel spatial feature map. After that, the spatial attention weight $w_{DSA}$ is generated by the Sigmoid activation function. Finally, the original feature $f_{CA}$ is multiplied element wise by the spatial attention weight $w_{DSA}$ to obtain the enhanced feature $f_{DSA}$.

$$k_{dyn}=\mathrm{Reshape}\left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{AvgPool}\left(f_{CA}\right)\right)\right)\right)\right)$$

(14)

$$w_{DSA}=\sigma \left({\mathrm{Conv}}_{dyn}\left(\mathrm{mean}\left(f_{CA}\right)\right)\right)$$

(15)

$$f_{DSA}=f_{CA}\odot w_{DSA}$$

(16)

The output of each RAB is added to the input feature via a local residual connection.

$$\begin{array}{cc}\hfill f_{R-g}& =f_{R-out}^5={\mathrm{Conv}}_{3\times 3}(RAB\_5(RAB\_4\hfill \\ \hfill & (RAB\_3(RAB\_2(RAB\_1\left(f_{init}\right))))\left)\right)\hfill \end{array}$$

(17)

$$f_{R-out}^i=f_{DSA}+f_{R-in}^i(i=1,2,3,4,5)$$

(18)

$$f_{res}=f_{R-g}+f_{init}$$

(19)

$$f_c^i={\mathrm{Conv}}_{1\times 1}\left(f_{res}\right)$$

(20)

The output of five serially connected RABs is integrated into $f_{R-g}$ through a $3\times 3$ convolution, then added to $f_{init}$ via a global residual connection to obtain $f_{res}$. Finally, a $1\times 1$ convolution is used to adjust the channel dimension, making it consistent with the $f_t^i$, yielding the fused feature $f_c^i$ of DRF.

3.5. Decoder

To ensure the integrity of the network architecture and achieve accurate mapping from high-level fused features to pixel-level saliency maps, this paper directly adopts the Saliency-Guided Attention Enhanced Decoder (SGAED) from the literature as the decoding module of the network [31]. This decoder can adapt to the feature format output by the encoder in this paper without additional modifications. Its core function is to receive the three groups of high-level fused features (${\mathbf{f}}_c^i\in {\mathbb{R}}^{256\times {\displaystyle \frac{112}{2^{i-1}}}\times {\displaystyle \frac{112}{2^{i-1}}}},i=2,3,4$) from the DRF module and the enhanced shallow feature (${\mathbf{f}}_p^1\in {\mathbb{R}}^{128\times 112\times 112}$) from the PCE module. In terms of the specific process, $f_t^4$ feature and $S_5(14\times 14)$ generated from the $f_t^4$ feature are fed into decoder4, followed by decoder4 to decoder1 that gradually perform feature upsampling and detail refinement (decoder4 does not perform upsampling, ${\mathbf{f}}_{de}^i\in {\mathbb{R}}^{C\times {\displaystyle \frac{112}{2^{i-1}}}\times {\displaystyle \frac{112}{2^{i-1}}}},S_i({\displaystyle \frac{112}{2^{i-1}}}\times {\displaystyle \frac{112}{2^{i-1}}}),i=2,3,4,{\mathbf{f}}_{de}^2\in {\mathbb{R}}^{128\times 112\times 112}$). Specifically, decoder0 is designed to optimize the $S_1(112\times 112)$ features at the resolution of $112\times 112$ for generating the high-detail $S_0(224\times 224)$, which supplies fine-grained supervision. Through repeated computation, key features are strengthened and representation capability is improved, while preserving the integrity of the five SGAED components in the original framework. Finally, all saliency maps are interpolated to $224\times 224$ and used as the supervised output and final predictions.

3.6. Loss Function

The Loss constructed in this paper combines CE Loss and IoU Loss [35,36]. And we design a novel weighting mechanism for the phase congruency map output by the phase congruency enhancement module.

1) Saliency Supervision We adopt CE Loss and IoU Loss to perform basic supervision on the last 4 layers of intermediate saliency maps, which is formulated as follows:

$${\mathcal{L}}_B=\sum _{i=2}^5\left({\mathcal{L}}_{CE}({\mathrm{Up}}_{k_i}\left(S_i\right),G)+{\mathcal{L}}_{\mathrm{IoU}}({\mathrm{Up}}_{k_i}\left(S_i\right),G)\right)$$

(21)

where $S_i$ represents the i-th saliency prediction map ($i=2,3,4,5$), G is the ground-truth label map, and ${\mathrm{Up}}_{k_i}(·)$ denotes the upsampling operation with an upsampling factor of $k_i$. The value of $k_i$ is defined as:

$$k_i=\left\{\begin{array}{cc}{2}^4,\hfill & \mathrm{if}i=5\hfill \\ 2^i,\hfill & \mathrm{if}i\le 4\hfill \end{array}\right.$$

(22)

2) Phase Congruency Map Weighting (pc_weight) Existing loss functions fail to focus on salient structural regions in ORSI, so we design a phase congruency map weighting mechanism, referred to as pc_weight. It strengthens supervision on key regions and mitigates the impacts of weak features of low contrast targets. It should be noted that we only apply this weighted loss to $S_0$ and $S_1$. Because the phase congruency enhancement module is embedded only in stage 1 of feature extraction, and its effect is limited to the shallow feature stage.

The pc_map refers to the single channel phase congruency map derived by averaging the 128 channels phase congruency map PC generated by Eq. (6) in Section 3.3 above. It reflects the distribution characteristics of salient structures such as edges and textures in the image. First, the original pc_map is interpolated to the size of the current prediction map and normalized to obtain the pixel-level weight $\omega$ (pc_weight), calculated as:

$$\omega ={\displaystyle \frac{{\mathrm{Up}}_s\left(\mathrm{pc}\text{\_}\mathrm{map}\right)-min\left({\mathrm{Up}}_s\left(\mathrm{pc}\text{\_}\mathrm{map}\right)\right)}{max\left({\mathrm{Up}}_s\left(\mathrm{pc}\text{\_}\mathrm{map}\right)\right)-min\left({\mathrm{Up}}_s\left(\mathrm{pc}\text{\_}\mathrm{map}\right)\right)+\beta }}$$

(23)

where ${\mathrm{Up}}_s(·)$ is the upsampling operation to interpolate pc_map to the size of the i-th prediction map. $\beta ={10}^{-8}$ is used to avoid denominator being zero. For the first two layers of high-resolution intermediate saliency maps ($S_i,i=0,1$), this weight is introduced to weight the loss, formulated as:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{weighted}}^{\left(i\right)}& ={\mathcal{L}}_{CE}\left({\mathrm{Up}}_{k_i}\left(S_i\right)\odot \omega ,G\odot \omega \right)\hfill \\ \hfill & +{\mathcal{L}}_{\mathrm{IoU}}\left({\mathrm{Up}}_{k_i}\left(S_i\right)\odot \omega ,G\odot \omega \right)\hfill \end{array}$$

(24)

$\omega$ is the normalized pixel-level phase congruency weight map (ranging in $[0,1]$), ${\mathrm{Up}}_{k_i}(·)$ denotes the upsampling operation with an upsampling factor of $k_i$, and ⊙ represents the element-wise multiplication operation.

3) Overall Loss Function Finally, the overall loss function ${\mathcal{L}}_{\mathrm{total}}$ of the model is formulated as follows:

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_B+\sum _{i\in \{0,1\}}{\mathcal{L}}_{\mathrm{weighted}}^{\left(i\right)}$$

(25)

4. Experiments

4.1. Experimental Settings

1) Dataset We adopt three mainstream benchmark datasets for ORSI-SOD, including ORSSD [37], EORSSD [38] and ORSI4199 [39]. These datasets cover scenes of varying complexity to comprehensively test the model’s detection performance.

ORSSD, the first public dataset for remote sensing salient object detection, with 800 pixel-annotated images (600 training, 200 testing). It covers regular scenes, verifying the model’s basic performance.

EORSSD is an extension of ORSSD, with 2000 images (1400 training, 600 testing). It adds targets with complex backgrounds and irregular structures, increasing detection difficulty to test the model’s adaptability to complex scenes.

ORSI4199 is a large-scale challenging dataset, with 4199 high-precision annotated images (2000 training, 2199 testing). It includes multi-attribute complex targets (large, small, low-contrast) in realistic scenes, verifying the model’s generalization ability.

2) Implementation Details To verify the proposed network’s performance in ORSI-SOD, experiments are implemented via the PyTorch framework on a workstation with NVIDIA RTX 3090 Ti GPUs, using Python 3.8, PyTorch 1.11.0 and related libraries. During training, images are augmented with random flip, rotation and Gaussian blur, resized to $224\times 224$. The Adam optimizer (initial learning rate $7\times {10}^{-5}$, ${\beta }_1=0.9$, ${\beta }_2=0.999$) is used, with learning rate adjusted via StepLR (step size=20 epochs, decay factor=0.5). Batch size is 8 and the model is trained for 100 epochs.

3) Evaluation Metrics Four mainstream metrics are adopted in the experiments to comprehensively evaluate the model performance.

S-Measure ($S_{\alpha }$) [40]: It measures the structural similarity between the saliency map and the ground truth, integrating object-level ($S_{oj}$) and region-level ($S_{re}$) similarity, with the formula as follows:

$$S_{\alpha }=\alpha \times S_{oj}+(1-\alpha )\times S_{re}$$

(26)

where the balance weight $\alpha$ is set to 0.5 by default.

F-Measure ($F_{\beta }$) [20]: It is a comprehensive metric that balances precision ($Pr$) and recall ($Re$), calculated by:

$$F_{\beta }={\displaystyle \frac{(1+{\beta }^2)\times Pr\times Re}{{\beta }^2\times Pr+Re}}$$

(27)

Following the mainstream setting, $\beta$ is set to 0.3.

E-measure ($E_{\xi }$) [41]: It considers both pixel-level correspondence and image-level statistical information simultaneously, with the formula:

$$E_{\xi }={\displaystyle \frac{1}{H\times W}}\sum _{x=1}^H\sum _{y=1}^W\varphi (x,y)$$

(28)

where H and W represent the height and width of the saliency map, and $\varphi (x,y)$ denotes the enhanced alignment function.

MAE (M) [42]: It calculates the average pixel deviation between the saliency map and the ground truth, defined as:

$$\mathrm{MAE}={\displaystyle \frac{1}{H\times W}}\sum _{x=1}^H\sum _{y=1}^W|S(x,y)-G(x,y)|$$

(29)

where $S(x,y)$ and $G(x,y)$ are the pixel values of the saliency map and the ground truth at position $(x,y)$, respectively.

4.2. Comparison with SOTA Methods

1) Comparison Methods Our proposed model (ours) and 23 state-of-the-art (SOTA) models are compared across all three benchmark datasets. The compared methods encompass a diverse range of models: RRWR [20] and RCRR [43] are two conventional NSI-SOD models. EGNet [44], MINet [45], and GatedNet [46] are three deep NSI-SOD models.

LVNet [37], DAFNet [38], MCCNet [47], CorrNet [48], ASTTNet [28], MJRBM [49], RRNet [40], EMFINet [50], ERPNet [44], ACCoNet [51], AESINet [27], DCCNet [52], LSHNet [53], MCPNet [54] are fourteen deep ORSI-SOD models. HFANet [55], ADSTNet [56], HFCNet [31], CMNFNet [57] are four Hybrid ORSI-SOD models. Table 1 lists all the quantitative results, which were generated by running the corresponding open-source codes provided by the author and adopting the default parameter configurations, or through calculations based on publicly accessible saliency maps.

2) Quantitative Comparisons and Discussions As shown in Table 1, our method (ours) performs prominently in key metrics across ORSSD, EORSSD, and ORSI4199 datasets. Overall, it outperforms most state-of-the-art methods. On the ORSSD dataset, our method ranks first in $S_{\alpha }$, $F_{\beta }$, $E_{\xi }$, and MAE (0.9540, 0.9305, 0.9888, 0.0071 respectively). Compared with CMNFNet, it achieves a 0.65% improvement in $S_{\alpha }$ and a 1.16% improvement in $F_{\beta }$. On the EORSSD dataset, our method ranks first in $F_{\beta }$(0.8943), $E_{\xi }$ (0.9843) and MAE (0.0048), and second in $S_{\alpha }$ (slightly behind HFCNet-R), while maintaining more balanced overall performance. On the ORSI4199 dataset, our method ranks first in two metrics, with $S_{\alpha }$(0.8858) and $F_{\beta }$ (0.8859). In summary, our method demonstrates competitive-ness across all datasets, verifying its effectiveness in ORSI-SOD.

3) Qualitative Comparisons and Discussions We select representative visual examples from the ORSSD, EORSSD, and ORSI4199 datasets. These examples are grouped by scene type (as shown in Figure 5) to qualitatively compare our method with state-of-the-art models.

For the first four rows representing low-contrast scenes, the target and background exhibit high texture similarity. In the 1st row, this similarity causes blurred edges in the saliency maps of ACCoNet and ERPNet. In the 2, 3, 4 rows, the subtle grayscale difference between target and background leads to missing targets or blurred adhesion in the saliency maps of CMNFNet and MJRBM. In contrast, our method enhances the discriminability of target features in low-contrast regions through the PCE module, producing saliency maps that accurately restore target contours and closely match the ground truth. For the last four rows representing complex background interference scenes, the complex background easily interferes with target detection. In Rows 5 and 6 with complex building scenes, models such as CMNFNet and MCCNet tend to introduce background noise spots. In contrast, our model separates the target from the background more clearly and remains robust to irrelevant background interference. In the 7th row featuring a dome-shaped scene, GatedNet suffers from edge distortion due to background element interference. While our dual module effectively resists redundant background interference and accurately aligns with the true target contour. In the 8th row, the target is extremely small and easily confused with the background, causing methods like CorrNet and MCCNet to miss the target or falsely activate background regions. In contrast, our method can detect it correctly.

4.3. Ablation Study

To verify the effectiveness of the proposed modules, core components, and loss functions, we conduct ablation experiments in this section. The ablations are designed at the module, component, and loss function levels. We further analyze the contribution of each part to model performance with visualization results.

1) Ablation study between different modules To verify the effectiveness of the proposed PCE and DRF modules, we designed four model variants on the ORSSD and EORSSD datasets: a) Base (Swin-B Encoder and SGAED Decoder), b) Base+PCE, c) Base+DRF, d) Base+DRF+PCE (ours).

The quantitative results (Table 2) demonstrate that: On the ORSSD dataset, the Base performs the worst. After integrating PCE, $S_{\alpha }$ is improved by 0.74%. When adding DRF, $S_{\alpha }$ increases by 0.64% while the $F_{\beta }$ and $E_{\xi }$ are slightly improved. Our method, which combines both modules, not only enhances the target discriminability in low-contrast regions via PCE but also aligns cross-scale features and filters background interference via DRF. Compared with the Baseline, our method achieves a 0.99% improvement in $S_{\alpha }$ and a 1.40% improvement in $F_{\beta }$. Consistent trends are observed on the EORSSD dataset, where our method improves $S_{\alpha }$ and $F_{\beta }$ by 0.67% and 2.73% respectively compared with the Baseline, confirming the targeted value of the two modules in addressing typical issues of optical remote sensing images.

In Figure 6, the role of our modules is further demonstrated: In the scene of the 1st row, the Base result exhibits target edge diffusion due to background interference. Adding DRF makes the predicted contours more compact and well defined, suggesting that DRF suppresses redundant background responses. In the 2nd row with a low contrast linear target, the Base result shows discontinuities. Schemes with PCE preserve the target shape more completely, indicating PCE’s ability to enhance low contrast features. In the 3rd row, schemes with a single module lose details. Combining DRF and PCE preserves the target shape much better, showing their synergy in suppressing interference and enhancing features for more accurate segmentation.

To verify the enhancement effect of PCE on features extracted by Swin Transformer Stage 1, we generate feature heatmaps with a Tkinter based feature map viewer, as shown in Figure 7. The differences between the original Stage 1 heatmaps and the enhanced ones are clear, reflecting improved target details and saliency. In Row 1, the original Stage 1 features show small brightness differences between target and background and blurry contours. After enhancement, the target region becomes brighter and more distinguishable. In Row 2, the original features contain only sparse dot like highlights. After enhancement, the target region becomes more uniform and its shape is more complete and better separated from the background. In Row 3, the original edge responses are discontinuous. After enhancement, continuous highlight bands appear along the target edges. These observations indicate that PCE improves target–background separability in low contrast scenes and strengthens edge integrity, providing a stronger detail basis for subsequent cross-scale fusion.

2) Ablation study within the modules: To verify the effectiveness of the core components in the PCE and DRF modules, we designed ablation variants with key components removed. And we analyzed their performance on the ORSSD and EORSSD datasets.

For the PCE module, we designed two variants: w/o PC calculation (we removed the Filter, Phase Congruency Calculation, Stripe and only retained the VSA) and w/o VSA (PCE without VSA). As shown in Table 3(a), either removing the VSA or only having VSA leads to performance degradation. As shown in Figure 8(a), in the 1st row, removing these two parts results in the loss of texture details inside the target. In the 2nd row, “ours” accurately reproduces the target shape, while “w/o VSA” suffers from detail loss and background interference. This indicates that the phase congruency calculation enhances target edges and textures while suppressing background noise, and VSA focuses on the target region to strengthen effective feature responses. The two parts work synergistically, serving as the core of the PCE module for low-contrast feature enhancement, and are indispensable for performance improvement.

For the DRF module, we designed two variants: w/o CA (DRF without CA) and w/o DSA (DRF without DSA). The quantitative results (Table 3(b)) indicate that the performance decreases after removing CA or DSA. For dome shaped building targets in Figure 8(b), the saliency maps of w/o CA contain background clutter, and the target contours of w/o DSA are distorted. The river scene is similar in situation. The complete DRF module generates saliency maps with clear and intact shapes, verifying that the synergy between CA and DSA effectively resolves background challenges.

In this paper, we select 5 stacked RABs for the DRF module, which is the optimal choice based on ablation experiments. Table 4 shows that with the number of RABs from 3 to 5 improves all three core metrics ($S_{\alpha }$, $F_{\beta }$, $E_{\xi }$) on the ORSSD dataset. This suggests that more RABs strengthen feature fusion and redundancy filtering. When the number of RABs exceeds 5, parameter redundancy causes gradient attenuation, and all metrics show a decline. Therefore, the stacked structure of 5 RABs enables the model to achieve the best performance.

3) The impact of our pc_weight on the loss function: This section verifies the effectiveness of pc_weight. In the original design, saliency map $S_0$ and $S_1$ use pc-weight combination of CE Loss and IOU Loss, while $S_2$–$S_5$ adopt ordinary CE and IOU Loss. Removing pc_weight in this ablation experiment means that $S_0$ and $S_1$ also use only ordinary CE and IOU Loss (consistent with $S_2$–$S_5$). The experimental results are shown in Table 5 and the Figure 8(c). From Table 5, it can be observed that after removing pc_weight, only the basic losses of CE Loss and IOU Loss remain. $S_{\alpha }$ of ORSSD decreases from 0.9540 to 0.9484, and $F_{\beta }$ drops from 0.9305 to 0.9244. This indicates that pc_weight effectively improves the model performance. The visualization of the ablation experiment for the proposed pc_weight is presented in Figure 8(c). For the low-contrast building targets, the prediction map of $w/o$ pc_weight loses the prominent structures of the buildings, which demonstrates that pc_weight strengthens the detail supervision of low-contrast targets. For bridge targets, the prediction map of $w/o$ pc_weight exhibits misinformation, proving that pc_weight constrains the structural integrity of the targets.

Table 5. The impact of our pc_weight on the loss function.

No.	Baseline	pc_weight	ORSSD			EORSSD
			$S_{\alpha }\uparrow$	$F_{\beta }\uparrow$	$E_{\xi }\uparrow$	$S_{\alpha }\uparrow$	$F_{\beta }\uparrow$	$E_{\xi }\uparrow$
1	✓		0.9484	0.9244	0.9844	0.9323	0.8854	0.9641
2	✓	✓	0.9540	0.9305	0.9888	0.9393	0.8943	0.9843

Table 5. The impact of our pc_weight on the loss function.

No.	Baseline	pc_weight	ORSSD			EORSSD
			$S_{\alpha }\uparrow$	$F_{\beta }\uparrow$	$E_{\xi }\uparrow$	$S_{\alpha }\uparrow$	$F_{\beta }\uparrow$	$E_{\xi }\uparrow$
1	✓		0.9484	0.9244	0.9844	0.9323	0.8854	0.9641
2	✓	✓	0.9540	0.9305	0.9888	0.9393	0.8943	0.9843

4.4. Computational Complexity Analysis

1) Complexity Changes from Module Superposition The model in this paper uses TF and SGAED as the Baseline. After adding the PCE module, FLOPs increase by 6.67G, and Params increase by 0.53M. The PCE module only introduces lightweight frequency-domain filter operations, resulting in minimal complexity overhead. After adding the DRF module: FLOPs increase by 21G, and Params increase by 13M. The multi-group RAB attention blocks in DRF are the main source of complexity, but they ensure the effectiveness of cross-scale fusion.

2) Complexity Comparison with Existing Methods Our proposed model (ours) has 126.94G FLOPs and 117.29M parameters. For high-performance methods such as EMFINet (176.87G) and ACCoNet (184.50G), our model achieves better performance while reducing the complexity by around 30%. For HFCNet, our model achieves better performance with fewer parameters and comparable FLOPs. For models with lower complexity (e.g., AESINet), although our model has slightly higher complexity, its performance is significantly better (see Section 4.2 for details).

Table 6. Model Computational Complexity Comparison: (a) complexity analysis of the proposed modules, (b) comparison of complexity with some state-of-the-art methods.

(a)
Models	FLOPs	Params
TF	71.44 G	86.64 M
TF+SGAED	99.27 G	103.76 M
TF+SGAED+PCE	105.94 G (↑6.67)	104.29 M (↑0.53)
TF+SGAED+PCE+DRF(Ours)	126.94 G (↑21)	117.29 M (↑13)
(b)
Models	FLOPs	Params
MCCNet	117.15 G	67.65 M
EMFINet	176.87 G	95.09 M
ERPNet	131.63 G	77.19 M
ACCQNet	184.50 G	102.55 M
AESINet	53.42 G	41.05 M
ASTTNet	43.12 G	23.35 M
ADSTNet	62.09 G	27.72 M
HFCNet	120.41 G	140.75 M
ours	126.94 G	117.29 M

Table 6. Model Computational Complexity Comparison: (a) complexity analysis of the proposed modules, (b) comparison of complexity with some state-of-the-art methods.

(a)
Models	FLOPs	Params
TF	71.44 G	86.64 M
TF+SGAED	99.27 G	103.76 M
TF+SGAED+PCE	105.94 G (↑6.67)	104.29 M (↑0.53)
TF+SGAED+PCE+DRF(Ours)	126.94 G (↑21)	117.29 M (↑13)
(b)
Models	FLOPs	Params
MCCNet	117.15 G	67.65 M
EMFINet	176.87 G	95.09 M
ERPNet	131.63 G	77.19 M
ACCQNet	184.50 G	102.55 M
AESINet	53.42 G	41.05 M
ASTTNet	43.12 G	23.35 M
ADSTNet	62.09 G	27.72 M
HFCNet	120.41 G	140.75 M
ours	126.94 G	117.29 M

5. Conclusion

In this paper, we propose a novel PCFNet to tackle the complex target detection challenge in ORSI-SOD tasks. We embed PCE and DRF into the Swin Transformer backbone to build a unified pipeline that combines frequency domain detail enhancement, cross scale semantic fusion, and refined supervision. Specifically, PCE extracts frequency-domain phase features, which breaks the limitation of traditional spatial enhancement that relies on brightness differences. It improves the detail representation capability of targets in low-contrast scenes. The dual attention block (RAB) designed in DRF fuses channel and dynamic spatial attention to resolve problems of complex background interference. Extensive experiments on 3 datasets validate the superior performance of PCFNet.