Stabilized Multi-Objective Optimization for Camouflage Attack in Remote Sensing

1. Introduction

Modern remote sensing systems integrate satellite and aerial imagery with deep learning for applications such as maritime surveillance, border security, environmental monitoring, and disaster response [1]. In these systems, heterogeneous perception pipelines may combine region-proposal-based object detectors, such as Mask R-CNN [2], with dense semantic segmentation networks, such as U-Net [3]. Although these architectures provide complementary representations of the same scene, their increasing use also raises concerns regarding the robustness of remote sensing perception systems to adversarial perturbations.

Deep neural networks are known to be vulnerable to carefully optimized perturbations that can substantially alter their predictions. In remote sensing, adversarial patch generation is particularly challenging because targets may appear at different scales, orientations, geometries, and Ground Sample Distances (GSDs). Existing attacks are typically optimized for a single prediction task or architecture, and their effectiveness can degrade when transferred between fundamentally different models, such as object detectors and semantic segmenters. In addition, jointly attacking multiple prediction objectives introduces competing loss terms with substantially different magnitudes and gradient directions, making optimization difficult.

To address these challenges, we propose Stabilized Multi-Objective Optimization (SMO), a geometry-aware adversarial camouflage framework that jointly targets object detection and semantic segmentation models. SMO employs a U-Net-based generator to synthesize a spatially coherent adversarial texture and applies it through a target-aligned mask so that the perturbation follows the geometry of the object. To improve optimization across heterogeneous objectives, we introduce a logarithmic-inversion adversarial loss and incorporate a feature-inconsistency objective. We further investigate several strategies for balancing the resulting loss terms, including uniform weighting, static heuristics, cyclic optimization, adaptive stochastic optimization, and GradNorm-based gradient normalization.

The proposed framework is evaluated on the Airbus Ship Detection, DOTA, and VisDrone datasets. Experiments show that SMO substantially degrades both detection and segmentation performance and outperforms the evaluated adversarial patch baselines on the primary Airbus benchmark. Cross-dataset and cross-architecture experiments further demonstrate transferability to YOLO-based detectors and several convolutional detection and segmentation architectures, while also revealing reduced effectiveness against some architecturally dissimilar models. Additional experiments analyze patch coverage, patch geometry, multi-patch configurations, optimization strategies, and the contribution of individual SMO components.

The main contributions of this paper are summarized as follows:

• We formulate adversarial camouflage against heterogeneous remote sensing perception systems as a unified multi-objective optimization problem that jointly targets object detection and semantic segmentation.
• We introduce a logarithmic-inversion adversarial objective together with feature inconsistency to improve optimization across multiple heterogeneous detector and segmentation losses.
• We develop a U-Net-based geometry-aware perturbation mechanism that produces target-aligned adversarial textures and accommodates variations in object scale, orientation, and geometry without relying on a fixed-size rectangular patch.
• We systematically investigate multiple fixed and adaptive loss-balancing strategies and analyze their influence on adversarial optimization.
• We validate the proposed framework on Airbus, DOTA, and VisDrone through baseline comparisons, cross-model transferability experiments, patch-configuration studies, and component-level ablations.

Figure 1. Comparison of single-task and joint-model adversarial suppression. A segmentation-optimized patch ($1^{\mathrm{st}}$ row) transfers poorly to the region-proposal-based detector, while a detection-optimized patch ($2^{\mathrm{nd}}$ row) does not consistently suppress the dense-prediction segmenter. In contrast, the proposed Stabilized Multi-Objective Optimization (SMO) framework ($3^{\mathrm{rd}}$ row) jointly optimizes the adversarial texture against both architectures, resulting in more balanced degradation across detection and segmentation.

Figure 1. Comparison of single-task and joint-model adversarial suppression. A segmentation-optimized patch ($1^{\mathrm{st}}$ row) transfers poorly to the region-proposal-based detector, while a detection-optimized patch ($2^{\mathrm{nd}}$ row) does not consistently suppress the dense-prediction segmenter. In contrast, the proposed Stabilized Multi-Objective Optimization (SMO) framework ($3^{\mathrm{rd}}$ row) jointly optimizes the adversarial texture against both architectures, resulting in more balanced degradation across detection and segmentation.

2. Related Work

2.1. Adversarial Attacks on Computer Vision

The vulnerability of deep neural networks to adversarial perturbations has been extensively studied [4,5,6,7]. Early approaches primarily considered image-wide additive perturbations, whereas later studies demonstrated that localized adversarial patches can effectively manipulate model predictions while modifying only a limited spatial region [8]. Adversarial patches have subsequently been extended to object detection, where they can suppress objectness scores, alter classifications, or interfere with localization [9,10,11,12]. Representative attacks target both two-stage detectors such as Faster R-CNN [13] and one-stage detectors such as YOLO [14]. However, many of these methods are developed for terrestrial imagery and do not explicitly address the large variations in scale and orientation encountered in aerial and satellite scenes.

2.2. Adversarial Attacks in Remote Sensing

Remote sensing imagery introduces additional challenges for adversarial attacks because objects can appear at arbitrary orientations and over substantial variations in spatial resolution and GSD. Recent studies have investigated adversarial camouflage and patch attacks against aerial targets including vehicles [15,16,17], pedestrians [18,19], and infrastructure [20]. Adversarial attacks have also been developed specifically for remote sensing object detection, including multi-scale and camouflage-oriented approaches [21,22,23,24,25].

A common strategy for improving robustness to spatial variations is Expectation over Transformation (EoT), in which transformations such as scaling, rotation, and translation are sampled during patch optimization. Scale-adaptive methods additionally modify the perturbation according to the expected target-size distribution [23]. Although these approaches improve robustness to geometric variation, most existing remote sensing attacks are optimized primarily against a single detection architecture or task. In contrast, our work considers a heterogeneous setting in which the same adversarial texture is optimized jointly against region-based object detection and dense semantic segmentation.

2.3. Generative Adversarial Synthesis

Generative models have increasingly been used to parameterize adversarial perturbations rather than optimizing patch pixels directly [26,27]. Architectures such as GANs and encoder-decoder networks can map latent representations or spatial masks to structured adversarial signals [28]. Such parameterizations can impose spatial priors on the perturbation and have also been explored for generating more naturalistic adversarial modifications [29].

In the proposed framework, a U-Net generator [3] is used as a structural prior for adversarial texture synthesis. Rather than generating an unconstrained fixed patch, the synthesized texture is applied through a target-aligned mask that conforms to the object geometry. This design allows the adversarial signal to adapt spatially to targets with different sizes and orientations and provides an alternative to explicitly applying EoT transformations during the proposed optimization procedure [30].

2.4. Multi-Task Learning and Gradient Balancing

Jointly optimizing several prediction objectives is closely related to multi-task learning, where different tasks may generate gradients with conflicting directions or substantially different magnitudes. Gradient interference can prevent simultaneous improvement across objectives and has motivated approaches such as gradient surgery [31], uncertainty-based task weighting [32], and gradient normalization methods.

This issue becomes particularly important in our adversarial setting because Mask R-CNN contains multiple prediction heads, including RPN objectness, bounding-box regression, ROI classification, ROI regression, and instance-mask prediction, while the segmentation network introduces an additional dense-prediction objective. Directly combining these losses can allow individual terms to dominate the optimization. We therefore couple the proposed logarithmic-inversion formulation with several fixed and adaptive loss-balancing strategies. In particular, we investigate uniform weighting, manually selected static weights, cyclic partitioning, adaptive stochastic optimization, and GradNorm [33]. Rather than assuming that a single weighting mechanism is universally optimal, we empirically compare these strategies and analyze their effects on adversarial patch optimization.

3. Methodology

3.1. Threat Model

We consider a white-box, digital targeted attack against systems designed to distinguish ship from background. Specifically, we assume the attacker can train against a surrogate model, with full access to the model architecture and representative training data, to design a single, fixed patch for a camouflage attack. This white-box setting serves as a conservative, worst-case evaluation. It is realistic in remote sensing contexts where architectures, pre-trained weights, and training recipes are often shared or can be approximated from public datasets. For practicality, the attacker is constrained to modifying only the ship surface area (e.g., replacing image pixel values of the ship with the trained patch) and cannot alter sensor hardware or adapt the patch at inference time. We adopt this threat model to: (i) provide a reproducible upper bound on system vulnerability, and (ii) evaluate the transferability of the learned patch to black-box victims, demonstrating risk even when exact model parameters remain unavailable.

3.2. System Overview

The proposed SMO framework aims to synthesize a geometry-aware adversarial texture that concurrently suppresses region-proposal and dense-prediction pipelines. As illustrated in Figure 2, the pipeline comprises three core functional modules: (1) a generative model G that maps a latent noise manifold z to a universal adversarial patch p, (2) a geometry-aware poisoning function P that integrates the patch onto the ship hull x to form the adversarial input $x^{\mathrm{adv}}$, and (3) a dual-paradigm victim suite consisting of a detector F (Mask R-CNN) and a segmentor U (U-Net). These interactions are formalized as follows:

$$p=G\left(z\right),x^{\mathrm{adv}}=P(x,p)$$

(1)

$$(y_d,m_s)=(F\left(x\right),U\left(x\right)),(y_d^{\mathrm{adv}},m_s^{\mathrm{adv}})=(F\left(x^{\mathrm{adv}}\right),U\left(x^{\mathrm{adv}}\right))$$

(2)

where $y_d$ denotes the set of bounding boxes, classification scores, and instance masks, while $m_s$ represents the pixel-wise binary segmentation mask.

Figure 2. Schematic of the proposed SMO framework. A generator G synthesizes adversarial textures p onto ships via a geometry-aware function P from noise input z. The resulting image $x^{\mathrm{adv}}$ simultaneously targets a detector F (Mask R-CNN) and segmenter U (U-Net). SMO employs a logarithmic-inversion to balance seven loss heads—including RPN, ROI, and feature inconsistency—ensuring stable gradient fusion and comprehensive multi-task evasion.

Figure 2. Schematic of the proposed SMO framework. A generator G synthesizes adversarial textures p onto ships via a geometry-aware function P from noise input z. The resulting image $x^{\mathrm{adv}}$ simultaneously targets a detector F (Mask R-CNN) and segmenter U (U-Net). SMO employs a logarithmic-inversion to balance seven loss heads—including RPN, ROI, and feature inconsistency—ensuring stable gradient fusion and comprehensive multi-task evasion.

3.3. Generator Model

To mitigate high-frequency noise and improve cross-sensor transferability, we optimize over a latent manifold, utilizing a U-Net architecture to impose a structural prior that ensures spatial coherency. At each iteration t, the generator transforms a trainable latent tensor $z_t$ into a spatially-consistent patch: $p_t=G\left(z_t\right)$. The optimization simultaneously updates the weights of G and the coefficients of z. To prevent overfitting and ensure generalization, we employ an early-stopping mechanism governed by a dual-objective criterion:

$${\parallel z_t-z_{t-1}\parallel }_2<ϵ\mathrm{or}\Psi \left(x^{\mathrm{adv}}\right)<\tau$$

(3)

where $ϵ$ is a latent stability threshold and $\Psi (·)$ represents a performance metric (e.g., AP for detection or IoU for segmentation). Upon convergence, the generator is frozen, yielding a universal patch p for camouflage attack during inference.

3.4. Geometry-Aware Poisoning Function

The adversarial texture synthesized by G is a global signal $p\in {\mathbb{R}}^{3\times H\times W}$, where H and W are the sizes of the input image. We define a geometry-aware poisoning function P to fit ship’s geometry. The adversarial image $x^{\mathrm{adv}}$ is obtained through a convex combination of the clean image and the patch signal:

$$x^{\mathrm{adv}}=P(x,p)=x\odot (1-M_{\mathrm{hull}})+p\odot M_{\mathrm{hull}}$$

(4)

where $M_{\mathrm{hull}}$ denotes a binary elliptical mask strictly localized within the ship’s hull boundaries, and ⊙ represents the Hadamard product.

3.5. Standard Object Detection and Segmentation Objectives

To achieve comprehensive camouflage attack, the framework must suppress the standard supervisory signals of both the regional-proposal detector F and the dense segmentor U. We briefly list these objectives as follows:

Detector Objectives (${\mathcal{L}}_F$): The Mask R-CNN detector [2] utilizes a two-stage optimization process. The first stage, the Region Proposal Network (RPN), minimizes the objectness score ${\mathcal{L}}_{\mathrm{rpn}-\mathrm{obj}}$ and the anchor box regression loss ${\mathcal{L}}_{\mathrm{rpn}-\mathrm{reg}}$. The second stage, the ROIHead, performs region-wise classification and refines bounding box regression by minimizing ${\mathcal{L}}_{\mathrm{roi}-\mathrm{cls}}$ and ${\mathcal{L}}_{\mathrm{roi}-\mathrm{reg}}$, respectively. Additionally, an instance-level mask head minimizes the binary cross-entropy ${\mathcal{L}}_{\mathrm{mask}}$ within each proposal. Together, these constitute the set of detector loss heads:

$${\mathcal{L}}_F=\{{\mathcal{L}}_{\mathrm{rpn}-\mathrm{obj}},{\mathcal{L}}_{\mathrm{rpn}-\mathrm{reg}},{\mathcal{L}}_{\mathrm{roi}-\mathrm{cls}},{\mathcal{L}}_{\mathrm{roi}-\mathrm{reg}},{\mathcal{L}}_{\mathrm{mask}}\}$$

(5)

Segmentor Objective (${\mathcal{L}}_{\mathrm{seg}}$): The U-Net segmentor performs pixel-level semantic classification by minimizing a hybrid Dice and Cross-Entropy objective:

$${\mathcal{L}}_{\mathrm{seg}}=\mathrm{DiceCE}(m_d,m)$$

(6)

These six task-specific losses define the standard performance manifold. In the following sections, we describe how our logarithmic-inversion scheme transforms these losses into a stabilized adversarial training objective.

3.6. Stabilized Adversarial Optimization

Logarithmic Inversion for Gradient Stability.

To perform camouflage attack, the goal of training is to compromise those standard objectives listed above. A primary challenge in multi-objective adversarial training is the instability arising from the disparate magnitudes of the loss signals. A naive sign-reversal (i.e., ${\mathcal{L}}^{\mathrm{adv}}=-\mathcal{L}$) often yields a volatile optimization landscape, characterized by gradient explosion when $\mathcal{L}$ is large or vanishing gradients when $\mathcal{L}$ approaches zero. To enforce a stable, bounded gradient profile, we propose a logarithmic-inversion scheme. For each standard task loss ${\mathcal{L}}_i\in \{{\mathcal{L}}_{\mathcal{F}},{\mathcal{L}}_{\mathrm{seg}}\}$, we define its adversarial counterpart as:

$${\mathcal{L}}_i^{\mathrm{adv}}=log\left(1+{\displaystyle \frac{\lambda }{{\mathcal{L}}_i+\alpha }}\right)$$

(7)

where $\lambda >0$ modulates the inversion strength and $\alpha >0$ serves as a numerical stabilizer. Since ${\mathcal{L}}_i^{\mathrm{adv}}$ is a monotonically decreasing function of ${\mathcal{L}}_i$, minimizing the adversarial objective inherently drives the generator to maximize the model’s prediction error. The superiority of this formulation lies in its derivative with respect to the task loss:

$${\displaystyle \frac{d{\mathcal{L}}_i^{\mathrm{adv}}}{d{\mathcal{L}}_i}}=-{\displaystyle \frac{\lambda }{({\mathcal{L}}_i+\alpha )({\mathcal{L}}_i+\alpha +\lambda )}}$$

(8)

This design ensures that when the attack is failing (i.e., ${\mathcal{L}}_i\to 0$), the gradient remains bounded away from zero: $|d{\mathcal{L}}_i^{\mathrm{adv}}/d{\mathcal{L}}_i|\approx \lambda /\left(\alpha (\alpha +\lambda )\right)$. This prevents gradient vanishing and provides a persistent update signal to the generator. Conversely, as the attack succeeds (i.e., ${\mathcal{L}}_i\to \infty$), the gradient naturally decays at a rate of $\mathcal{O}\left({\mathcal{L}}_i^{-2}\right)$, providing a smooth saturation that prevents the optimizer from over-focusing on already-evaded loss heads.

Feature Inconsistency and Joint Objective.

Beyond surface-level predictions, we aim to disrupt the latent representations within the detection backbone. We introduce a feature-inconsistency loss, formulated as the squared $L_2$ distance between the RPN feature embeddings of the clean and adversarial inputs:

$${\mathcal{L}}_{\mathrm{feat}}={\parallel \mathrm{RPN}\left(x\right)-\mathrm{RPN}\left(x^{\mathrm{adv}}\right)\parallel }_2^2,{\mathcal{L}}_{\mathrm{feat}}^{\mathrm{adv}}=log\left(1+{\displaystyle \frac{\lambda }{{\mathcal{L}}_{\mathrm{feat}}+\alpha }}\right)$$

(9)

By maximizing the divergence in the feature manifold, we encourage large perceptual shifts that further degrade the robustness of the region proposal mechanism. The final total adversarial loss is constructed as a weighted composition of the seven stabilized adversarial terms:

$${\mathcal{L}}^{\mathrm{adv}}=\sum _{i\in \mathcal{S}}{w}_i{\mathcal{L}}_i^{\mathrm{adv}}+w_{\mathrm{feat}}{\mathcal{L}}_{\mathrm{feat}}^{\mathrm{adv}}$$

(10)

where $\mathcal{S}$ represents the set of detector losses and the segmentation loss. This joint objective ensures that the synthesized patch p is optimized to simultaneously bypass ship detection of both the detector F and the segmentor U.

3.7. Adversarial Training Dynamics and Multi-Objective Strategies

The adversarial objective ${\mathcal{L}}^{\mathrm{adv}}$ is composed of seven distinct loss terms, creating a complex optimization landscape where individual objectives may compete or vary significantly in scale. We investigate six strategic configurations designed to stabilize the training of the adversarial patch and ensure robust convergence:

• Setting A (Primitive Optimization): Directly optimizes p from noise z while bypassing G, establishing an unconstrained baseline for adversarial efficacy.
• Setting B (Uniform Weighting): Optimizes both G and z with an equal weight for each of the components in ${\mathcal{L}}^{\mathrm{adv}}$, assuming equal contribution among them.
• Setting C (Static Heuristics): This approach utilizes fixed weights determined through extensive trial and error, thereby leveraging empirical intuition to mitigate the negative influence of several dominant loss signals.
• Setting D (Cyclic Partitioning): Optimizes a single loss term per iteration, with an aim to decouple the objectives to prevent gradient interference.
• Setting E (Adaptive Stochasticity): Utilizes a probabilistic sampling strategy where the optimization focus is biased toward high-impact terms, allowing the model to dynamically address the most significant deficiencies.
• Setting F (Dynamic Gradient Normalization): Implements GradNorm [33] to automatically calibrate weights in real-time. By balancing gradient magnitudes across the seven terms, this strategy ensures that no single objective overwhelms the learning process, maintaining a stable and consistent descent.

A comprehensive comparative analysis of these strategies is detailed in Section 5.5, while the finalized training procedure is formalized in Algorithm 1.

Algorithm 1 Proposed Adversarial Patch Training

1: Input: Target detectors $\{F,U\}$ (Mask R-CNN, U-Net), patch generator G, dataset ${\mathcal{D}}_{\mathrm{train}}$, batch size b.  2: Output: Optimized adversarial patch p.  3: Initialize: Latent noise tensor $z\sim \mathcal{N}(0,I)$.  4: while training objective not converged do  5:     Sample a mini-batch $\mathcal{X}={\left\{x_i\right\}}_{i=1}^b$ from ${\mathcal{D}}_{\mathrm{train}}$.  6:     Synthesize Patch: $p=G\left(z\right)$.  7:     Adversarial Projection: For each $x_i\in \mathcal{X}$, generate $x_i^{\mathrm{adv}}=P(x_i,p)$.  8:     Loss Computation: Evaluate multi-objective adversarial loss ${\mathcal{L}}^{\mathrm{adv}}$ by processing $\{\mathcal{X},{\mathcal{X}}^{\mathrm{adv}}\}$ through F and U.  9:     Optimization Step: Update weights of G and latent vector z via the selected strategy (e.g., GradNorm) as detailed in Section 3.7. 10: end while

3.8. Baseline Competitors and Comparative Objectives

We benchmark our approach against four general-purpose adversarial patch attacks: 1) Universal Patch (Brown et al. [8]), designed for classification and adapted here for segmentation, 2) DPatch (Liu et al. [11]), which directly optimizes pixels to minimize detector confidence, 3) ShapeShifter (Chen et al. [34]), which introduces multi-objective optimization to attack both classification and localization, and 4) Scale-Adaptive Attack (Lu et al. [23]), which conditions optimization on an empirical scale distribution. We also compare against several remote sensing and aerial camouflage methods  [21,24,25,35]. While all baselines rely on the Expectation-over-Transformation (EoT) framework [8] to explicitly model spatial variance via stochastic sampling, our method bypasses this requirement. By leveraging a U-Net generator to produce hull-aligned textures, the model implicitly internalizes geometric and scale invariance during synthesis. This enables robust attacks across diverse orientations and resolutions without the computational overhead of explicit spatial augmentations. Please refer to Section A.1 of the Appendix for further details.

Core Methodological Distinctions: Our proposed SMO framework diverges from the aforementioned baselines in three fundamental aspects:

• Joint optimization: Existing attacks focus on isolated tasks either for classification or detection, our approach executes a joint optimization across detector and segmentation, and prevents the "objective interference".
• Generative Patch Parameterization: Baseline methods optimize raw pixel values p directly. We parameterize the adversarial manifold via a generator G, ensuring the synthesis textures are more resilient to satellite surveillance.
• Implicit Geometric Invariance: Existing methods rely on explicit EoT sampling to handle various variations. Our method utilizes hull-aligned extraction to conform naturally to the ship’s variance of orientation and scale.

4. Experimental Setup and Evaluation Metrics

4.1. Maritime and Aerial Benchmarks.

We primarily evaluate on the Airbus ship dection dataset [36], filtering for the 42,556 images containing valid ship instances (40,428 training/2,128 testing). The dataset’s example images and multi-instance distribution are shown in Figure 3 and Figure 4. To validate generalizability and ensure a rigorous comparison against standard remote sensing camouflage attacks [21,35], we extend our evaluation to the DOTA [37] and VisDrone [38] benchmarks. This allows us to confirm the transferability of our SMO optimization method over different object types and dataset. Table 1 summarizes the information about all the three dataset.

4.2. Victim Model Architectures and Implementation Details.

We target a Mask R-CNN [2] with a ResNet-101 FPN backbone as the primary victim model. To evaluate patch transferability, we further benchmark against ResNet-50 and ResNeXt backbones, as well as YOLOv2 and YOLOv3 [14] paradigms. All optimization pipelines are built on the Detectron2 framework [39], with our generator G and SMO logic integrated into a PyTorch-based pipeline. This setup facilitates concurrent gradient backpropagation from heterogeneous models, ensuring a rigorous evaluation across distinct architectural designs. More details about dataset and hyperparameters used during the training can be found in Appendix sections A, B.

4.3. Performance Metrics

To evaluate adversarial impact across detection and segmentation, we employ a suite of standard metrics based on Intersection-over-Union (IoU). For object detection, we classify predictions as True Positives (TP) or False Positives (FP) at an IoU threshold of 0.50, while semantic segmentation metrics are computed at the pixel level. Based on these counts, we report Precision, Recall, and the F1-score to capture the balance between localization and classification. Our primary measure of adversarial efficacy is Average Precision (AP), representing the area under the precision-recall curve. Specifically, we report ${\mathrm{AP}}_{50}$ and the COCO-standard mAP, averaged across IoU thresholds from 0.50 to 0.95, to assess performance across varying levels of spatial rigor.

5. Results and Discussions

5.1. Results on Airbus Dataset

The comparative performance on the Airbus Ship Detection dataset is summarized in Table 2, where we delineate efficacy across object scales using ${\mathrm{AP}}_s$ (small), ${\mathrm{AP}}_m$ (medium), and ${\mathrm{AP}}_l$ (large). SMO demonstrates a decisive advantage, consistently degrading detection performance across the entire scale spectrum. A critical observation is the scale-sensitivity of traditional baselines. For instance, while DPatch [11] exhibits a significant reduction in ${\mathrm{AP}}_s$, this is primarily a consequence of its fixed $32\times 32$ spatial footprint, which physically occludes a majority of small ship instances. However, its effectiveness diminishes precipitously for larger vessels (minimal reduction in ${\mathrm{AP}}_l$), as the fixed-size patch occupies an insignificant fraction of the object’s feature map. In contrast, our approach utilizes a generative, geometry-aware extraction mechanism that adaptively scales the adversarial texture relative to the ship’s hull.

This ensures a balanced degradation across all categories without requiring total occlusion. SMO maintains adversarial capability on large targets, underscoring the importance of our "hull-aligned" synthesis. Visual comparisons of the patches generated by each baseline are provided in Figure 5, illustrating the transition from rigid, square perturbations to our fluid, geometry-conforming textures.

5.2. Impact of Patch Coverage Percentage

To quantify the sensitivity of the victim models to adversarial surface area, we evaluate performance as a function of hull coverage ($0\%-100\%$). As detailed in Table 3, we observe a non-linear degradation in all metrics, with a critical ‘collapse threshold’ occurring beyond $50\%$ coverage. At full occlusion ($100\%$), precision and recall plummet to $2.3\%$ and $4.4\%$, respectively. These results validate that while the patch is most potent when conforming to the majority of the ship’s hull, the SMO framework begins to compromise feature extraction logic even at partial coverage, demonstrating high efficiency in adversarial settings.

5.3. Results on DOTA and VisDrone Datasets

To assess the generalizability of the proposed SMO framework beyond satellite-based maritime contexts, we benchmark against state-of-the-art camouflage attacks [21,35] on the DOTA [37] and VisDrone [38] datasets. In these experiments, we train the adversarial patch by jointly optimizing through Mask R-CNN and U-Net and utilize YOLOv2 and YOLOv3 as victim detectors to evaluate the attack’s efficacy against one-stage, anchor-based architectures. As summarized in Table 4, our method consistently induces a more profound degradation in ${\mathrm{AP}}_{50}$ than the baseline competitors across all configurations.

A notable result occurs on the DOTA benchmark using the YOLOv2 detector: while the strongest existing baseline achieves a 57.1% reduction in AP, our SMO framework collapses the detection performance from an 88.0% baseline to a mere 6.0%, representing a 93.1% relative decrease. This significant performance gap suggests that while traditional camouflage attacks successfully incorporate physical-realism priors (e.g., TV and NPS losses), they lack the gradient stability and multi-objective alignment necessary to fully compromise the detector’s feature extraction logic. The consistent superiority of our approach across disparate YOLO versions further validates the transferability of our synthesized textures, confirming that our generative strategy successfully captures universal adversarial vulnerabilities in the remote sensing domain.

5.4. Results on Transferability Experiments

To evaluate the generalizability of our SMO-synthesized textures, we deployed patches trained on the Mask R-CNN (ResNet-101) victim against a diverse suite of architectures. As shown in Table 5, the adversarial potency remains exceptionally high across structurally related convolutional frameworks, with the source model experiencing a catastrophic collapse in $F_1$ and mIoU (dropping from 70.2/86.4 to 16.3/28.5). We observe significant cross-task transfer to U-Net and FPN backbones, confirming that our multi-objective optimization captures fundamental geometric vulnerabilities within the CNN manifold. Conversely, a ’robustness gap’ emerges in Vision Transformer (ViT) and MobileNet paradigms. This suggests that the global self-attention mechanisms of ViTs are inherently more resilient to the locally-optimized, hull-aligned perturbations intended for convolutional kernels—a finding that highlights architectural diversity as a critical defense for maritime surveillance.

Figure 6. Patches generated by different training strategies.

Figure 6. Patches generated by different training strategies.

5.5. Comparison of Patch Optimization Strategies

We evaluated the training strategies described in Section 3.7 with a 40% coverage. Table 6 shows that the fixed-weights strategy performs best. The GradNorm and stochastic variants also show promise, though they are less aggressive.

GradNorm helped identify suitable coefficients for each loss, and the results indicate that U-Net–based optimization outperforms direct optimization (Adam) in both control and effectiveness. These findings underscore the importance of structured, multi-loss balancing when optimizing adversarial patches.

5.6. Effect of Patch Shape

We evaluate the influence of patch shape on adversarial efficacy using a Mask R-CNN and U-Net pipeline (Table 7). The oval (elliptical) patch proved most effective, reducing segmentation F1 to 30.6 and detection F1 to 16.6. This suggests that elongated contours more effectively disrupt the spatial context and boundary features prioritized by these architectures. In contrast, circular and rectangular geometries were less disruptive, with the rectangular patch preserving the highest performance across both tasks. These results indicate that patch geometry is a critical factor in adversarial robustness for shape-sensitive models.

5.7. Large Patch vs. Small Patches

Covering 60% of a ship’s surface with a single monolithic patch may be impractical due to complex superstructures. We therefore compare a single-patch strategy against multi-patch configurations (comprising two or three disjoint patches) while maintaining a fixed total coverage budget of 60% to ensure a fair evaluation. Figure 7 shows that distributed layouts achieve slightly higher adversarial effectiveness (Table 8), suggesting that multi-patch designs not only accommodate physical structural constraints but may also disrupt feature extraction more effectively, making them a preferred choice for practical camouflage applications.

5.8. Ablation Study

The proposed method consists of three components: the U-Net generator G to generate the adversarial patch, the log-adversarial loss {${\mathcal{L}}_i^{\mathrm{adv}},i\in \mathrm{Standard}\mathrm{loss}$}, and the feature inconsistence loss ${\mathcal{L}}_{\mathrm{feat}}^{\mathrm{adv}}$. We performed an ablation study to investigate the contribution of each component. In experiments, when the log-adversarial loss was omitted, the traditional negative adversarial loss was employed. Table 9 demonstrates that including each component improves the proposed model for camouflage attack, and the most significant component is the U-Net generator model, indicating that utilizing a U-Net to generate the adversarial patch is much more effective than optimizing the patch directly.

6. Conclusions

This work introduces a geometry-aware and resolution-robust adversarial attack framework for maritime surveillance systems. The proposed method generates hull-aligned adversarial textures using a stabilized multi-objective optimization (SMO) strategy, enabling a single attack to simultaneously disrupt detection, localization, and segmentation tasks. To address conflicts between different network objectives, the framework employs a logarithmic-inversion loss and dynamic gradient balancing, improving attack stability across varying image scales and resolutions. Experiments on multiple remote sensing benchmarks show that the approach significantly outperforms conventional attacks targeting only detection or segmentation models. The results reveal major security vulnerabilities in modern multi-task maritime monitoring systems and highlight the need for stronger geometry-aware defense strategies.

Appendix A.1. Baseline Competitors and Comparative Objectives

We benchmark our approach against general-purpose adversarial patch attacks:

Brown et al. [8] (classification-based attack): Brown et al. generate a universal adversarial patch under the Expectation over Transformation (EoT) framework $\mathcal{T}$. For our ship-detection setting, we adapt the original image-classification objective to binary ship/background segmentation:

$$\underset{p}{min}{\mathbb{E}}_{T\sim \mathcal{T}}{\mathcal{L}}_{\mathrm{cls}}\left(f\left(T\right(P(x,p)\left)\right),y\right)$$

(A1)

where p is the adversarial patch, $P(x,p)$ overlays p onto image x, T denotes random physical transformations, $f(·)$ is the victim model, and y is the target ship label.

DPatch [11] (object-detection attack): DPatch suppresses detector objectness/confidence scores by directly optimizing patch pixels under EoT:

$$\underset{p}{min}{\mathbb{E}}_{T\sim \mathcal{T}}{\mathcal{L}}_{\mathrm{obj}}\left(f\left(T\right(P(x,p)\left)\right),y\right)$$

(A2)

where ${\mathcal{L}}_{\mathrm{obj}}$ minimizes objectness or detection confidence for true ship instances.

ShapeShifter [34] (detection and localization attack): ShapeShifter jointly attacks detector classification and localization objectives:

$$\begin{array}{cc}\hfill \underset{p}{min}{\mathbb{E}}_{T\sim \mathcal{T}}& [{\mathcal{L}}_{\mathrm{cls}}\left(f\left(T\left(P(x,p)\right)\right),y_{\mathrm{cls}}\right)\hfill \\ \hfill & +{\lambda }_{\mathrm{loc}}{\mathcal{L}}_{\mathrm{loc}}\left(f\left(T\left(P(x,p)\right)\right),y_{\mathrm{loc}}\right)]\hfill \end{array}$$

(A3)

where ${\mathcal{L}}_{\mathrm{loc}}$ is the bounding-box regression loss and ${\lambda }_{\mathrm{loc}}$ balances classification and localization objectives.

Scale-Adaptive [23] (multi-scale detection attack): The scale-adaptive method improves robustness across object sizes by sampling scales $s\sim S$ during optimization:

$$\underset{p}{min}{\mathbb{E}}_{s\sim S}{\mathbb{E}}_{T\sim \mathcal{T}}{\mathcal{L}}_{\mathrm{task}}\left(f\left(T_s\left(P_s(x,p)\right)\right),y\right)$$

(A4)

where S is the empirical object-scale distribution, and $T_s,P_s$ denote scale-conditioned transformations and patch placement operations.

Camouflage Attacks for Aerial Imagery [21,24,25,35]: These methods extend detector-oriented patch attacks by incorporating physical-realism constraints such as smoothness and printability. All methods optimize adversarial patches against a pretrained YOLO detector [40]:

$$\begin{array}{cc}\hfill \underset{p}{min}{\mathbb{E}}_{s\sim S}{\mathbb{E}}_{T\sim \mathcal{T}}& [{\mathcal{L}}_{\mathrm{task}}\left(f\left(T_s\left(P_s(x,p)\right)\right),y\right)\hfill \\ \hfill & +{\lambda }_{\mathrm{TV}}{\mathcal{L}}_{\mathrm{TV}}\left(p\right)+{\lambda }_{\mathrm{NPS}}{\mathcal{L}}_{\mathrm{NPS}}\left(p\right)]\hfill \end{array}$$

(A5)

where ${\mathcal{L}}_{\mathrm{task}}$ includes detector classification and box-regression losses, ${\mathcal{L}}_{\mathrm{TV}}$ enforces spatial smoothness, ${\mathcal{L}}_{\mathrm{NPS}}$ improves printability, and ${\lambda }_{\mathrm{TV}},{\lambda }_{\mathrm{NPS}}$ control their relative importance.

Appendix A.2. Dataset Statistics and Examples

We evaluated our method on three datasets covering ships, airplanes, and vehicles. Figure A1, Figure A2 and Figure A3 show representative examples with bounding-box annotations.

Figure A1. Examples from the Ship dataset with detected ships.

Figure A1. Examples from the Ship dataset with detected ships.

Figure A2. Examples from the DOTA dataset with detected planes.

Figure A2. Examples from the DOTA dataset with detected planes.

Figure A3. Examples from VisDrone with detected cars.

Figure A3. Examples from VisDrone with detected cars.

Appendix A.3. Hyperparameters for Model Pretraining and Adversarial Patch Training

Table A1 summarizes the hyperparameters used to train all pre-trained models. Most networks are trained using a fixed learning rate of $1\mathrm{e}-4$. For deeper or higher-capacity models, we adopt the WarmUpMultiStepLR schedule, which starts at $1\mathrm{e}-6$ and increases by a factor of 10 after the first 1000 iterations until reaching $1\mathrm{e}-4$. This stabilizes early optimization for large architectures.

In Section 3.7, we described different adversarial patch training strategies, and Section 5.5 presented their performance. Table A2 lists the coefficients for the different strategies.

Table A1. Hyperparameters used for model pretraining. Larger models used the WarmUpMultiStepLR schedule.

Model Structure	Backbone	# Parameters	Learning Rate
UNet	ResNet34	24M	$1\mathrm{e}-4$
FPN	ResNet34	23M	$1\mathrm{e}-4$
Mask R-CNN FPN	ResNet101	101M	WarmupMultiStepLR
Faster R-CNN FPN	RetinaNet50	38M	WarmupMultiStepLR
DeepLabV3+	MobileNet	4M	$1\mathrm{e}-4$
Faster R-CNN C4	ResNet50	32M	WarmupMultiStepLR
R-CNN FPN	ResNetXt	142M	WarmupMultiStepLR
PAN	EfficientNet	6M	$1\mathrm{e}-4$
Mask-RCNN	MViTV2-T	43M	WarmupMultiStepLR
Mask R-CNN	ViTDet-B	110M	WarmupMultiStepLR
Cascade Mask R-CNN	Swin-B	139M	WarmupMultiStepLR

Table A1. Hyperparameters used for model pretraining. Larger models used the WarmUpMultiStepLR schedule.

Model Structure	Backbone	# Parameters	Learning Rate
UNet	ResNet34	24M	$1\mathrm{e}-4$
FPN	ResNet34	23M	$1\mathrm{e}-4$
Mask R-CNN FPN	ResNet101	101M	WarmupMultiStepLR
Faster R-CNN FPN	RetinaNet50	38M	WarmupMultiStepLR
DeepLabV3+	MobileNet	4M	$1\mathrm{e}-4$
Faster R-CNN C4	ResNet50	32M	WarmupMultiStepLR
R-CNN FPN	ResNetXt	142M	WarmupMultiStepLR
PAN	EfficientNet	6M	$1\mathrm{e}-4$
Mask-RCNN	MViTV2-T	43M	WarmupMultiStepLR
Mask R-CNN	ViTDet-B	110M	WarmupMultiStepLR
Cascade Mask R-CNN	Swin-B	139M	WarmupMultiStepLR

Table A2. Coefficients of the Loss Components for Adversarial Patch Training. Values in GradNorm represent the converged coefficients.

Training Strategy	${\mathit{\alpha }}_{\mathbf{rpn}-\mathbf{obj}}$	${\mathit{\alpha }}_{\mathbf{rpn}-\mathbf{reg}}$	${\mathit{\alpha }}_{\mathbf{feat}}$	${\mathit{\alpha }}_{\mathbf{roi}-\mathbf{cls}}$	${\mathit{\alpha }}_{\mathbf{roi}-\mathbf{reg}}$	${\mathit{\alpha }}_{\mathbf{mask}}$	${\mathit{\alpha }}_{\mathbf{seg}}$
Uniform Weighting	1.0	1.0	1.0	1.0	1.0	1.0	1.0
Static Heuristics	1.0	1e-3	1e-3	1.0	1e-3	1e-2	1e-2
GradNorm	1.0	1e-8	1e-8	1e-8	1e-8	1e-8	1e-8

Table A2. Coefficients of the Loss Components for Adversarial Patch Training. Values in GradNorm represent the converged coefficients.

Training Strategy	${\mathit{\alpha }}_{\mathbf{rpn}-\mathbf{obj}}$	${\mathit{\alpha }}_{\mathbf{rpn}-\mathbf{reg}}$	${\mathit{\alpha }}_{\mathbf{feat}}$	${\mathit{\alpha }}_{\mathbf{roi}-\mathbf{cls}}$	${\mathit{\alpha }}_{\mathbf{roi}-\mathbf{reg}}$	${\mathit{\alpha }}_{\mathbf{mask}}$	${\mathit{\alpha }}_{\mathbf{seg}}$
Uniform Weighting	1.0	1.0	1.0	1.0	1.0	1.0	1.0
Static Heuristics	1.0	1e-3	1e-3	1.0	1e-3	1e-2	1e-2
GradNorm	1.0	1e-8	1e-8	1e-8	1e-8	1e-8	1e-8

Appendix A.4. Additional Results on the Transferability of Adversarial Patches

We investigated the transferability of adversarial patches optimized with different models. In this experiment, we obtained three adversarial patches by optimizing them with Faster R-CNN, U-Net, and jointly with both models, respectively. Note that Faster R-CNN detects objects through region proposals, whereas U-Net performs detection via segmentation. Using these three adversarial patches, we attacked both the Faster R-CNN and U-Net models. Table A3 and Table A4 list the performance of the camouflage attacks. We observe that the jointly trained patch yields a more balanced degradation in detection performance for both the Faster R-CNN and U-Net models. In contrast, patches trained solely on Faster R-CNN show limited transferability to the segmentation model, while U-Net–trained patches heavily degrade segmentation but do not reliably affect detection.

Table A3. Performances of camouflage attacks on Faster R-CNN using patches optimized with different models, where “Joint Optimization” denotes that the patch was optimized using both Faster R-CNN and U-Net.

Applied Patch	Precision	Recall	F1	mIoU
No patch	76.7	96.6	85.5	87.8
Patch with Faster R-CNN	31.7	39.0	34.9	39.3
Patch with U-Net	78.5	95.8	86.3	82.0
Joint Optimization	25.9	36.2	30.2	38.4

Table A3. Performances of camouflage attacks on Faster R-CNN using patches optimized with different models, where “Joint Optimization” denotes that the patch was optimized using both Faster R-CNN and U-Net.

Applied Patch	Precision	Recall	F1	mIoU
No patch	76.7	96.6	85.5	87.8
Patch with Faster R-CNN	31.7	39.0	34.9	39.3
Patch with U-Net	78.5	95.8	86.3	82.0
Joint Optimization	25.9	36.2	30.2	38.4

Table A4. Performances of Camouflage Attacks on U-Net Using Patches Optimized with Different Models.

Applied Patch	Precision	Recall	F1	mIoU
No patch	86.9	91.9	88.2	80.9
Patch with Faster R-CNN	82.2	39.3	49.7	35.0
Patch with U-Net	42.1	1.8	8.9	1.82
Joint Optimization	74.1	19.8	32.8	17.8

Table A4. Performances of Camouflage Attacks on U-Net Using Patches Optimized with Different Models.

Applied Patch	Precision	Recall	F1	mIoU
No patch	86.9	91.9	88.2	80.9
Patch with Faster R-CNN	82.2	39.3	49.7	35.0
Patch with U-Net	42.1	1.8	8.9	1.82
Joint Optimization	74.1	19.8	32.8	17.8

Appendix A.5. Additional Camouflage Attack Examples

Figure A5 presents additional ship-detection results by Mask R-CNN and U-Net under camouflage attack using the adversarial patch jointly optimized by the proposed model, illustrating successful, partially successful, and failed adversarial cases. Figure A4 shows examples from the DOTA dataset where YOLOv2 is subjected to camouflage attacks, with the adversarial patch optimized by the proposed method.

Appendix A.6. Discussions

Results in appendix strengthen the core findings of the paper. The dataset examples further demonstrate that the method generalizes across object types and imaging conditions, reinforcing the practical relevance of the proposed adversarial design.

Figure A4. Detection results on DOTA images. The first row shows YOLOv2 predictions, and the second row shows Mask R-CNN predictions, for both clean and adversarial inputs.

Figure A4. Detection results on DOTA images. The first row shows YOLOv2 predictions, and the second row shows Mask R-CNN predictions, for both clean and adversarial inputs.

Figure A5. Camouflage attack examples for Mask R-CNN (detection) and U-Net (segmentation) on clean and adversarial ship images.

Figure A5. Camouflage attack examples for Mask R-CNN (detection) and U-Net (segmentation) on clean and adversarial ship images.