Spatiotemporal Alignment for Remote Sensing Image Recovery via Terrain-Aware Diffusion

1. Introduction

Remote sensing imagery is a critical data source for environmental monitoring, agricultural planning, and disaster response [1,2]. However, satellite observations are frequently disrupted by cloud cover, sensor failure, or acquisition gaps [3,4], especially in high-resolution or high-frequency scenarios. These disruptions yield temporally and spatially incomplete datasets that reduce analysis accuracy, complicate multi-source fusion, and undermine reliable long-term monitoring. In climate-vulnerable and data-scarce regions, such deficiencies further exacerbate global inequities in access to environmental intelligence.

Traditional solutions, including interpolation and deep learning-based image completion, focus on pixel-level plausibility [5,6] but often fail to preserve structural, spectral, or temporal consistency, especially in semantically complex or geographically diverse regions [7]. Moreover, many approaches overlook physical priors such as terrain morphology or acquisition geometry and rely heavily on large labeled datasets, leading to reconstructions that diverge from geophysical reality and limiting scientific and operational utility.

To address these challenges, we reformulate satellite image reconstruction as a spatiotemporal alignment problem, aiming to infer content consistent with both spatial landscape and temporal dynamics. This perspective emphasizes context-aware reasoning and geophysically grounded alignment, moving beyond standard completion paradigms.

We present AlignDiff, a diffusion-based framework that treats remote sensing image reconstruction as a spatiotemporal alignment task. AlignDiff integrates Digital Elevation Models (DEMs) as terrain priors and uses location–time prompts to guide generation, while a lightweight alignment adapter enforces distributional consistency between generated and observed regions. Our main contributions are:

• Casting reconstruction as a unified spatiotemporal alignment problem for simultaneous spatial and temporal recovery;
• Proposing AlignDiff, combining terrain priors and prompt-based conditioning for physically consistent completion;
• Designing an alignment adapter to improve generalization across terrains and missing patterns.

Figure 1. The AlignDiff pipeline integrates DEM-guided diffusion generation, LoRA-based prompt adaptation, and VGG-based distribution alignment for reconstructing missing satellite data across spatial and temporal gaps.

Figure 1. The AlignDiff pipeline integrates DEM-guided diffusion generation, LoRA-based prompt adaptation, and VGG-based distribution alignment for reconstructing missing satellite data across spatial and temporal gaps.

2. Proposed Framework: AlignDiff

We propose AlignDiff, a unified diffusion-based generative framework that reconstructs missing satellite observations through three complementary alignment mechanisms. (1) spatial alignment via terrain-aware priors; (2) distribution alignment to enforce perceptual consistency; and (3) semantic alignment through prompt-conditioned generation.

2.1. Problem Formulation and Task Setting

Let $x\in {\mathbb{R}}^{C\times H\times W}$ be the complete image and $x_{\mathrm{obs}}$ its partially observed version with missing regions $\mathcal{M}\subset \mathrm{\Omega }$. The goal is to generate $\widehat{x}$ that fills $\mathcal{M}$ while preserving spatial, spectral, and contextual consistency. We learn the conditional distribution

$$p(\widehat{x}\mid x_{\mathrm{obs}},a_{\mathrm{spatial}},a_{\mathrm{dist}},a_{\mathrm{semantic}})$$

(1)

where $a_{\mathrm{spatial}}$ encodes DEM-based terrain priors, $a_{\mathrm{dist}}$ enforces distributional alignment, and $a_{\mathrm{semantic}}$ provides prompt-based semantic conditioning.

We address two tasks: Task-1 (Spatial Completion): reconstruct missing regions across spatially distinct tiles at a fixed time t; Task-2 (Temporal Completion): recover observations at missing timepoints $\left\{t_k\right\}$ for a fixed location s using semantics and context.

2.2. Three-Way Alignment Strategy

To address the complex and heterogeneous nature of missing patterns in satellite imagery, AlignDiff integrates three alignment modules: spatial alignment with terrain priors, distribution alignment for perceptual fidelity, and semantic alignment via prompt-aware adaptation. Each module contributes complementary inductive biases, as detailed below.

2.2.1. Spatial Alignment with DEM Conditioning.

To incorporate physical topography and enforce terrain-aware realism, we adopt a ControlNet-style branch [8] that conditions the diffusion process on Digital Elevation Models (DEMs). This spatial alignment guides generation in mountainous, coastal, and varied terrain regions.

Let $c_{\mathrm{spatial}}$ denote the DEM-based conditioning input. The conditional denoising process is formulated as:

$$p(\widehat{x}\mid c_{\mathrm{spatial}})=\prod _{t=1}^{T}p(x_t\mid x_{t-1},c_{\mathrm{spatial}})$$

(2)

where $x_t$ denotes the latent image at timestep t. The spatial reconstruction is supervised via a pixel-wise MSE loss:

$${\mathcal{L}}_{\mathrm{recon}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\parallel x_i-{\widehat{x}}_i\parallel }^2$$

(3)

2.2.2. Distribution Alignment with VGG Adapter.

To correct brightness imbalance, texture distortion, and spectral inconsistencies, we introduce a VGG-based adapter that aligns feature distributions between generated and ground-truth images.

Let $\varphi (·)$ denote features extracted from a pretrained VGG-19 network. We minimize the Maximum Mean Discrepancy (MMD) between the empirical feature distributions:

$${\mathcal{L}}_{\mathrm{dist}}={∥{\mathbb{E}}_x\left[\varphi \left(x\right)\right]-{\mathbb{E}}_{\widehat{x}}\left[\varphi \left(\widehat{x}\right)\right]∥}^2$$

(4)

Additionally, we include a multi-level style loss to regularize fine-grained texture via Gram matrix alignment:

$${\mathcal{L}}_{\mathrm{style}}=\sum _{l=1}^L{\displaystyle \frac{1}{4C_l^2{H}_l^2{W}_l^2}}{∥G_{\widehat{x}}^l-G_x^l∥}_F^2$$

(5)

where $G^l$ is the Gram matrix of VGG features at layer l.

2.2.3. Semantic Alignment with Prompt.

To condition the generation process on high-level semantics—such as location, seasonality, or band type—we introduce a lightweight adaptation module using Low-Rank Adaptation (LoRA) [9]. A task-specific token <AlignDiff> is prepended to the text encoder, enabling fine-grained prompt control across domains.

For each task, the injected prompt is structured as:

Task 1: <AlignDiff>Region, Band

Task 2: <AlignDiff>Region, Date, Day, Band

LoRA adapts the attention weight matrix $W\in {\mathbb{R}}^{d\times d}$ by injecting a low-rank perturbation:

$$W^{\prime }=W+AB,A\in {\mathbb{R}}^{d\times r},B\in {\mathbb{R}}^{r\times d}$$

(6)

This design ensures efficient parameter tuning while maintaining generalization and cross-domain adaptability.

2.3. Unified Loss Function and Training

To jointly optimize spatial realism, statistical fidelity, and semantic control, we formulate a unified training objective that balances all three alignment mechanisms. The total loss is defined as:

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_{\mathrm{recon}}+{\lambda }_{\mathrm{dist}}{\mathcal{L}}_{\mathrm{dist}}+{\lambda }_{\mathrm{style}}{\mathcal{L}}_{\mathrm{style}}$$

(7)

Here, ${\lambda }_{\mathrm{dist}}$ and ${\lambda }_{\mathrm{style}}$ are hyperparameters used to control the influence of perceptual distribution and style consistency. In practice, we set ${\lambda }_{\mathrm{dist}}=1.0$ and ${\lambda }_{\mathrm{style}}=100.0$ based on validation performance.

3. Experiment

3.1. Implementation Details

Dataset Description. Task 1 – Landsat-8 Multi-Region. We use Landsat-8 imagery and DEM data from 10 diverse regions covering various climates and land covers. Images (30 m resolution) are filtered in Google Earth Engine (cloud $<1\%$) and tiled into $512\times 512$ patches with 50% overlap. Task 2 – EarthNet2021. EarthNet2021 [10] offers 200k+ Sentinel-2 sequences for spatiotemporal modeling. Samples with $>5\%$ invalid pixels are removed, and splits follow the official IID/OOD protocol. Masking. We adopt a unified masking strategy: (1) remove cloudy pixels, (2) randomly mask 10–50% valid pixels to simulate missing data, and (3) store indices for reproducible evaluation across both tasks.

Experimental Settings. Experiments ran on a single NVIDIA A100 (80 GB). The diffusion model was trained at $512\times 512$ resolution with learning rate $5\times {10}^{-5}$. Inference used DDIM (50 steps, guidance 0.9, classifier-free scale 1.0, $\eta =1.0$). STCNN baselines used batch size 16, learning rate $1\times {10}^{-4}$, 100 epochs. An 80:20 train-test split was applied for both tasks.

Evaluation Metrics. We evaluate AlignDiff using these metrics: Root Mean Square Error (RMSE) [11] and Mean Absolute Error (MAE) [12] for pixel-level accuracy, Peak Signal-to-Noise Ratio (PSNR) [13] and Structural Similarity Index (SSIM) [14] for image quality and structural similarity, and Learned Perceptual Image Patch Similarity (LPIPS) [15] for perceptual similarity.

3.2. Comparison with Existing Methods

Task-1: Spatial Completion across Diverse Regions. We compare with state-of-the-art methods including natural image inpainting models, unconditioned diffusion models, and terrain-aware diffusion baselines. Palette [16] and LaMa [17], originally designed for natural image inpainting, demonstrate limited transferability to the satellite domain. Despite LaMa showing improved robustness among the two, both methods fail to preserve geophysical structure in large or topographically varied missing areas. Stable Diffusion (SD) [18], used as an unconditioned diffusion baseline, achieves the highest SSIM ($0.5402\pm 0.012$) among all models. However, its low PSNR ($17.1599\pm 0.84$) and high RMSE ($0.1448\pm 0.007$) reflect blurred reconstructions and spectral distortion, due to the absence of terrain- or time-specific conditioning. ControlNet [8], adapted with DEM guidance, significantly improves generation quality, yielding PSNR of $21.1847\pm 0.72$ and RMSE of $0.0873\pm 0.005$. This demonstrates the value of integrating physical priors into the generation process.

AlignDiff improves upon ControlNet by introducing a VGG-based adapter for distribution alignment, achieving the best PSNR ($23.04\pm 0.69$), RMSE ($0.0713\pm 0.004$), and MAE ($0.0500\pm 0.002$). Despite slightly lower SSIM, qualitative results (Figure 2) show more coherent and terrain-consistent outputs, confirming the benefit of combining geospatial priors with perceptual alignment.

Task-2: Spatiotemporal Completion under Structured Constraints. In this setting, we evaluate the model’s ability to restore missing observations across both spatial and temporal dimensions. We consider two reconstruction scenarios: (1) temporal prediction from prior frames, and (2) spatial reconstruction from DEM priors. The results are averaged across 100 globally sampled locations with varied seasonal and spectral conditions.

Under the temporal setting, interpolation-based methods exhibit poor performance, with PSNR below $12.0$ and high RMSE above $0.25$, failing to exploit semantic or physical consistency. STCNN improves moderately (PSNR=$14.5317\pm 0.65$), but still lacks high-frequency details. AutoEncoder [19] yields strong metrics (SSIM=$0.6090\pm 0.013$, PSNR=$18.2038\pm 0.78$), yet suffers from hallucination and texture mismatch in regions with significant temporal gaps.

In spatial reconstruction, diffusion models demonstrate superior stability. ControlNet achieves SSIM of $0.3787\pm 0.011$ and PSNR of $22.7866\pm 0.73$, substantially better than unconditioned models (SD: SSIM=$0.2819\pm 0.012$). However, ControlNet alone fails to ensure distributional consistency, often introducing spectral shifts and texture inconsistencies.

AlignDiff addresses these issues holistically by combining spatial priors, semantic prompts, and perceptual alignment. It outperforms all baselines in every metric—SSIM ($0.5704\pm 0.011$), PSNR ($24.3429\pm 0.71$), RMSE ($0.0642\pm$$0.003$), MAE ($0.0479\pm 0.002$), and LPIPS ($0.0469\pm 0.002$). Compared to ControlNet, this represents a 50.68% improvement in SSIM and 11.56% reduction in RMSE. The integration of alignment modules enables AlignDiff to handle complex missing patterns across seasons, land types, and spectral bands—making it a robust candidate for real-world satellite observation recovery.

3.3. Ablation Study

The proposed VGG-Adapter serves as a style alignment module to mitigate distributional discrepancies between generated outputs and reference data. As shown in Figure 4, omitting the adapter results in artifacts such as overexposed regions and brightness shifts, particularly in areas with complex land cover and elevation variability. Quantitatively, the mean brightness of reconstructed images without the adapter reaches $\mu =123.63\pm 6.02$, significantly deviating from the reference ($\mu =95.19\pm 3.87$). In contrast, AlignDiff with VGG-Adapter restores this value to $\mu =95.27\pm 3.82$, closely matching the ground truth distribution.

Table 1 (averaged over 100 sites) further corroborates this improvement across all metrics. Notably, the red band—which is highly sensitive to spectral shifts—shows the most significant gain in PSNR (+11.01 dB) and RMSE reduction (-0.12), as illustrated in Figure 3. These results confirm that the VGG-Adapter contributes both perceptual and statistical alignment, and plays a critical role in enhancing output consistency across diverse geographies.

Figure 3. Comparison for Task-2 with selected missing data days. The above images are the results after brightness adjustment and gamma correction, with a coefficient of 1.2. The original image was used for calculating the evaluation metrics.

Figure 3. Comparison for Task-2 with selected missing data days. The above images are the results after brightness adjustment and gamma correction, with a coefficient of 1.2. The original image was used for calculating the evaluation metrics.

Figure 4. Ablation study on the VGG-Adapter.

Figure 4. Ablation study on the VGG-Adapter.

Table 1. Ablation study on the VGG-Adapter module. Bold indicates the best result; Underline denotes the second-best.

Table 1. Ablation study on the VGG-Adapter module. Bold indicates the best result; Underline denotes the second-best.

Table 2. Performance comparison on Task-1. Bold indicates the best, underline the second-best result.

Table 2. Performance comparison on Task-1. Bold indicates the best, underline the second-best result.

Table 3. Performance comparison on Task-2. The upper part uses the timestep as input; the lower uses DEM guidance. Bold denotes best, underline second-best.

Table 3. Performance comparison on Task-2. The upper part uses the timestep as input; the lower uses DEM guidance. Bold denotes best, underline second-best.

4. Conclusion

We propose AlignDiff, a diffusion-based framework for remote sensing image reconstruction under missing data. It addresses both spatial completion and temporal recovery by leveraging Digital Elevation Models (DEMs) as structural priors for terrain-consistent generation. A VGG-Adapter mitigates distributional shifts via perceptual alignment, improving realism and spectral fidelity. Experiments on global Landsat-8 imagery and the EarthNet2021 benchmark show consistent gains over Stable Diffusion, ControlNet, and AutoEncoder across SSIM, PSNR, RMSE, MAE, and LPIPS. AlignDiff provides a scalable, generalizable, and geophysically grounded solution for satellite image recovery, especially in persistently data-scarce regions.