A Deep Learning-Based Method for Enhancing the Signal-to-Noise Ratio of Star Sensor Images

1. Introduction

Star sensors are core optical sensors for spacecraft attitude measurement. The attitude information provided by star sensors directly affects the geocoding accuracy of remote sensing images and therefore influences downstream remote sensing applications. During operation, star sensors capture stellar images, extract the centroid coordinates of star spots, and determine spacecraft attitude or detect space targets through star-map recognition [1,2,3]. To improve measurement accuracy and data update rates, star sensors commonly operate in window tracking mode during on-orbit observation. In this mode, the positions of known stars in the current image are predicted from the attitude information at the previous moment, and small windows centered on these predicted positions are cropped. A field-programmable gate array (FPGA) then processes each window in parallel and independently calculates the centroid of the corresponding star spot [4,5,6]. Under ideal conditions, each window contains a single star, with its centroid located at the center of the window. and the grayscale distribution follows a two-dimensional Gaussian point spread function (PSF) [7,8]. Figure 1 shows the small windows cropped by the FPGA from the original star map.

In actual on-orbit environments, however, the quality of window images can be strongly degraded by stray-light interference from Earth airglow, direct sunlight, and internal scattering in the optical system, together with detector readout noise and dark current. Under such interference, star-spot signals are readily submerged in background noise. The resulting degradation reduces centroid extraction accuracy and may ultimately cause attitude determination failure [9,10].

Among existing stray-light suppression methods, multi-frame stacking remains the most widely used technical route in engineering applications [11,12]. Common implementations include mean stacking, median stacking, and weighted stacking [13]. Mean stacking can suppress Gaussian noise effectively, but this strategy is sensitive to outlier frames, namely abnormal observations caused by sudden and short-duration interference in a continuously acquired image sequence. Median stacking is robust to transient noise such as cosmic rays, but this strategy cannot remove spatially continuous background gradients. Weighted stacking assigns weights according to the inter-frame signal-to-noise ratio, yet the weight-estimation process is itself easily disturbed by noise [14,15]. Overall, multi-frame stacking methods share three limitations: the statistical assumptions for stray light and noise are overly idealized, adaptive compensation for subpixel jitter of star spots is limited, and high-frequency star-spot details are often weakened during background suppression. These limitations may lead to missed detection of dim stars or photometric distortion [16,17].

Methods beyond multi-frame stacking also have clear constraints. Wavelet-threshold denoising [9] suppresses noise by exploiting sparsity in the wavelet domain, but threshold selection is empirical and the method is prone to pseudo-Gibbs artifacts. Non-local means (NLM) filtering [11] performs weighted averaging based on image self-similarity and is effective for uniform-background denoising, although the computational complexity is high and texture preservation remains limited. Three-dimensional block matching (BM3D) and the volumetric extension BM4D [14,15] achieve strong denoising performance through collaborative filtering of grouped blocks. However, these methods require accurate block matching, are prone to failure in star maps affected by severe stray-light interference, and are difficult to deploy in real-time star-sensor systems. In general, these traditional methods depend on manually designed priors, provide limited adaptive modeling of spatially non-stationary stray light and spectral aliasing between stray light and star-spot signals, and often incur high computational cost. Therefore, this study uses multi-frame stacking, the most common and computationally simple engineering method, as the representative traditional baseline to evaluate the improvement provided by deep learning.

Deep learning has recently provided a promising route for star-map restoration. Rather than relying on manually specified priors, deep learning methods can learn restoration priors directly from image pixels and are therefore less constrained by the assumptions that limit traditional algorithms. Convolutional neural network (CNN)-based methods have shown good performance in star-spot detection and centroid positioning [18,19]. Transformer architectures have further improved image-restoration capability. Restormer [20] uses Multi-Dconv Head Transposed Attention (MDTA) to perform global interaction along the channel dimension and combines MDTA with a Gated-Dconv Feed-Forward Network (GDFN), thereby improving restoration quality while maintaining computational efficiency. The cutting-edge achievement model ASTERIS [21] extends Restormer to 3D spatiotemporal sequences and uses self-supervised learning to improve the detection sensitivity of the James Webb Space Telescope by one magnitude.

Despite these advances, directly applying existing deep learning methods to star-sensor window tracking remains challenging, particularly under severe stray-light interference that submerges star spots in noise. Two limitations are especially important. First, existing methods provide insufficient adaptive compensation for subpixel-level jitter of star spots. In orbit, platform jitter and optical distortion cause small inter-frame shifts of star spots, preventing accurate registration and alignment of multi-frame star images and consequently degrading the denoising performance of models such as ASTERIS [21]. Ordinary convolution responds weakly to such displacement, leading to feature misalignment. Second, existing methods lack an explicit mechanism for preserving the high-frequency energy of star spots. The star spot signal appears as a medium-to-high frequency component in the window image [22]. During training, a network may weaken star-spot sharpness to obtain a smoother background, which appears as reduced peak intensity and an increased PSF diffusion radius. To address these limitations, this study introduces two improvements.

First, frame-by-frame spatial deformable convolution, hereafter referred to as deformable convolution, is incorporated into the decoder to replace standard 3D convolution. In the decoder upsampling stage, a spatial offset field is predicted independently for each frame and then used for two-dimensional deformable convolution. This design enables the sampling positions of the convolution kernel to adaptively follow the actual locations of star spots. The strategy is inspired by deformable convolution [23] and its lightweight improvement [24]. Placing deformable convolution in the upsampling stage is advantageous because high-resolution feature maps support more accurate offset estimation, and deformable convolution close to the output end can more directly correct positional errors accumulated by the encoder. The operation compensates only for displacement in the spatial dimensions and does not alter the temporal axis, which is consistent with the physical fact that star-spot displacement occurs within the image plane.

Second, a frequency-domain constraint is introduced into the loss function. A complex-domain frequency-domain loss and a high-frequency weighted mask, hereafter referred to as the frequency-domain loss constraint, are used to explicitly constrain the high-frequency components of star spots. This design draws on the structural similarity index [25] and joint wavelet-frequency-domain loss [26]. Strong denoising performance is obtained when the magnitude of this loss is comparable to that of the spatial-domain loss. Recent methods based on diffusion models [27], implicit neural representations [28], and joint loss functions integrating frequency-domain and perceptual losses [29] also support the effectiveness of frequency-domain constraints.

Because noise-free ground-truth star maps are difficult to obtain during on-orbit operation, self-supervised learning provides a practical training strategy. Self-supervised frameworks such as Noise2Noise [30], Noise2Void [31], and Noise2Self [32] establish the theoretical basis for training without clean targets. CNN-based denoising methods such as DnCNN [33] and FFDNet [34s] have also informed subsequent research. Both convolution-based 3D Res U-Net [19] and Transformer-based Restormer [20] can be trained within a self-supervised framework. ASTERIS adopts this self-supervised spatiotemporal denoising strategy to enhance deep-space images without ground truth. With ASTERIS as the baseline, this study also uses a self-supervised strategy and trains the model with multi-frame noisy sequences acquired in the window tracking mode of star sensors.

Interference in actual on-orbit environments can be broadly divided into two categories [9,10]:

The first category is detector readout noise, which refers to random fluctuations generated by electronic circuits during image-sensor signal readout, including reset noise, amplification noise, and quantization noise. This type of noise approximately satisfies a zero-mean independent and identically distributed assumption in the temporal domain and appears spatially as an isotropic random shot-like pattern.

The second category is spatially non-stationary stray light, which refers to background radiation entering the optical system from the external spacecraft environment, including Earth airglow, moonlight, and internal scattering in the optical system. This interference forms a spatially nonuniform but temporally slowly varying or stable distribution on the image plane. Its typical characteristics include spatial gradients or local highlights, approximate constancy within a short-time window, and spectral concentration in the low-frequency region.

Traditional multi-frame stacking methods suppress zero-mean random noise by exploiting temporal redundancy. For spatially non-stationary stray light, however, stacking can only preserve the mean background and cannot improve spatial nonuniformity, even when the interference is independent between frames [11,12,13,14,15]. The effectiveness of self-supervised deep learning methods also depends on the assumption that inter-frame noise is independent and identically distributed. For interference with a nonzero mean, such as dark-field noise or fixed-pattern noise, any method based only on temporal independence cannot remove the mean component, because both stacking and self-supervised learning preserve this component.

The simulation dataset used in this study mainly contains detector readout noise and spatial random stray-light interference, both of which are independently generated for each frame and have an approximately zero statistical expectation. Under these conditions, deep learning methods can fully exploit their advantages. The remainder of this paper is organized as follows: Section 2 reviews related work, Section 3 describes the proposed method, Section 4 presents the experimental design and results, and Section 5 summarizes the study and discusses future directions.

2. Related Work

2.1. Traditional Star-Image Denoising Methods

Traditional star-image denoising methods can be divided into spatial-domain filtering, transform-domain filtering, and multi-frame stacking. Spatial-domain filtering methods, including Gaussian, median, and bilateral filtering, are simple to implement but have clear limitations. Gaussian filtering blurs edges, median filtering cannot handle gradually varying backgrounds, and bilateral filtering requires careful parameter tuning [11,12]. Transform-domain filtering methods, such as DCT and wavelet-threshold denoising [9], depend strongly on threshold selection and basis-function matching and are prone to distortion when spectral overlap occurs [13,14]. Multi-frame stacking is the most widely used approach. Mean stacking is susceptible to outlier frames, median stacking cannot remove spatially continuous backgrounds, and improved methods based on image autocorrelation [16] depend on the success rate of initial star-spot extraction. Overall, traditional methods rely on overly idealized statistical assumptions about stray light. This dependence makes it difficult to preserve the PSF morphology of star spots while suppressing the background and can lead to missed detection of dim stars or grayscale distortion [17].

2.2. Deep Learning-Based Methods

Self-supervised learning: Noise2Noise demonstrates that, under zero-mean inter-frame independent noise, training can be performed using only pairs of noisy images. The theoretical basis is as follows: for a noisy pair in the form x₁=y+n₁ and x₂=y+n₂, where y denotes the ground truth and n₁ and n₂ denote noise interference, minimizing the difference between x₁ and x₂ is equivalent to learning a mapping from noisy images to the clean signal y. Noise2Void and Noise2Self further enable training from a single noisy image by constructing self-supervised signals through blind-spot networks or pixel-masking strategies. Noise2Noise does not use temporal-axis information in multi-frame sequences, whereas 3D models such as ASTERIS stack sequences into 3D tensors and use spatiotemporal self-attention to jointly model spatial details and temporal dependencies. This strategy provides stronger performance under low signal-to-noise ratios. The method can also reduce the influence of outlier frames in multi-frame stacking, because the grayscale distribution of an outlier frame differs substantially from that of other frames in the sequence. To minimize fitting error, the network tends to treat such frames as noise and suppress their contribution.

3D Res U-Net architecture, abbreviated as 3DRU-Net: This architecture introduces a 3D encoder-decoder structure with residual connections [19], which alleviates gradient vanishing and has been widely used in medical image segmentation and denoising. However, its convolution operations are local operators, and long-range spatiotemporal dependencies must be transmitted indirectly through repeated downsampling and upsampling. Under the operating conditions considered in this study, this limitation can cause loss of high-frequency star-spot details. Static convolution kernels also assign the same spatial weights to all inputs, which makes adaptive discrimination between star spots and stray-light backgrounds difficult.

Transformer architecture: Restormer [20] uses MDTA to perform global interaction along the channel dimension and combines MDTA with GDFN to improve restoration quality efficiently. MDTA transposes self-attention computation from the spatial dimension to the channel dimension, reducing the computational complexity from O(N²) to O(C²), where N is the number of pixels and C is the number of channels, with C much smaller than N. ASTERIS [21] extends Restormer to 3D spatiotemporal sequences and adopts self-supervised spatiotemporal denoising. Unlike 3DRU-Net, which relies on local convolution and indirect transmission of long-range dependencies, the Restormer-based 3D Transformer enables each spatial position to interact directly with all positions in the image through MDTA, thereby providing a global receptive field. Dynamic attention weights can also focus adaptively on star-spot regions according to the input signal. Experiments show that, under low signal-to-noise ratios, ASTERIS achieves a substantially greater PSNR improvement than 3DRU-Net [21].

Deformable convolution: Deformable convolution [23] enables convolution kernels to adaptively follow target deformation by learning sampling-point offsets and can compensate for subpixel displacement of star spots. These offsets are predicted from the input feature map by an additional convolutional layer. Directly using 3D deformable convolution, however, introduces unnecessary offsets in the temporal dimension and entails high computational overhead. Recent research [24] has explored the placement of deformable convolution in the decoder stage. Inspired by this strategy, this study applies frame-by-frame spatial deformable 2D convolution in the upsampling stage. The experiments show that this placement is superior to applying deformable convolution in the downsampling stage, although the downsampling-stage comparison is not discussed in detail.

3. Proposed Method

3.1. Problem Formulation

Given an input sequence of window images with a window size of H×W, the ideal grayscale distribution of the star spot follows the PSF [7], whereas the observed image is degraded by stray light and noise interference. The objective is to learn the mapping between the degraded observation and the ideal star-spot signal so that a clean image can be restored.

Each noisy image frame ${\mathrm{I}}_{\mathrm{t}}$ can therefore be regarded as the superposition of three types of signals as follows:

$${\mathrm{I}}_{\mathrm{t}}{=\mathrm{S}}_{\mathrm{t}}{+\mathrm{B}}_{\mathrm{t}}{+\mathrm{N}}_{\mathrm{t}}$$

where ${\mathrm{S}}_{\mathrm{t}}$ denotes the clean star-spot signal, whose grayscale distribution approximately follows the PSF; ${\mathrm{B}}_{\mathrm{t}}$ denotes spatial random stray light, which is independent between frames and has an approximately zero mean; and ${\mathrm{N}}_{\mathrm{t}}$ denotes detector readout noise, which approximately satisfies a zero-mean Gaussian independent and identically distributed assumption in the temporal sequence as follows:

$${\mathrm{N}}_{\mathrm{t}}~\mathrm{N}(0,{\sigma }_N^2)$$

The objective of this study is to design a trainable mapping function that takes a multi-frame sequence ${\mathrm{I}}_{\mathrm{t}}$ as input and outputs the corresponding restored multi-frame image ${\mathrm{S}}_{\mathrm{t}}$.

3.2. Baseline ASTERIS Model

This study adopts ASTERIS as the backbone network [21]. The core of ASTERIS is a 3D Transformer encoder-decoder structure. The input sequence is stacked into a 3D tensor X (T×H×W), processed by a 3D convolutional layer, and then fed into the MDTA module. MDTA performs self-attention along the channel dimension, enabling each channel to interact with all spatiotemporal positions and thereby providing a global receptive field. GDFN enhances nonlinear representation through a gating mechanism [20]. By stacking multiple Transformer blocks, the network progressively learns the mapping from noisy observations to the clean signal.

ASTERIS uses a self-supervised learning strategy in which adjacent-frame noisy image pairs serve as supervisory signals. When the noise satisfies zero-mean and inter-frame independence assumptions, learning inter-frame mappings enables the network to converge toward the denoised clean signal. The loss function jointly optimizes two spatial-domain losses. L1_stack is a smooth L1 loss that constrains pixel-level differences between the single-frame output and the target, thereby preserving high-frequency details and star-spot edges. L2_mean is a mean squared error loss for the sequence-mean image, which suppresses the accumulation of inter-frame random noise and improves the signal-to-noise ratio. These two losses provide complementary constraints.

3.3. Theoretical Advantages over Multi-Frame Stacking

Traditional multi-frame stacking linearly combines pixel values along the temporal axis, and its denoising capability is derived from temporal redundancy. This strategy has three inherent limitations. First, for spatially non-stationary backgrounds, stacking can only preserve the mean distribution and cannot improve spatial nonuniformity. Second, stacking cannot compensate for subpixel jitter of star spots, so direct stacking increases the energy diffusion radius. Third, stacking is essentially a low-pass filtering operation that weakens the high-frequency morphology of star spots. The improved ASTERIS-based deep learning method used in this study addresses these limitations through four aspects.

First, spatial information helps suppress non-stationary backgrounds. Through 3D convolution and attention mechanisms, the deep learning model learns the spatial distribution of the background within a single frame and uses a large receptive field to smooth the background adaptively. Even when the background mean is nonzero, the network can use spatial context to attenuate the visual impact of the background. By contrast, multi-frame stacking fully preserves the mean background and is therefore less effective at improving the signal-to-noise ratio.

Second, deformable convolution provides adaptive compensation for subpixel jitter. By learning spatial offsets for each sampling point, deformable convolution enables the sampling grid of the convolution kernel to align actively with the center of the star spot and supports precise subpixel-level fusion.

Third, the frequency-domain loss explicitly preserves the high-frequency morphology of star spots. This loss constrains the amplitude and phase spectra simultaneously and strengthens the mid- and high-frequency components through a high-frequency weighted mask. This design helps avoid excessive smoothing, whereas the low-pass characteristics of multi-frame stacking tend to weaken star-spot edges.

Fourth, MDTA provides joint spatiotemporal modeling. This mechanism enables each spatial position to perceive information from all temporal frames simultaneously, allowing the network to distinguish temporally varying star spots from temporally stable backgrounds and to suppress the background spatially. The experiments in Section 4 verify these advantages.

3.4. Frame-by-Frame Spatial Deformable Convolution

3.4.1. Design Motivation

Deformable convolution is applied in the upsampling stage for three reasons. First, the feature maps in the upsampling stage have higher resolution, which supports accurate prediction of subpixel offsets. Second, the decoder fuses details lost during downsampling, and deformable convolution can directly correct accumulated positional errors. Third, gradient propagation is more direct near the output end, which improves training stability. Platform vibration, optical distortion, and other factors can also cause residual subpixel inter-frame registration errors, usually within 0.2 pixels. Deformable convolution compensates for these errors adaptively by learning spatial offsets for each sampling point, thereby improving cross-frame fusion accuracy. Replacing standard 3D convolution in the decoder with frame-by-frame 2D deformable convolution does not affect temporal modeling, because temporal dependencies are mainly captured by encoder MDTA and transmitted to the decoder through residual skip connections [21]. This design also reduces the number of parameters by approximately one quarter without increasing time overhead. Therefore, deformable convolution is integrated into the upsampling stage of the ASTERIS decoder.

Deformable convolution can also suppress background noise. Star-spot energy is locally concentrated, whereas background noise is spatially diffuse. Standard convolution samples background noise indiscriminately on a regular grid. By contrast, deformable convolution learns spatial offsets that guide sampling points toward star-spot regions and away from the background. During training, the offset-prediction network shifts sampling points near star spots toward the star-spot center, whereas sampling points in background regions are guided toward informative star-spot regions or kept at zero offset. The contribution of the background is therefore relatively suppressed. In essence, this operation performs adaptive spatial resampling. By minimizing the reconstruction loss, the sampling grid gradually aligns with the star-spot region and focuses on star-spot features. Unlike traditional denoising methods such as mean filtering, which smooth the entire image indiscriminately, deformable convolution provides an active selection mechanism that bypasses the background adaptively and collects information from the vicinity of star spots. This mechanism suppresses background noise without weakening star-spot energy.

3.4.2. Implementation Principle

Let the input feature map be X (N×C×T×H×W). The procedure is described as follows:

1. Frame-by-frame splitting: the feature map is split along the temporal dimension into T two-dimensional frames X_t (N×C×H×W).

2. Offset prediction: for each frame, a two-dimensional convolution with shared weights, Convoffset, predicts the spatial offset O_t. The number of output channels is 2G×k², where k is the convolution kernel size and G is the number of groups, which is usually set to 1. Each offset is a floating-point value that represents the subpixel displacement of a sampling point in the x and y directions. The offset-prediction formula can be written as O_t = Convoffset(X_t), and the shape of O_t is (N, 2G×k², H, W). All T frames share the same Convoffset, thereby reducing the number of parameters.

3. Two-dimensional deformable convolution: the torchvision.ops.deformconv2d function provided by Python is used to perform deformable convolution on each frame. The function takes the input feature map X_t, the offset O_t, and the weight W_dc and the bias b_dc of Convoffset as inputs. The output Y_t has the shape (N, C', H', W'). All T frames share W_dc and b_dc.

4. Temporal stacking: Y_t is concatenated along the temporal dimension to obtain Y (N, C', T, H', W').

5. Offset initialization: the offsets are initialized to 0; that is, W_dc and b_dc are initially set to 0. During training, the offsets are learned progressively from small to large values. This process is consistent with the natural optimization trajectory, facilitates convergence to a favorable local optimum, and improves training stability. Figure 2 shows the improved ASTERIS model.

3.5. Frequency-Domain Loss Constraint

3.5.1. Complex-Domain Loss

Let the image size be H×W. The predicted image and the target image are transformed from the spatial domain to the frequency domain by a two-dimensional Fourier transform F. In this representation, low-frequency components mainly describe smooth background variations, including interferences such as noise and stray light, whereas high-frequency components encode star-spot edges and fine details. The amplitude spectrum $\left|\mathrm{F}\right|$ characterizes the energy of each frequency component, whereas the phase spectrum encodes positional information. For star-spot centroiding, the phase spectrum is particularly important because phase determines the position of the star spot and the symmetry of the PSF. A constraint imposed only on the amplitude spectrum, namely $\left|\mathrm{F}\right|$, can preserve high-frequency energy but cannot ensure phase consistency, which may introduce positional shifts or shape distortion. Therefore, the complex-domain loss constrains amplitude and phase simultaneously as follows:

$${\mathrm{L}}_{\mathrm{complex}}={\displaystyle \frac{1}{\mathrm{H}\cdot \mathrm{W}}}{\displaystyle \sum _{\mathrm{u}=0}^{\mathrm{H}-1}{\displaystyle \sum _{\mathrm{v}=0}^{\mathrm{W}-1}}}\left|{\hat{\mathrm{F}}(\mathrm{u},\mathrm{v})-\mathrm{F}}_{\mathrm{gt}}(\mathrm{u},\mathrm{v})\right|$$

where (u,v) denotes the frequency-domain coordinates. This loss penalizes amplitude and phase discrepancies simultaneously. Specifically, the squared modulus of the complex difference can be expanded as follows, where ${(\mathsf{\theta }}_{\wedge }{-\mathsf{\theta }}_{\mathrm{gt}})$ denotes the phase difference:

$${|\hat{\mathrm{F}}-\mathrm{F}}_{\mathrm{gt}}{|}^2{=\left|\hat{\mathrm{F}}\right|}^2{+|\mathrm{F}}_{\mathrm{gt}}{|}^2{-2\left|\hat{\mathrm{F}}\right||\mathrm{F}}_{\mathrm{gt}}{\left|\cos \right(\mathsf{\theta }}_{\wedge }{-\mathsf{\theta }}_{\mathrm{gt}})$$

3.5.2. High-Frequency Weighted Mask

The useful high-frequency energy of a star spot is mainly distributed near the boundary frequency of the PSF, whereas low-frequency components are largely associated with background variation and stray-light interference. To ensure that the frequency constraint focuses on star-spot morphology rather than smooth background components, the normalized frequency radius is defined as follows:

$$\mathrm{r}(\mathrm{u},\mathrm{v})=\sqrt{{\left({\displaystyle \frac{\mathrm{u}}{\mathrm{H}}}-0.5\right)}^2+{\left({\displaystyle \frac{\mathrm{v}}{\mathrm{W}}}-0.5\right)}^2}$$

The radius r lies in [0, 0.707]. Given a threshold r0, the mask is set to M(u,v) = 1 when r > r0 and M(u,v) = 0 otherwise. The weighted frequency-domain loss is defined as follows:

$${\mathrm{L}}_{\mathrm{freq}}={\displaystyle \frac{1}{\mathrm{H}\cdot \mathrm{W}}}{\displaystyle \sum _{\mathrm{u}=0}^{\mathrm{H}-1}{\displaystyle \sum _{\mathrm{v}=0}^{\mathrm{W}-1}\mathrm{M}(\mathrm{u},\mathrm{v})\cdot |}}\widehat{\mathrm{F}}{(\mathrm{u},\mathrm{v})-\mathrm{F}}_{\mathrm{gt}}(\mathrm{u},\mathrm{v})|$$

The mask assigns greater emphasis to the mid- and high-frequency components that contain star-spot boundary and shape information. At the same time, the mask suppresses low-frequency background interference and reduces unnecessary computation. The experiments in this study indicate that r0 = 0.2 provides the best restoration of star-spot details. The detailed threshold-selection procedure is not discussed further.

3.5.3. Loss-Balance Coefficient

The frequency-domain loss ${\mathrm{L}}_{\mathrm{freq}}$ is calculated in a manner consistent with L1_stack in ASTERIS [21]. Because the spatial-domain losses L1_stack and L2_mean are multiplied by coefficients on the order of 1e6 in the baseline model, a balance coefficient lambda_freq is introduced for L_freq. The total loss is expressed as follows:

L_total = 0.125e6 × L1_stack + 1e6 × L2_mean + lambda_freq ×L_freq

A grid search over lambda_freq ∈ {0, 100, 500, 1000, 2000, 5000} identifies 1000 as the optimal value.

3.6. Network Training

Model training uses the Adam optimizer with an initial learning rate of 1e-4, which is annealed to 1e-5 through a cosine schedule. The batch size is set to 3, and the network is trained for 20 epochs. The learning rate of the deformable-convolution offsets is set to 1e-5 to avoid unstable offset updates and reduce the risk of convergence to a noise-induced local optimum.

4. Experimental Validation

4.1. Dataset and Evaluation Metrics

Dataset construction method: Real window images obtained through ground calibration are used, and the window size is 48×48. The star-spot energy and diffusion radius are randomly set. For each star spot, 4000 windows are calibrated, and the calibrated centroid is allowed to vary within 0.2 pixels around the window center, with coordinates (25,25). Setting all centroids at the window center simulates ideal registration and alignment of all frame window images, whereas centroid fluctuation within 0.2 pixels represents the alignment result under real conditions. Detector readout noise and spatial random stray light, which are uncorrelated between frames and have an approximately zero expected value, are then added to each window image through a simulation program to simulate real on-orbit operating conditions. The dataset does not include inter-frame gradually varying stray-light interference with a nonzero mean caused by solar illumination conditions. The peak signal-to-noise ratio (PSNR) of images containing stray light and readout noise ranges from -6 to 15 dB, with an average of 8 dB. Among these images, 800 windows are randomly selected for testing, with every 8 images forming one group for denoising and restoration, including multi-frame stacking. This setting yields 100 groups of test conditions, whereas the remaining images are used for training. A dataset of 30 star spots is generated in total, namely 30×4000 groups of images that cover star spots with different brightness levels and morphologies. The dataset for each star spot is divided according to the 4:1 ratio described above, and the training datasets of all star spots are mixed for model training to improve generalization across different star spots. The test dataset contains 30×100 groups of test data. The calculation formula for the peak signal-to-noise ratio is as follows:

$$\mathrm{PSNR}=10\cdot {\log }_{10}\left({\displaystyle \frac{{\mathrm{MAX}}^2}{\mathrm{MSE}}}\right)$$

$$\mathrm{MSE}={\displaystyle \frac{1}{\mathrm{mn}}}{\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{m}-1}{\displaystyle \sum _{\mathrm{j}=0}^{\mathrm{n}-1}{\left[\mathrm{I}(\mathrm{i},\mathrm{j})-\mathrm{K}(\mathrm{i},\mathrm{j})\right]}^2}}$$

where MAX denotes the maximum pixel value of the star spot, m×n denotes the image size, I denotes the original image before noise addition, and K denotes the noisy image to be evaluated.

Evaluation metrics: The evaluation metrics include peak signal-to-noise ratio, centroiding-error standard deviation, and noise standard deviation (${\mathsf{\sigma }}_{\mathrm{n}}$) [35]. The centroiding-error standard deviation is defined as the standard deviation of the deviations between the centroid obtained from the denoised image and the window center, in pixels. The “Local-window Centroiding Method” is used for centroid extraction. The noise standard deviation is calculated as follows, where a smaller value indicates a smoother denoised image:

$${\mathsf{\sigma }}_{\mathrm{n}}=\sqrt{{\displaystyle \frac{1}{\mathrm{N}}}{\displaystyle \sum _{\mathrm{i}=0}^{\mathrm{N}-1}{{(\mathrm{K}}_{\mathrm{i}}{-\mathrm{I}}_{\mathrm{i}})}^2}}$$

where N denotes the total number of pixels, I denotes the original image before noise addition, and K denotes the noisy image to be evaluated.

4.2. Experimental Results and Comparison with Multi-Frame Stacking (Mean) and the Baseline Model

Each star spot in the test set is processed using multi-frame stacking, the baseline model, and the improved baseline model, namely deformable convolution combined with the frequency-domain loss constraint. Table 1 summarizes the statistical results for the signal-to-noise ratio of the window images and the centroiding-error standard deviation.

The experimental comparison shows that the improved baseline model achieves much higher signal-to-noise ratio improvement and higher centroiding accuracy than the other two methods. This result verifies that the two proposed improvements further enhance the denoising capability of the baseline model. The results also support the theoretical analysis in Section 3 regarding the advantages of the proposed deep learning method over multi-frame stacking. The noise standard deviation indicates that multi-frame stacking leaves more residual noise after denoising and produces poorer background smoothness.

4.3. Ablation Study

Ablation experiments are conducted separately on the two proposed improvements, and the results are shown in Table 2 and Table 3. When both improvements are ablated, the processing results are the same as those of the baseline model in Table 1.

Table 2 and Table 3 show that ablating deformable convolution has a greater impact on the signal-to-noise ratio and noise standard deviation. This result is consistent with the theoretical analysis that deformable convolution effectively suppresses background noise interference. By contrast, ablating the frequency-domain loss constraint has a greater impact on centroiding accuracy. This result is also consistent with the analysis that this loss encourages the model to focus on high-frequency image information and is therefore conducive to restoring the PSF morphology of star spots. Together, the two proposed improvements are complementary, and their combined use enhances the denoising capability of the baseline model.

For one star spot in the test set with relatively weak energy, restoration is difficult; this subset contains 100 groups × 8 noisy frames. Figure 3 and Figure 4 show the statistical curves of the window-image signal-to-noise ratio and centroiding-error standard deviation after processing by the five methods. In these figures, the curves labeled Multiframe Method, DFConv+Freq, DFConv, Freq, and Org denote multi-frame stacking, the improved baseline, the baseline with only deformable convolution, the baseline with only the frequency-domain loss constraint, and the baseline model, respectively. The corresponding data are listed in Table 4.

Figure 5 shows a group of eight noisy test images for this star spot. The star spot is almost submerged by noise, making centroid extraction impossible.

Figure 6 shows the denoised images obtained after processing the group of noisy images above by the five methods, together with the original noise-free star-spot image.

As shown in Figure 6, the proposed method achieves a substantially higher signal-to-noise ratio than the other methods after denoising, and the star spot is restored more accurately with clearer edges. By contrast, multi-frame stacking yields a markedly lower signal-to-noise ratio and poorer background smoothness.

5. Discussion and Future Work

This study proposes a deep learning method based on frame-by-frame spatial deformable convolution and complex-domain frequency-domain constraints for star-sensor window images degraded by detector readout noise, which follows a Gaussian distribution, spatially random stray-light interference, which is independent between frames and has an approximately zero mean. With ASTERIS as the baseline, the proposed method retains the advantages of the baseline model while introducing two improvements: frame-by-frame spatial deformable convolution in the decoder upsampling stage and a complex-domain frequency-domain loss constraint. These improvements compensate for subpixel jitter of star spots, enhance denoising capability, preserve the high-frequency morphology of star spots, and reduce the number of model parameters by approximately one quarter. In the experiments, the proposed method achieves substantially better denoising performance than multi-frame stacking and the baseline model. The proposed method improves PSNR by approximately 60-fold, and the centroiding error reaches approximately 0.1 pixels, which is smaller than the artificially added centroid-jitter range in the test set, approximately 0.2 pixels.

For interference with a nonzero mean, any method that relies only on temporal independence, including multi-frame stacking and self-supervised learning, cannot remove the mean component. Although the proposed method theoretically cannot eliminate this component, its spatially adaptive filtering capability enables better suppression of background spatial nonuniformity than multi-frame stacking and can reduce star-spot centroiding errors.

Future work will introduce a background estimation branch into the model, allowing the static background to be subtracted at the input end through a learnable background-separation network. This extension is expected to improve the adaptability of the proposed method to complex on-orbit environments and diverse remote sensing tasks. Lightweight network design and dynamically adaptive frequency-domain loss weighting are also important directions for future research. Real on-orbit star image data will be acquired from various on-orbit missions to construct more diverse test benchmarks, to verify and improve the generalization capability against unknown noise distributions and stray light patterns of the model, thereby laying a solid foundation for eventual engineering deployment.