Continuous Active Correction of Deployable Space Telescopes Using Machine Learning

1. Introduction

The image quality of high resolution space telescopes is dependent upon the size of the primary mirror. Under diffraction-limited conditions, the angular resolution is inversely proportional to the mirror diameter, making it a key factor in optomechanical design. Higher spatial resolution allows for the detection of finer details - e.g. cars, buildings, infrastructure - useful for urban planning and land classification [1,2,3]; global environmental change and geospatial mapping for climate control and weather detection [4,5,6]; surveillance and monitoring for applications in defence and security. This is also impacted by the ground sampling distance (GSD), the physical size each pixel represents on the ground, dependent on sensor pixel dimensions and the orbital height of the satellite. State of the art satellites, namely the WorldView series [7], can achieve a panchromatic GSD of 0.31 m for a 1.1 m aperture in panchromatic band of 450 - 800 nm, providing high resolution imaging from sun-synchronous low earth orbit (LEO). The HiRISE [8] mission for Mars imagery, achieves a similar GSD due to a lower orbit of 300 km with an aperture of 0.5 m.

Although larger apertures increase resolution, scaling up the optics size is challenging because of manufacturing, material expenses and launch constraints. Deployable segmented optics are a potential solution. Deployable segments are engineered to deploy from a stored position within the satellite payload once in orbit, reaching a pre-calibrated position that collectively approximates a monolithic aperture. By using unfolding booms and segmented mirrors, aperture sizes difficult or impossible to achieve with large monolithic mirrors can be reached, while adhering to the initial volume constraints of launch. Segmentation also eases mass constraints of the overall system, with mirror mass scaling closer to the square of the diameter, rather than the cubic relationship for monolithic mirrors. Studies into boom constraints and miniaturisation of mechanics [9,10,11] aim to optimise systems. Segmented primary mirror research [12,13,14] focuses instead on direct enhancement of the diameter and thus resolution, with some satellites incorporating both for high performance. Previous large scale implementations include JWST [15,16], as well as the ground telescope Keck [17,18]. Such cases demonstrate that precise segment co-alignment is essential in reaching diffraction-limited performance and the intended resolving power.

In order to benefit from the large physical diameter, the mirror segments must approximate a monolithic mirror form to fine precision. Diffraction-limited performance requires alignment to sub-wavelength precision, on the order of tens of nanometers for visible wavelengths. Alignment to achieve this is termed `phasing’ and is performed on telescope commissioning after launch and mirror deployment. As well as initial mirror phasing, thermal deformations once in space contribute to constant drifts and misalignments throughout the mission life. Assuming rigid mirror segments, misalignments can be characterised as piston, tip and tilt (PTT) errors. Piston refers to a displacement of the entire mirror surface along the optical (’z’) axis, and tip/tilt are rotations around the x and y axes respectively. Combinations of PTT introduce wavefront error (WFE) into the system and can drastically decrease the telescope’s performance. In principle, misalignments can be corrected if a reliable method can measure and distinguish their PTT errors and actively apply adjustments while in orbit.

Phase retrieval [19] algorithms, including Gerchberg-Saxton [14,20] and conjugate-gradient methods, can be employed for the correction of present aberrations. These typically require observation of a point source [16], as the shape and intensity of the PSF provides a performance metric related to the system’s wavefront aberrations. For a space telescope a point source can be observed by imaging an isolated star. However, if the primary purpose is Earth observation (EO), such calibration would require a complete reorientation between calibration and operational imaging.

An alternative algorithm, Phase Diversity, can operate by observing an extended scene. As demonstrated by Gonsalves [21], by using a second image with a known additional phase (e.g. defocus) in relation to the first, sign ambiguities can be removed. Phase Diversity approaches using multiple wavelengths, as well as sensor-less solutions [22] have been investigated.

Iterative methods can also be applied to extended scenes to correct for present misalignments. Performance metrics are defined based on image quality or intensity sharpness, and optimisation [23] can be applied to reduce PTT errors. This approach was demonstrated in PistonCam [24], which used a Nelder Mead simplex model [25] to iteratively co-align three segmented mirrors by optimising the sharpness of an extended Earth scene. While effective at reducing aberrations, this method relies on tracking a fixed point, which may be challenging depending on the required alignment duration. With 400-700 segment positions evaluated, an actuator operating at 10 Hz - three per segment - would take up to 70 seconds. Iterative optimisation methods are computationally expensive to implement on-board, particularly for smaller scale (<1-2 m) satellites where power is a limited resource. Repeat actuation also speeds up mechanical wear and reduces actuator lifetime and repeatability.

This study builds upon the work presented in [26], in which a convolutional neural network (CNN) [27,28] was specifically designed and trained to interpret PSFs and recover piston errors within two iterations. It was developed as a fine alignment procedure after a coarse alignment had been carried out. This work was applied to a telescope composed of four primary mirror segments as illustrated in Figure 1. The approach assumed the telescope was imaging a distant point source, represented as an isolated star, or an artificial calibration source located on Earth. In this work, an implementation of an on-board calibration system (IMPACT) is proposed, enabling continuous PSF imaging and wavefront reconstruction. The CNN has been extended to sense and correct not only piston but also tip and tilt. The calibration system incorporates an inner facet structure integrated into each primary mirror segment, which is specifically designed to return light from an on-board illumination source [29] onto a dedicated PSF-sensing alignment detector. Such a system can be referred to as ’self-aligning’, capable of correcting misalignments without an external source. The facets enables continuous alignment without interrupting nominal telescope operations. Simultaneous imaging and alignment avoids slewing and reduces the stability requirements of the structure, for example in areas when exposed to thermal gradients. The CNN model was developed using PyTorch [30] and is applied to the correction of piston, tip and tilt aberrations, achieving optimal convergence within a maximum of two iterations. The application of machine learning to the segmented facet design is motivated by a fast, computationally efficient alignment method that reduces the number of iterative corrections required.

This paper is organised as follows: Section 2.1 presents the design of the facetted mirror and associated optical layout, followed by a description of PSF data simulation in 2.2, and the CNN model architecture in 2.3. The complete alignment procedure is then outlined. Section 3 presents the experimental results. Section 4 discusses potential applications of the proposed approach highlighting both its benefits and limitations, as well as prospective directions for future work. Finally, Section 5 concludes the paper by reiterating the overall motivation and summarising the key results.

2. Materials and Methods

2.1. Design

Small satellites and Cubesats have strict mass and volume constraints, and so would benefit greatly from deployable mirror technology. In this study, a four-segment primary mirror is used as a case study. When folded, the four segments can be tightly stowed along the four sides of a cubic structure. Once in orbit, they may be deployed and unfold in conjunction with the extension of a secondary mirror, mounted to a boom. The resulting ray trace is shown in Figure 1a. This aperture was modelled in [26] where a CNN was applied to correct relative piston aberrations. Although excellent correction was achieved - improving the mean Strehl from 0.62 to 0.99 - the approach had two key limitations. First, only piston values were considered under the assumption that tip and tilt aberrations had already been aligned via a coarse PSF overlap procedure. Second, the alignment relied on observation of a reference star or dedicated calibration target, requiring the telescope’s attitude to be pointed away from Earth’s surface and therefore reducing valuable science time. To address these limitations, a new procedure has been developed to correct PTT using a small inner facet.

Figure 1b shows a variation of the initial design. A curved outer edge is added for ease of manufacturing. The main change is including mirror facets, an inner section specifically designed for autocollimation on each segment to provide feedback on their position for alignment. It is located on the inner side of the primary mirror and isolated from the science beam reflected from the main part of the segment. The full segment is a single congruent piece so any PTT values applied to the segment are equally transferred to the facet. The alignment relies on this inner ring reflecting the light from the on-board fibre located within the obscured region at the centre of M2.

The alignment concept is shown in Figure 2. Light emitted from a fibre source passes through a beam splitter, is reflected by the facet and directed towards an alignment detector. The fibre will be capable of multi-wavelength emissions, allowing the acquisition of PSFs in two colour channels. Keeping the configuration separate from the primary science detector allows alignment to be performed without interruption.

2.2. PSF Simulation

Machine learning (ML) models require large input data for training and validation. Since it is not practical to obtain all of this data through measurement alone, a digital twin is used to model the system and generate the required datasets. The twin is employed to generate representative PSFs under a wide range of misalignment conditions. In this work, the complete telescope design is simulated with a two-dimensional pupil function representing each active segment. The full aperture has a diameter of 30 cm, compatible with standard Cubesat or small satellite platforms, and each facet has a thickness of 3 cm. The telescope is assumed to be perfectly stigmatic and only the primary is modelled. Light emitted from the fibre is simulated in the far-field regime, appropriate for the small aberrations considered in this study. Field aberrations are neglected by only considering the on-axis beam. A reference segment - defined as the segment to which all others are aligned - is introduced into the digital twin to break degeneracies arising from the symmetry of the system. This reduces the number of active parameters and consequently computational burden of the models. Each unique combination of the 9 PTT values (three for each active segment) generates a distinct PSF, given by:

$$\mathrm{PSF}(x^{\prime },y^{\prime })\propto {\left|\mathcal{F}\left\{A(x,y)e^{i\varphi (x,y)}\right\}\right|}^2\mathrm{where}A(x,y)=A_{0}P(x,y)$$

(1)

where (x’,y’) and (x,y) denote both the focal plane and aperture plane coordinates; $\mathcal{F}$ represents the Fourier transform; A is an aperture function defined by the nominal amplitude $A_0$ and normalised pupil function P; and $\varphi$ is the phase error produced by the PTT applied to each segment.

Figure 3 shows examples of the apertures and their simulated PSF images when aberrations are both present and corrected for. The first column shows the aperture functions for the full segmented M1 and then for the inner facet ring. Because the inner ring covers a smaller effective area, produced PSF images are larger than that of the full aperture. When designing a telescope, the ratio between the detector pixel and Airy disk FWHM (Q ratio) is a key parameter. In this work, PSFs are generated with a Q ratio of 2. This is achieved by oversampling the aperture (via padding) and then binning down to a desired detector pixel size. A Q of 2 provides high sensitivity of small PTT errors on order of $\sim \lambda /10$. As seen in Figure 3b, PTT can affect the overall PSF shape to such an extent that the sub-PSFs associated with individual segments can become separated. These effects are less pronounced when considering images produced by the inner ring, which, due to its smaller size and the selected sampling, generates larger PSFs. The PTT predicted from the alignment PSFs are then used to correct the full aperture PSF.

System performance is evaluated using the peak intensity ratio (Strehl ratio), defined as the ratio of aberrated to unaberrated peak intensity, normalised to the total intensity. The nominal reference is shown in Figure 3c. A threshold of 0.8 is adopted as a baseline of diffraction-limited performance, with the objective of achieving $100\%$ success in correcting cases to exceed this value.

Unlike piston which corresponds to an axial displacement of the optical surface along the system’s optical axis, and does not depend on aperture shape, tip and tilt are generally coupled with piston and depend on the location of the pivot point. As such, different phase jumps and thus PSFs can be generated. When simulating square segments alone, as in [26], tip and tilt were initially assumed to be symmetric about the centre of the square. When applying this assumption to extended asymmetric segments, phase jumps are introduced on the facet regions. This discontinuity is shown in Figure 4a when applying a global tilt to the system. The left and right facet segments vary by the maximum amount of phase, introducing unwanted effects into the system when the annulus is masked. As a result, the gradients are applied centered about the facet ring, as shown in Figure 4b. Not only does this resolve phase errors but is also more representative of real deployable architecture due to the segments requiring a hinge on the inner edge for unfolding.

2.3. CNN Model Architecture

A CNN model to measure segment PTT errors is developed, based on that presented in [26]. Here, the model is expanded to include sensitivity to tip and tilt aberrations as well as piston. Both models are created using the PyTorch [30] library, and development and testing performed on a standard laptop with an AMD Ryzen 5 PRO 7535U processor (6 cores, 12 threads, 29-4.55 Ghz) and 16 GB of DDR5 RAM.

The general structure of a CNN consists of passing input data through convolutional layers and fully-connected or ’dense’ layers, and finally outputting pre-defined parameters. Convolutional layers apply a convolution kernel across the input data, playing a key role in the prediction of PTT from the image structure. By incorporating more layers, the model becomes deeper and more capable of non-linear analysis.

In the model presented in [26], three convolutional layers were used, each halving the spatial extent as the data progressed through them. This fast downsampling is applicable to linear inputs and so was effective in correcting piston values alone. To obtain similar results with the complexity of tip and tilt aberrations present as well, IMPACT is developed to include four additional layers, totalling seven for the overall the architecture. This new new model also differs in altering the rate at which layers downsample, and increasing the number of feature channels with depth, ending the convolution block with 32. The IMPACT CNN can be seen in Figure 5, with the data flow through each layer represented with arrows. Additional layers of convolution make the model deeper and more capable of analysing and classifying the many features of the PSFs. Initial high spatial resolution preserves the fine structures and asymmetry related to the tip and tilt, while the deeper layers increase the perceptive field for global piston effects. Progressive abstraction of features and their relations to PTT can be continued through each step of the model, with hierarchical learning applied effectively. This is most notable between layers where spatial extent is kept constant. Preserving feature channels allows global structures and related aberration effects to be classified, improving expressiveness prior to regression. Fully connected layers then condense the extracted features and map them to specific PTT coefficients. The resulting output values are then used to find a residual PTT (difference to truth values) and analyse the full system’s PSF.

The overall architecture consists of 27,123 parameters, of which all are trainable in the model. Considering forward and backward passes, and the size of the parameters themselves, the model is estimated to use a total of 0.8 MB in memory.

2.4. Data Preparation

Simulated PSF images are generated for four channels: two wavelength channels with an in-focus and defocus of each. In order to effectively sense piston values greater than a magnitude of $2\pi$, two wavelength channels are used. Here the testing wavelength is 532 nm, and so $2\pi \equiv \lambda =532$ nm. A secondary wavelength of 632 nm is used to ensure at least one channel is always out of phase [26], excluding when the mirrors are truly co-phased. The defocus channels allow the distinction between positive and negative piston inputs. A defocus of $0.8\lambda$ is added, ensuring enough physical change is induced for distinct PSFs while aiming to minimise the defocus added into the system. This can be added using additional optics, discussed further in Sec. 4. Using the architecture shown in Figure 5, two models were trained to analyse the four channel inputs, each in specific PTT regimes. Initially, a full ’coarse’ model is trained on piston values within [-400,400] nm, covering 800 nm in total and extending across the $2\pi$ ambiguity range. Tips and tilts are bound between [-200,200] nm, or equivalently [-6.6,6.6] $\mu$radians for the 10 cm segments. These values were chosen to visually separate each sub-PSF on the simulated focal plane, reflecting a poor coarse alignment prior to the implementation of this stage. Larger values are expected to be corrected by the deployment mechanism and so are not present here. A second model labelled the ’fine’ model is similarly trained but on a smaller PTT bound.

Figure 6 summarises the overall alignment procedure. PSF images from the alignment camera, here named as ’ring PSFs’ are generated and act as the main tool for alignment, undergoing a first pass in the coarse model. Once PTT outputs are returned, the residual error is calculated as the difference between initial truth values and the outputs. The residuals are simulated on the full aperture segments and the resulting PSF is then generated with Eq. 1. If the measured Strehl ratio is not above 80%, a second pass of the model is performed using a `fine’ mode. This process is repeated until the Strehl ratio is above 80% resulting in full correction of PTT errors.

3. Results

3.1. CNN Comparison

Figure 7. Histogram comparing preliminary results from a comparison of two CNN models. Initial Strehl counts are plotted in light blue. Results after correction via the 3-layer CNN presented in [26] are shown in dark blue, with orange showing results from IMPACT. The same dataset in used in each case, with piston error bounded between [-300,300] nm and tip/tilt bounded between [-6.6,6.6] $\mu$rads. Here 450 nm and 650 nm wavelengths are used for a direct comparison to work in [26], with the histogram reporting Strehl values for $\lambda$ = 450 nm.

Figure 7. Histogram comparing preliminary results from a comparison of two CNN models. Initial Strehl counts are plotted in light blue. Results after correction via the 3-layer CNN presented in [26] are shown in dark blue, with orange showing results from IMPACT. The same dataset in used in each case, with piston error bounded between [-300,300] nm and tip/tilt bounded between [-6.6,6.6] $\mu$rads. Here 450 nm and 650 nm wavelengths are used for a direct comparison to work in [26], with the histogram reporting Strehl values for $\lambda$ = 450 nm.

A preliminary study was undertaken into the applicability of the previous 3-layer CNN. A dataset was compiled with piston and tip/tilt values bounded by $[-300,300]$ nm and $[-6.6,6.6]$$\mu$rads respectively. Piston bounds were limited in this test alone (compared to ±400 nm in proceeding results) for direct comparison with previous work [26]. The initial (input) Strehl ratios are shown in light blue, before any correction. Results when compensating using measurements from the previous model (CNN-3) are shown in dark blue, and those compensated using measurements from IMPACT in orange. CNN-3 improves the Strehl ratio, but rarely with the required accuracy to achieve a Strehl ratio of over $0.8$. The mean reached only $0.48$, and only $3\%$ of the dataset was corrected to a Strehl ratio of $\ge 0.8$. Compensations from the IMPACT model correct to a higher degree of precision. Here the predictions are more accurate, with a corrected Strehl mean of $0.84$ with nearly $75\%$ of cases above the diffraction-limited threshold of $0.8$. While still requiring further passes, the correction is sufficient in many cases, highlighting the benefit of deeper layers and justifying the increase in parameters.

Figure 8 shows the overall metrics used to quantify the performance of IMPACT’s correction procedure. Firstly, root mean squared (RMS) errors for the full PTT parameters are plotted in Figure 8a. Initial aberrations prior to correction spanned up to an RMS of 423 nm, corresponding to near 0 Strehl ratio. The mean of the data was $276.7$ nm, with initial Strehl ratio always below $0.4$.

When the first pass was applied, RMS errors decreased drastically, reaching a mean of only $58.5$ nm. The range of data also decreased, with standard deviation dropping from $53.6$ nm to $17.4$ nm. The coarse model is capable of predicting the PTT values to a high precision, yet still some uncertainties remain. By implementing a second pass (IMPACT-2), these remaining errors are reduced further, down to a mean of just $11.1$ nm and standard deviation of $5.5$ nm. This progression in performance is mirrored in the Strehl results of Figure 8b. The initial values in light blue are biased heavily $<0.4$ caused by high RMS. After the first pass (IMPACT-1), the Strehls increased with nearly $62.5\%$ of the dataset corrected to above $0.8$. Depending on the applications and use cases of the imaging satellite, the Strehl improvement caused by the first pass shows excellent correction and may already be sufficient. In order to increase the performance as much as possible - to alleviate error budget elsewhere in the satellite - the second pass can be applied. Here the Strehl ratio is increased to near $1.0$, with $99.9\%$ of cases corrected above the diffraction-limit. IMPACT-2 Strehls had a standard deviation of $0.02$, being highly precise. These results show both the accuracy of the prediction and alignment scheme, but also the reliability when applying this method to a varying array of initial aberrations. The large range of initial aberrations ensures nearly all cases can be corrected to a sufficient Strehl ratio.

Visualisation of the correction procedure is shown in Figure 9, with a full system PSF carried through each step of the process. The initial PSF is generated from PTT aberrations incident on each segment. The relative Strehl is calculated to be $0.15$, with an RMS error of 230 nm. Such poor performance is evident through the distorted image and separation of sub-PSFs. The first pass (shown in the middle panel of the image) corrected the Strehl to $0.77$, and RMS to 51 nm. While the image appears to be fully corrected, residual aberrations $\sim \lambda /10$ and smaller are still present. Hence, a second pass is applied for further correction and RMS optimisation. The third panel shows the second pass PSF, with a Strehl of $0.99$ and RMS of only 10 nm. The PTT RMS is reduced by a factor of 20, showing the high potential of the CNN in distinguishing small piston values for fine correction.

4. Conclusions

This study employs a facet design in order to decouple PSF measurement and analysis from the main science detector, allowing continuous alignment. PSF images have been generated for PTT aberrations of varying magnitude, prepared in a four channel data set. Two wavelengths, of 532 nm and 632 nm, and a defocus channel of each have been used to test compact CNN models. The results show this method is capable of correcting RMS errors from as large as $423.4$ nm down to $5.8$ nm, corresponding to a relative Strehl improvement from $0.18$ to $1.0$. Here the worse case RMS is highlighted demonstrating excellent applicability of IMPACT.

Small ground resolving distances and thus high resolution images depend on the sub-micron alignment of the segmented primary. A reduced RMS aberration error ensures diffraction-limited performance for highly detailed imaging, and high ground resolution. Carrying out the full alignment procedure optimises the RMS with excellent efficiency.

Even with the application of the coarse first pass model, mean RMS is reduced from 234 nm down to 49 nm showing accurate predictions of all low order aberrations present. Depending on system requirements, a first pass alone may reach suitable levels of correction. However, in the interest of reducing error budget elsewhere, and aiming for optimised alignment, a second pass is easy to apply and returns excellent results. A mean of 11 nm is achieved with minimal additional resource cost.

The CNN models consist of additional layers for the analysis of non-linear tip and tilt effects. This extends the models to 0.8 MB each, still compact for onboard alignment. Since training and testing is carried out prior to launch, the satellite must only contain the model themselves which will process the images and return actuator controls. Limited compute resources require small models and so their size fits within requirements. The actuator controls will be applied with either one or two passes, greatly reducing iterations compared to other iterative phase retrieval/diversity techniques. Actuator wear will be minimised and thus continuous alignment - to compensate for thermal drift and deformation throughout the mission life - can be performed for an extended amount of time.

Many considerations are worth noting for application to a satellite. The specific satellite design will need to be modelled, however the principle has been shown effective and will be applicable to additional segments or varying sizes. By machining the facets as part of the full congruent segment, the manufacturing will pose little challenge. Also, the use of two wavelengths fits with the capabilities of the satellite. Two lasers can be fed through the fibre, using readily available wavelengths. A defocus can be introduced via additional optics or out of focus positioning. Prior to launch, extensive testing of the alignment system can be carried out; in comparison to satellites which use ground only calibration methods different to in-orbit procedures, this satellite will use the exact same method in both cases and so rigorous tests can ensure alignment success.

While showing excellent alignment, the simulated results rely on highly precise (nanometer level) actuators. Currently, capable technology may not be available or space qualified to achieve such results. Investigation into the uses and limitations of Piezo-legs or similar actuator technology will provide knowledge on the state of the art and feasibility on board. Additionally, ensuring the light incident upon the facetted ring is solely from the fibre may be challenging. The inclusion of a baffle could impose engineering challenges, and so further study into stray light and the effect on measuring PSF images will be highly beneficial. A further limitation imposed by the simulation of the PSF images is the negation of noise. In reality, camera noise, dark current, and other environmental effects will impact the quality of the image acquisition. These effects have thus far been negated from the simulation, and so further testing may be required for the results to be more applicable to real life data.

This study highlights the potential of using only a specified section of the primary mirror, and how the use of a CNN model can predict and thus correct PTT aberrations of the full system. While employing assumptions and a simplified segment model, this work contributes to phase correction of segmented telescopes as a whole, building upon [26] and reinforcing the effective use of ML as a method of PSF analysis for aberrations.