Landsat-8/9 Atmospheric Correction Reliability Using Scene-Statistics

David Groeneveld; Tim Ruggles; Bo-Cai Gao

doi:10.20944/preprints202404.1745.v1

Submitted:

25 April 2024

Posted:

26 April 2024

You are already at the latest version

Abstract

Smallsats are becoming the predominant electro-optical Earth observation (EO) imaging platforms. While atmospheric correction of smallsat data enhances its utility, an established pathway to do so may constrain accuracy and utility. The alternative, Closed-form Method for Atmospheric Correction (CMAC), developed for smallsat application provides surface reflectance derived solely from scene statistics. In a prior paper, CMAC closely agreed with Land Surface Reflectance Code (LaSRC) software for correction of the four VNIR bands of Landsat-8/9 images for conditions of low to moderate atmospheric effect over quasi-invariant warehouse-industrial targets. Those results were accepted as surrogate surface reflectance to support analysis of CMAC and LaSRC reliability for surface reflectance retrieval in two contrasting environments: shortgrass prairie and barren desert. Reliability was defined and tested through a null hypothesis: the same top-of-atmosphere reflectance under the same atmospheric condition will provide the same estimate of surface reflectance. Evaluated against the prior surrogate surface reflectance, the results found decreasing error with increasing wavelength for both methods. From 58 comparisons across the four bands, LaSRC average absolute error ranged from 0.59% (NIR) to 50.30% (blue). CMAC error was well constrained from 0.01% (NIR) to 0.98% (blue) sustaining the null hypothesis for reliability.

Keywords:

surface reflectance

;

retrieval

;

LaSRC

;

CMAC

;

scene statistics

;

near real-time

;

spectral diversity

Subject:

Engineering - Aerospace Engineering

1. Introduction

Through smallsats, electro-optical Earth observation (EO) is rapidly expanding, enabled by advances in electronics, imaging sensors, data transmission and miniaturization of components. The resulting smallsat constellations provide rapid repeat imagery that are especially needed to better understand and manage the unprecedented planetary-scale threats from climate change. Perhaps the greatest challenge for all EO data application is that it is obtained through the atmosphere that variably corrupts the data. The solution is to correct the data to surface reflectance, a process that seeks to remove the atmospheric effect entirely resulting in clear images and restored digital signals. However, atmospheric correction is problematic for the many hundreds of EO smallsats without onboard equipment to calibrate sensor output permitting direct conversion to surface reflectance.

A proposed pathway for smallsat surface reflectance correction applies the Land Surface Reflectance Code (LaSRC) and cross calibration to the data of two research grade satellite platforms atmospherically corrected by LaSRC: Landsat-8/9 and Sentinel-2 A/B [1,2,3]. Such cross calibration can introduce uncertainty due to mismatched overpass timing and spectral responses between sensor platforms. LaSRC currently requires ancillary data to assess the degree of atmospheric effect on the day of the smallsat’s image acquisition, both for calibration and application. This potentially adds another layer of uncertainty in operational surface reflectance retrieval due to any temporally mismatched image collection for the ancillary data. Ancillary data also delays image processing and output and may have coarser granularity that reduces spatial sensitivity. CMAC was formulated to avoid these sources of uncertainty.

The Closed-form Method for Atmospheric Correction (CMAC) was developed to deliver surface reflectance with no delay upon image download. Its development was prompted by a seminal observation of atmospherically driven reflectance changes. The novel CMAC pathway was tested against Sen2Cor for Sentinel-2 correction [1] and LaSRC for Landsat-8/9 correction [2]. Corrected Landsat data by LaSRC, L2A, are available only through Earth Explorer and represent the general current state of the art in surface reflectance retrieval. CMAC proved accurate and precise for higher levels of atmospheric effect estimated from each image’s spectral data, alone. This present paper investigates an additional inquiry for CMAC application, whether the reliability of surface reflectance output from one area of interest (AOI) applies to all other environments, especially those with very different spectral characteristics. As a yardstick for this comparison, CMAC reliability is compared to the state-of-the-art LaSRC correction for the four visible and near infrared bands (VNIR) of Landsat-8/9.

The paucity of surface reflectance data is a challenge for evaluating atmospheric correction. A few such datasets exist but virtually never in time and space to support sustained, focused testing to compare methods, thus necessitating a workaround for investigating the reliability of atmospheric corrections. The workaround for this investigation relies upon a truism relative to atmospheric correction: correction accuracy is greatest for scenes taken through a “clean” atmosphere. This is partially because clean images require much less adjustment to achieve surface reflectance, but also because the engineering tolerances to accurately retrieve surface reflectance become tighter and the solution accuracy more critical as atmospheric effects increase. In the prior investigation [2], CMAC and LaSRC closely agreed on images acquired under relatively clean atmospheric conditions. These data were accepted as surrogate surface reflectance to provide a reference to construct the datasets to support this analysis.

This paper investigates (1) the reliability of CMAC to provide surface reflectance output for environments widely different from where CMAC was developed and calibrated, and (2) to evaluate the functionality of CMAC in relationship to the widely accepted state of the art, LaSRC. A null hypothesis was formulated to express atmospheric correction reliability: Equivalent top-of-atmosphere reflectance from images of different environments collected under the same level of atmospheric effect, when atmospherically corrected, will yield equivalent surface reflectance (i.e., no difference). This hypothesis can be stated more simply as “the same input affected by the same conditions will yield the same output.”

A point of continual reference in past papers that we use again here is application of image appearance that includes clarity and color balance of atmospherically corrected images [1,2]. Though a subjective interpretation, images that appear clear are logically closer to surface reflectance than those containing visible haze. Figure 1 is an example of an extremely hazy image portrayed as top-of-atmosphere reflectance (TOAR) and CMAC and LaSRC-corrected views and illustrates the importance of scene appearance for judging atmospheric correction. While the lack of surface reflectance groundtruth in appropriate time and at an appropriate scale can be problematic for cross checking the accuracy of retrieved surface reflectance, image appearance is useful as a qualitative guide for correction accuracy.

2. Materials and Methods

CMAC is a recently developed method intended for atmospheric correction of smallsat images. CMAC processing applies reflectance in two steps that first map and then reverse atmospheric effect spatially for each image. These steps and how they were developed are described in Appendix A and the interested reader can also consult previous journal papers [1,2] for additional information.

This investigation applies an earlier comparison of CMAC to LaSRC for 31 relatively clear Landsat-8 and 9 images of five AOIs of warehouse/industrial districts in Southern California (SoCal) known to have consistent surface reflectance [2]. In that analysis, the average cumulative distribution functions (CDFs) for these two disparate methods agreed to such a close extent that they plotted virtually atop one another (Figure 2). Those paired datasets are employed here as surrogate estimates of surface reflectance. By averaging and interpolating the values for CDF extractions from images of the 31-image cohort, datasets were constructed to assess CMAC and LaSRC output under the same atmospheric conditions from the same TOAR input from completely different environments than where these methods were in close agreement. The high dark-to-bright dynamic range of reflectance in the SoCal dataset is an important distinction because the AOIs selected for comparison of method reliability have extremely low dark to bright dynamic spectral range.

Landsat-8 and 9 data were not considered separately given the close agreement of these paired satellites [4]. LaSRC applies a radiative transfer (RadTran) based workflow that is documented in readily attainable remote sensing literature that readers are urged to consult [5,6]. RadTran calculations account for the various reflectance, absorbance, transmittance, etc., components to estimate the amount of light, radiance, measured by the sensor.

Two locations were selected for analysis of method reliability, both with extremely low dark to bright dynamic spectral range very different from other areas where CMAC has been calibrated or applied. An AOI just west of Lake Newell, Alberta, Canada was chosen to represent shortgrass prairie, a vegetation cover occupying a band of semiarid climate that runs north to south for over 2000 km across the United States and Canada. A second site adjacent to the El Pinacate volcanic uplands in Sonora, Mexico represents profound desert of exposed sand with sparse, widespread shrubby trees constituting less than three percent cover within the AOI investigated; such profound deserts are found in significant proportions of South America, Asia, Africa and Australia. Shapefiles were mapped to enclose homogeneous cover for both AOIs (Figure 3). A regional view from the 7-04-2022 Landsat-8 El Pinacate region in Appendix B provides a wider view of the TOAR, CMAC-corrected and LaSRC-corrected examples and further context for atmospheric correction over deserts with low dynamic spectral reflectance contrasting with adjacent areas surrounding the Gulf of California shore exhibiting high dynamic spectral reflectance.

Three Landsat-8 and 9 images were selected for each AOI from the Landsat archives and downloaded as both uncorrected and LaSRC-corrected versions from Earth Explorer (Table 1). These six images were corrected by CMAC v1.1L, and calibrated for Landsat-8/9 application; the same processing was also applied in the previous CMAC-to-LaSRC comparison [2]. The dataset to test the null hypothesis was constructed through back comparison to the SoCal datasets from AOIs with quasi-invariant reflectance where CMAC and LaSRC showed close agreement. Any of the five SoCal AOIs would work for this application because the 31 images were affected by relatively moderate atmospheric effect, and because these two disparate methods resulted in virtually the same corrected data distributions per AOI. Two were selected, Fontana and Rochester, whose reflectance distributions are shown in Figure 2. Since the true surface reflectance is unknown and there were slight differences between CMAC and LaSRC in the SoCal results, this analysis regarded the SoCal surface reflectance estimates as most appropriate for application per atmospheric correction method: CMAC for CMAC and LaSRC for LaSRC. This ensured that the choice of datasets did not bias the results. For these comparisons, Atm-I was applied to measure the atmospheric effect rather than the aerosol optical thickness ancillary data applied in LaSRC from the Moderate Resolution Imaging Spectroradiometer (MODIS) [3]; however, the LaSRC datasets downloaded from Earth Explorer were generated with the workflow that applies MODIS AOT for assessment of atmospheric effect [5,6].

Pixel values for the blue, green, red and NIR bands were extracted from within the AOI polygons shown in Figure 3 and exported to spreadsheets for analysis. Reflectance distributions from each image for three treatments (i.e., TOAR, corrected CMAC and LaSRC) were extracted and represented as CDFs in 21 percentiles from 1%, 3% and in 5-percentile steps between 5% and 95%. Data for the Rochester AOI was matched with the 8-14-2023 Lake Newell image to check for bias from selection of the SoCal AOI, none was found in comparison to data for the Fontana AOI that was matched with the other five images (Table 1).

Construction of the dataset for testing the null hypothesis began by finding the Atm-I of multiple images of the SoCal datasets whose averages equaled the median Atm-I’s from the images selected for Lake Newell and El Pinacate. The individual values of the 21 percentile steps for the distributions are arrayed in columns in the spreadsheets, one column per image, ranked by increasing Atm-I. This format facilitated averaging image values to support the comparisons by pairing the experimental image TOAR, CMAC and LaSRC values with the corresponding averaged values for the SoCal images.

Appendix C provides portions of the combined 7-29-2023 Lake Newell and Fontana datasets that were reformatted and annotated to support explanation of the workflow to identify values for the three defining properties of the null hypothesis: (1) Atm-I conditions in the SoCal dataset were selected whose averages equaled the experimental datasets, thereby achieving the same atmospheric conditions; (2) interpolation to identify the exact TOAR and its percentile position in each of the six experimental images to match the SoCal TOAR input for atmospheric correction; and (3) identifying the corresponding atmospherically corrected output values from the SoCal dataset for testing the null hypothesis. Interrelating the TOAR data and the surface reflectance calculated from it was accomplished within each dataset using their percentile positions. The 7-29-2023 Lake Newell spreadsheet contained in the Supplementary Materials can be compared to Appendix C to assist following the calculation workflow.

Error for the atmospherically corrected data was estimated by treating the Fontana and Rochester-corrected surface reflectance estimates as the standard to assess CMAC and LaSRC error: % error = 100 x (value – standard)/standard. This comparison was judged to be valid because the atmospheric correction results for the spectrally diverse SoCal AOIs were accepted as surrogate surface reflectance. “value” in this formula represents the Lake Newell and El Pinacate surface reflectance estimates.

This statistical distribution-based workflow was repeated for all four bands for each Lake Newell and El Pinacate image. In this manner, a series of common TOAR values, and the CMAC and LaSRC surface reflectances estimated from them, were interpolated from these two datasets.

3. Results

CDFs for the four bands of the three treatments of the three images per AOI afford a comprehensive look at the responses per correction method (Figure 4). The TOAR CDFs for Lake Newell illustrate diverse reflectance due to the vegetated shortgrass prairie in comparison to El Pinacate, where the reflectance remained consistent for the ground surface virtually devoid of perennial vegetation.

The extremely low variability of the El Pinacate distributions in Figure 4 illustrates several trends. The CMAC-corrected bands have greater spacing and are positioned left of LaSRC. All three treatments portray red reflectance distributions as having less coherence.

The Lake Newell CMAC distributions are tighter than LaSRC. The unexpected discrepancy for NIR observed in the Lake Newell data was due to rain prior to the 8-14-2023 image that was investigated and confirmed as described in Appendix E. Lake Newell LaSRC CDFs for the highest atmospheric effect (8-06-2023, 1044 versus 920s for the other two dates) are displaced to the right. This discrepancy is an indication that the increase of Atm-I from 920s to 1044 resulted in under correction by LaSRC; an interpretation based on the fact that atmospheric correction reduces the brightening effect of backscatter by moving the CDFs to the left. Hence under correction results in the 8-06-20223 being displaced rightward in relation to the corrected images from 7-29-2023 and 8-14-2023 . The CMAC corrections of the visible bands were unaffected by Atm-I and maintain consistency and close agreement as would be expected for the reflectance of midsummer shortgrass prairie when vegetation growth is essentially static.

Figure 5 presents the bandwise Lake Newell CDFs plotted with the surface reflectance points reconstructed from the SoCal AOIs identified through the workflow described earlier. For all bands, the reconstructed SoCal surface reflectance points of CMAC lie on the CDFs for Lake Newell. Many of the LaSRC points reconstructed in the same workflow, also lie on or close to the Lake Newell CDFs, partially corroborating that CMAC provides accurate surface reflectance estimates, though disagreeing with the LaSRC CDFs. Thus, for the Lake Newell comparisons, the null hypothesis that CMAC provides output equivalent to the surface reflectance surrogate dataset of SoCal is accepted. Judged by the data plots in Figure 5, any error between the SoCal and the Lake Newell datasets was slight.

In contrast to CMAC results, the points displayed for LaSRC reconstructed from the SoCal dataset TOAR values disagree with the Lake Newell reflectance distributions in all twelve graphic comparisons in Figure 5. The Lake Newell analysis was performed first. The analysis for the El Pinacate AOI was begun to verify the same relationships for a different environment, one of profound aridity and almost no vegetation cover.

El Pinacate data plotted in Figure 6 confirmed the results from the CMAC visible band in Figure 5 calculated from the shared SoCal TOAR reflectance values. Fontana CMAC surface reflectance estimates lie close to the CMAC El Pinacate surface reflectance distributions for blue, green, and red. The SoCal LaSRC points plotted closer to the CMAC distribution than to the LaSRC El Pinacate distributions. Figure 6 LaSRC NIR points plot differently than for Lake Newell (Figure 5) instead essentially lying on the reflectance distribution for El Pinacate TOAR, indicating that virtually no correction for NIR occurred.

Error for surface reflectance estimation by CMAC and LaSRC was calculated by treating the surrogate SoCal reflectance values as true surface reflectance that are presented in Table 2 and Table 3. For CMAC, the Lake Newell and El Pinacate surface reflectance estimates agree well with the SoCal surrogate true surface reflectance; CMAC error was low and almost evenly distributed between positive and negative values, hence unbiased. CMAC results were comparable between the Lake Newell and El Pinacate datasets. The average absolute value of CMAC surface reflectance error did not exceed 1% for the blue band that experienced the greatest error. The average absolute value of LaSRC error for the Lake Newell shortgrass prairie was severe, around 50% for the blue band. The error for LaSRC was lower for El Pinacate but still an order of magnitude greater than CMAC. Error for both CMAC and LaSRC decreased with increasing wavelength.

Sufficient data are presented in Supplementary Materials to allow the interested reader to reconstruct and verify the workflow and the results. These include averages, interpolations and error calculations and spreadsheets. Values derived through this analysis are summarized in tables within Appendix D. Spreadsheets and shapefiles of the Fontana and Rochester AOIs are provided along with spreadsheets and shapefiles for the Lake Newell and El Pinacate AOIs. Cloud-based image browsing, selection, and CMAC correction and download of Landsat-8/9 and Sentinel-2 VNIR bands can be accessed through a link in Supplementary Materials.

4. Discussion

The CMAC surface reflectance estimates for Lake Newell and El Pinacate were within 99% agreement with the CMAC SoCal surrogate surface reflectance estimates in all 58 TOAR-based comparisons across the four VNIR bands (agreement was calculated as 100% minus the % error). The strong agreement for CMAC results between datasets from widely diverse environments validates accuracy and reliability for CMAC processing and for its constituent assessment of atmospheric effect and conceptual model-derived workflow that reverses it. Further corroborating CMAC accuracy is the observation that the LaSRC surface reflectance in Figure 5 and Figure 6 for Lake Newell and El Pinacate lie closer to the CMAC distributions than to the LaSRC distributions, in many cases plotting atop the CMAC SoCal points.

The CMAC data demonstrate accuracy independent of the dynamic spectral range (highest minus lowest reflectance values): the SoCal AOIs had extremely wide ranges of values (Figure 2) while the spectral ranges for Lake Newell and El Pinacate were extremely narrow (Figure 4). For the clear to moderately hazy conditions examined here, the null hypothesis is accepted: CMAC analyses produced the same surface reflectance estimates from the same TOAR input under the same atmospheric conditions despite differences in the two terrestrial environments examined.

The LaSRC analysis demonstrated surface reflectance estimates with average agreement as low as 50%; hence, the null hypothesis is rejected – LaSRC was not reliable for estimation of surface reflectance across the two environments. This discrepancy may be related to the low dynamic spectral range of the Lake Newell and El Pinacate locations; however, this issue is more complicated because the same bandwise dynamic spectral ranges were comparable between these two experimental AOIs, but the degree of error for Lake Newell was about four times that of the El Pinacate error.

Atmospheric correction of satellite imagery by LaSRC, widely viewed as the state of the art in radiative transfer application for EO imagery is proposed as the basis for smallsat atmospheric correction through a cross calibration process with harmonized data from Landsat-8/9 and Sentinel-2 [3,7]. However, reliance upon LaSRC for smallsat applications can be expected to incorporate the same problems that reduce LaSRC accuracy. These problems will include loss of accuracy at higher levels of Atm-I that was found for LaSRC under conditions of increasing haze from wildfire [2] and from these results, lack of reliable accuracy hypothetically related to low spectral diversity environments.

CMAC is a unique pathway for atmospheric correction and its testing here and in the previous two journal papers has found that its performance was more accurate over a wider range of atmospheric effect than Sen2Cor and LaSRC. Rather than delaying surface reflectance output waiting for ancillary data, CMAC can process images immediately upon download from the satellite. Due to its robust and simple mathematical structure, CMAC can readily be calibrated for Smallsat application for any VNIR band combination due to its robust mathematical structure. CMAC will be adapted to correct data from hyperspectral sensors in a next-generation program that will include accurizing the Atm-I model, reliable overwater correction, and development/application of a calibration target and the technology to apply it under automation.

The greatest source of uncertainty in the CMAC workflow is for the measure of atmospheric effect, Atm-I. While Atm-I can be shown to be far more sensitive than the ancillary data currently in use by LaSRC from MODIS [2] it was generated by a static assumption of reflectance of a reference crop rather than actual reflectance measurements. The key to this upgrade is extensive groundtruth. Likewise, extensive groundtruth will also permit spectral modeling to isolate and remove the specular reflectance component of water surfaces that could yield accurate water-leaving reflectance. Though CMAC generally performs well over water, the overarching effect of image geometry has not yet been characterized.

Calibration is the key for application of surface reflectance retrieval. This step is presently done vicariously requiring the use of master images of Sentinel-2 compared to proxy images from smallsats. Even though these steps are automated, this program is inefficient because it requires visual assessment steps to assure accuracy. Vicarious calibration can be replaced by a workflow that employs a well-engineered, constructed, managed, and monitored calibration target. Such a target is expected to yield greater precision, accuracy and automation for CMAC calibration. The benefit of periodic automated calibration is that it allows detection and compensation of episodic in-orbit radiation-related sensor degradation. [8].

5. Conclusions

This investigation confirms that CMAC provides reliable surface reflectance retrieval for all environments. When judged by the appearance of many varied images, CMAC has accurately corrected all terrestrial environments, including deserts, arctic and alpine tundra, tropical and temperate forests, savannahs, grasslands, and farmland on six continents. CMAC has also produced excellent results over the ocean as can be seen in Appendix B, Figure B-3. Smallsats can be calibrated readily for direct application of CMAC without incorporating additional uncertainty of ancillary data.

6. Patents

Currently, one CMAC patent is granted. Two additional patent applications are pending before the US Patent Trade Office with one of these filed internationally through the Patent Cooperation Treaty (International Search Report - language approved as filed).

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org: “live” Excel workbooks for Fontana, Rochester, El Pinacate and Lake Newell locations, Collations for Bandwise CDFs for the two experimental locations, and the shapefiles of these four locations to facilitate data extraction. Sentinel-2 and Landsat-8/9 images can be browsed, selected and corrected by the user at https://strato.advancedremotesensing.com/app (accessed on 21 April 2024).

Author Contributions

D.P.G., and T.A.R.; methodology, D.P.G. and T.A.R.; software, T.A.R. and D.P.G.; Formal analysis, D.P.G.; data curation, T.A.R. and D.P.G.; visualization, D.P.G. and B.-C.G.; original draft, D.P.G.; review and editing, D.P.G., T.A.R. and B.-C.G.; resources, D.P.G. and B.-C.G.; project administration, D.P.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the U.S. National Science Foundation Small Business Innovation Research program through grants #1840196 and #1950746.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Please see Supplementary Materials for accessing the analyzed data, shapefiles for extraction of data and the Cloud-based site for test correction and download of Landsat-8/9 images.

Acknowledgments

We thank the Landsat programs for the steady stream of free, high-quality imagery and its excellent documentation and celebrate Landsat’s pathfinder role for imaging satellite technology. Our heartfelt thanks in memory of Dr. Thomas Loveland (dec. 13 May 2022), a central figure in Landsat applications, for his friendship, encouragement and insights early in our R&D process.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A, Brief Description of CMAC Workflow

A.1 Atmospheric Effect Mapped as a Grayscale

The first CMAC step employs the spatially discrete spectral band statistics from the scene as input to a model that assesses atmospheric effect in images. The Atm-I model was developed using dense dark vegetation (DDV) measured by field spectrometry to establish a standard value for a ubiquitous index crop. This crop was identified and sampled under a wide range of atmospheric and surface cover conditions that resulted in a robust scene-based statistical model [1]. Application of the Atm-I model generates a grayscale map that expresses the degree of effect across the image as a numerical scalar for the correction needed to return each pixel’s TOAR to its original surface reflectance.

A.2 Reversing the Mapped Atmospheric Effect to Deliver Surface Reflectance

The second CMAC step reverses the atmospheric effect based upon a conceptual model derived from an observed phenomenon that initially prompted CMAC development. For a hazy and clear pair of TOAR images over an AOI whose surface reflectance has remained relatively consistent, their CDFs will vary systematically as shown in Figure A.1. Imposition of increasing aerosol causes the distributions to rotate counterclockwise, and for decreasing aerosol, clockwise. This observation prompted development of a conceptual model as the basis for reversal of atmospheric effect. To facilitate referencing this phenomenon, it was dubbed the “pinwheel effect.” This conceptualization was valuable only as a first step in formulating the CMAC approach for atmospheric correction.

Figure A1. CDFs of two Sentinel-2 spectral bands extracted from an AOI with consistent surface reflectance across both image snapshots. Increasing. Arrows show the direction of CDF rotation from increasing haze. This effect occurs in all VNIR bands.

In the pinwheel effect, increasing dark reflectance in response to increasing haze is due to aerosol backscatter of light. The decrease of bright reflectance is due to attenuation from absorption and diffuse shading by aerosol particles. When viewed as CDFs, changing levels of haze cause the distribution to rotate around an axis point where scatter and attenuation balance and the value remains unchanged. Subsequent observation indicated that this axis point migrates rightward with increasing Atm-I. hypothetically due to forward scatter as a property of the target’s brightness. Target brightness influences the magnitude of the reflected energy and its interaction with atmospheric aerosol. In the context of EO, this forward scatter is the illumination of aerosol from below that increases in proportion to ground target brightness. Forward scatter is still somewhat hypothetical and is the subject of ongoing focus; however, it may hold the key to detecting and reversing the effect of specular reflectance over water. The Atm-I model is sensitive to forward scatter and performs well to remove specular reflectance from water (Appendix B).

Application of data distributions was a key factor for development of CMAC, since atmospheric correction seeks to return the range of TOAR pixel values to surface reflectance. CDFs are a robust means to approach the atmospheric correction problem since individual pixel values could be correct in a distribution that is incorrect, but not vice versa. Additionally, the ranked position of any value within reflectance distributions remains the same through various treatments; hence mathematically translated values can be identified afterward by their percentile position in the distribution. This feature enabled finding equivalent properties for intercomparison of experimental data extracted from very different environments to test the null hypothesis.

The observed reflectance behavior of Figure A.1 was expressed as a graphic model by inversion and adjustment of the well-known empirical line method [9]. The resulting linear model in Figure A-2 has precedence in a 40-year-old paper by two prominent researchers (Figure A-.3).

Figure A2. The CMAC conceptual model. Blue arrows indicate the rotational direction for increasing atmospheric effect. The dashed line represents all pixels, dark to bright, under one atmospheric effect for a single spectral band.

Figure A3. Figure 2 copied from Fraser and Kaufman, 1985 [10]. The solid lines represent common atmospheric aerosol and are equivalent to the dashed line in Figure 4., Dashed lines represent highly absorptive carbon particles to illustrate the importance of aerosol absorption upon reflectance.

The conceptual model of Figure 4 was translated into the CMAC Equation that reverses atmospheric effect for each pixel of an image (Equation 1). The pinwheel effect of Atm-I in Figure 2 is represented by the upward/downward blue arrows in Figure 4 that results in linear deviation of surface reflectance due to atmospheric effect. Slope and offset uniquely define any TOAR deviation line and are the parameters applied in Equation 1 to reverse atmospheric effect to retrieve surface reflectance. Slope and offset responses are unique for each sensor band and are determined through calibration. Presently, calibration is accomplished through image-to-image methods using a Sentinel-2 CMAC master calibration that was generated from several years of painstaking effort. Image-to-image calibration will eventually be replaced with a well-engineered and managed calibration target providing increased accuracy and precision so that data from one to several sensor overpasses can be applied in an automated process. Automated recalibration will guard against episodic radiation-induced sensor changes well-known to occur in orbit [11].

SR = (TOAR - b) / (m + 1)

where m is slope and b the offset of the TOAR deviation line.

Appendix B, El Pinacate Region Data Processed by CMAC and LaSRC for Visual Comparison

The 7-04-2022 Landsat-8 image displayed in TOAR, Atm-I and CMAC and LaSRC corrected views.

Figure B1. TOAR View. The moderate level of haze obscures ground features over the desert. Light colored features of the ocean result from a mix of entrained sediments in the water column and (hypothetically) from specular reflectance of the sky, This was influenced by wind, in evidence as streaks from the northwest.

Figure B2. Atm-I Grayscale View. This atmospheric model output was applied to scale the degree of correction removing the atmospheric effect in the 7-04-2022 image: the brighter the response, the greater the correction. Of note is the brightness of the grayscale over the Sea of Corez hypothetically induced by specular reflectance. Atm-I is a statistical representation of the atmospheric effect and, as such, has lower resolution than the original image, hence the faint streaks from wind effects visible in the TOAR view are smeared in this view.

Figure B3. CMAC Corrected View. With specular reflectance largely removed from the TOAR view of the ocean, patterns of entrained sediments and green-tinted water are now visible. Terrestrial features of windblown dunes and the complex hydrology surrounding the bay are visible after CMAC processing that were indistinct in the TOAR view. Future research is expected to define a relationship for the atmospheric statistical model (Atm-I) measurement of specular reflectance to enable reliable atmospheric correction over water.

Figure 4. LaSRC corrected View. Like the CMAC correction, finer features of the image are visible after LaSRC correction. Image artifacts over the water are a common feature created by LaSRC correction. Such artifacts are also visible in the LaSRC view of Figure 1 (panel c).

Appendix C. Annotated Spreadsheet Explaining Dataset Construction

This annotated example spreadsheet shows the calculation workflow for comparison of extracted L8 blue band data. Note that the TOAR, CMAC, and LaSRC tables are stacked vertically in the original spreadsheet but rearranged here for ease of illustration. Tables of five images are from the Fontana AOI whose TOAR values correspond with the TOAR column for the Lake Newell AOI (shaded). Portions of the distribution are enclosed in boxes defining where the average TOAR of Fontana was found by interpolation of the Lake Newell distributions. The process began at A with selection of five sequential images of Fontana whose median Atm-I values were averaged and found to equal the median Atm-I of the Lake Newell AOI. The interpolations of the data in the shaded column boxes were performed graphically in the original spreadsheets that are downloadable from supplementary materials.

Appendix D, Summaries of Spreadsheet Statistics

Data tables for three images each of the El Pinacate and Lake Newell AOIs. Derivation of these data can be found in the spreadsheets available in Supplementary Materials. Column 2 presents the TOAR shared values in common among the SoCal, El Pinacate or Lake Newell datasets. Corresponding percentiles for these shared TOAR values are presented in the 3rd column. Interpolated reflectance values based upon shared TOAR are provided for SoCal CMAC (col. 4 and 5) and LaSRC (col. 7 and 8), are points plotted in Figure 5 and Figure 6 and summarized in Table 2 and Table 3 from col. 6 and 9.

Table D1. El Pinacate Statistics.

Table D2. Lake Newell Statistics (following page).

Appendix E, Why the Disconnect for 8-14-23 NIR Band CDF?

Change detection confirmed rainfall between prior to 8-14-2023 visible as a darkened smear to the west of Lake Newell. Figure 5 provides a color reference for the 8-06-2023 image. This analysis was performed to confirm the validity of the extracted NIR values to explain why the 8-14 NIR results did not conform with the other two dates in Figure 4. The linear features that cross the area west of Lake Newell are gravel roads (confirmed on Google Earth) that drain rapidly and dry much quicker than the surrounding prairie. The prominence of these roads on the change detection image confirms that the darker area is not an atmospheric issue. Lakes appear black here because the TOAR reflectance was elevated due to haze (higher Atm-I) and are brighter on the 8-06-2023 .

Steps in this image analysis were:

Subtract the 8-06-2023 NIR raster from the 8-14-2023 NIR raster. Minimum and maximum display values were selected to accentuate the display of wetter conditions guided by the magnitude of the differences visible for NIR in Figure 4.

2.: The resulting dark pattern was then compared against other variables across the AOI to ensure that this pattern was not explainable by some other image property – none were found.
3.: The analysis supports the conclusion that the AOI received rainfall from a localized thunderstorm likely within several days prior to 8-14-2023.

References

Groeneveld, D.P.; Ruggles, T.A.; Gao, B.-C. Closed-Form Method for Atmospheric Correction (CMAC) of Smallsat Data Using Scene Statistics. Appl. Sci. 2023, 13, 6352. https://doi.org/10.3390/app13106352 (accessed 16 April 2024).
Groeneveld, D.P.; Ruggles, T.A. An Algorithm Developed for Smallsats Accurately Retrieves Landsat Surface Reflectance Using Scene Statistics. Appl. Sci. 2023, 13, 12604. https://doi.org/10.3390/ app132312604 (accessed 16 April 2024).
Claverie, M.; Ju, J.; Masek, J. G.; Dungan, J. L.; Vermote, E. F.; Roger, J.-C.; Skakun, S. V.; Justice, C. The harmonized Landsat and Sentinel-2 surface reflectance data set. Remote Sens. Environ. 2018, 219 pp. 145-161.
Gross, G.; Helder, D.; Begeman, C.; Leigh, L.; Kaewmanee, M. and Shah, R. Initial cross-calibration of Landsat 8 and 9 using the simultaneous underfly event. Remote Sens. 2022. Volume 14, 2418, https://doi.org/10.3390/rs14102418 (accessed 16 April 2024).
Landsat Mission. Landsat 8-9 Collection 2 Level 2 Science Product Guide. March 30, 2023. https://www.usgs.gov/media/files/landsat-8-9-collection-2-level-2-science-product-guide (accessed 16 April 2024).
Vermote, E.;Justice, C.; Claverie, M.; and Franch, B. Preliminary analysis of the performance of the Landsat 8/OLI land surface reflectance product. Remote Sens. Env. 2016. Volume 185 https://doi.org/10.1016/j.rse.2016.04.008 (accessed 16 April 2024).
Kington, J.; Collison, A. Scene level normalization and harmonization of Planet Dove imagery; 2022. Available online: https://earth.esa.int/eogateway/missions/planetscope/objectives (Accessed 16 April 2024).
Karpouzli, E.; Malthus, T. The empirical line method for the atmospheric correction of IKONOS imagery. Int. J. Remote Sens. 2003, 24, 1143–1150, . https://doi.org/10.1080/0143116021000026779 (accessed 16 April 2024).. [Google Scholar]
Fraser, R.S.; Kaufman, Y.J. The relative importance of aerosol scattering and absorption in remote sensing. IEEE Trans. Geosci. Remote Sens. 1985, GE-23, 625–633.
Kabir, S.; Leigh, L.; Helder, D. Vicarious methodologies to assess and improve the quality of optical remote sensing images: a critical review. Remote Sens. 2020, 12, 4029. https://doi.org/10.3390/rs12244029 (accessed 16 April 2024).
Xapsos, M. The single event effects in the environment of space. PowerPoint for Texas A&M University Bootcamp. 2021NASA Goddard Space flight Center. 2020. Available online: https://ntrs.nasa.gov/api/citations/20205011676/downloads/TAMUBootcampSEE.pdf. (accessed on 21 April 2024).

Figure 1. The quality of atmospheric correction can be judged by image clarity and color balance. Landsat 8 image of Kelowna, British Columbia, 8-08-2021: (a) uncorrected TOAR, (b) CMAC corrected, and (c) LaSRC corrected.

Figure 2. Bandwise reflectance CDFs for the four VNIR bands averaged for all 31 images of Landsat-8 and 9 of TOAR, CMAC and LaSRC treatments extracted from AOIs in two Southern California municipalities. The resulting curves show nearly complete agreement between CMAC and LaSRC. High dynamic spectral range, dark to bright, for each band is demonstrated by the wide range of extracted reflectance values.

Figure 3. AOIs of Lake Newell (L8, 8-06-2023; top) and El Pinacate (L8, 7-04-2022; bottom) shown for three treatments: (a) TOAR; (b) CMAC; and (c) LaSRC. The area inside the Lake Newell AOI is 27.46 km² and 12.62 km² for the El Pinacate AOI.

Figure 4. Extracted reflectance by treatment with dot, dash and solid patterns that identify each image evaluated by the two methods. Other than TOAR for Lake Newell, the bands in the graphs stack from left to right according to increasing wavelength. These AOIs have extremely low spectral dynamic range compared to Figure 2 and plot here as almost vertical lines.

Figure 5. Lake Newell CDFs per band displayed with the points determined from the SoCal AOIs: Rochester (for 8-14-2023) and Fontana (for 7-29-2023 and 8-06-2023). The scaling of the x-axes is optimized to provide equivalent reflectance intervals for the same range per band. Where points overlie each other, the LaSRC SoCal points are depicted in white and of smaller size.

Figure 6. Per band El Pinacate CDFs displayed with the points reconstructed from the SoCal AOI of Fontana. The scaling of the x-axes is optimized to provide equivalent reflectance intervals for the same range per band. Where points overlie each other, the LaSRC SoCal points are depicted in white and of smaller size.

Table 1. Data for the six Landsat-8 and 9 images that were selected for this analysis.

Table 2. Error calculated for surface reflectance estimation of three images of El Pinacate calculated from the average of absolute values. Each cell is the error in surface reflectance output from a TOAR input. This comparison included 21 individual comparisons across the four bands for each method.

Table 3. Error calculated for surface reflectance estimates for each band of three Lake Newell images. Overall averages were calculated from absolute values. Each cell is the error in surface reflectance output from a TOAR input. This comparison included 37 individual comparisons across the four bands for each method.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.