Improving The Spatiotemporal Transferability of Hyperspectral Remote Sensing for Estimating Soil Organic Matter by Minimizing The Coupling Effect of Soil Physical Properties on The Spectrum

1. Introduction

As the largest carbon reservoir among the terrestrial ecosystems, soil (pedosphere) constitutes the global carbon cycle with hydrosphere, atmosphere, biosphere, geosphere, and lithosphere [1]. Minor soil carbon pool changes can significantly alter atmospheric CO₂ concentration, thereby affecting global carbon cycling and climate [2]. The vast majority of carbon stored in soil is organic carbon as the carbon component of organic matter [3,4,5]. Soil organic matter (SOM) is also the primary source of biological nutrients and energy in soil, and the SOM content is often used as a vital soil fertility evaluation indicator [6,7]. Therefore, precisely understanding the SOM content and spatial distribution is crucial for promoting sustainable agricultural development, enhancing soil carbon sequestration potential, and regulating global climate change [8,9].

Remote sensing is a low-cost, high-accuracy, real-time method for multi-angle, multi-temporal, and large-area earth observation [10,11,12]. At present, the ability of hyperspectral remote sensing to predict and map SOM contents has been confirmed in many studies [13,14,15]. With the rapid growth of remote sensing data and the urgent need for large-scale soil surveys, the research on remote sensing-based soil element content prediction has gradually shifted from constructing high-precision prediction models to establishing prediction models with strong spatiotemporal transferability [16,17,18]. Imaging spectra are the most important data source, and their characteristic response to soil chemical composition is an important foundation for hyperspectral remote sensing-based SOM content prediction [18,19,20]. However, imaging spectra are not affected by soil composition alone but comprehensively reflect the soil physical properties and chemical composition within the ground sample, and soil physical properties and chemical composition exert a coupling effect on the response to the spectrum [21,22,23]. Research has shown that the scattering contribution of soil physical properties, such as soil moisture (SM) and surface roughness properties (e.g., root mean squared height, RMSH), to spectral reflectance seriously affects the sensitivity of the hyperspectral data to the SOM content [24]. The near-infrared spectrum is very sensitive to a small amount of water or hydroxyl group, easily causing irregular radiation characteristics. As the SM content increases continuously until saturation, the soil reflectance decreases first and increases due to the specular reflection effect [25]. In addition, the RMSH increments enhance light scattering and transmission on the soil surface, thus decreasing the reflectivity, especially in the visible and near-infrared wavelength range [26]. Meanwhile, long-term high-intensity mechanized planting increases the soil bulk weight (SBW) of cultivated land. The concomitant changes in SM, SBW, and spectral reflectance exhibit a complex relationship. Generally, the increase in SM, SBW, and RMSH decreases the spectral reflectance, showing a coupling effect [27,28]. Noteworthy, the effect of SOM content on the soil spectrum is far weaker than that of soil physical properties [29]. Due to the satellite revisit period and soil physical condition uncertainties, the noise interference of soil physical properties on the spectrum greatly limits the accuracy and spatiotemporal transferability of the remote sensing-based SOM evaluation model, which needs to be solved urgently.

The data reliability and completeness when mapping information about the prediction target are often the keys to the generalization ability of the model [30,31]. To alleviate the influence of soil physical properties on hyperspectral data and improve the spatiotemporal transferability of the SOM content prediction model, scholars have fused hyperspectral data of long time series to reduce the sensitivity of the model to spectral differences caused by soil physical property changes [17,32,33,34]. Ge et al. attempted to introduce soil physical parameters as input variables into the element content prediction model [35]. Pan et al. established SOM content prediction models based on different SM ranges [36]. Most of these innovative improvement attempts at the macro scale have been successful, while the in-depth development and analysis of satellite hyperspectral images at the pixel scale are still lacking. Considering the reflectance difference caused by soil physical properties, Minasny et al. developed a spectral correction model based on the EOP method to eliminate the influence of SM [37]. Castaldi et al. synthesized the dry soil spectrum by calculating the statistical variability of dry and wet soil, thereby improving the model prediction accuracy [21]. Although these methods have corrected the hyperspectral data to a certain extent, they are mostly based on one physical parameter and ignore the coupling response of different soil physical properties to the spectrum. Due to the scarcity of soil physical data from satellite ground synchronization experiments, the potential of hyperspectral correction methods comprehensively considering multiple soil physical properties has not been fully explored.

Developing a soil spectral correction method alleviating the coupling effect of surface physical properties on soil pixel spectra is a long-term solution to improve the spatiotemporal transferability of the SOM prediction model. The complex interactions between the various surface physical properties and electromagnetic waves make it difficult to simulate the relationship between soil physical properties and the spectrum with physical models [28]. Therefore, this study aims to separate the physical and chemical soil information in the spectral data with data-driven methods. Studies have indicated various functional relationships between soil physical parameters (SM, RMSH, and SBW) and spectral reflectance, including logarithmic, exponential, and power exponential functions [26,38]. However, most previous studies were based on multi-spectral data, while the effect of soil physical properties on reflectance remained to be clarified based on hyperspectral data with more continuous and dense bands [39,40]. Although these functional relationships may change slightly due to differences in soil types, components, etc., they generally reflect the spectral characteristics induced by soil physical properties and are an essential basis for representing soil physical property information in the spectral data [41,42,43]. Based on this, spectral forward modeling can be carried out using soil physical properties, providing prior data for the soil spectral correction [44]. Moreover, separating the spectral information derived from soil physical and chemical properties requires spectral data only responding to soil chemical properties in the corresponding pure pixels. The dried and ground soil samples have the same uniform SM, SBW, and RMSH, and the soil spectra at this time is considered the “pure spectra “ that only reflects the soil chemical composition information [6]. To effectively decompose the soil physical and chemical information in the pixel spectrum, nonlinear parameter regression and machine learning models have been used to simulate the coupling relationship between soil physical property spectra, “pure spectra”, and pixel spectra, respectively. These two data-driven methods search for the rules between data through statistical analysis and machine learning training, respectively. In this way, the unknown data can be predicted, and the generalization ability of the soil spectral correction method can be guaranteed by the regression equation. For the SOM prediction model, the soil spectral correction redistributes the observation information of the original hyperspectral remote sensing to ensure the uniformity of the soil physical properties of all pixels, which helps improve the spatiotemporal transferability of the model [45,46,47].

This work seeks to establish a hyperspectral SOM prediction model with high spatiotemporal transferability, which can guide soil investigation and parameter prediction. The objectives of this study are: i) evaluating the impact of soil physical properties on satellite hyperspectral data and their contribution to the bias in SOM content prediction; ii) developing soil spectral correction methods alleviating the coupling impact of soil physical properties on satellite spectrum; and iii) determining the spatiotemporal transferability potential of satellite hyperspectral data for SOM retrieval based on a soil spectral correction strategy. Data-poor regions might benefit from the proposed SOM prediction model with strong spatiotemporal transferability when mapping SOM to develop appropriate policies.

2. Materials and Methods

Before establishing the SOM content prediction model with high accuracy and strong spatiotemporal transferability, a soil spectral correction method was developed to alleviate the coupling effect of surface physical properties on the soil pixel spectrum (Figure 1). Firstly, parameter estimation equations were used to establish empirical relationships between satellite hyperspectral data and the three main soil physical parameters SM, RMSH, and SBW. Three sets of simulated soil spectral data based on SM, RMSH, and SBW were obtained by correlating soil physical parameters with satellite hyperspectral images using empirical relationships. Then, a soil pixel spectral correction model was constructed based on the simulated spectrum, soil pixel spectrum, and ground spectrum using multi-order polynomials and various machine learning models to separate soil physical and chemical information in the pixel spectral data. Finally, the SOM prediction model was constructed using XG-Boost based on the original and corrected soil spectral data. Site 2 soil samples were used to evaluate the spatiotemporal transferability of the spectral correction models and SOM prediction models established with soil samples from Site 1.

2.1. Study Area

Site 1 is in the protected black soil cultivated land of Heilongjiang Province, Northeast China (131°30’-132°03’ E, 46°36’-46°49’ N), as shown in Figure 2, which has an area of 1095 km². The area has a temperate monsoon climate, with an annual precipitation of approximately 614 mm. According to the World Reference Base for Soil Resources (WRB), the cultivated land is mainly Chernozems with a sedimentary layer under the topsoil, which has a clayey and heavy texture and poor permeability, often forming surface saucer water during precipitation periods [17]. The heavy sediment layer formed by the downward leaching of dark organic matter in the clay particles intensifies the water retention on the surface. The cultivated land surface is covered by a layer of black humus of over 10 cm. The soil has extremely high fertility and is rich in organic matter, which is suitable for crop growth [48].

Site 2 is in Changchun City, Jilin Province, Northeast China (125°24’-125°43’ E, 44°36’-44°46’ N), as shown in Figure 2, which has an area of 713 km². Its terrain is flat, with an elevation between 189 and 237 m. Due to the influence of geographical location and atmospheric circulation, the region has a temperate continental monsoon climate, with a frost-free period of about 135 days and an average annual precipitation of about 580 mm. The region has rich river systems, relatively abundant agricultural water resources, and strong SM spatial heterogeneity. The soil in the region is mainly Phaeozems with a fertile cultivated layer, and maize and rice are the main crops [49,50]. Site 2 has significantly different soil type, surface characteristics, and other environmental factors than Site 1, which can verify the spatiotemporal transferability of the SOM content prediction model in this study.

2.2. Datasets

2.2.1. Soil Sampling and Topsoil Parameter Measurement

A total of 104 soil samples were collected from Site 1 on October 29, 2022 (Figure 2b). On April 14, 2023, 40 soil samples were collected from Site 2 (Figure 2c). Among them, 80 soil samples from Site 1 were used as the training set for the soil spectral correction model and SOM prediction model, and the remaining 24 samples were used as the validation set. Meanwhile, the 40 soil samples from Site 2 were used to test the spatiotemporal transferability of the spectral correction model and SOM prediction model. All soil samples were collected from the cultivated land portion of the study area during the “bare soil period.” First, one 3D laser scanner (Trimble TX8, maximum standard range: 120 m; Scanning speed: 1 million points per second) was installed at the midpoint of each edge of the quadrat to scan the soil surface structure (Figure 3). The sampling was conducted after the scanning to ensure the natural state of the soil surface structure within the sampled area. Next, nine subsamples were collected with a ring knife (depth 5 cm and volume 200 mL) in each 30 × 30 m quadrat. The real-time kinematic (RTK) survey technique was used to record the longitude and latitude of the quadrat midpoint.

After being transported to the laboratory, the SM and SBW of the nine subsamples in each quadrat were obtained through weighing and drying, and the average of the subsamples was calculated to represent the overall level of the quadrat. Then, the nine subsamples were mixed into one composite sample, ground, and sieved to a size of ≤ 0.2 mm for subsequent spectral measurements and SOM content testing [51]. The SOM content was determined using the potassium dichromate heating method. Soil spectral reflectance was measured with an ASD FieldSpec 4 spectrometer in the darkroom. To ensure the same SBW of each sample, soil samples were loaded in a disposable culture dish (60 mm diameter) for spectral measurements. Each soil sample was measured 10 times, and the average value was taken as the soil ground spectral data. The soil surface point cloud data from 3D laser scanning were spliced, cut, and filtered, and a 3D relative coordinate system was established (Figure 3b). After processing, the point cloud density was greater than 3 points/cm³, and the relative coordinate system accuracy of the point cloud was less than 2 mm. The Z coordinate of the point cloud data within the sample quadrat was extracted, and the standard deviation was calculated as the RMSH of the quadrat.

2.2.2. Hyperspectral Image Data Acquisition and Data Preprocessing

The Ziyuan1-02D (ZY1-02D) hyperspectral image data were acquired from the Aerospace Information Research Institute, Chinese Academy of Sciences. According to the soil sampling time of the two regions, the images generated on October 29, 2022 (Site 1) and April 14, 2023 (Site 2) were selected as data sources. All images have less than 1% cloud coverage and meet the characteristics of the “bare soil period.” The spatial resolution of the hyperspectral images is 30 m, with a total of 166 spectral channels and a spectral range of 400–2500 nm (Table 1). The sensor suffers strong noises in the wavelengths of 400 to 450 nm and 2460 to 2500 nm and is affected by atmospheric water vapor absorption in the wavelengths of 1290 to 1408 nm and 1828 to 1963 nm [52]. Therefore, the 450 to 1290 nm, 1408 to 1828 nm, and 1963 to 2460 nm bands were selected as the spectral bands in this study. The images were subjected to stripe removal, geometric correction, and atmospherical correction using Radiometric Calibration and FLAASH in the Environment for Visualizing Images 5.6 to obtain the original reflectance data. The bidirectional reflectance distribution function (BRDF) effect of the images is corrected by calculating the zenith angle and azimuth angle of the sun (and satellite). The kernel-driven BRDF model is used to normalize ZY1-02D reflectance to reduce the effect of observation geometry on reflectance [53].

2.3. Spectral Correction Strategy

The image pixel spectrum comprehensively reflects soil physical properties (e.g., SM, RMSH, and SBW) and chemical composition within the ground quadrat. Spectral correction aims to separate the reflection features attributed to the physical and chemical properties of the soil in the pixel spectral data, thus alleviating the coupling effect of soil physical properties on the spectrum. Firstly, linear, exponential, power exponential, and logarithmic parameter estimation equations were used to establish empirical relationships between satellite hyperspectral data and the three soil physical parameters SM, RMSH, and SBW on a band-by-band basis. These parameter estimation methods for fitting the relationship between soil physical properties and spectral reflectance have been verified in several studies [38,40].

By using the empirical relationships to associate soil physical parameters with satellite hyperspectral data, three sets of simulated soil spectral data based on SM, RMSH, and SBW were obtained. The soil ground spectrum measured with dried and ground soil samples is regarded as a “pure spectrum” that only reflects the soil chemical composition information [54]. Based on this, a spectral correction model was constructed, which took the pixel spectrum and three sets of soil physical parametric simulated spectrum as input and ground spectrum as training targets. Through multi-order polynomials and various machine learning algorithms, the correction relations between the pixel spectrum and the ground spectrum were established to strip the reflection information attributed to soil physical properties in the pixel spectrum. The multi-order polynomial equation is as follows:

(1)

where R_G is the ground-based spectral reflectance of a certain band, R_SM is the spectral reflectance simulated based on SM, R_RMSH is the spectral reflectance simulated based on RMSH, R_SBW is the spectral reflectance simulated based on SBW, R_P is the spectral reflectance of the pixel, i is the polynomial order, a_i, b_i, c_i, d_i, and e are regression coefficients, respectively.

2.4. Machine Learning Models

2.4.1. Competitive Adaptive Reweighted Sampling (CARS)

CARS is adopted to extract sensitive bands corresponding to SOM in the hyperspectral data. CARS imitates the “survival of the fittest” principle of Darwin’s evolutionary theory. Through adaptive weighted sampling, it screens out the wavelengths with large absolute coefficients of the PLS model and removes the wavelengths with small weights, thus obtaining many subsets of wavelength variables. Next, the subset of wavelengths with the lowest root-mean-square error is selected via cross-validation as the optimal subset [46,55]. CARS can effectively retain the best wavelength combination related to the measured characteristics.

2.4.2. eXtreme Gradient Boosting (XG-Boost)

XG-Boost is an ensemble learning model based on the Boosting strategy, which combines several CART trees into a strong learner. As an ensemble algorithm framework, it supports the parallel gradient lifting of the base learner, thus greatly improving the model training speed. The Newton method is used to solve the extreme value of the loss function, which is expanded to the second order using the Taylor formula. The loss function is optimized with the first-order gradient function and second-order gradient function to reduce model complexity [56]. Simultaneously, the probability of over-fitting is reduced through regularization, significantly improving the model’s generalization ability.

2.4.3. Model Validation

In this study, the coefficient of determination (R²), root mean square error (RMSE), and residual predictive deviation (RPD) were selected as evaluation indices, as expressed below:

$$R^2=1-\sum _{i=1}^n{(y_i-Y_i)}^2/\sum _{i=1`}^n{(y_i-\overline{y})}^2$$

(2)

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^n{(y_i-Y_i)}^2}$$

(3)

RPD = SD/RMSE

(4)

where n is the number of samples; y_i and Y_i represent the measured and predicted values, respectively; $\overline{y}$ denotes the measurements on average.

3. Results

3.1. Description of Soil Physical Parameters and SOM Content

The statistical results of soil physical parameters and SOM content are listed in Table 2. At Site 1, the mean SM, RMSH, and SBW values were 0.25 cm³/cm3, 2.49 cm, and 0.98 g/cm³, respectively, with coefficients of variation (CV) of 31.99%, 30.92%, and 15.31%. The moderately high CV and SD indicate the combined influence of structural and anthropogenic factors on soil surface physical properties, showing strong spatial heterogeneity. The SOM content varied significantly from 25.84 to 75.97 g/kg, with a standard deviation (SD) of 10.51 g/kg and a CV of 24.30%. Site 2 had significantly different soil properties from Site 1. The average SM, RMSH, and SBW were 0.37 cm³/cm³, 3.65 cm, and 1.13 g/cm³, respectively, which were significantly higher than Site 1 and had stronger variability. The SOM content at Site 2 ranged from 27.40 to 72.97 g/kg, averaging 41.57 g/kg, which was lower than Site 1.

3.2. Effect of Soil Physical Properties on Soil Spectra

To verify the reliability of the ZY1-02D hyperspectral image, the soil pixel spectrum was compared with the soil ground spectrum (Figure 4). Although the soil pixel spectrum has a similar shape to that of the soil ground spectrum, it has some noise and a relatively low smoothness, especially in the VNIR wavelength range. In addition, the spectral reflectance in the soil pixels was slightly lower than that measured in the laboratory. The Spearman correlation coefficients (SCCs) and Pearson correlation coefficients (PCCs) between soil pixel reflectance and soil ground reflectance in each band were calculated. The results showed that the PCCs between the two sets of spectral data were below 0.5 in most wavelengths, while the SCCs in the visible light and short-wave infrared wavelength range were basically greater than 0.5, indicating a possible nonlinear relationship between the pixel spectral reflectance and the ground spectral reflectance in the same wavelength range. To further reveal the factors affecting the pixel spectrum, the differences in soil reflectance between different physical property gradients were compared. With the increase of SM, the soil spectral reflectance decreased significantly, especially in the 500 to 1300 nm and 1450 to 1700 nm wavelength ranges (Figure 5). The soil spectral reflectance decreased relatively slightly with the increase in SBW. The effect of RMSH on the soil spectrum was the most significant, and the reflectance decreased significantly with the increase of RMSH. In summary, the coupling effect of multiple soil physical properties on the spectrum is an important reason for the deviation of the two sets of spectral data, which seriously limits the acquisition of soil “pure spectrum” by the imaging spectrometer. Therefore, it is necessary to separate soil physical and chemical information in the pixel spectral data and improve the SOM prediction accuracy of hyperspectral remote sensing.

3.3. Empirical Relationship between Satellite Hyperspectral Image and Soil Physical Properties

The empirical coefficients were regressed based on the field data and soil pixel spectrum to determine the relationship of soil reflectance with SM, RMSH, and SBW (Figure 6). Among the fitting equations between SM and soil reflectance, the exponential equation has the best fitting effect. Except for the 2000 to 2500 nm wavelength range, the fitting results were good, with R² of 0.49 to 0.68. In the 2000 to 2500 nm wavelength range, the fitting between SM and reflectance was not good, possibly due to the absorption of clay minerals to the spectral characteristics caused by the hydroxyl groups in soil. With higher clay mineral contents, the water retention capacity of the soil was greater. According to the fitting results between SBW and soil reflectance, the exponential equation fitted the best in the 450 to 1800 nm wavelength range, with the R² of 0.50 to 0.69, while the power exponential equation fitted the best in the 2000 to 2500 nm wavelength range. In terms of the entire wavelength range, RMSH had the strongest fitting relationship with soil reflectance among the three groups of soil physical parameters, implying its most significant effect on the soil spectrum. Among the four equations, the logarithmic equation had the best fitting effect, with the R² of 0.55 to 0.69. In general, the best empirical relationship of soil reflectance with SM is exponential, that with RMSH is logarithmic, and that with SBW is exponential in the 450 to 1800 nm wavelength range and power exponent in the 2000 to 2500 nm wavelength range. Three sets of soil reflectance data were simulated based on the empirical relationship between soil physical parameters and soil spectra, respectively (Figure 7). The soil reflectance simulated based on SM showed an almost uniform trend between 2000 and 2500 nm, implying that the effect of SM on reflectance in this wavelength range was suppressed by other factors, resulting in insignificant spectral features. Other than that, the remaining simulated soil spectra showed significant differences. These soil spectra simulated through empirical equations based on soil physical properties were used to construct the soil spectral correction model.

3.4. Modeling of Soil Spectral Correction

Empirical coefficient models and machine learning models were employed to establish the correction relationship between the soil pixel spectrum and soil “pure spectrum”. The original pixel spectrum and three sets of soil spectrum simulated based on SM, RMSH, and SBW were used as input spectral data, and the ground-based soil spectrum was used as the training target to build the soil spectral correction model band by band. The multi-order polynomial-based soil spectral correction model showed improved accuracy with the increasing order, and its RPD and RMSE were optimal in all bands at the fourth order (Figure 8). An excessively high order renders the empirical equations too complex, leading to over-fitting, reduced adaptability to new data, and decreased accuracy. The RPD of the fourth-order polynomial model was above 1.5 in all bands, indicating its good correction effect on the soil spectrum.

Four machine learning algorithms, namely, support vector machine regression (SVR), extreme learning machine (ELM), back propagation neural network (BPNN), and XG-Boost, were used to construct soil spectral correction models in the same way. The best soil spectral correction model was determined by comparing the mapping ability of the different machine learning algorithms to the coupling relationship between multiple soil spectra (Figure 9). The correction results showed that the four machine learning algorithms differed significantly in the soil spectral correction accuracy. The accuracy fluctuations around 1000 nm wavelength may be caused by other noise in the spectral data. Other than that, all soil spectral correction results were good, and the overall accuracy was high relative to the polynomial-based models. As a representative algorithm of the ensemble learning model, XG-Boost achieved the best spectral correction results, with R² above 0.6 and RPD above 1.6 for all bands. The second best was ELM, while SVM and BPNN performed poorly.

Based on the soil spectral correction accuracy, the soil spectral correction results of the fourth-order polynomial model and XG-Boost model were selected for further analysis. The results showed that the soil spectra corrected with the XG-Boost model were smoother than those corrected with the fourth-order polynomial model and fitted the spectral shape of the soil “pure spectrum” more closely (Figure 10). The calculated correlation coefficients between the corrected soil pixel spectrum and the soil “pure spectrum” showed that the PCCs in most wavelengths were above 0.8, which was greatly improved compared with the correlation between the original pixel spectrum and the ground spectrum. Therefore, after spectral correction, the spectral response induced by soil physical properties in the soil pixel spectrum was alleviated, and the proportion of information on soil chemical composition response signals in the pixel spectral data was significantly increased. In terms of the spectral shape of the soil pixel spectral correction results and their correlation with the soil “pure spectrum,” the correction results of the XG-Boost model are slightly better than those of the fourth-order polynomial model. However, the accuracy improvement effect of these methods on hyperspectral SOM prediction needs further analysis through modeling.

3.5. SOM Content Prediction Accuracy Based on Different Spectral Data

Four types of soil spectral data, namely, pixel spectrum, fourth-order polynomial corrected spectrum, XG-Boost corrected spectrum, and ground-based spectrum, were used to establish the SOM content prediction models, respectively. In order to reduce the data dimensionality and improve the computational efficiency of the model, the spectral data were first subjected to feature extraction. Pearson’s correlation coefficient threshold was used to determine the sensitive bands of SOM. The correlation coefficient distribution between the four sets of soil spectral data and SOM content showed a relatively consistent trend. Specifically, the correlation coefficient decreased with the increasing wavelength before the 800 nm wavelength and increased after the 800 nm wavelength (Figure 11a). The bands with absolute correlation coefficients above 0.5 were selected as the sensitive bands of SOM. The sensitive spectral bands corresponding to SOM in the four spectral data sets of pixel spectrum, fourth-order polynomial corrected spectrum, XG-Boost corrected spectrum, and ground-based spectrum were concentrated in the wavelength range of 628 to 1023 nm, 524 to 1223 nm, 542 to 1560 nm, and 550 to 1762 nm, respectively. CARS was adopted to further extract the optimal subset of features containing the least redundant information in the sensitive bands. The optimal number of CARS iterations was determined by the RMSECV of multiple regression (Figure 11b). The bands listed in Table 3 are the spectral bands selected by CARS for SOM inversion modeling and validation analysis.

The four sets of spectral bands selected through CARS and the SOM contents were used as the input data of the model. The XG-Boost algorithm is used to construct the SOM prediction model. The results indicated that these two spectral correction methods significantly improved the SOM prediction accuracy based on the original pixel spectrum. Among all SOM prediction results, the ground spectral data had the highest prediction accuracy, with the $R_p^2$, RMSEP, and RPD of 0.79, 4.89 g/kg, and 1.97, respectively (Figure 12). The prediction accuracy evaluated based on $R_p^2$ was 0.64 when the original pixel spectral dataset was used as model input data. Adopting the soil spectral correction strategy based on fourth-order polynomials increased the prediction accuracy ($R_p^2$) by 0.05, decreased RMSEP by 2.28 g/kg, and increased RPD by 0.38. The soil spectral correction strategy based on the XG-Boost model had a greater SOM prediction accuracy improvement, with an $R_p^2$ increase of 0.12, an RMSEP decrease of 3.10 g/kg, and an RPD increase of 0.59. The SOM prediction accuracy with the corrected spectrum came close to that with the ground spectrum, implying that alleviating the coupling effect of soil physical properties on the soil pixel spectrum can effectively improve the hyperspectral SOM prediction accuracy.

4. Discussion

4.1. The Transferability of the Soil Spectral Correction Model and SOM Prediction Model

The soil pixel spectral correction methods based on empirical coefficient and machine learning models provided new strategies for improving the SOM prediction accuracy based on hyperspectral images. The two soil spectral correction methods based on different models have their advantages and limitations. The soil spectral correction method based on XG-Boost significantly affected the SOM prediction accuracy improvement, but its correction process and principles were difficult to express mathematically. Despite its weak SOM prediction accuracy improvement, the soil spectral correction method based on the fourth-order polynomial model expressed the improved method with the coefficient equation, which was more conducive to its promotion. The high transferability of the soil spectral correction method is a key prerequisite for constructing a SOM prediction model with strong generalization ability [57]. To verify their spatiotemporal transferability, 40 groups of Site 2 soil pixel spectra and ground experimental data were imported into the two spectral correction models. The spectral correction results showed that the corrected soil spectra were very consistent with the shape of the soil “pure spectra” (Figure 13). Compared with the soil spectrum after fourth-order polynomial correction, the soil spectrum corrected with the XG-Boost model is smoother. According to the accuracy of the model migration test results, the soil spectral correction model based on XG-Boost showed better migration performance, with RPD above 1.4 for all bands. The machine learning algorithms represented by XG-Boost were much better than the coefficient models in terms of the calculation ability to establish the coupling relationship between multiple soil spectra and its adaptability to new data. The reason is that this ensemble learning model comprehensively utilizes all the eigenvalues of each soil sample point and continuously adjusts the weight of the tree through iteration to explore the optimal solution of the coupling relationship between soil physical properties and the pixel spectrum [45,58].

The poor spatiotemporal transferability of traditional SOM prediction models is mainly attributed to their poor applicability to different spatiotemporal spectral data [59]. Evaluating the improvement effects of soil spectral correction methods on the spatiotemporal transferability of SOM prediction models is the direct basis to prove the effectiveness of spectral correction methods [60]. The Site 2 soil samples and spectral data were used to evaluate the transferability of the SOM prediction model established with the Site 1 data. The SOM prediction model based on ground spectra exhibited the best transferability, with RMSEP only increasing by 0.39 g/kg (Figure 14). However, the transferability of SOM prediction models based on the original pixel spectrum is extremely poor as surface physical property changes cause spectral reflectance deviations. After transferability verification, RMSEP increased by 8.05 g/kg, while RDP decreased by 44.04%. Adopting the two soil spectral correction strategies significantly improved the transferability of the prediction model based on the original pixel spectrum, with an RPD of over 1.4 for model transferability validation. The SOM prediction model based on the XG-Boost correction spectrum had greater transferability. Compared with the model transferability validation based on the pixel spectrum, RMSEP was reduced by 60.27%, and RPD was increased by 150.82%. These findings proved the effectiveness of the soil spectral correction methods and the feasibility of the corrected satellite hyperspectral data to predict SOM content. The SOM prediction model based on corrected satellite hyperspectral data can be used even at two sites with different soil types, soil physical properties, SOM contents, and spatiotemporal features. The core of this soil spectral correction method is to comprehensively consider the coupling effect of various soil properties on spectral reflectance to restore the true spectral characteristics of the research target. For different research objectives, the main factors affecting the spectral response of the target can be analyzed according to the actual environment and imaging conditions [39]. Therefore, the proposed method is not limited to SOM prediction and can provide valuable insights for soil property prediction based on satellite hyperspectral data.

4.2. Contribution of Soil Physical Properties to SOM Content Prediction Bias

The soil spectral correction method proved to greatly improve the SOM content prediction accuracy and spatiotemporal transferability of the pixel spectrum. In other words, soil properties (SM, RMSH, and SBW) may be the main factors leading to errors in SOM estimation based on original pixel spectra. This section investigated the error dependence of the original pixel spectrum and two sets of corrected spectral data on SM, RMSH, and SBW, and their contribution to SOM content prediction bias was estimated through the stepwise regression method. The results showed that the cumulative deviation contribution rate of these three soil properties to the SOM prediction results based on the original pixel spectrum was over 70% at both sites (Figure 15). Thus, soil physical properties are the main error source of SOM prediction [53]. The contribution of SM to SOM bias was the highest, followed by RMSH and SBW. This is related to the most significant response of the pixel spectrum to SM within the sensitive wavelength of SOM and possibly the stronger spatial heterogeneity of SM compared to RMSH and SBW. The stronger spatial heterogeneity of soil physical properties leads to greater differences in the impact on pixel spectra and, thus, greater deviation in SOM prediction [10]. Adopting the spectral correction strategy significantly reduced the bias of SM, RMSH, and SBW in SOM prediction. Soil spectral correction based on XG-Boost more significantly reduced the SOM prediction bias caused by soil physical properties than the soil spectral correction based on fourth-order polynomials. This result fundamentally explains the higher prediction accuracy and stronger spatiotemporal transferability of the SOM prediction model based on the XG-Boost corrected spectrum. Despite the increased relative contribution of random errors, the total bias in terms of the accuracy of the predicted results was significantly reduced. Therefore, the spectral correction strategy did not introduce more error sources but only improved the relative contribution of other error factors to SOM prediction bias, such as hyperspectral image processing uncertainty and field data acquisition uncertainty [61,62]. Judging from the relative contribution rate of soil physical properties to SOM prediction deviation before and after spectral correction, the contribution rate of SM to SOM prediction bias decreased the most by over 10%, followed by RMSH. The spectral correction method based on polynomials and XG-Boost reduced the average relative contribution rate of RMSH at the two sites by 10% and 14.5%, respectively. Although the declined contribution rate of SBW to SOM prediction bias was the smallest, the reduction exceeded 6%. By comparing the improvement effects of different input variables on SOM prediction bias, the improvement effects of soil physical properties on SOM content prediction accuracy ranked as SM > RMSH > SBW, consistent with the order of spatial heterogeneity of these three soil physical properties and the order of sensitivity of soil spectra to them in the VNIR range. Thus, soil physical properties with strong spatial heterogeneity and sensitive spectral response should be prioritized in soil spectral correction.

4.3. The Potential and Limitations of the Soil Spectral Correction Model

As soil spectral correction methods are designed to address the coupled effects of surface physical properties on hyperspectral images, they are suitable for remote sensing image processing of various soil chemical composition predictions. Such methods suppress the sensitivity of spectral data to SM, RMSH, and SBW and reduce the possibility of SOM prediction results falling into the local optimum. Another advantage is that determining the empirical relationship of SM, RMSH, and SBW with the hyperspectral soil reflectance of ZY1-02D satellite spectra improves the generalization ability and application efficiency of the method. In addition, this study also has two potential applications. 1) It enables the application of optical and radar remote sensing combined in soil physicochemical property estimation, and 2) it provides a solution to the spatiotemporal heterogeneity of spectral data due to uncertain changes in surface physical conditions in multi-source remote sensing data fusion. Although the soil spectral correction model can restore most soil “pure spectrum” characteristics, some uncertainties may remain. The applicability of the proposed method to airborne hyperspectral sensors or other hyperspectral satellites requires evaluation with more data. In addition, this experiment only considered the influence of soil properties within 5 cm of the surface layer on the spectrum, while the spectrum may have different sensing depths for different soil properties [63]. Even though the spectrum only directly detects SM changes in shallow soil (about 0 to 2 cm), this depth also changes under different SBW and RMSH conditions [20,64]. The vertical heterogeneity of SM and SBW may be the main factor causing soil spectral correction errors. Adopting a hierarchical strategy to establish spectral correction models for soil properties at different depths or assigning different weights to soil physical properties at different depths may maximize the effectiveness of the model within the specified depth and range.

4.4. Future Work and Suggested Next Steps

This study, for the first time, used the “pure spectrum” derived from soil spectral correction considering SM, RMSH, and SBW for SOM content prediction and confirmed its excellent spatiotemporal transferability. The strategy of alleviating the influence of soil physical properties on spectral coupling may provide a paradigm for remote sensing-based soil element content prediction in the future. Soil spectral correction of the whole hyperspectral image requires real-time high-resolution soil physical parameter data at the regional scale. The strong sensitivity of synthetic aperture radar remote sensing to soil physical parameters makes it possible to obtain rich information on soil physical parameters in practical applications [65]. Future research may combine the advantages of hyperspectral imaging and radar remote sensing to improve the prediction accuracy of soil physicochemical parameters [66]. Given the type and dimension heterogeneity of optical and radar data, a new link to combine the two data was provided. Meanwhile, some interesting new directions have also emerged. Due to the difference in imaging time between radar and hyperspectral images, the corresponding surface physical properties also change, especially SM, which is highly susceptible to weather. Therefore, combining optical and radar data to eliminate the bias due to the temporal phase may be the optimal strategy to solve this problem. In addition, the perception depth of optical and radar remote sensing for soil properties needs further clarification. As radar sensors have better penetrability than optical sensors, quantifying the vertical heterogeneity of SM and SBW with radar remote sensing could be the key to further improving the soil spectral correction accuracy [67,68]. These strategies reduce errors in large-scale soil physical and chemical parameter surveys to support regional strategic arrangements for sustainable agricultural development.

5. Conclusions

This study utilized satellite and ground hyperspectral data and soil physical parameters data to construct two soil spectral correction models based on fourth-order polynomial and XG-Boost, respectively, to alleviate the coupling effect of soil physical properties on the pixel spectrum. The performance of the soil spectral correction models and their influence on the accuracy and spatiotemporal transferability of the SOM prediction model were evaluated using data from two sites. The main conclusions are as follows. (1) The soil pixel spectral reflectance is nonlinearly related to soil ground spectral reflectance. The difference in surface physical properties is the main factor for the deviation of the two spectral data. RMSH has the most significant effect on the soil pixel spectrum, followed by SM and SBW. (2) The fourth-order polynomial and XG-Boost models have good soil spectral correction accuracy. The soil spectral correction model based on XG-Boost has higher accuracy and stronger spatiotemporal transferability as it considers all the features to continuously adjust the weight of the tree and prevent the result from falling into the local optimum. (3) Soil spectral correction significantly alleviates the coupling effect of soil physical properties on soil pixel spectra, effectively improves the accuracy of the SOM prediction model, and, more importantly, greatly enhances the spatiotemporal transferability of the SOM prediction model based on pixel spectrum. Considering the response of satellite hyperspectral imaging to soil physical properties helps to understand their roles in SOM content prediction. This work provides a new research paradigm for predicting soil property parameters in other regions.