Analyzing and Predicting the Agronomic Effectiveness of Fertilizers Derived from Food Waste Using Data-Driven Models

1. Introduction

A circular economy approach promotes nutrient recovery from food waste (FW), reducing reliance on mined phosphorus (P) and synthetic nitrogen (N) fertilizers [1]. Despite the environmental impact of mineral fertilizers, many nutrient-rich waste sources remain underutilized. Integrating bio-based fertilizers into agriculture could enhance soil quality and support sustainable food production [2,3,4,5].

The disposal of FW without recycling leads to the loss of valuable nutrients that could otherwise support agricultural productivity. Globally, FW represents approximately one-third of total food production, amounting to 1.3 billion tons annually [6]. FW’s disposal via landfills and incineration causes severe environmental harm, while recycling focuses on converting FW into valuable chemicals and bioenergy [7,8]. FW can be converted into biofertilizers through composting and anaerobic digestion (AD). Composting produces nutrient-rich organic matter, while AD generates biogas and nutrient-dense digestate, offering a sustainable alternative to synthetic fertilizers [9]. Research has explored FW recycling through AD, thermal hydrolysis, and dehydration to create biofertilizers as substitutes for inorganic fertilizers [4,5,10,11,12]. Biofertilizers often result from fermenting or composting kitchen waste alongside other organic materials [13].

Fertilizers significantly influence plant yield (PY), making accurate predictions essential for agricultural planning and food security. Machine learning (ML) has emerged as a powerful tool for prediction in environmental sciences [14,15,16,17] and, specifically, for optimizing crop yields based on fertilizer use [18,19,20]. For example, X. Li et al. [18] analyzed 35 years of fertilizer application and grain yield data in China, finding that backpropagation neural networks (BPNN) outperformed other ML models with an RMSE below 0.12 and R² > 0.80. Wang et al. [19] optimized planting density and fertilizer rates using BPNN and regression methods. Thai et al. [21] applied analysis of variance, linear mixed-effects models, and regression trees to evaluate winter wheat yields under different fertilizer practices. Similarly, L. Meng et al. [20] demonstrated that Random Forest (RF) and adaptive boosting significantly improved yield predictions (R² = 0.85−0.98).

In addition to PY, the fertilizer value of FW after various treatments was assessed using internal efficiency of nitrogen utilization (IENU). IENU evaluates how efficiently crops use nitrogen for growth, measuring sustainability and optimization in crop production [22,23]. PY represents the agricultural output harvested per unit area, making it a widely used metric for assessing fertilizer impact on crop performance. Both metrics are crucial for evaluating the effectiveness of FW-based fertilizers and FW mitigation programs. However, while PY is frequently applied in such assessments, IENU has received limited attention in practical applications. No studies have employed ML models to predict FW-driven crop yields using PY and IENU metrics.

The study aims to achieve three key objectives: first, to develop a comprehensive dataset of fertilizers derived from FW; second, to identify and analyze crop yield data influenced by FW-based fertilizers; and third, to estimate the agronomic effectiveness of these fertilizers using statistical and ML models, particularly through PY and IENU. By addressing these objectives, the research aims to enhance the understanding of how FW-driven fertilizers influence crop productivity and nutrient utilization efficiency, ultimately providing valuable insights to improve waste recycling efforts and optimize agricultural practices

2. Materials and Methods

2.1. Experiments About Kitchen Waste Recycling as Fertilizer for Plant Growth

2.1.1. In-House Experimental Procedures

The data for our experiments includes 102 cases from our research, published in previous studies [13], and an additional 159 unpublished cases. The experimental methods for these data were based on the following methodology.

The preparation of the final fertilizer began with creating a model waste mixture composed of 100 g each of banana, tomato, lettuce, fruit juice, bun, and apple, along with 200 g each of flowers and paper. The ingredients were pre-cut with a knife and ground twice using a meat grinder to form a homogeneous paste. The model waste paste exhibited a total solids (TS) content of 54% and a volatile solids (VS) content of 90% (based on TS).

Various pre-treatment methods were applied before fertilizer production, involving combinations of microbial inoculation, natural decay, sterilization, and fermentation:

• In the first scenario, effective microorganisms (EM) were added directly to the model waste.
• In the second scenario, the waste was left to decay for 12 days before a double dose of EM was introduced.
• In the third and fourth scenarios, the waste was sterilized after 12 days; however, only the third scenario included EM addition.
• In the final scenario, the waste was allowed to decay and was sterilized but not inoculated with EM.

After pre-treatment, the waste was pre-dried, pelletized, and dried to produce the final fertilizer.

2.1.2. External Investigations

To supplement our experimental data, we extracted and standardized information from 20 additional published studies (Table 1). Together, these datasets formed the comprehensive database for modeling purposes.

Table 1 summarizes key variables across the studies, including treatment types, seasons, growth durations, and plant species. Treatments involved various composting and digestion processes applied to crops such as ryegrass, lettuce, oat, winter wheat, corn, Kai choy, and barley. Growth periods ranged from 0.5 to 180 months, covering both warm and cold seasons. Some studies also evaluated combinations of FW, garden waste, and municipal solid waste for their effects on plant growth and productivity.

2.2. Dataset Preparation and Processing

The initial dataset consisted of 448 instances with seven features: nitrogen content (NC), volatile solids (VS), season, growth time (T), plant type, fertilizer dose (D), plant yield (PY), and input efficiency of nitrogen use (IENU). Due to a high proportion of missing values (45.5%), the VS variable was excluded from further analysis.

After removing missing target values, 420 instances remained for PY prediction and 342 for IENU prediction. While PY prediction utilized data from all 20 studies, IENU prediction excluded three studies [28,29,36].

The study focused on two output parameters:

• PY: Represented in kilograms of total solids per hectare (kg/ha), PY reflects the agricultural output per unit area.
• IENU: Measured in kilograms of nitrogen per hectare (kg N/ha), IENU evaluates the effectiveness of nitrogen use in supporting crop growth.

Three explanatory features were analyzed:

• NC: The nitrogen percentage in fertilizer affects crop growth and nitrogen efficiency.
• T: The crop growth duration (in days) influences nutrient uptake and development.
• D: The fertilizer amount applied per hectare impacts yield outcomes.

The data contain several missing and outlier values handled using different methods. Four versions of the dataset were created. The first version, Data 1, was created by discarding any records with missing values. Data 2 was then derived from Data 1 by eliminating outliers using the boxplot method. The third version, Data 3, was generated by imputing missing values to maintain the dataset’s integrity. Finally, Data 4 was produced by removing outliers from Data 3, similar to the procedure used to create Data 2, ensuring the dataset’s quality for further examination. These processed datasets were each randomly divided, allocating 80% of the cases to a training set and the remaining 20% to a testing set. The normalized, log, standardized transformation methods were deployed to test the change in error of prediction models.

2.3. Machine Learning Modeling Approach

This study utilized four ML models to predict PY and IENU: Gradient Boosting (GB), Cubist (CB), RF, and Extreme Gradient Boosting (XGB). These models were chosen to leverage their collective strengths in handling the diverse and complex data necessary for accurately predicting PY. They have been widely utilized in the prediction of agricultural production, demonstrating their effectiveness and reliability.

• Gradient Boosting:

GB is a method ML that addresses regression and classification challenges by creating an ensemble of weak models, typically decision trees. The approach incrementally builds the model by optimizing a differentiable loss function at each step [42]. GB has been reported as an effective model for predicting corn yields [43].

• Cubist:

CB is a regression-based predictive model incorporating nearest-neighbor corrections to enhance its predictions [44,45]. It constructs decision trees where the terminal leaves define rule-based “if-then” conditions. Each rule in the Cubist algorithm corresponds to a multiple linear regression model, improving predictive accuracy. CB has two key parameters that can be either default or fine-tuned: (1) committees, which determine the number of boosting iterations, and (2) neighbors, which specify the number of instances used to adjust rule-based predictions [46,47].

• Extreme Gradient Boosting:

XGB is an optimized and scalable implementation of Gradient Boosting. It is specifically designed to improve performance and computational efficiency. It implements ML algorithms under the Gradient Boosting framework and is renowned for its performance and speed [48]. XGB includes L1 and L2 regularization to prevent overfitting. In agriculture, XGB has been successfully applied to yield prediction, demonstrating high accuracy and the ability to model complex relationships in crop data [49]. Studies have shown that XGB performs comparably to, or even outperforms, deep learning models when predicting yields using remote sensing data [50]. Additionally, XGB has explicitly been utilized to predict PY in agricultural research [51].

• Random Forest:

RF is an ensemble learning technique that generates multiple decision trees during training. For classification tasks, it predicts the mode of the classes, and for regression tasks, it calculates the mean prediction from all the individual trees. By averaging the outcomes of several trees, it mitigates overfitting and enhances generalization (Breiman, 2001). RF has been widely used for crop prediction due to its ability to handle large datasets with many features and its resistance to overfitting [52].

2.4. Evaluation metrics

In addition to ML models, linear regression (LR) models were employed for comparison. A variety of metrics were employed to assess the performance of these models. R² (Coefficient of Determination) and RMSE (Root Mean Squared Error) were used for ML models. R² quantifies the proportion of the variance in the response variable that the independent features can explain. RMSE evaluates the square root of the average of the squared differences between observed and predicted values, reflecting the model’s prediction accuracy. R², RSE (Residual Standard Error), RMSE, and AIC (Akaike Information Criterion) were used for LR models. RSE assesses the standard deviation of the residuals, indicating the typical size of the errors. AIC measures the relative quality of statistical models, balancing model complexity and goodness of fit. Combining these metrics mitigates individual disadvantages and sensitivities, enhancing evaluation efficiency [15]. These metrics are presented in Equations 1 – 4 [53,54,55].

$${\mathrm{R}}^2=1-{\displaystyle \frac{\sum _{\mathrm{i}=1}^{\mathrm{n}}{{(\mathrm{y}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{i}})}^2}{\sum _{\mathrm{i}=1}^{\mathrm{n}}{{(\mathrm{y}}_{\mathrm{i}}-\overline{{\mathrm{y}}_{\mathrm{i}}})}^2}}$$

(1)

$$RMSE=\sqrt{{\displaystyle \frac{\sum _{i=1}^N{(y_i-{\widehat{y}}_i)}^2}{N}}}$$

(2)

$$RSE=\sqrt{{\displaystyle \frac{\sum _{i=1}^N{(y_i-{\widehat{y}}_i)}^2}{N-p-1}}}$$

(3)

$$AIC=2k-2\mathrm{l}\mathrm{n}\left(\widehat{L}\right)$$

(4)

where:

y_i is the observed value;

${\widehat{y}}_{\mathrm{i}}$ is the predicted value;

N is the number of observations;

p is the number of predictors (independent variables);

k is the number of parameters in the model;

$\widehat{L}$ is the maximum value of the likelihood function for the model.

3. Results and Discussion

3.1. Exploratory Data Analysis

Table 2 encompasses six variables, each representing different aspects of agricultural research. The targets, PY and IENU, exhibit high variability with coefficients of variation (CV) of 136.61% and 286.76%, respectively. PY, with a mean of 2268.42 kg TS/ha and a range up to 18008.11 kg TS/ha , and IENU, averaging 32.26 kg N/ha with a range extending to 1285.71 kg N/ha , underscore the broad spectrum of outcomes in agricultural productivity and efficiency. Predictor variables NC, T, and D also show significant variability.

Table 3 outlines the frequency of various crops studied for growth using kitchen waste as fertilizer, reflecting multiple agricultural interests. Ryegrass dominates the dataset with a substantial frequency of 73.77%. “Grass” (unspecified species) accounted for 10.76%, while oats and lettuce followed with frequencies of 4.87% and 2.02%, respectively. Other crops, including barley, corn, and specific varieties of potato and rice, were less frequently represented (<2%), indicating a narrower research focus on these species.

The correlation analysis was conducted using all available data. Given the non-normal distribution of the data and the presence of a significant number of ties, Kendall’s Tau was employed. Kendall’s Tau is less influenced by outliers, making it suitable for this dataset. For datasets with numerous ties, Kendall’s Tau-b provides a refined correlation measure by adequately accounting for these ties in its calculation. Examining the correlations between targets and predictors, D demonstrated positive, significant correlations with PY and IENU, with r-values of 0.42 and 0.44, respectively (P < 0.01) (Figure 1). In contrast, VS and T had negative correlations with PY and IENU, with r-values around -0.24 (P < 0.01). NC showed no significant correlation with the targets. The absence of strong correlations among predictors indicates their independence, deeming them appropriate for PY and IENU prediction.

3.2. Prediction of Plant Yield

The initial dataset comprises 420 rows and includes four variables: NC, T, D, and PY. Following this omission, the dataset underwent several transformations, yielding four distinct versions tailored for detailed comparative studies. These versions are denoted as Data 1 (382 cases), Data 2 (324 cases), Data 3 (324 cases), and Data 4 (299 cases). These processed datasets were each randomly divided, allocating 80% of the cases to a training set and the remaining 20% to a testing set. To ensure consistent distributions of the target variable, mitigate data leakage, identify biases, and promote model generalization, the practice of checking the distributions of training and test data in ML prediction was implemented. The histograms illustrate the “PY” frequency distributions across four datasets, revealing right-skewed patterns in all training sets. Notably, Data 2 is the most promising for predictive modeling due to its training and test set distributions aligning more closely (Fig. S1).

3.2.1. Linear Regression

LR models applied to the four datasets produced R² values ranging from 0.18 to 0.41 (Table 4), indicating modest explanatory power. Data 2 and 4, in particular, exhibited lower AIC values, suggesting a more suitable balance between fit and complexity than those for Data 1 and 3. However, introducing RMSE provides additional insight into the model’s predictive performance, with Data 2 and 4 showing lower errors, thus reinforcing their efficacy.

Despite the moderate explanatory power demonstrated by all models, the consistency in results between the transformed and untransformed predictors for Data 2 underscores the model’s robustness (lowest RSE, AIC, and RMSE). This aspect is particularly noteworthy as it highlights the model’s capability to effectively capture the underlying relationship between the predictors and PY, regardless of the scale or distribution of the predictor variables. However, the model’s explanatory power is limited, as the three variables collectively explain only 24% of the variation in the data. Analysis of the coefficients reveals that T has the most substantial impact on PY (62.42), followed by NC (-8.05), and then D (2.05).

The diminished predictive capacity of LR underscores the need for advanced modeling techniques to more accurately estimate PY in the context of highly variable crop yield data influenced by numerous factors [56,57]. LR typically yields modest results in such scenarios [58]. Moreover, the inherent variability in crop yield, characterized by its specificity to each period and its intrinsic nonlinearity, further complicates prediction [59].

3.2.2. Machine Learning

Initially, ML algorithms, including CB, XGB, RF, and GB, were employed to identify the most suitable dataset for PY prediction. Subsequent analysis revealed significant disparities among the four datasets (Table 5). Data 2 consistently outperformed others, with the RF model yielding the lowest RMSE (493.22) and the highest R² (0.77). Therefore, Data 2 emerges as the optimal choice for subsequent prediction tasks.

To enhance the rigor of our analysis of Data 2, four ML models were trained using 80% of the data and the remaining 20% for testing. However, due to the random nature of the split, each 80/20 split varied, leading to inconsistent model performance outcomes. To mitigate this variability, 30 separate runs of the train/test splits were conducted, and the results were reported as the mean of these runs. This methodology aligns with the approach used by [60], who applied a similar strategy in their study using RF to predict soybean yield; however, they also included multiple repetitions to optimize hyperparameters. Each run involved training and tuning the ML models using 10-fold cross-validation and normalization transformations (Table S1). The RF and Cubist models exhibited superior generalization capabilities, with low RMSE and high R² values (0.79–0.83 in training and testing), demonstrating robustness in handling unseen data.

In contrast, XGB and GB models, despite their competitive performance during training, showed signs of slight overfitting, as evidenced by an increase in RMSE from training to testing phases. This indicates that while these models effectively capture complex patterns in training data, they may require careful tuning to enhance generalizability. Notably, the RF model consistently performed well, effectively capturing nuances in the training data while maintaining robust accuracy during testing, achieving a balanced trade-off between fitting and generalization.

The CB and RF models effectively predicted PY, with optimal parameters identified as committees 50 and neighbors 01 for CB, and ntree 100 and mtry 02 for RF. Previous research has employed various techniques and predictors in ML models for estimating PY, including cross-sectional and time-series predictions and predictors based on remote sensing and ground truth data (experimental plot-based designs) [60,61]. These diverse approaches have led to inconsistent findings and varying performance levels in ML models for PY prediction. For example, RF and Support Vector Machine models showed similar efficacy, capturing between 50% and 70% of the spatial and temporal variance in yields for crops such as silage maize, winter barley, winter rapeseed, and winter wheat [62]. Furthermore, deep learning and RF models demonstrated strong performance at the field level for large-scale crop yield estimation, achieving mean R² values of 0.71 and 0.66 and RMSE values of 1127 kg/ha and 956 kg/ha, respectively [63]. The RF model, in particular, presented promising results in wheat yield prediction, with an RMSE of 434, normalized RMSE of 0.149 grams per plot, and an R² of 0.74 [64]

3.3. Prediction of IENU

The initial dataset contains 341 entries with four variables: NC, T, D, and PY. It was transformed into four distinct versions to facilitate comparative analysis: Data 1 (302 entries), Data 2 (256 entries), Data 3 (341 entries), and Data 4 (286 entries). Histograms demonstrated the distribution quality within these datasets (Fig. S2). Notably, Data 2 and Data 4 were identified as the most suitable for predictive modeling due to the close alignment between their training and testing distributions. The removal of outliers rendered the data more effective for predicting IENU.

3.3.1. Linear Regression

Data 2 exhibited the lowest testing RMSE (8.29), suggesting accurate predictions, and a comparatively low training RSE (6.864), indicating a robust model fit. Data 4, with a testing RMSE of 7.82, outperformed Data 2 in testing accuracy but has a slightly higher training RSE of 8.073, though still below Data 3’s. Conversely, Data 1 showed the poorest performance, with the highest testing RMSE (66.86) and training RSE (90.78), indicating the least accurate predictions and model fit. Data 3, despite the highest R² value during testing, also had high testing RMSE (45.96) and training RSE (98.55), reflecting lower accuracy and fit.

Overall, Data 2 was the top performer, effectively balancing prediction accuracy and model fit, while Data 4 served as a viable alternative, although its low R² value indicates sensitivity in this metric. Despite the moderate explanatory power demonstrated by all models, the model’s explanatory power is limited, as the three variables collectively explain only 27 % of the variation in the data. In contrast, a multiple LR study by [65] accounted for 74% of the variation in nitrogen use efficiency. Analysis of the coefficients reveals that T had the most substantial impact on IENU (2.96), followed by NC (0.12), and then D (0.03).

3.3.2. Machine Learning

The analysis demonstrates that Data 2 consistently outperformed other datasets across various regression models, yielding the lowest RMSE values and the highest R² scores (Table 4). For instance, Data 2 exhibited an average RMSE of approximately 3.91 and an average R² of around 0.81, indicating superior predictive accuracy compared to other datasets. Conversely, Data 3 and 4 presented higher RMSE values ( 55.06 and 4.47, respectively) and lower R² scores (0.74 and 0.72, respectively), suggesting greater complexity or noise in these datasets. Therefore, based on these findings, Data 2 was the most favorable for regression modeling tasks.

The ML models were assessed based on their performance metrics, including the mean RMSE and R², along with their respective standard deviations, for both the training and testing datasets (Figure 2). CB demonstrated the lowest mean RMSE on the training data at 3.62 ± 0.22, indicating its ability to make accurate predictions while maintaining consistency across iterations. Regarding R² on the training data, both XGB and CB achieved high mean values of 0.82, suggesting their effectiveness in explaining the variance in the training data. On the testing dataset, XGB maintained its strong performance with a mean RMSE of 3.62 and a mean R² of 0.81, indicating its capability to generalize well to unseen data. CB also performed well on the testing dataset with a mean RMSE of 3.83 and a mean R² of 0.78. These results suggest that both XGB and CB exhibited promising performance in capturing the underlying patterns in the training data and making accurate predictions on new, unseen data. RF and GB models also demonstrated competitive performance, with mean RMSE values of 3.76 and 5.10, respectively, and mean R² values of 0.80 and 0.62, respectively, on the training dataset. However, their performance slightly decreased on the testing dataset, indicating potential overfitting or a lack of generalization ability compared to XGB and CB. While no studies have targeted predicting IENU precisely, several researchers have focused on nitrogen use efficiency [66,67,68]. However, the metrics used in these studies are not directly comparable to those needed in this context. The results presented here build on the study by [13], which applied only the Monod model to predict plant dry matter yield (kg d.m./ha) at each harvest based on plant nitrogen (N) utilization data (kg N/ha). The model, fitted to all available data, showed a strong overall fit, suggesting that nitrogen effectively drives yield dynamics. However, since Monod was applied to the entire dataset without independent validation, there is a risk of overfitting. As a result, its predictive accuracy for new, unseen inputs remains uncertain and may lead to incorrect estimations under different conditions.

4. Limitations and Practical Applications

4.1. Limitations

This research, while innovative, confronts several limitations that future studies need to address. Firstly, the data variability and completeness issues stem from the datasets primarily from controlled experiments, which may not capture the full variability and complexities of real-world agricultural settings. Additionally, the high rate of missing data and the exclusion of certain variables could impact the robustness and generalizability of the findings. Secondly, the ML models, despite showing promising results, carry a potential risk of overfitting, particularly noted in models like XGB and Gradient Boosting when applied to unseen data. Enhancing the models’ generalization capabilities to various agricultural environments remains critical for future research. Thirdly, the experiments were conducted on a relatively small scale, and scaling these findings to more extensive commercial agricultural operations could introduce new challenges, such as logistical, economic, and climatic variations that were not accounted for in this study. Lastly, the effectiveness of FW-derived fertilizers is heavily dependent on the specific pre-treatment processes used, and different treatments may yield varying effectiveness across different soil types and crop species, which this study did not fully explore.

4.2. Practical Applications

Despite the limitations above, this study offers valuable insights with significant practical applications. It supports the shift towards more sustainable agricultural practices by demonstrating the effectiveness of FW-derived fertilizers, encouraging recycled organic waste, reducing reliance on chemical fertilizers, and minimizing environmental impact. The findings can help inform policymakers in developing guidelines and regulations that promote organic waste recycling into valuable agricultural inputs, encouraging circular economy practices in the farming sector [69].

Furthermore, the application of ML models in predicting the outcomes of different fertilizer treatments can aid farmers and agricultural practitioners in optimizing the use of resources. This optimization leads to better crop yields and more efficient use of nitrogen, enhancing food security and sustainability. ML has recently been gradually used to develop decision support tools for modern agricultural systems, covering nutrient management to improve yields while reducing expenses and environmental impact [61]. It can provide cost-effective and thorough nutrient assessment and decision-making solutions [70,71].

Additionally, integrating ML into agricultural practices offers an excellent opportunity for educational programs to teach new technologies in agronomy, fostering a new generation of tech-savvy farmers and researchers. Lastly, there is potential for commercializing the FW treatment processes and the related ML prediction models developed in this study. These can be offered to agricultural businesses as part of a service or technology package, enhancing their productivity and environmental sustainability.

5. Conclusions

This study provides key insights into the potential of FW-derived fertilizers for agronomic effectiveness and predicting PY and IENU via data-driven models. By rigorously filtering out missing values and outliers, Data 2 delivered reliable and consistent results. Correlation analysis revealed that decomposition duration positively impacted PY and IENU, while temperature and volatile solids had negative correlations. ML models outperformed LR, with RF and CB achieving high PY prediction accuracy and XGB excelling in IENU prediction. By integrating predictive simulations into fertilization planning, this study contributes to sustainable agriculture by enhancing resource efficiency and reducing reliance on synthetic fertilizers. Future research should explore larger datasets and additional environmental factors to refine ML-based fertilizer optimization strategies.