6. Benchmarking with Artificial Neural Networks
After developing and aggregating the partial models into a comprehensive analytical model (AM) for predicting the tensile modulus of polypropylene compounds, the next step is to benchmark this model against an Artificial Neural Network (ANN) approach. This comparison will evaluate the efficacy of the analytical model relative to the ANN in terms of prediction accuracy and robustness, particularly in scenarios where data is limited or noisy.
For the ANN, the MLPRegressor from the scikit-learn library (version 1.5.1) was employed, using Python version 3.10 [
23]. The typical approach in machine learning involves splitting the dataset into a training set, used to train the model, and a test set, used to evaluate its performance. For this study, an 80/20 split was chosen, with 80% of the data allocated for training and 20% reserved for testing.
To ensure a robust comparison, the dataset was split randomly into training and test sets ten times, mitigating any bias that might arise from a single dataset split. This randomisation helps to prevent scenarios where, for example, all data points for a specific additive end up in the training set, skewing the results. The aggregated model (AM) was evaluated on the same splits as the ANN for consistency.
The analysis was performed on two versions of the dataset: the full dataset, which includes all data points, and a reduced dataset, from which outliers with a measurement deviation exceeding 50 N/mm² were removed. This allowed for an examination of how well each model handles data variability and outliers.
The results of the Mean Absolute Error (MAE) and the Coefficient of Determination (R²) for both models are summarised in
Figure 13 and
Figure 14.
Figure 13 illustrates the MAE values for both the AM and ANN models across all dataset splits, for both training and testing. As expected, the MAE for the test datasets is generally higher than that for the training datasets, reflecting the challenge of generalising the model to unseen data. However, the AM consistently outperforms the ANN, achieving lower MAE values across both the full and reduced datasets. For the full dataset, the AM demonstrated an average MAE of 25.70 for the training data and 38.41 for the test data. In comparison, the ANN produced an average MAE of 33.93 for training and 45.21 for testing. These results indicate that the AM is more accurate overall, with performance close to the inherent variability in the data, as the measurement error for the full dataset was 24.07. When examining the reduced dataset, which excludes outliers with significant measurement deviations, the AM's performance improved further.
The model achieved a minimum MAE of 24.00 for the training data and 28.29 for the test data. In contrast, the ANN showed only a modest improvement under the same conditions, with average MAE values of 32.86 for training and 42.47 for testing. This suggests that the AM is more robust in handling variability and outliers in the data.
Figure 14 presents the R² values for both models, which indicate the proportion of variance in the dependent variable that is predictable from the independent variables. For the full dataset, the AM achieved an average R² of 0.97 for the training data and 0.93 for the test data, indicating a strong fit to the data. The ANN's performance was slightly lower, with R² values of 0.96 for training and 0.91 for testing, further demonstrating that the AM is more effective in capturing the underlying relationships in the data.
The reduced dataset results highlight the AM’s superior performance, where it exhibited an R² of 0.98 for the training data and 0.96 for the test data, suggesting an excellent fit even when data variability is reduced. In contrast, the ANN's R² values were lower, at 0.97 for training and 0.93 for testing, underscoring the AM’s robustness and effectiveness in predictive modelling.
The benchmarking results reveal that the aggregated analytical model (AM) not only matches but often exceeds the performance of the Artificial Neural Network (ANN), especially in terms of Mean Absolute Error (MAE). This outcome is somewhat unexpected, given that ANNs are typically well-suited to handle complex, non-linear relationships within large datasets. However, the AM’s superior performance, particularly with reduced datasets, underscores its robustness and efficiency in predicting the tensile modulus of PP compounds.
One of the critical advantages of the AM lies in its simplicity. Unlike the ANN, which relies on the optimisation of numerous neuron weights, the AM is governed by only 14 fitting parameters. This simplicity enables the AM to achieve high accuracy with fewer data points, which is a significant advantage in real-world applications where extensive datasets are often unavailable. Moreover, the AM’s better generalisation to unseen data suggests that it may be less prone to overfitting compared to the ANN. This characteristic is crucial for practical applications, where the ability to predict the performance of new formulations based on limited data is highly valuable.
The results further indicate that the AM is particularly effective at handling variability and outliers within the data. When outliers were removed, the model’s performance improved noticeably, achieving lower MAE values and higher R² scores. This robustness in the face of data variability makes the AM an attractive option for industrial applications, where the quality of data can often be inconsistent.