Machine Learning in Sensory Analysis of Mead. Case Study: Ensembles of Classifiers

1. Introduction

Honey has been a valued food product for centuries, renowned for its exceptional taste, aroma and health benefits. As the only available sweetener, honey has been an essential part of the diet since the beginning of Homo sapiens[1,2]. Its diversity is due to different nectar sources, natural environments, production techniques and fermentation processes. Depending on the type of plant from which bees collect nectar, honey can take on other colours, flavours and aromas, making it a unique product in each region. The processes of fermentation and storage of honey affect its final properties, including texture, shelf life and nutritional content. The main alcoholic beverage prepared from honey is mead, known for its unique and distinctive aroma and taste. The literature points to a number of research efforts to better understand and classify mead. The research by the team of Cicha-Wojciechowicz et al. (2024) focused on the effects of fermentation techniques and honey type on mead sensory profile [3], while the team of Pereira et al. (2019) studied the composition of volatile aromatic compounds [4]. Furthermore, Starowicz and Granvogl (2022) analyzed the effect of heat treatment on wort aroma changes[5], which is an important step in understanding the technological processes that influence the quality of this product. These research efforts help to know how different factors in different types of honey can affect the sensory and chemical properties of mead. Nowadays, approaches using modern analytical technologies and machine learning methods are becoming crucial in the systematic analysis of honey.

The application of machine learning methods to the analysis of mead offers new possibilities for classifying honey types [6,7,8], identifying aromatic compounds, and conducting future experiments in the optimization of production processes. Machine learning algorithms such as classifiers based on supervised learning such as Random Forest (RF) [9,10], Adaptive Boosting (AdaBoost), Bootstrap Aggregating (Bagging), K-Nearest Neighbours (KNN) and Decision Tree (DT) [11,12,13] among others, can be applied to the analysis of chemical and sensory data sets [14,15]. DTs are some of the most widely used algorithms in data analysis, which allow the creation of simple yet explainable decision models. With DT, it is possible to both classify and predict the properties of food products based on the various descriptors identified in a given research question. In a transparent and easy to interpret way[16], DTs can be useful in the data analysis process, where the result must be understandable and practical for DTs to serve as a tool for implementation in the production process. In combination with other methods, such as RF or AdaBoost, decision trees form so-called ensembles of classifiers. This approach helps to improve the efficiency of classification and can be a crucial tool for improving mead production processes, assessing mead quality and distinguishing between different types of mead based on sensory and chemical data characteristics.

In order to improve the performance of the models, an optimization technique was applied by tuning the hyperparameters using the GridSearch method, which makes it possible to effectively identify relevant features of the data, even in the case of high complexity. This type of optimization not only improves the generalizability of the model, but also significantly increases its efficiency. As pointed out by Liu et al. (2021) [17], machine learning algorithms can exhibit poor performance due to inappropriate selection of hyperparameters. Optimization of hyperparameters becomes crucial for achieving better model performance [18].

The aim of this research was to investigate the possibility of using machine learning to classify mead types based on their sensory analysis, in particularly employing cluster map and k-means methods to identify relationships between mead groups based on their sensory characteristics. This approach will not only allow an accurate classification of mead, but also improve understanding of the factors that influence its quality.

This research contributes to the development of interdisciplinary research on food products, combining advanced methods of chemical analysis, sensory evaluation and artificial intelligence.

2. Results and Discussion

2.1. Hierarchical Analysis of Mead Aroma Compounds

A cluster map of odor related characteristics and odor compound concentration characteristics was generated (Figure 1). It was found that one variable had the highest value relative to the others. This was the odor sensory attribute, General Odor Intensity, which was the focus of the next stage of the experiments. Based on this analysis, combinations were made to create learning sets and then a model mead classification process. It was also observed that for the ‘buckwheat’ type of mead, in addition to General Odor Intensity, sensory attributes such as Malty, Rum, Honey, Fermented and Alcoholic achieved high scores. For comparison, the sensory attributes listed for the other types of tilia and acacia mead were also higher for buckwheat honey than for the other attributes. This means that these sensory aroma trait variables were more dominant than the aroma compound concentration traits. This made it possible to accurately understand and prepare classification models for the selected aroma-sensory attributes. It was also found that the dominant decision class for mead would be the variable responsible for the type of buckwheat.

2.2. Interpretation of K-Means of Honey Aroma Compounds

In the next stage of research, K-means analysis (Figure 2) was carried out between the descriptors (odor sensory attributes) identified from the clustermap patterns. In Table 1, it can be observed that cluster 2 had the highest value of variance for the senor trait General odor intensity. This was confirmed from the clustermap analysis that General odor intensity strongly stands out in between cases in this cluster. It is possible that this descriptor plays a key role in identifying the type of mead. In this cluster, the General Odor Intensity interaction may facilitate the clustering of cases for other sensory attributes, which is reflected in the high values for other sensory attributes. For cluster 1, learning cases the sensory traits of mead aroma showed low intensity among the characteristics. This means that for this cluster 1, the learning cases do not distinguish among the characteristics. The highest variance value in this cluster 1 is also determined by the General Odor Intensity, whose mean value was 3.21. Cluster 0 gave an inter-mediate value between cluster 0 and cluster 1, suggesting that the General odor intensity descriptor was moderately expressed in the cases from this cluster. The learning cases in this cluster 0 may have been particularly characteristic of the moderate value of the aforementioned descriptor. In conclusion General Odor Intensity indeed represented an important sensory property for mead classification.

2.3. Machine Learning

In the analysis carried out on the basis of 21 different combinations of sensory features on the input variables classifying the mead, different machine learning algorithms were evaluated for their performance based on four key measures: accuracy (Acc), precision (Precision), sensitivity (Recall) and the F1-score (Table 2). The highest scores were obtained by the Random Forest (6 times) and KNN (6 times) algorithms. It can be concluded that these models were the most effective in classifying mead. It can be assumed that Random Forest, due to its use of multiple decision trees and random feature selection technique, had a high resistance to over-fitting. This translates into its stability and good performance in different configurations with mead based features [19]. On the other hand, KNN, based on nearest neighbour voting, also achieved high performance, especially in the evaluation of the performance metrics of this model [7,20,21]. Research shows that Random Forest, especially when combined with suitable feature extraction methods, achieves excellent results (e.g., Acc above 93%) [8,11,19,22,23]. Employing voting schemes and different distance metrics can further enhance classification performance. However, it should be noted that KNN is more sensitive to an increase in the number of features and may be less computationally efficient with large datasets.

In summary, both Random Forest and KNN can be very effective in sensory clas-sification tasks, but their advantage is due to different mechanisms of operation: RF from the strength of the tree ensemble and randomness, and KNN from local neighbour voting in feature space [22].

The rest of the machine learning algorithms such as Bagging (6 times), Naive Bayes (3 times) and Decision Tree (3 times) also performed well, but were not able to outperform Random Forest and KNN in terms of stability of results in different combinations. Bagging, a technique that involves training multiple models on different data samples, showed very robust results, but slightly less stable than the other algorithms. Naive Bayes and Decision Tree, although effective, achieved higher results, but only for selected correlations.

However, the highest metric for assessing the effectiveness of the model was ob-tained with Decision Tree for the sensory features General odor intensity, Yeasty, Floral, which reached an accuracy of 0.909, a precision of 0.929, a recall of 0.909 and an F1-score of 0.911. It was observed that when optimizing the algorithms with the Grid Search method for the most effective decision tree model, the value for the tree depth hyperparameter reached a score of 6. Depending on the combination of sensory features, the Grid Search method produced different results. This is due to the fact that for the Decision Tree model with data on these sensory features, the optimal tree depth was 6, whereas in other configurations, both for the Decision Tree and Random Forest models, values of less or more were obtained. The aim of this was to avoid overfitting the model, as an appropriate tree depth affects the ability of the model to generalize, and too much depth can lead to the model fitting the noise in the data rather than capturing the true relationships.

In contrast, the worst results were obtained with the AdaBoost algorithm. In fact, the AdaBoost model achieved the lowest accuracy value of 0.515 for aroma features such as General Odor Intensity, Rum and Yeasty, indicating its weaker performance compared to other algorithms. AdaBoost, which is a boosting algorithm, aims to improve the accuracy of the classifier by iteratively increasing the weights of incorrect examples and creating new weak classifiers.

In Figure 3, it is observed that the Random Forest and K-Nearest Neighbours (KNN) algorithms generally achieve the highest accuracy results in classifying mead types. In Figure 4, the Random Forest and KNN algorithms also demonstrate high performance in terms of recall rate. Overall, these algorithms also provide high precision (Figure 5) scores and F1-Score (Figure 6). Although it is worth noting that for specific sensory characteristics of the mead, the Decision Tree algorithm obtained the highest single value for accuracy, recall, precision, and F1-Score. The results clearly indicate that choosing an appropriate classification model significantly impacts the performance of the mead identification process in practical applications. Additionally, selecting suitable sensory descriptors is crucial for achieving optimal mead identification performance.

When tuning the models using the K-NN algorithm, it was observed that the ‘Manhattan’ hyperparameter was the most frequently selected. The reason for this is that it gives a better assessment of model performance and stability [24]. In the case of the Random Forest algorithm, as in the case of the Decision Tree algorithm, the choice of hyperparameter was influenced by the avoidance of over-fitting and, at the same time, the goal of obtaining high model performance. In the case of the Naive Bayes model, the Grid Search method determined the exact optimal value of var_smoothing equal to 1e-09 to improve the stability of the model, as well as its generalizability, avoiding overfitting problems or computational errors associated with zero variance. In summary, the selection of different hyperparameter values using different machine learning algorithms made it possible to test on the basis of which sensory characteristics the model effectively classifies the type of mead. In light of the above and observations in the literature, it is worth investigating the selection of hyperparameters suitable for a specific task [25]. In the age of modern machine learning and deep learning tools, it allows that optimization through Grid Search has become rapid. Observations and literature show that when choosing an optimization technique, it is also worth paying attention to the size of the dataset for a given problem [26].

2.4. Analysis of Classifier Performance Based on Confusion Matrix Results

In machine learning, confusion matrices are an important tool for evaluating the performance of classifiers. Confusion matrices allow a detailed analysis of the predictive performance for a test set to be performed. It has also been observed in the literature that it is the most commonly used visualization to present among others this in-formation [27,28,29,30,31]. It was found that it is usually presented to represent a single model. In our study, the recognition behavior of the selected mead type was also compared between models, which is crucial for selecting the appropriate model in a future implementation in an application [11,27].

In this experiment, the number of correct and incorrect classifications of mead in-to a particular class (buckwheat, acacia, tilia) was accurately determined. This analysis led to an understanding of the extent to which the model correctly classified learning instances related to aroma characteristics and where it went wrong. The confusion matrix also made it possible to identify specific difficulties that may have occurred with certain types of mead. Such a solution will be helpful in optimizing the model, which would lead to an assessment of the overall performance of the classifier, while attempting to improve these algorithms. A total of 126 confusion matrix plots were generated (Figure: S1-S126), corresponding to 21 combinations of sensory odor features generated from the similarity map analysis and the K-means method. Each com-bination contained 6 models and for each model a confusion matrix was plotted for the test set. In each matrix the class number corresponded to the type of mead, i.e., '1' was acacia, '2' was buckwheat and '3' was tilia. In order to explicitly identify individual types of mead, the classification analysis of the attributes 'General_odor_intensity', 'Honey' and 'Malty' from Table 1, Random Forest (Figure S1) and Bagging (Figure S4) successfully classified 11 cases for acacia and at the same time made an error in 1 case by assigning tilia to mead on the test set. In the class corresponding to buck-wheat-based mead, 8 cases were classified in the class corresponding to this buck-wheat, while 2 cases were incorrectly classified in the class corresponding to tilia. In the case of the last type of honey, 9 cases were clearly classified in favor of tilia honey, while only 2 cases were incorrectly classified in favor of tilia honey. In the second case (Table 3), for the characteristics 'general_odour_intensity', 'honey', 'fermented', the Random Forest model was the most successful, classifying acacia honey in 11 cases, buckwheat-based mead in 9 cases and tilia honey in 7 cases (Figure S7). In the third case (Table 3), considering the features 'general_odor_intensity', 'honey', 'rum', the Bagging model was the one that correctly classified acacia honey in 11 cases, buck-wheat-based mead in 9 cases and tilia honey in 8 cases (Figure S16). The analysis of the confusion matrix between the models and the selection of sensory attributes showed that the algorithms were more successful in classifying acacia honey. In the case of buckwheat and tilia honey, this depended on the choice of model and the sensory characteristics of the mead. Considering the most effective decision tree model for the sensory attributes 'General_Odor_Intensity', 'Yeasty', 'Floral' (Figure S109-S114), the confusion matrix on the test set correctly classified acacia honey in 10 cases, buckwheat-based mead in 9 cases and lime tree honey in 11 cases (Figure S110). In fact, it can be seen that the difficulty in classifying mead is due to the type of mead, i.e., tilia honey, where in case 19 (Table 3) tilia honey was classified more correctly than acacia honey. In the group of characteristics related to General_Odor_Intensity, Rum, Yeasty, where the AdaBoost model performed the worst (Figure S93), the KNN algorithm proved to be the most successful model correctly classifying tilia mead in 11 cases and buckwheat and tilia mead in 9 cases (Figure S95).

3. Materials and Methods

3.1. Data Source

For the current research, data from a previously published study were used, including a quantitative olfactory profile analysis and the quantitation of important odor-active compounds for twelve types of mead. These meads were produced using three different honey varieties (acacia, buckwheat, and tilia) and three fermentation methods: spontaneous fermentation, inoculation with Saccharomyces cerevisiae yeast, and inoculation with Galactomyces Geotrichum molds. Additionally, both heated and unheated wort variations were analyzed. The methods used for sample analysis were thoroughly described before [3].

3.2. Data Collection

The research examined a series of sensory attributes: 'general odor intensity', 'honey', 'malty', 'fermented', 'rum', 'yeasty', 'floral', 'alcoholic'. A total of these attributes were created. This process facilitated the creation of multiple input variables sets used for training the cluster models. The combinations aimed to deepen the under-standing of relationships between the different sensory characteristics and potentially enhance the performance of the algorithms. By varying these patterns, it was possible to better capture important features that influence the accuracy of mead classification. As a result, 21 learning sets were designed for which the input variables were 3 sensory characteristics. The output variable in each set was the type of honey (acacia, talia, backwheat). The dataset consisted of 108 learning cases. The train_test_split function was used to split the set into a training set and a test set. This means that 30% of the learning cases belonged to the test set and 70% of the learning cases belonged to the training set. The selection of learning cases from the training set was done by the aforementioned function, which automatically decides which cases should go into the learning set and which into the test set. The only criterion set by the user is the proportion that determines how much data is included in the training set and how much in the test set. A random seed (random_state) was also specified to ensure reproducibility of the distribution, allowing results to be replicated in future experiments.

3.3. Parameter Selection for Classification Models

In the design of algorithms using ensembles of classifiers, an appropriate architecture was developed for each model (Table 3), which was then used to the classify the different types of mead. In addition, during the learning process, the selection of the hyperparameters of each model was automated in order to obtain only the best model in terms of the evaluation metrics. In this way, it was possible to optimize the performance of each classifier to obtain the best possible results in the classification of mead types. The automated hyperparameter selection process also ensured greater consistency and repeatability of results, eliminating the risk of errors due to manual calibration.

In view of the fact that machine learning algorithms often contain several hyperparameters, this study used the optimization of selected hyperparameters to assess the performance of the models [32]. Within the Random Forest model, a set of hyperparameters was selected that included different values for the maximum tree depth (max_depth) and the number of estimators (n_estimators). This allowed different configurations to be tested and the optimum balance between accuracy and model complexity to be found.

In the research question for decision trees, only the maximum tree depth (max_depth) was considered as a hyperparameter, with specific values tested (Table 1). These values were intended to strike a balance between the complexity of the tree and its ability to extract meaningful patterns from the data.

Two important variables were introduced for the AdaBoost model: the number of estimators (n_estimators) and the learning rate (learning_rate). These values were tested over a wide range to optimise the learning rate and the number of weak classifiers in the ensemble.

The K-Nearest Neighbours algorithm used a more complex parametric grid, including the number of neighbours (n_neighbours), the distance measure (metric) and the way the neighbours were weighted (weights) (Table 3). Each of these parameters affected classification accuracy and computation time, so different values were tested to select the best settings.

Bagging (param_grid_bag) considered the parameter n_estimators, which deter-mines the number of estimators in the ensemble. Three different values were tested (Table 1) in order to find the optimal number of submodels that best influenced the classification performance.

In the case of the Naive Bayes model, the var_smoothing hyperparameter was used, which was responsible for adding a small value to the variance of the features to avoid the problem of dividing by zero. The tested values (Table 3) allowed to check the effect of this parameter on the stability and accuracy of the model.

3.4. Model Training and Testing

In the next step, the learning process of the designed ensembles of classifiers [33,34,35]was carried out in order to evaluate their performance in recognizing mead types. During validation on the test set, the quality of learning was assessed using evaluation metrics such as accuracy, precision, recall and F1-score. A confusion matrix was also used to analyze the predicted results. The process of model estimation and learning was carried out using Python version 3.11.11 [36,37,38,39].

3.5. Statistical Analysis

In the process of clustering the data using the sklearn.cluster library, the clustering for the cases defining the different sensory characteristics was plotted on a graph. In the next step, the PCA (Principal Component Analysis) algorithm was used to re-duce the data to two principal components, allowing them to be visualized in a two-dimensional graph. The K-Means algorithm [20,40] grouped the sensory features and the results of the clustering were presented in a graph with colour-coding for different groups separately. This visualization facilitated the evaluation of the structure of the learning instances corresponding to the sensory features and allowed us to understand the preparation of learning sets based on the three sensory features.

As part of the analysis of the sensory feature data, correlations between the different features were also determined. A cluster map of the sensory features and odors was created to identify feature similarities that could influence the classification results. Based on the clustermap results, it was possible to prepare learning sets for selected odor features.

4. Conclusions

Within the scope of our research, we identified mead on the basis of selected sensory characteristics of aroma, which allowed us to accurately classify this type of mead. By analyzing the performance of different classification algorithms, we found that Random Forest and K-Nearest Neighbours (KNN) algorithms proved to be the most effective models for mead recognition. Both models showed high performance, but it was the Decision Tree algorithm that achieved the highest accuracy value, suggesting its potential for accurate classification based on odor features. These results indicate that the choice of an appropriate classification model can significantly affect the performance of the mead identification process in practical applications. Analysis of the confusion matrix also showed that acacia honey was more easily identified by the algorithms than tilia or buckwheat mead. It is worth noting that the high performance of the decision tree algorithm influenced the fact that the number of classified cases using the confusion matrix method was higher for lime honey than for acacia honey. The highest model performance result was obtained using the decision tree for odor sensory attributes such as General odor intensity, Yeasty, Floral, which resulted in an accuracy value of 0.909, a precision of 0.929, a recall of 0.909 and an F1-score of 0.911.

Future research directions may focus on the optimization of mead production processes using machine learning methods, which open up new opportunities for experiments aimed at increasing the efficiency and quality of this beverage. A deeper understanding of the factors influencing the quality of mead both in terms of its chemical and sensory characteristics is also an important area for further analysis. In the future, it is also worth focusing on the selection and optimization of classification models used in practice, because, as the results so far have shown, the appropriate choice of algorithm significantly affects the effectiveness of mead identification. The difficulties in distinguishing between buckwheat and lime mead require special attention analysis of the confusion matrix has shown that they are classified less precisely than acacia honey, which may indicate the need for further research into improving the classification performance of these particular types of mead.

It is also worth highlighting the importance of developing interdisciplinary re-search that combines advanced methods of chemical analysis, sensory evaluation and artificial intelligence. This approach fosters the creation of modern research tools and the advancement of knowledge on the quality and characteristics of food products, including mead.