Groundnut (ARACHIS HYPOGAEA L.) Seed Defect Classification Using Ensemble Deep Learning Techniques

2. Related Works

Scholars (Huang et al., 2019; Szczypinski et al., 2017) run their study on crop defect recognition and identification during storage using computer technology. This section briefly discusses existing research approaches and methodologies in the context of machine learning-based defect inspection and classifications of groundnut seeds. So many researchers have done on groundnut seeds using images from different perspectives such as defect detection(Guchi, 2015), variety classifications (H. Yang et al., 2021) as well as genetic characters inheritance (Aworinde et al., 2007; District, 1935), and oil content quality (Sarvamangala et al., 2011; Shasidhar et al., 2017), etc. before and after harvesting. This study is limited to groundnut seed defect classifications after harvesting.

The study (Ziyaee et al., 2021) investigated effective image processing along with computational techniques for detecting a defect with an aflatoxin-producing fungus as well as the growth of molds using three machine learning algorithms which are support vector machine(SVM), Artificial neural network(ANN), and the adaptive neuron-fuzzy inference system (ANFIS) using 1500 images. The study was conducted on a small dataset and labeling of defects is subjective. Similarly (Manhando et al., 2021) employed an optical coherence tomography (OCT) imaging technique for the detection of mold-contaminated peanuts and the growth of molds during the incubation period with SVM trained on features extracted using a pre-trained Deep Convolutional Neural Network (DCNN). Conversely, hyperspectral imaging technology (Manhando et al., 2021; Zou et al.,2022) was used for non-destructive variety classification and mold detection of peanut seeds by stack ensemble learning mechanism. They achieved high detection performance on five varieties of peanut seed, more over the study (Liu et al., 2022) verified that augmentation can improve the detection accuracy on K-nearest neighbors (KNN), support vector machines (SVM), and MobileViT-xs models.

However, none of them addresses other peanut defect types such as defects with a virus, physical damage, infection with pests, and seed burning with high heat were not considered. The researcher(Y. Wang et al., 2023) developed a classical Faster Recurrent Convolutional Neural Network(R-CNN) model to detect defects on 1300 groundnut seed images based on textural features. The finding was achieved using texture enhancement and texture-based attention modules on the neural network of the backbone network VGG16. However, Faster R-CNN with a VGG16 backbone predominantly focuses on learning and extracting deep convolutional features rather than explicitly focusing on texture features. Nevertheless, considering a combination of features, including texture, shape, context, and semantic information, to achieve robust and accurate classifications in Faster R-CNN is important. The work by (Y. Wang et al., 2023) used Faster R-CNN for the detection of groundnut defects including moldy, mechanically damaged, and germinated using texture features sensitive to change in scale. As a scientific contribution, the findings provide valuable insights, novel approaches, and techniques for seed defect classifications as listed below.

• In the preprocessing phase, techniques are chosen based on image quality evaluation metrics value to assess the impact of preprocessing techniques.
• The GAN-based Augmentation technique was developed to increase dataset size and diversity for model performance and generalization.
• The deep ensemble classification model was developed utilizing VGG16 and InceptionV3, incorporating classical and deep-based features through concatenation.
• We collected 6050 images that served as defect grading for future research.

3. Methodology

The performance of the proposed model is based on the adjustment of the independent variables. This experimental research design involves the detail of dataset preparation, preprocessing, implementation, and performance evaluation as the working architecture of Figure 1 stated below.

The proposed architecture in Figure 1:1 outlines a classification model design consisting of four stages. The stages are preprocessing, feature extraction, model development, and model evaluation. Each stage involves specific techniques to ensure the successful completion of the task.

3.1. Preprocessing

The collected groundnut seed image suffered from burling low insight intensity and had varying sizes with a limited imbalanced collection. These challenges required noise filtering, resizing, enhancement, and other preprocessing techniques. To choose the most effective preprocessing technique image quality evaluation metrics, such as, Mean Square Error (MSE) Structural Similarity Index (SSI), and Peak Signal Noise Ratio (PSNR) were utilized.

Image quality evaluation: In Preprocessing, the result of the image may lead to a loss of important information. Image quality evaluation (Padilla et al., 2020) can be subjective to relying on human judgment, or objective utilizing image evaluation techniques. The study (Hu et al., 2020) suggested that the subjective method is efficient in cases like multimedia images, still for other image-based applications, it may be time-consuming and result in a loss of important information (Z. Wang et al., 2004). Instead, objective methods are best suited to compare and evaluate processed images with original images or image processing algorithms. Some of these evaluation metrics include Mean Squared Error (MSE), Signal to Noise Ratio (SNR), Peak Signal-to-Noise Ratio (PSNR), Structural Content (SC), etc.

Mean squared Error (MSE): The MSE is a method used to assess the dissimilarity between two images by calculating the average of the squared differences between corresponding pixel values in a reference (original) image and a distorted (processed) image. A lower MSE value indicates a higher similarity between the distorted and the original image. MSE is a mathematical measure that does not always correlate well with human perception, especially for view and small differences between images (Hu et al., 2020). It can be expressed as follows.

$$\mathbf{M}\mathbf{S}\mathbf{E}=(1/\mathbf{N})\ast \mathsf{\Sigma }[{\left({\mathbf{I}}_{\mathbf{o}\mathbf{r}\mathbf{g}\left(\mathbf{x},\mathbf{y}\right)}-{\mathbf{I}}_{\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}\left(\mathbf{x},\mathbf{y}\right)}\right)]}^2$$

(1)

where N is the total number of pixels in the image, I_org(x, y) is the pixel value at coordinates (x, y) in the original image, and I_pro(x, y) is the pixel value at coordinates (x, y) in the processed image.

PSNR: It assesses and calculates the ratio between the maximum signal power and the MSE between the original and distorted images. It is typically expressed in decibels (dB) and serves as an indicator of the amount of noise or distortion present in the image. A higher PSNR value suggests that the image has a reduced amount of noise or distortion. PSNR can be calculated as

$$\mathbf{P}\mathbf{S}\mathbf{N}\mathbf{R}=10\ast \mathit{l}\mathit{o}\mathit{g}10\left(\right(\mathit{m}\mathit{a}\mathit{x}\_\mathit{p}\mathit{i}\mathit{x}\mathit{e}\mathit{l}\_\mathit{v}\mathit{a}\mathit{l}\mathit{u}\mathit{e}^2)/\mathit{M}\mathit{S}\mathit{E})$$

(2)

where $\mathit{l}\mathit{o}\mathit{g}10$ Function converts the ratio of the peak signal power to MSE into a logarithmic scale and multiplying by 10 scales it to decibels (dB).

Structural Similarity Index: It assesses how similar the structures in two images are unlike conventional metrics that, only consider pixel differences on factors like luminance, contrast, and the overall structure of the image (Horé & Ziou, 2010). It captures important visual characteristics instead of just individual pixel values. When the SSIM value is higher, it signifies that the original and distorted images have a stronger resemblance in terms of their structure, implying a higher level of perceptual quality. Index similarity can be calculated as.

$$\mathbf{S}\mathbf{S}\mathbf{I}\mathbf{M}\left(\mathbf{x},\mathbf{y}\right)=\left[\mathbf{L}\left(\mathbf{x},\mathbf{y}\right)\ast \mathbf{C}\left(\mathbf{x},\mathbf{y}\right)\ast \mathbf{S}\left(\mathbf{x},\mathbf{y}\right)\right]$$

(3)

where: x and y represent the corresponding pixel values in the original and processed images. L(x, y) is the luminance similarity, which measures the similarity in mean pixel intensities between x and y. C(x, y) is the contrast similarity, which evaluates the similarity in variances of pixel intensities between x and y. S(x, y) is the structure similarity, which calculates the similarity in covariance of pixel intensities between x and y. The values of L(x, y), C(x, y), and S(x, y) are typically calculated using the mean, variance, and covariance of pixel intensities in the corresponding patches of the images. The final SSIM value is obtained by taking the product of the three similarity components.

3.1.1. Histogram Equalization

It is an image processing technique that aims to enhance contrast and improve the visual quality of an image. Rajendran et al. (2015) illustrated that a histogram is used to segment an image into several regions by thresholding and redistributing the intensity values across the entire range. We applied Adaptive Histogram Equalization to address brightness issues in the collected image.

3.1.2. Resizing

Image resizing is necessary to increase or decrease the total number of pixels. Making all experimental images uniform in dimensions helps to further preprocessing speeds in image retrieval (Gordo et al., 2017). Images are resized to 224 × 224 pixels with an aspect ratio of 2:1 by comparing resizing techniques, which include Nearest Neighbor, Bilinear Interpolation, Bicubic Interpolation, and Lanczos based on image evaluation metrics.

However, selecting the optimum resizing technique using experimental compression of objective evaluation metrics is the best strategy rather than relying on subjective evaluation in a large dataset. Table 1 shows the values of the four resizing techniques in a single image comparison. Based on the results, the bicubic interpolation technique has been selected.

Figure 2. Resized image quality evaluation metrics value.

Figure 2. Resized image quality evaluation metrics value.

3.1.3. Noise Reduction

Noise removal is a technique used to eliminate unwanted detail (Motwani et al., 2004) that depreciates and destroys the original image quality. Similar to resizing techniques we have selected noise reduction techniques based on the experimental result of evaluation matrices. Based on noise reduction experimental information and evaluation results. Gaussian Adaptive Bilateral Filter (GABF) is selected from the comparison result of mean, bilateral, Gaussian, and Gaussian adaptive bilateral filtering techniques. The evaluation metrics result of PSNR and MSE in Gaussian filtering is larger than others and bilateral filtering has smaller values of MSE and PSNR from this the combination of the two has an optimum value than other techniques.

3.1.4. Segmentation

Image segmentation includes the process of digital images partitioning into meaningful multiple regions (Kunaver & Tasič, 2005) offering easy dimensional reduction. It represents some kind of information to the user in the form of color, intensity, or texture offering easier ways (Yogamangalam & Karthikeyan, 2013)for further analysis and comparison of images. Three segmentation techniques are compared through experiments on three segmentation techniques, which are watershed, threshold, and edge-based segmentation. Watershed segmentation is selected based on the characteristics of the images, which are irregular shapes and clear intensity, images with minimal overlap, and the noise is filtered by good noise reduction techniques in the previous preprocessing.

4. Experimental Result

After collecting and preparing the images; each model is individually trained with end-to-end feature extraction. After training and evaluating each model; the results show varying classification abilities. The base model uses voting aggregation to develop an ensemble model with the same data. The model is trained and evaluated using three-split ratios of 70%-15%-15%, and 80%-10%-10% for training; testing; and validation respectively with hyperparameter setup using grid search optimization. The model is trained with both GAN-based and classical augmentation techniques. However; we achieved the best result using 80%-10%-10% split ratio using GAN-based augmentation. The total number of instances used for model development is 7800 images. The validation set was calculated to be 780; while the testing set also consists of 780 image instances. The remaining 6240 image instances are allocated to the training set. Each class in the validation and testing set contained 156 images; while the training set had 1248 images for each class. The accuracy of the base and ensemble model using both augmentations is summarized in the following table

Table 1. The result of augmented data.

Augmentation	Base models	Base model accuracy	Ensemble model accuracy
GAN based	VGG16 model	94.51%	96.25%
	Inception V3 model	91.31%
Classical	VGG16 model	93.56%	94.2%
	Inception V3 model	90.32
No augmentation	VGG16 model	89%	87%
	Inception V3 model	83%

Table 1. The result of augmented data.

Augmentation	Base models	Base model accuracy	Ensemble model accuracy
GAN based	VGG16 model	94.51%	96.25%
	Inception V3 model	91.31%
Classical	VGG16 model	93.56%	94.2%
	Inception V3 model	90.32
No augmentation	VGG16 model	89%	87%
	Inception V3 model	83%

The result of the augmented data has significant differences for both the base model and ensemble model as shown in the above table. The accuracy and loss graphs of the experimental results with augmented data are represented in Figure 1.4 (a) and (b).

Figure 3. Accuracy and loss graph of ensemble model with GAN-based and classical augmentation.

Figure 3. Accuracy and loss graph of ensemble model with GAN-based and classical augmentation.

The model achieved 97.22%, 92.05%, and 96.25% train, validation, and test accuracy respectively. The model with GAN-based augmentation has better generalization than the classical one when we have seen the accuracy and loss graph of the model.

5. Error Analysis

The ensemble learning classification model achieved high accuracy by correctly classifying 6067 instances of training images, 718 validation images, and 751 testing images. However, misclassifications amounted to 173 images in the training dataset, 62 validation images dataset, and 29 images in the testing dataset. Most errors and misclassifications in the training, validation, and test data were due to cracked classes. The model often mistook cracked images as pest defects, classifying them as pest defective instead of pest defect images. Another error occurred with two classes, where pest-defected images were learned as cracked, and virus-defected images were considered fungal defective instead of their actual classification. To improve the model's accuracy and robustness, we suggested removing images that resemble other classes and finding additional feature extraction techniques to differentiate them. Additionally, the VGG16 pre-trained model exhibited weaker classification ability compared to InceptionV3.

6. Conclusions

In this study, Groundnut seed image defects are classified into five categories virus, fungus, pest damage, physical damage, and health using a deep ensemble model. The collected images are enhanced with Global Histogram Equalization (GHE), and noise filtered using Gaussian adaptive Bilateral Filtering (GABF) techniques. The images are resized to 224 × 224 using bicubic interpolation with an aspect ratio of 2:1. Preprocessing techniques are chosen based on image evaluation metrics. Regions of interest are detected using YOLOv3. The initial dataset of 6050 images is augmented using traditional and GAN-based methods, resulting in a total of 7800 images. The classification model is trained using features extracted from a comparison of classical techniques (GLCM & HOG) with deep-based techniques (VGG16 & InceptionV3). The experiment showed that different base learners performed differently depending on the feature concatenation with InceptionV3 performing well on deep-based and HOG features. The model evaluation without augmentation, GAN-based augmentation, and traditional augmentation achieved test accuracies of 87%, 96.25%, and 94.2%, respectively. The model’s performance is limited by the similar appearance of different defects and the weaknesses of the classification base learners. The challenges make it difficult for researchers to claim performance improvements and the effectiveness of the model.