Design and Parameterization of a Computational Model Based on Convolutional Neural Networks for Mango Selection

1. Introduction

Mango (Mangifera indica L.) is known as the most traded tropical fruit in the world [1]. It is produced in more than 90 countries and ranks fifth in the world after bananas, apples, grapes and citrus fruits, gradually increasing its international trade in the last decade [2,3,4]. This fruit is highly valued by consumers worldwide for its attractive color, unique flavor and juicy pulp. It also contains several vitamins, high fiber content and several functional compounds that can improve our body’s defenses, fight cellular oxidation and promote physical health [5,6,7,8].

In Latin America, mango has stood out as one of the main export fruits, with Mexico, Brazil and Peru representing the largest production, offering high quality fruits in order to satisfy world demand [9]. However, mangoes are sensitive during post-harvest processing, since their shelf life is greatly reduced by physical factors, such as shocks, compression and/or sunburn, which, although they do not immediately destroy the integrity of the fruit, cause the onset of a degenerative process of its organoleptic properties [10,11], and ripening factors, since being a climacteric fruit, it reaches its highest respiration peaks a few days after being harvested, quickly triggering biochemical reactions that cause increased acidity, weight loss, rotting, pulp softening, cell wall degradation, among others [12,6], which directly affect their quality during storage and transportation before reaching the target market. The color tone plays a crucial role in the selection of this fruit for commercialization, which merits special care due to the subjectivity and fatigue of the workers [13,14]. The sales value of mango can vary; at harvest maturity stage, it can present a slightly yellow flesh color that is not related to its green-yellow or reddish-green skin, and on the other hand, at the consumption maturity stage, the flesh acquires yellow-orange tones and a yellow-reddish skin, being very versatile changes in the internal (flesh) and external (skin) color, as well as in its flavor. These changes are attributable to each mango variety [15,16,17]. Figure 1 shows workers during the mango harvesting season in Peru.

The high cost of extending the shelf life of fresh mangoes through the cold chain calls for precise solutions for the identification of fruit at the right maturity, accurately selecting mangoes of suitable quality for trade as a market articulation strategy. As is well known, color is an important factor in quality assessment as it can be used to estimate the maturity of fresh fruit before marketing [18,19]. Therefore, the use of non-invasive technologies such as artificial vision systems together with Deep Learning can contribute to finding solutions to various problems, thus promoting the improvement of quality control processes, productivity and flexibility in business management in agriculture and food industry [20,21,22,23]. Machine vision approaches have been studied for the detection of different post-harvest problems when sorting fruits by color, such as techniques based on identifying patterns by dimension using the circular Hough transform [24,25], by color in different thresholds [14,26] and textural characteristics on the surface of the fruit [27,28]. Thus, it is well known that classification approaches often use structure matching, support vector machines [29] and recently on CNN [30,18,31,32], the latter being the one that offers promising results.

A CNN is structurally composed of different layers, and these try to find the ideal numerical values that identify a learning characteristic [33,34,35], this learning if supervised allows trial and error based evaluation to achieve higher accuracy rates in each layer [36,37]. In the area of image acquisition, recognition and interpretation, CNNs have performed efficiently and even better than other machine learning techniques in the identification of various qualities to extract and combine typologies that establish the maturity categorization in fruits and vegetables [38,39,40,41].

2. Materials and Methods

2.1. Acquisition architecture

Image acquisition was performed in a semi-controlled environment. Natural light near neutral (5500K) was used because it is the one that adds the least tonality components to the images [42], avoiding direct sunlight on the fruit as this is a factor that hinders image processing [43]. A camera with a tone depth of 8 bits was used, reaching 24 bits per pixel and a spatial resolution of 1200 x 800 pixels, considering 3 channels (RGB) in "jpeg" format, the spatial resolution was approximately 0.02 mm/pixel, the evaluation of these images was performed on a laptop computer (Legion Intel® Core™ i7-10750H, Lenovo, China).

2.2. CNN architecture

The computational model was designed based on standard convolutional neural network architecture, which was parameterized for subsequent implementation using the Python programming language and the IDE tool Google Colab using the cloud service. The structure of the CNN consists of a Convolutional Layer 1, which receives the input image, applying 32 filters with a 3 x 3 pixel mask, an equal fill type so that the output has the same dimension as the input image, the mask offsets are every 3 pixels. A reduction layer follows where it receives the filters from the previous layer and performs a reduction based on the max pooling function, with a filter of 2 x 2 pixels, with a mask offset of 2 pixels.

I11	I12
I21	I22

f (I11, I12, I21, I22) = Maximum value (x, y)

The flattening layer receives the filters from the last layer and oversees converting the multidimensional data into a 1D vector, which is propagated to the other hidden layers, using an activation function called ReLU, because it eliminates negative values and leaves the positive ones, which are the values of an image.

$$\mathit{f}\left(\mathit{x}\right)=\mathbf{m}\mathbf{a}\mathbf{x}\mathbf{}(\mathbf{0},\mathit{x})\left\{\begin{array}{c}\mathbf{0}, \mathit{to} \mathit{x}<\mathit{0}\\ \mathit{x}, \mathit{to} \mathit{x}\ge \mathit{0}\end{array}\right.$$

Subsequently the hidden layers have full connectivity, a Dropout of 0.5 is used, to regularize and reduce the overfitting in the neurons, avoiding bias or memorization in learning [44]. Finally, we arrive at the Output Layer, which is composed of 2 neurons, corresponding to the 2 classes to be classified (mature and green), configured with a SoftMax activation function (Figure 2).

2.3. Analysis and validation of datasets

The dataset structuring consists of 201 images, 50 % are used for training, 50 % for validation, as described in Table 1.

The images of the training and test dataset are normalized with values between 0 and 1. To improve the training and model results, the "Data Augmentation" technique is used, which allows us to augment our dataset by introducing transformations on the original data. The transformations performed a rotation, horizontal and vertical shift, zoom and a horizontal flip. To analyze the uniformity of image acquisition, the environment was evaluated using histograms, which provide information about the distribution of shades in RGB (Figure 3).

At the end of the acquisition stage, a Data set composed of a total of 201 images was obtained, which were labeled according to the criteria of physiological maturity: "mature" or "green" based on regulations useful for export [45].

3. Results and Discussion

In the present work, a computational model based on a standard CNN architecture has been designed, on which a dataset consisting of 210 images was deployed to train and achieve learning for mango classification. To find the best CNN classification performance, configurations were evaluated, which is called parameterization. Five models have been parameterized, where different combinations of layers (convolution and reductions) have been applied repeating 15 epochs with a mini batch size of 15 images, thus calculating the accuracy on the test and training dataset.

3.1. Number of Convolution and Reduction Layers

According to [46], it suggests that there should always be a convolution and reduction layer, so the number of layers has varied, as seen in Table 2. Figure 4 shows that 4 types of layer configuration were evaluated, always considering a convolution and reduction, reaching the best performance in type 1, consisting of 1 convolution and reduction block with an accuracy of 94.94%.

[47] studied an 8-layer CNN for classifying fruits, obtaining a classification accuracy of 95.67 %, which compared to the present research, the CNN, only with 1 convolution layer and 1 reduction layer achieved an accuracy of 94.94 % demonstrating the potential of CNNs in automatic classification in an objective and fast way, although still to be improved. In another case [48], they evaluated mango price forecasting using a combined computational model based on back propagation learning and a long-term memory CNN, where they determined that the mean square error, mean absolute percentage error and coefficient of determination R² of the model were 0.02, 0.14 % and 0.99 respectively, demonstrating the functionality of this technique and corroborating the flexibility of these designs.

3.2. Spatial resolution of the input images.

The data set consisted of images acquired with a spatial resolution of 1280 x 800 pixels, in 3 channels (RGB).

As shown in Figure 5, 4 types of resolutions were evaluated based on which the input images were rescaled, obtaining the best result with the resolution of 32 x 32 pixels, followed by a minimum difference of 1.03 % and 1.18 % respectively with the resolution of 64 x 64 pixels and 150 x 150 pixels, observing that when the resolution increases, the accuracy decreases. According to the authors [49,50,51], input images to a convolutional neural network with small dimensions are adequate, as long as the original images are also small, for such reason the comparisons made in the work of [52] can consider dimensions up to 28 x 28 pixels, due to the data set with which they perform the evaluations which have images of 256 x 256 pixels. Similarly, [46] used CNN to classify sour lemon images, where they found that with 16×16 pixels, an accuracy of 99.8 % was obtained, moreover, the accuracy of this model was 100 % in the classification of 32 × 32 and 64 × 64-pixel sour lemon images. Therefore, the best resolution to use in the research was 32×32 pixels, as shown in Figure 5, reaching an accuracy of 96.12 % attributed to the complexity of the fruit.

3.3. Number of epochs used in training and validation.

The CNN requires repetitive cycles for training and validation which are called epochs.

As shown in Figure 6, the epochs started to be evaluated from the 10th cycle, obtaining an accuracy of 96.12 %, in the 20th and 30th epoch the indicator improves, with 60 and 160 reaches 100 %, but analyzing the internal behavior of the CNN, it apparently enters a state of overlearning, which is not recommended in any case of study, for this reason they should not be considered. [46] found that the number of training repetitions to achieve prediction accuracy in epochs 1-9 has a high error rate, and in epochs 9-40 the error rate is minimal (close to 0). As can be seen in Figure 6, the best number of epochs are in the range of 10 to 60, which means that to obtain a minimum error (between 3.88 % to 0 %), a number of epochs present in that range should be considered.

3.4. Parameterization of the CNN model

From the previous evaluations, it has been possible to obtain information about the constructed structure to design a parameterized CNN to improve the level of accuracy in the handle selection process. For this purpose, the following parameters are proposed: Number of convolution and reduction layers: 1 level, convolution and reduction layer per layer, a spatial resolution of the input images of 32 x 32 pixels and a number of 20 epochs. The result of the designed CNN can be seen in Figure 7 and 8.

The optimized accuracy in the CNN-based computational model designed is 96.04 %, possibly as a consequence of the use of back propagation, this is argued because CNNs have a structure that adapts and strengthens in digital image processing thanks to its convolution and reduction layers. In similar and recent works; [52,54], the models manage to achieve a classification accuracy and quality rating of 96.7 % and 95.11 %, probably because they did not use a single model for the evaluation of their factors and that the data set with which they work is prepared to evaluate image processing cases, on the other hand are fruits with very distinguishable characteristics such as color, textures, etc., in the present investigation the images were acquired without any previous preparation being an important aspect for a CNN since it improves with the use because it can learn not only the color characteristics but also the shape and texture approaching what is simulated in real time [27], this also improves the processing time since in the traditional computer vision scheme it decomposes them into stages [55], demanding more computational resources [56,57]. The work of [58], is oriented to classify various types of fruits, 25 categories, reaching 100 % accuracy, in this aspect they obtain a higher % than achieved in the present work, it should be considered that the data set with which they work is prepared to evaluate cases of image processing, on the other hand, they are fruits with very distinguishable characteristics by their color, textures, dimensions, etc., in the present research the images were acquired without prior preparation approaching a more real context.