Mold Detection on Food Surfaces Using YOLOv5

: The study aimed to identify different molds that grow on various food surfaces. As a result, we conducted a case study for the detection of mold on food surfaces based on the “you only look once (YOLO) v5” principle. In this context, a dataset of 2050 food images with mold growing on their surfaces was created. The dataset was trained using the pre-trained YOLOv5 algorithm. In comparison to YOLOv3 and YOLOv4, this current YOLOv5 model had better precision, recall, and average precision (AP), which were 98.10%, 100%, and 99.60%, respectively. The YOLOv5 algorithm was used for the first time in this study to detect mold on food surfaces. In conclusion, the proposed model successfully recognizes any kind of mold present on the food surface. Using YOLOv5, we are currently conducting research to identify the specific species of the detected mold.


Introduction
Mold is dangerous to human health and a serious threat to food supply chains. They can grow on a wide variety of acidic products, including fruits or fruit juices, as well as foodstuffs with intermediate moisture content, like breads and bakery products, which many other microorganisms, such as bacteria, cannot [1]. The prompt detection of fungal presence to prevent further harmful effects is very important in food processing.
In the field of computer vision and image processing, object detection is considered as a significant branch. Object detection algorithms have also received a lot of attention in deep learning [2]. There are roughly two types of visual object detection. The first is the region proposal method (e.g. R-CNN [2], Fast R-CNN [3], Faster R-CNN [4], etc.), and the second is the end-to-end method (e.g. YOLO, SSD, RetinaNet etc.) [3]. In terms of object detection speed, the end-to-end method showed better performance than the region proposal methods [4]. As object detection technology has evolved, the YOLO (You only look once) series of algorithms with very high precision and speed have been used in various scene detection tasks [5]. At the same time, the YOLO system computes all of the image's features and predicts all of the objects. YOLOv5 is the fifth generation of YOLO, written in Python programming language [6]. According to various studies YOLOv5 outperforms the rest of the YOLO model in terms of accuracy and speed [5,6,7]. To the best of our knowledge, this is the first time YOLOv5 has been used to detect mold on food surfaces in the present study. However, in some recent studies, YOLOv5 was used to detect various objects such as apples [5], mushrooms [7], ships [4], face masks [8], safety helmets [9], and vehicles [10], etc. Hence, the aim of this study was to detect the molds from the captured images and count the total number of spores in these images.

Data description
There are two types of data file in the dataset. A) Raw digital images which contain a total of 2050 images with a high diversity of poses, angles, lighting conditions, weather conditions, and backgrounds. The images are all in JPG format. B) Image annotation files containing 2050 images. These files specify the precise locations of the objects with labels in the corresponding image. The annotation was done manually, and the annotated values were also saved in txt files. The data files were divided into two folders: images & labels. Each folder was then divided into two subfolders: train and valid. On the other hand, the dataset preparation consists of four steps: 1) data collection, 2) data preprocessing, 3) data annotation, and 4) data augmentation. The Model building and detection consist of eight steps: 1) importing libraries, 2) importing dataset, 3) cloning YOLOv5 repository, 4) installing required libraries for YOLOv5, 5) training YOLOv5 model with mold dataset, 6) plotting metrics in tensor board, 7) detecting mold in images using trained model, and 8) plotting detected images.

Importing and installing libraries
Some popular Python libraries were imported for matrix operations, plotting, and file handling. import numpy as np import matplotlib.pyplot as plt from glob import glob There is a 'requirements.txt' file in the YOLOv5 directory that lists the required libraries and versions to run the YOLOv5 model. Required library was installed using the bash command '!pip install -qr <directory>'. It is good to mention that 'pip' is a python package manager. !pip install -qr '/content/yolov5/requirements.txt'

NumPy
NumPy's data formats include matrix and multidimensional arrays. NumPy can perform mathematical operations on arrays such as statistical, algebraic, and trigonometric routines. It provides a highly functional multidimensional array and even the necessary tools for computing and regulating the arrays [11].

Matplotlib
Matplotlib is a portable 2D plotting and imaging package largely used to visualize scientific, engineering, and financial data. The software is free to download, use, and distributeIt has a wide range of applications. Most users are familiar with the command-line interface for creating plots and images interactively. This interface displays and manipulates data in a simple pop-up window [12].

Data Collection
Images of foods with mold growing on their surfaces were collected. 750 images of various foods stored in the laboratory of Department of Food Engineering and Technology of Sylhet Agricultural University, Sylhet, Bangladesh were collected. The images were captured using an action camera (GoPro HERO 9 5K Ultra HD). The remaining images were obtained from various internet sources. Both paid and free stock photos were used in this case.

Data pre-processing and annotation
After taking photos of various mold-affected food surfaces, the images were all converted to JPG format. Tzutalin's [13] popular annotation tool LabelImg was used to cautiously label the images in this final phase. First, each image is opened in this tool one at a time. Then, a rectangular shape was manually drawn to the boundary of an object in order to specify its exact location in that image by x_center y_center width height. Finally, each object has been given a label, such as 'bread mold', or, 'cake mold' etc. In LabelImg, annotated values were saved as txt files in YOLOv5 format.

Data augmentation
Data Augmentation was done in order to increase the quantity and diversity of data. It aided in the reduction of over fitting in small datasets. To generate new images from the mold dataset, a few data augmentation techniques such as flipping, cropping, and color space transformation were used. The datasets were also uploaded to the roboflow web tool for data augmentation, preprocessing, and train-test split.

Plotting detected images
After running inference, images was saved in 'runs/detect/exp/' directory. Using 'matplotlib' images were plotted.

Training YOLOv5 model with mold dataset
The mold dataset was designed to train by using Google Colab, which provides free access to powerful GPUs. We used a notebook developed by Roboflow.ai [14] which is based on YOLOv5 [15] and uses pre-trained COCO weights. Suitable number of epochs was chosen to train newly developed mold dataset. To train the model 205 epochs was selected which was taken approximately 40 min. In YOLOv5 directory there is a 'train.py' file for training YOLOv5 model. Using bash command '!python train.py <parameters>' the model was trained accordingly.
The details of training the YOLOv5 model are as follows.

•
Image size: 640 • batch: size of mini batch of images to feed in one iteration.
• epochs: the number of training iteration.
• data: (train, validation) data directory and number of class and class name was described in this YAML file.
• cfg: models are described in YAML model configuration files in 'model' directory. There is four version of model of different size. 'YOLOv5s.yaml' has been used to train.
• name: a model name has been given.

Plotting metrics in tensor board
Tensor Board is a helpful tool for visualizing metrics related to planning, validation, and testing. To visualize mAP, recall, precision metrics 'tensorboard' was loaded using magic command '%load_ext tensorboard'. And '%tensorboard -logdir runs/' was used to visualize metrics from log directory. %tensorboard --logdir runs/

Detecting mold in Images using trained model
After training, different types of new and unseen images were used to test the model performance to detect the mold on food surfaces. We took more 100 images as testing set. There is a 'detect.py' file 'yolov5' directory to run inference on images. Inference was performed on the image source directory using '!python detect.py parame-ters>'.
!python detect.py --source /content/valid/images --weights 'runs/train/FoodMold/weights/best.pt Figure 1 shows that the model can detect molds on food surface to a higher degree of certainty.

Model performance and comparison
The YOLOv5 model performs really well with 205 epochs. After that with the increase of epochs all the losses like classification loss, box loss and objectness loss were increased. And the model performance was decreased. YOLOv5 was tested to detect mold on various food surfaces after the data set was created. To train the model, various image resolutions were used, and a suitable image resolution of 500×500 pixels was chosen. Before creating the data set, the images were labeled to attain the optimum mold detection. The detected molds' precision, recall, and average precision (AP) were calculated and compared to other models. Results are shown in Table 1. Metrics are calculated with the help of following equations [7].  As a YOLOv5 feature enhancement, the model is better at detecting small to large molds. The activation of Mish+SPP in the models showed an increase in precision, recall, F1-score and AP. It was also discovered that when the image quality was high, the performance was better.

Ongoing Research
There are still a lot of things that can be done to improve the mold detection system with YOLOv5. We are currently working on improving the model to detect specific species of food mold. Furthermore, once the system detects and declares any type of foodborne mold, we will attempt to make the entire system more convenient for food industry applications. Data Availability Statement: All of the data used to generate the results is included in the article, and no additional source data is required.