Feature Extraction Using Deep Learning Techniques to Identify Microplastics in Open Sewer Systems

1. Introduction

Microplastics are tiny plastic particles that measure less than 5 millimeters in size, commonly found in marine environments, including oceans, lakes, and rivers [13]. They are usually the result of the breakdown of larger plastic materials or the direct disposal of microplastics into the environment. Microplastics can accumulate in water bodies and soil, and their consumption by marine life and other microorganisms can lead to death [14]. They pose a significant threat to aquatic organisms and have been known to cause harm to human beings when consumed directly or indirectly [13]. The impact of microplastics on the environment has been widely researched, and various approaches have been proposed to identify and mitigate their presence [14]. In this paper, we focus on extracting features that can be used to build a model for identifying microplastics in images taken from open sewers that lead to the Indian Ocean.

The paper presents the process of feature extraction from images of microplastics in open sewer systems using Scale-Invariant Feature Transform (SIFT) algorithm. The extracted features will be used to build a model for identifying the presence of microplastics in the environment and thus help in their control.

The first goal of this research was to discover the characteristics that influence the presence of microplastics in photographs taken from open sewer systems. Microplastics characteristics can be divided into two categories: physical and chemical where dimensions and colour are physical attributes, while structure and polymer composition are chemical attributes, [1,7]. The emphasis was to investigate the physical attribute of microplastics, specifically variations in size rather than colour, as the primary feature to be analyzed in the data. Foam is one of the physical characteristics that can be used to identify microplastics [15,16].

When organic materials such as leaves or twigs decompose in water, they release surfactants that disrupt the surface tension and facilitate the mixing of air and water, ultimately leading to the formation of natural foam. Although not all foam is naturally occurring, certain human-made products such as detergents can generate foam that is visually similar to natural foam but contains harmful chemicals that pose a threat to aquatic life, including fish [16]. Therefore, it is crucial to distinguish between natural and human-made foam in water bodies to prevent potential environmental harm.

2. Related Work

This review focused on feature extraction techniques for object detection in images. Tiny objects such as microplastics in images have been detected using various techniques. One such study explored IoT devices and instance segmentation for detecting suspicious objects in hyperspectral images and then classifying them using Neural Networks, to detect, classify, and quantify microplastics pollution in freshwater ecosystems [9]. The study by Sundar magnified the images to detect the objects, this approach did not require feature extraction [9]. Another study used YOLOv5, a Convolutional Neural Network (CNN), a deep learning model to detect and classify microplastics and microorganisms in the marine environment. They trained the model on a dataset of marine images marked using Label-Studio and achieved accurate detection and classification of microplastics [8].

A different approach employed electrical impedance spectroscopy (EIS) enhanced by support vector machines (SVMs) to detect microplastics in water. Stationary and dynamic measurements were taken on water samples contaminated with different plastic concentrations. The model achieved a high classification accuracy of above 90% for detecting microplastics using SVMs [4]. A similar approach used Electrical impedance spectroscopy (EIS) enhanced by a classifier based on support vector machines (SVM) to detect microplastics. This method successfully distinguished different plastic materials and particle sizes [1]. Another interesting approach that begins with Polarization-resolved digital holography was employed to discriminate synthetic microfibers for detection of the microplastics using machine-learning. A high accuracy of more than 85% in classifying microfibers was achieved [12].

A study using deep learning techniques was conducted on general small object detection [2]. A further study used an improved Single Short Detection (SSD) algorithm which incorporates DenseNet and feature fusion, being improved versions of Convolutional Neural Network (CNN). The study demonstrated advancements in object detection algorithms [6]. These advanced deep learning techniques can be applied to microplastics detection tasks. The current study employed one of the advanced deep learning techniques called Scale-Invariant Feature Transform (SIFT) algorithm [17] to extract features of microplastics in images taken from sewer systems.

3. Methodology

3.1. Data Selection

Previous studies of pollution along the Kenya coastline identified three counties; Kilifi, Mombasa, and Kwale, as major inlets into the ocean as a result of the increased tourism activity in these areas, [5]. As a result, the counties of Kilifi, Mombasa, and Kwale were chosen as data collection points for this study.

This study employed stratified sampling [10]. Sewer points were identified in the three counties where pictures were taken. The dataset was gathered from the various locations in Kilifi, Mombasa, and Kwale. All sewer points flowing into the Indian Ocean were represented by the selected points. The Nyali Bridge and Fort Jesus were selected in Mombasa [11]. In Kilifi, Kilifi Creek and Malindi Beaches were selected, while in Kwale County, Kwale river tributary was selected. Photographs were taken at these locations and analyzed.

3.2. Data Collection

Data was collected using photographic images. A high-resolution camera was used to capture images from the identified locations. The study used Canon EOS 2000D DSLR 24.1MP Camera with EF-S 18-55mm f/3.5-5.6 II USM Lens Camera Cover R-F-3, Camera strap, EW-400D Battery Pack, LP-E10 Scene Intelligent Auto 3.0LCD Screen. One thousand (1000) pictures were taken using the same camera. The pictures were then subjected to auto-cropping using an algorithm written in python to isolate the microplastics. Since the study did not capture images of persons there was no need for consent form for participants.

Raw sewage was clearly present, particularly along the beaches of Kilifi and Kwale, however, the precise locations where the sewer flowed into the ocean were not obvious. Regardless, the odour of sewage and the presence of non-dissolving foam were clear indicators of microplastics’ presence. In Mombasa the collection points, particularly those on the island, were noticeably more accessible. To avoid potential bias in the dataset, data on foam was collected, as a specific type of physical characteristic of microplastics data.

Figure 1. Flow chart for image processing.

Figure 1. Flow chart for image processing.

3.3. Data Preprocessing

Data preprocessing is critical because it serves as a foundation for the subsequent stages of the deep learning workflow. A thorough data processing stage ultimately leads to improved model accuracy and precision. Figure 2 shows image samples that provide a visual representation of the unprocessed data.

The following activities were undertaken to transform the dataset into a usable form: (i) Importing Relevant Python Deep Learning Libraries, (ii) Renaming of the image files, (iii) Resizing of images, (iv) Detecting key points and computing descriptors, (v) Looping over the key points and adding them to the list, (vi) Converting the list to a Pandas Dataframe, and (vii) Saving the Data Frame to a CSV file. The details of the steps are presented next.

a)

Importing Relevant Python Deep Learning Libraries: these libraries play a crucial role in facilitating the efficient manipulation and analysis of the data.

b)

Renaming of the image files: the image files were renamed to enhance ease of reference and to adhere to established naming conventions that align with the requirements of machine learning algorithms.

c)

Resizing of images: the images underwent a resizing procedure. The objective of this step was to standardize the dimensions of the images, aligning them with the specific input layer prerequisites of the neural network. By achieving uniformity in image size, the model's ability to accurately interpret intrinsic features within the dataset was heightened.

d)

Detecting key points and computing descriptors: this stage involves the detection of key points within the images and the subsequent computation of descriptors associated with these points. These descriptors encompassed crucial numerical characteristics detailing aspects such as orientation, scale, and spatial location. Through dedicated training, the model would acquire proficiency in identifying key points and calculating their corresponding descriptors.

e)

Converting the list to a Pandas Dataframe: the identified key points and their descriptors were consolidated within a well-structured list. This list served as a malleable foundation for further analysis and transformations. To facilitate streamlined data manipulation and exploration, the list was then transmuted into a Pandas Data Frame. This tabular structure endowed researchers with the capability to perform intuitive data manipulation operations.

f)

Saving the Data Frame to a CSV file: the prepared data was saved as a Comma Separated Value (CSV) file, a versatile data format that supports seamless integration into diverse software applications. This final step empowered subsequent phases of analysis and model training by providing a coherent and accessible dataset for further investigations.

The activities (shown in Figure 1) involved in transforming the dataset into a usable format highlight the significance of data preprocessing in deep learning. The steps taken aimed to standardize the data and extract relevant features, ultimately improving the model's accuracy and precision in object recognition and classification. The TensorFlow tool with openCV was used to capture the shape of microplastics, and bounding boxes were used to annotate the microplastics in the same images. Furthermore, the Scale-Invariant Feature Transform (SIFT) algorithm was used to extract relevant features from the images. Specific features were keypoints, size, angle and 128 descriptors, which gave insightful information about the object in the picture. The output of the process was a dataset of features for model building to identify microplastics in images. The dataset included features such as the size, shape, texture, and color of the microplastics.

4. Results

To achieve the goal of determining features that show the presence of microplastics in the images, one of the most important questions was:

“What characteristics are identified in photos taken from open sewer systems that indicate the presence of Microplastics?”

This study concentrated on the physical properties of microplastics. These properties are tangible when implementing Deep Learning algorithms for object detection in images because they can be easily depicted using visual imagery. It was revealed that microplastics (MPs) have the following physical properties: fibre, sphere, foam, sheet, fragment, bead, and film. The vast majority of fiber-shaped MPs are naturally derived. The clothing industry is the primary source of microfiber release, owing to washing procedures used at various stages of production. Foam-like MPs, on the other hand, can be produced by the degradation of certain plastic-based products, whereas plastic synthetic bags and packaging materials generate MPs with a film-like structure.

The foam was selected as a natural form of physical microplastics for inclusion in the study. This research used the foam characteristic and collected 1,000 photographs of foam at the selected locations.

4.1. Feature Extraction from Images

Because of their small size, deformities, and partial occlusion, identifying microplastics in images can be difficult. The Scale-Invariant Feature Transform (SIFT) algorithm was used to extract features from images to address this challenge. SIFT is a computer vision algorithm that can extract key features from images regardless of size or rotation. It accomplishes this by detecting key points that are invariant to scale and rotation and describing them with a set of noise and occlusion-resistant features. The resulting SIFT features can then be used to identify microplastics' distinct visual patterns and textures in images. These characteristics can then be used to train machine learning models, such as neural networks, to classify images that contain microplastics.

Following the extraction of features from the images, a dataset of key features was created in order to develop a model to detect microplastics. The features are: Keypoints, size, angle, and a set of 128 descriptors that capture the unique characteristics of the microplastics present in the images. The descriptors were generated by examining the local gradients of the pixels surrounding the key points and creating an orientation histogram to capture the dominant gradient direction. The descriptors were designed to be scale, rotation, and lighting condition insensitive, ensuring their robustness in identifying microplastics in images. Descriptors are groups of feature vectors that represent the key points in an image. These descriptors are calculated by analyzing the local gradients of the pixels surrounding the key point and creating an orientation histogram to capture the dominant gradient direction. The descriptor is then created by concatenating the histograms from multiple spatial bins around the key point. The resulting descriptor is a high-dimensional vector that captures the key visual characteristics of the local region surrounding the key point, such as texture, contrast, and shape. Figure 2 depicts the results of the microplastics detection process, with bounding circles indicating the location of the detected objects.

Figure 3. Extracted features with bounding circles.

Figure 3. Extracted features with bounding circles.

Figure 4 depicts a snapshot of the key features, such as keypoints x and y coordinates, size, angle, and 128 descriptors (descriptor 0 to descriptor 127). The feature dataset is stored in a CSV file for further analysis. The descriptors represent the images' unique visual patterns and textures, and they are critical for accurate identification and classification. The descriptors are built to be resistant to changes in scale, rotation, and lighting, ensuring their robustness in identifying microplastics in images.

The dataset was created by loading and preprocessing the input data for a machine learning model. The input images were read from a CSV file, resized to 224 x 224 pixels, and appended to a list of images. Each image's keypoints were extracted from the CSV file and added to a list of keypoints. After that, the images and keypoints were converted to NumPy arrays

5. Discussion

This study has shown that Deep learning techniques can be used to detect microplastics in images. SIFT algorithm has proved to have the capacity to extract features for detecting microplastics in open sewer images that can be used to create a dataset. The study identified the key characteristics of microplastics, such as size variation, colour, structure, and polymer composition, as well as their physical and chemical properties. Physical properties of microplastics was the main interest, which can be easily visualised in pictorial images and used to detect objects using Deep Learning techniques.

In this study, the Scale-Invariant Feature Transform (SIFT) algorithm was used to extract features from images for the purpose of identifying microplastics. The resulting SIFT features can be used to identify the unique visual patterns and textures of microplastics in images by detecting key points that are invariant to scale and rotation, and describing them with a set of noise and occlusion-resistant features. Descriptors are groups of feature vectors that represent key points in an image. The descriptors were created to be insensitive to scale, rotation, and lighting, ensuring the model's robustness in identifying microplastics in images. They are generated by analyzing the local gradients of the pixels surrounding the key point and creating an orientation histogram to capture the dominant gradient direction. A high-dimensional vector capturing the key visual characteristics of the local region surrounding the key point is created by concatenating the histograms from multiple spatial bins around the key point. Because of their resistance to changes in lighting, scale, and rotation, the descriptors are useful for image matching and object recognition tasks.

This method could be applied in environmental monitoring and pollution management, especially in cities with large populations, because it provides a reliable and automated method for identifying microplastics in aquatic environments. The SIFT algorithm for feature extraction combined with deep learning models has the potential to detect and monitor microplastics in the environment.

6. Conclusions and Recommendation

This paper presented a method for detecting microplastics in images taken from open sewers using deep learning techniques. The application of the SIFT algorithm for feature extraction was shown to be effective in capturing distinctive features of microplastics in the images.

Based on the findings of this study, the use of high-resolution cameras to capture photographic images of open sewers flowing into the ocean can be an effective method for detecting microplastics. This study emphasizes the significance of choosing appropriate sampling sites to ensure that the data collected is representative of the population of interest. To gain a more comprehensive understanding of the prevalence and distribution of microplastics in coastal areas, future research should consider a broader range of sampling locations.

The SIFT algorithm for feature extraction proved to be effective in capturing distinctive features of microplastics in the images. The proposed method can be used to extract features from images of microplastics in open sewer systems. The extracted features can be used to build a model for identifying microplastics in images. The model can be used to monitor the presence of microplastics in the environment and help in their control. Further research can be conducted to extract more features using different algorithms and build models for identifying microplastics in images.