Real-Time Automatic Identification of Plastic Waste Stream for Advanced Waste Sorting Systems

1. Introduction

Despite being in use for over a century, the global production and consumption of plastics are still increasing exponentially [1]. This ultimately contributes to the massive generation of plastic waste, which raises many environmental concerns associated with prolonged degradation time. However, it is possible to significantly reduce the negative environmental impacts by utilizing plastic's recycling potential, allowing for multiple recycling cycles while preserving its original value and functionality [2]. For recycling to be feasible, a crucial preliminary action is the efficient sorting of collected waste [3] to separate different types of plastics for further processing. Despite the availability of numerous advanced technologies designed to support this process[4,5], successful waste sorting remains a complex and demanding task due to unresolved issues related to the inherent uncertainty of the waste stream.

Solid waste stream indicates high heterogeneity and variability in composition [6]. These attributes are consistently identified as key factors contributing to the challenges in optimizing waste sorting processes aimed at maximizing the recovery of recyclable materials [7]. The primary challenge lies in identifying the composition of the stream, which consists of a wide variety of materials. For this reason, numerous studies have been published in recent years reviewing this issue. A summary of such publications highlighting the key conclusions and limitations of existing developments is presented in Table 1.

In [3], more than fifty studies were reviewed to identify the physical enablers, datasets, and machine learning algorithms used for waste identification in indirect separation systems. A broader coverage is included in [8], where almost two hundred papers were analyzed over a 20-year period on the subject of optical sensors and machine learning algorithms for sensor-based material flow characterization. Similarly, [10] reviewed over two hundred studies that used Machine Learning algorithms for municipal solid waste management, including automatic stream identification. In [9], an extensive review was presented addressing the development trajectory, current state, and future opportunities and challenges of utilizing computer vision for waste sorting. In another [11], commonly utilized sensor-based technologies for solid waste sorting, including spectroscopic- and computer vision-based classification, can be found. It should be noted that all these review publications from the past few years highlight similar limitations of existing solutions for automated waste stream identification. The most frequently reported limitations across all studies include testing under controlled laboratory conditions and challenges arising from using small-scale datasets, such as failure to account for waste diversity or insufficient generalization of models. On the one hand, this highlights the necessity of developing large-scale datasets that account for the specific characteristics of waste originating from different regions. On the other hand, laboratory conditions should be adjusted to replicate real-world scenarios strongly.

Automatic waste stream identification requires real-time, rapid detection of multiple objects within a single frame, accounting for variations in size and orientation. In recent years, convolutional neural networks (CNNs) have been widely employed for automated waste detection and classification, with the YOLO series being one of the most commonly utilized frameworks [13]. Table 2 summarizes recent publications on automatic waste identification using YOLO.

In [14] an improved YOLOv5 model was proposed considering the requirements of construction waste i.e., small object detection and inter-object occlusion. Household waste was subjected to automatic detection and classification in [15,19,20], where self-made datasets were used to train and validate YOLO models. Among studies of this type, the issue of detecting river floating waste can also be found [18]. Plastic waste is also the subject of research. However, it is typically limited to only four [16] or two types [17] of plastics. The analysis of the summary presented in Table 2 reveals three significant limitations. As already highlighted by review articles, a recurring constraint in many studies is that model testing is conducted under laboratory conditions. Most studies evaluate the model by partitioning a portion of the utilized dataset (what has been referred to as static validation), where waste remains stationary during testing. Only in [17] was dynamic validation performed, in which waste moved along a conveyor belt. However, the detection was limited to only PET and PET-G classes. Another limitation concerns the omission of contaminants present in the waste stream, which should serve as the background of the prepared samples rather than being the target of detection. This approach was implemented only in [14,20], where access to recordings from a real-world system was available. The final observed limitation is the low level of detail in the classification of plastic waste. Plastic is most commonly treated as a single class or, at best, divided into a maximum of four classes without distinguishing, for instance, different colors of PET, which is crucial from a recycling perspective.

Considering the identified limitations, this paper aims to develop a model for the automatic identification of plastic waste streams that takes into account the requirements of real-world waste sorting facilities, such as the presence of contaminants and the transportation of waste on a conveyor belt with variable speed. Due to the drawbacks of publicly available datasets, the model will incorporate a preparation process of the dataset. Additionally, the model will be based on CNNs, as they have achieved the best results in image classification in recent years. The main contributions of our paper are the following:

• We based the YOLO model training process on a self-build dataset consisting of over 4,000 images of waste, including contamination.
• We have distinguished six classes reflecting the most commonly separated types of plastic, i.e., blue PET, white PET, green PET, HDPE, tetrapak, and, in addition to plastic, aluminum waste.
• We developed a set of recommendations for dataset creation aimed at automatic waste identification.
• We have applied a combination of three YOLO models to the multi-class waste classification issue.

The model validation was conducted in a laboratory environment using a test setup and waste samples designed to closely replicate real-world conditions, with waste moving on a conveyor and being detected in real-time by a camera.

2. Materials and Methods

The considered automatic waste stream identification is part of an ongoing project funded by the National Centre for Research and Development, entitled “Cyber-physical waste stream control system”. The project aims to develop a cyber-physical control system for municipal waste sorting, i.e. a Digital Twin of a waste sorting system. The structure of the Digital Twin consists of three fundamental components: the physical system, its digital replica, and a bidirectional link connecting the two [21]. A detailed description of the indicated elements and the Digital Twin framework for waste sorting systems are presented in [22]. The foundation of the Digital Twin's operation is the automatic collection of data from the real-world system. Automatic identification of waste stream is therefore one of the key components of the data collection and control layer. The scope of this publication can thus be placed in the Digital Twin framework, as shown in Figure 1.

Based on the results of the literature review, waste characterization, and stream monitoring requirements, the automatic identification of the stream was developed using CNN, specifically the YOLOv5 model.

2.1. Automatic Waste Identification Model Based on YOLOv5

The YOLO series are CNN-based frameworks commonly used in object detection and classification, including waste detection. Its three core components include backbone, neck and head. The backbone, a convolutional neural network, encodes image data into multi-scale feature maps. These maps are then refined by the neck, which consists of layers that enhance and integrate feature representations. Finally, the head module utilizes the processed features to predict object bounding boxes and class labels [23]. The general architecture of YOLOv5 is shown in Figure 2.

The automated waste stream identification model proposed in this paper is based on YOLOv5. Due to the complexity of detecting multiple waste classes and based on the results of our preliminary studies, we employed a combination of three YOLO models. Instead of burdening a single model with all the classes, the division into smaller models allows focusing on a smaller set of classes in each case. This approach enables more precise tailoring of the model to the specific characteristics of the waste. Such a strategy significantly improves detection accuracy and prevents situations where a single model might struggle to distinguish very similar classes. The use of three models allows for more accurate differentiation between classes, which is crucial in the context of waste sorting. Initially, the image is processed by two parallel models (1a and 1b). Each of them detects the same four waste classes (PET, HDPE, tetrapak, and Al), but these models are parameterized differently in terms of confidence score so that one focuses on PET, tetrapak, and Al, while the other focuses on the HDPE class. The other classes are ignored in both models. In the case of detecting the PET class, the output from model 1a is further processed by model 2, which assigns one of three colors to this class: blue, white, or green. The architecture of the proposed model is illustrated in Figure 3.

2.2. Dataset Preparation

A large-scale dataset including many plastic waste images of different types should be implemented in detection and classification models. There are several publicly available datasets with waste images, but they are insufficient for advanced sorting systems. Indeed, effective detection and classification of waste moving along the conveyor requires considering factors such as piling of waste, high irregularity of shapes or presence of contaminants that are not subjected to sorting. The development of a new complex dataset is the first major action needed when considering the problem of automatic waste identification. In the case of self-build datasets, the procedure for their creation is typically described only briefly in publications. Since the dataset significantly impacts both the learning efficiency and the subsequent performance of the model in a real-world system, this study presents the sequential steps required for the development of a self-build dataset. The main stages of dataset creation for automatic waste identification are presented in Figure 4.

The first two stages are related to the data acquisition process. The input data for detection and classification models consists of images. Therefore, it is necessary to collect waste and subsequently capture images of it. The collected waste should include not only the target objects (Figure 5a) but also contaminants (Figure 5b) that typically appear in real-world sorting facilities due to errors in source separation. Such errors arise because people often misclassify waste, incorrectly assigning non-plastic items to plastic waste bins. Additionally, it is necessary to collect multiple instances of the same type of waste to account for the high variation in shape, primarily resulting from different levels of compression.

The considered waste classes include blue PET, white PET, green PET, HDPE, Tetrapak, and additionally, aluminum. For each class, waste was collected while ensuring a balanced number of instances across all categories. The collected waste was then photographed in small groups, positioned differently on the conveyor belt—both in the center and near the edges. Additionally, images were captured with overlapping waste items to reflect real-world conditions.

The collected images require preprocessing before being used for model training, which consists of two main steps: augmentation and labeling. Augmentation is optional, however, it is commonly performed to expand the dataset by applying transformations such as rotation or scaling. In contrast, labeling is a mandatory step for training the YOLO model, as it provides the necessary annotations for object detection. In the case of data augmentation for waste moving along a conveyor belt, geometric transformation is a key augmentation technique. In our dataset, we applied rotation. The final dataset, after augmentation, consisted of 2,242 images used in models 1A and 1B. This dataset was further expanded with additional images for model 2, specifically for detecting blue PET, white PET, and green PET, resulting in a total of 4,042 images. Each image underwent a labeling process involving the annotation of the detected classes. The summary of the applied labels is presented in Figure 6.

The final step in preparing the dataset for use in the YOLO model is to split it into two parts: the training and test sets, with the most commonly accepted ratio being 80/20.

As a summary, Table 3 presents a set of recommendations for each stage of the dataset creation process for waste identification.

2.3. Experimental Environment

In advanced sorting systems, waste is transported between separating points (i.e., NIR separators, sorting robots, etc.). Effective sorting requires automatic identification at multiple stages of the sorting process (at the beginning – input stream, at the end – sorting residues, and before subsequent separation points). Regardless of the sorting stage, automatic identification using computer vision requires placing an RGB camera above the conveyor transporting the waste. To best replicate real-world conditions for the final model validation, a test bed representing a fragment of a real sorting system was constructed, namely, a manual waste sorting cabin. The components of the developed test bed include:

• conveyor belt (speed 0-0.5m/s, 1.2x5m),
• RGB camera positioned above the conveyor (4K, 30 fps, 5-50mm zoom lens),
• computer,
• input stream feeder,
• output stream receiver,
• two manual sorting stations (not considered in this paper)
• hyperspectral camera (not in the scope of the study).

The scheme of the test bed and its real appearance are shown in Figure 7 and Figure 8, respectively.

2.4. Waste Sample Preparation for Model Validation

The primary limitation of previously applied approaches in automatic waste identification models is the testing of solutions under laboratory conditions, based on a designated subset of the utilized dataset. The final validation of the model should be conducted under real-world conditions, considering the movement of waste along a conveyor. In the absence of access to a real sorting system, this was substituted by constructing a representative fragment of a real system and using waste samples that reflect the morphology of the Polish plastic waste stream.

In Poland, waste segregation is implemented at the source of generation. This means that before collection, waste is placed in designated containers for paper, glass, mixed waste, and plastics. Metal waste is also disposed of in the plastic waste container; therefore, aluminum cans were included in the developed dataset. The plastic container is also designated for metals; therefore, aluminum cans were included in the developed dataset.

For the final validation of the developed model, a 5kg waste sample reflecting the morphology of Polish municipal waste was prepared based on the Institute of Environmental Protection – National Research Institute (IOS-PIB) report. The morphology of the prepared sample is shown in Figure 9.

The presented figure confirms the significance of considering contaminants in the development of automatic identification models. In the average morphology of the Polish plastic waste stream, contaminants account for more than 50%.

3. Results

3.1. Model Performance Based on Validation Data Separated from the Self-Build Dataset

The proposed automatic waste stream identification model is based on three YOLOv5 models. The performance of each model was evaluated using Precision, Recall, and mean Average Precision (mAP), which are commonly used evaluation metrics for detection problems [27]. In the learning stage of the model, the indicated metrics represent a global perspective without considering each class separately.

Precision indicates the ratio of correct object detections (TP) to the total number of detections (both correct TP and incorrect FP), defined as follows:

$$Precision={\displaystyle \frac{TP}{(TP+FP)}}$$

(1)

Recall represents the ratio of correct object detections to the total number of actual objects, defined as follows:

$$Recall={\displaystyle \frac{TP}{(TP+FN)}}$$

(2)

Mean Average Precision (mAP) denotes the average performance of the detector across all recall values, which is equivalent to the area under the precision-recall curve. It can be mathematically expressed as (4):

$$AP={\int }_0^{1}P\left(R\right)dR,$$

(3)

$$mAP={\displaystyle \frac{\sum _{i=1}^{n}A{P}_i}{n}}$$

(4)

and it measures the performance of an object detection model by averaging the area under the Precision-Recall curve across all classes. In this context, mAP_0.5 represents the average precision computed at a fixed IoU threshold of 0.5, whereas mAP_0.5:0.95 is a more rigorous and comprehensive metric, averaging precision over multiple IoU thresholds from 0.5 to 0.95 in increments of 0.05.

The progression of Precision, Recall, and mAP throughout the YOLOv5 training process for each of the three models used is presented in Figure 10.

The comparison of the final performance metrics for each of the three models is presented in Table 4.

Additionally, Figure 11 presents the confusion matrices generated by the trained YOLO models.

3.2. Model Performance Under Replicated Real-World Conditions

The lack of access to the real system was replaced by its replica in laboratory conditions using the developed test bed (described in Subsection 2.3). The model was tested using waste sample with average Polish morphology (described in Subsection 2.4) moving at a speed of 0,2 m/s on the conveyor. The sample was fed onto the conveyor eight times consecutively. The visualization of the model's performance under conditions replicating a real-world sorting facility is presented in Figure 12.

The final results (model output) were analyzed to determine the Accuracy, Precision, Recall, and F1-score for each detected class. These metrics are calculated based on four groups of results for each class separately:

• TP – True Positives (e.g., HDPE detected, which is actually HDPE),
• FP – False Positives (e.g., HDPE detected, which is actually a different fraction),
• TN – True Negatives (e.g., a fraction other than HDPE detected, which is actually a fraction other than HDPE),
• FN – False Negatives (e.g., HDPE not detected).

The comparison of the final performance metrics for each class is presented in Table 5.

4. Discussion

Primary performance measures of models 1a, 1b and 2 shows that Model 2 is the best overall, achieving the highest recall and mAP values, making it the most accurate and generalizable. Model 1b offers a good balance between precision and recall, slightly outperforming 1a in mAP. The combination of high recall (1a), high precision (1b), and strong classification accuracy (2) suggests an optimized pipeline that balances detection and classification accuracy. Model 1a focuses on detecting Tetrapak, Aluminum, and PET, while Model 1b is trained primarily for HDPE detection. This approach improves the accuracy of HDPE detection, which is often more challenging. Since Model 1b disregards all non-HDPE objects, and Model 1a does not output HDPE, the system minimizes false detections and enhances classification confidence. Model 2 improves the accuracy of sorting PET based on color, which is crucial for recycling.

The confusion matrices indicate that the additional model trained for HDPE successfully eliminated the misclassification of HDPE as PET. It is also evident that Model 1a struggles with detecting HDPE, often confusing it with Tetrapak and aluminum. Additionally, each model occasionally detects background objects due to contamination resembling the target fractions. Therefore, the proposed model effectively detects the assigned waste classes, but a significant challenge remains in misclassifying contaminants as sortable objects. This highlights the importance of introducing contaminants into self-built datasets for automatic waste identification, as real-world waste streams often contain more contaminants than recyclable materials—a situation particularly relevant in countries such as Poland. In further research, a more comprehensive examination of contaminants in the stream is planned.

The final validation eliminated a key limitation present in most models available in the literature, which are typically not tested under real-world conditions. A binary classification accuracy ranging from 0.94 to 0.98 indicates a high level of correct detections among all identified objects. If an object is classified, the probability of assigning it to the wrong class remains low. The achieved precision values, ranging from 0.89 to 1.00, demonstrate high classification accuracy for each waste fraction. Once an object is detected, the model very rarely misclassifies it. The highest probability of such an error, at 0.11, was observed for the Tetrapak fraction. The recall values can also be considered high. The lowest recall was recorded for HDPE (0.76), while for the remaining fractions, recall values ranged between 0.85 and 0.9. These values indicate a low probability of failing to detect an object of a given class. PET Classes (Blue, Green, White) show strong overall performance, with particularly high precision, but there is room to improve recall, especially for green and white PET. Tetrapak and Aluminum have a good balance of precision and recall, with minor improvements possible in precision for Tetrapak. HDPE is the weakest in terms of recall, which could mean that more HDPE objects are not being detected.

Under conditions that replicate real-world scenarios, the results remain within acceptable limits, indicating a high potential for the practical applicability of the developed model in waste sorting systems.

5. Conclusions

In advanced waste sorting systems, the automatic identification of the waste stream is crucial for the efficient separation of materials intended for recycling. It is primarily used by automated and robotic sorting components but also for monitoring the waste stream at various stages of processing, which enables the control of the system. The waste detection and classification model presented in this paper is one of the components of the municipal waste sorting system Digital Twin, which is a part of the ongoing project “Cyber-physical waste stream control system”. The proposed model is based on the combination of three YOLOv5 models focused on classifying elements of the waste stream into the following classes: blue PET, white PET, green PET, HDPE, tetrapak, and aluminum. The model was trained using a self-built dataset, and its performance was validated in a replica of a real-world sorting system fragment.

The next step in the development of the data collection layer of the Digital Twin is the application of a lidar sensor for measuring the volume of transported waste. The final fusion of the RGB camera with the lidar sensor will enable real-time collection of all key characteristics of the waste stream, including morphology, as well as volumetric and mass measurements.