Object Detection and Predictive Maintenance in Autonomous Cars in YOLO, A Survey

1. Introduction

Every year, the World Health Organization (WHO) publishes data on the number of people injured in vehicle accidents. Approximately 1.3 million people lose their lives worldwide each year and between 20 and 50 million experience severe injuries, with young men under 25 years of age making up the majority 73%, of fatal car accidents. Thorough investigations into the alarming rate of traffic-related deaths have resulted in advances in Intelligent Transportation Systems (ITS) with enhanced features. The forefront of modern automotive technology is now defined by Automated Driving Systems (ADS) and Autonomous Vehicles (AVs). AVs, particularly self-driving cars, have sparked considerable excitement in the realms of intelligent transportation, computer vision, and artificial intelligence. No prior invention has influenced the automotive sector as significantly as self-driving vehicles. According to projections shown in Figure 1 [1], the self-driving car market is expected to increase from 23.80 million units in 2022 to 80.43 million units in 2032.

Autonomous Vehicles (AVs) can benefit from AI analytics in a number of ways, including increased efficiency and safety [2]. This calls for reliable object detection across a range of settings. Autonomous vehicles’ ability to detect and avoid objects is essential to their growth since it allows them to sense and comprehend their environment. Recognizing and classifying objects is crucial to AV systems’ fundamental decision-making processes. Road safety and navigation are directly impacted by people, bicycles, and road signs. This entails improving object detection in AV contexts by utilizing a variety of Internet of Things (IoT) and artificial intelligence technologies. The evolution of obstacle avoidance in Autonomous Vehicles (AVs) has been significantly impacted by developments in sensor technologies, including as Light Detection and Ranging (LiDAR), radar, and cameras, which offer vital information for identifying and evaluating possible obstructions in the vehicle’s path. These sensors provide vast amounts of data, which are then processed and examined by AI and machine learning algorithms to detect and categorize nearby obstacles. Computer vision plays a key role in obstacle avoidance by allowing it to observe and comprehend its surroundings. Using cameras and other optical sensors, AVs can comprehend and evaluate visual data from their surroundings thanks to computer vision, a branch of artificial intelligence.

Artificial intelligence’s field of computer vision allows AVs to use cameras and other optical sensors to comprehend and evaluate visual data from their surroundings. Computer vision algorithms analyze visual data to identify and detect objects in the route of a vehicle, including bicycles, cars, pedestrians, and other pertinent objects. Finding potential risks and impediments requires this.

The capacity of deep learning-based techniques to extract complex patterns and attributes from huge datasets has made them well-known for object detection. Real-time decision-making algorithms, intelligent control systems, and sensor data processing enable obstacle avoidance. Strategies such as "You Only Look Once" (YOLO) are being employed more and more to improve AVs’ ability to identify and recognize things on the road [3].

1.1. Survey Objective

This article endeavors to examine the object detection techniques which have been adopted in autonomous vehicles. It also examines object detection in adverse weather conditions. It also explores the way in which predictive maintenance can be adopted in object detection. In the last, an object detection frame work is presented comprising of predictive maintenance. Specifically, this review aims to achieve the following objectives:

• Exploration of Object detection Applications in Autonomous vehicles: This involves the investigation of real-world use cases in object detection.
• Exploration of YOLO Applications in Diverse Weather Conditions: We then explore the specific applications of YOLO variants in diverse weather conditions.
• Predictive maintenance: In this survey, we have proposed to imply predictive maintenaince techniques in object detection algorithms. Predictive maintenance has been made extremely difficult by the intricacy and risks of autonomous car systems. Regular maintenance should be carried out to ensure human safety because malfunctioning hardware and software in autonomous vehicle systems could result in fatal collisions. Large-scale product design for automotive systems depends heavily on anticipating future failures and adopting preventative measures to preserve system reliability and security.
• Proposed Methodology for Object Detection: In last the general framework of object detection is mentioned which shows how PM can be adopted in AV.

2. Predictive Maintenance

Maintaining systems is essential to improve product efficiency and continuity. There are different types of system maintenance: reactive, planned, and predictive as depicted in Figure 2 [4].

Reactive maintenance solely addresses system breakdowns or malfunctions. The issue is detected, and the necessary repairs are carried out. Planned maintenance involves periodic inspections and maintenance tasks at predetermined intervals to extend system life and save repair costs, regardless of whether the system has shown failure signs. Predictive Maintenance (PM) uses advanced analytics on sensor data to predict system failures and optimise maintenance intervals, reducing malfunction time and increasing system reliability.

PM has shown significant growth and advancements. Artificial Intelligence (AI) has been adopted in PM. AI based PM systems boost network efficiency and safety. Researchers are motivated to pay attention to AI models and techniques to improve the autonomy and adaptability of complex and dynamic wireless environments [5] [6].

2.1. Predictive Maintenance in Autonomous Vehicles

Autonomous vehicles employ artificial intelligence (AI) to analyse their surroundings and make real-time driving decisions. In order to evaluate their operating environment and make decisions about how to drive in real-time, autonomous cars combine input from several sensors [7]. Object detection is an important component in autonomous driving. Autonomous vehicles need to perceive their environment to drive safely and effectively. This perception uses object detection algorithms to identify nearby items, including pedestrians, automobiles, traffic signs, and barriers. Deep learning-based object detectors play a vital role in finding and localizing these objects.

Missed object detection across subsets of data can have unexpected harms, reducing the reliability, safety, fairness, and trust in machine learning (ML) systems. For example, an image content moderation program may have excellent overall accuracy, but fail to perform well in specific lighting circumstances or environment effects.

The Key Components of Predictive Maintenance are as follows.

• Integration with IoT: Autonomous vehicle (AV) predictive maintenance system architectures rely significantly on Internet of Things (IoT) frameworks. IoT frameworks make it possible to gather data in real time and interpretation of data from a variety of car sensors, including those that track battery health, tire pressure, and engine performance. [9]. Predictive maintenance with IoT integration makes it easier to monitor continuously and identify irregularities early. AV predictive maintenance systems use Internet of Things frameworks to gather information from a variety of sensors mounted all over the car, such as cameras, gyroscopes, and accelerometers. After being sent to a central processing unit, this data is examined using machine learning algorithms to find trends and anticipate any problems. Real-time monitoring and diagnostics are made possible by predictive maintenance systems that use IoT frameworks, allowing for early problem identification and prompt response. In addition to improving the safety and dependability of AVs, this reduces maintenance expenses and downtime.
• Sensor Networks: An Internet of Things (IoT)-based predictive maintenance system’s sensors are usually tiny, wireless devices that are simple to put on equipment and cars. They have several sensing components that are able to identify modifications in the equipment’s working circumstances. Through cellular networks, Bluetooth, Wi-Fi, and other wireless communication protocols, the data gathered by these sensors is sent to a central processing unit (CPU).
• Real-time Monitoring IoT frameworks’ real-time monitoring features are essential for improving the dependability and security of driverless cars. These frameworks allow for the early identification of any issues by continuously gathering and analyzing data from a variety of sensors and vehicle components. IoT frameworks reduce the risks associated with unplanned breakdowns by detecting any deviations from standard operating conditions in real-time and enabling fast intervention and corrective actions.

Figure 3. An illustration of Autonomous Vehicle Equipment [10].

Figure 3. An illustration of Autonomous Vehicle Equipment [10].

2.2. You Look Only Once , YOLO

In recent years, the convergence of computer vision and agriculture has led to significant advancements, ushering in a new era of precision farming and smart agricultural management. A central contributor to this transformation is the development of the You Only Look Once (YOLO) algorithm—a series of object detection models known for their remarkable speed and accuracy.

Initially introduced by Joseph Redmon in 2015, YOLOv1 represented a breakthrough in object detection by enabling real-time processing. Unlike traditional two-stage detectors, YOLO employs a single-stage approach that divides an image into a grid and simultaneously predicts bounding boxes and class probabilities. This novel design significantly improved detection speed without compromising accuracy, laying the foundation for future iterations.

As the YOLO series evolved, each version brought improvements that addressed the shortcomings of earlier models while introducing innovations in network architecture, training techniques, and optimization strategies. These advancements have been especially effective in overcoming challenges such as detecting small or occluded objects and enhancing performance across varied datasets. Collectively, the YOLO family has become a powerful tool in advancing agricultural applications through more accurate and efficient visual recognition systems.

3. Object Detection in Autonomous Vehicles

Real-time object detection, identification, and tracking present significant challenges in the field of computer vision algorithm development. The ability to accurately recognize and identify objects from just one or two image frames is particularly essential for Advanced Driver Assistance Systems (ADAS) in modern vehicles [8]. These systems support drivers in detecting objects that might otherwise go unnoticed due to distractions, unfamiliar shapes, or adverse lighting conditions. Poorly designed algorithms can compromise safety, increasing the risk of accidents, especially in high-speed or congested traffic environments. To address these challenges, ADAS in autonomous vehicles must incorporate robust and adaptable object classification and detection techniques suitable for various driving scenarios. Beyond the automotive sector, object detection plays a key role in areas such as advanced robotics, security and surveillance systems, defense technologies, facial recognition, and space exploration.

In this manuscript, in-depth analysis of object detection techniques have been done in two contexts, first exploration of object detection applications in autonomous vehicles. This involves investigating real-world use cases in object detection. Second exploration of YOLO applications in different weather conditions. Then predictive maintenance has been explored in object detection techniques.

According to [11], object detection and recognition in computer vision typically involve four key stages:

• Object classification within an image,
• Localization of objects,
• Detection of objects
• Image segmentation.

In addition to these steps, self-driving vehicles equipped with ADAS utilize advanced sensor fusion methods to identify and analyze features of both moving and stationary objects. Numerous detection algorithms have been developed based on fuzzy logic, statistical approaches, and neural networks. These methods often involve complex theoretical frameworks and require in-depth understanding, practical implementation, and extensive experimentation.

To train and evaluate these algorithms, large datasets consisting of real-time images and videos are used. These datasets simulate real-world environments and help determine the distance and direction of objects relative to the vehicle. The detection and response mechanisms are integrated by combining feature extraction modules with decision-making or response models.

An essential function of the system includes object counting, which estimates in real-time the number of objects present in a frame. Once the objects are detected, classified, localized, and counted, the algorithm proceeds to track them using the extracted features.

Performing all these tasks in real-time requires high-performance computing hardware. In this study, [11] image segmentation is used to enable accurate object tracking without errors, implemented using MATLAB’s automated toolbox.

The algorithm in [8] was tested using 19 short video clips obtained from the Automated Driving Lab, covering scenarios such as sharp curves, highway ramps, and high-traffic conditions. It demonstrated superior performance in tracking vehicles, pedestrians, and lane markings when compared to the ACF (Aggregate Channel Features) detector. Additionally, the algorithm was evaluated using the Vision Sensor Dataset and during a full road test lasting over one minute. However, in situations involving low-light conditions or extremely congested roads, the algorithm exhibited certain limitations in accurately detecting and tracking objects. These challenges highlight the need for a sensor fusion approach, which combines data from multiple sources—such as vision cameras, radar, and inertial measurement units (IMUs)—to enhance the reliability and effectiveness of the vehicle’s forward collision warning system.

The capacity of deep learning-based techniques to extract complex patterns and attributes from huge datasets has made them well-known for object detection [3,4]. Real-time decision-making algorithms, intelligent control systems, and sensor data processing enable obstacle avoidance. Strategies such as "You Only Look Once"

(YOLO) are being employed more and more to improve AVs’ ability to identify and recognize things on the road [6,7].

Table 1. Comparison of Object Detection Approaches in Autonomous Vehicles.

Paper	Approach	Performance Metrics	Outcome
[19]	Latency was reduced using mobile net and SSD	Latency
[24]	Comprehensive examination of object detection strategies designed for self-driving cars, covering both conventional and deep learning-based strategies as well as new developments.	Adaptibility and reliability	The study concludes by outlining potential future paths, such as the use of edge computing, transformer-based designs, and continuous learning to increase adaptability.
[27]	Sand weather makes it extremely difficult to detect objects because of environmental factors such opacity, low visibility, and shifting lighting. The performance of various activation functions in sandy weather was assessed in using the You Only Look Once (YOLO) version 5 and version 7 architectures.	mean average precision (mAP), recall, and precision	According to their findings, YOLOv5 with the LeakyReLU activation function outperformed other designs in the original DAWN dataset in terms of the documented research findings in sandy conditions, achieving 88% mAP. YOLOv7 with SiLU obtained 94% mAP on the expanded DAWN dataset.
[20]	The study in [20] tackles the problem of accurate object detection in foggy conditions, which is a significant challenge in computer vision. We introduce a new method utilizing a real-world dataset gathered from various foggy weather scenarios, emphasizing different levels of fog density.	a mean average precision (mAP)

Table 1. Comparison of Object Detection Approaches in Autonomous Vehicles.

Paper	Approach	Performance Metrics	Outcome
[19]	Latency was reduced using mobile net and SSD	Latency
[24]	Comprehensive examination of object detection strategies designed for self-driving cars, covering both conventional and deep learning-based strategies as well as new developments.	Adaptibility and reliability	The study concludes by outlining potential future paths, such as the use of edge computing, transformer-based designs, and continuous learning to increase adaptability.
[27]	Sand weather makes it extremely difficult to detect objects because of environmental factors such opacity, low visibility, and shifting lighting. The performance of various activation functions in sandy weather was assessed in using the You Only Look Once (YOLO) version 5 and version 7 architectures.	mean average precision (mAP), recall, and precision	According to their findings, YOLOv5 with the LeakyReLU activation function outperformed other designs in the original DAWN dataset in terms of the documented research findings in sandy conditions, achieving 88% mAP. YOLOv7 with SiLU obtained 94% mAP on the expanded DAWN dataset.
[20]	The study in [20] tackles the problem of accurate object detection in foggy conditions, which is a significant challenge in computer vision. We introduce a new method utilizing a real-world dataset gathered from various foggy weather scenarios, emphasizing different levels of fog density.	a mean average precision (mAP)

Table 2. Comparison of Object Detection Approaches in Autonomous Vehicles

Paper	Approach	Performance Metrics	Outcome
[8]	19 short videos obtained from automated driving lab were tested	Performance compared with Aggregate channel feature detector in terms of reliability and effectiveness	The model outperformed ACF detector. It was recommended to use sensor fusion approach to enhance reliability and effectiveness of collision warning system.
[12]	The object detection accuracy was improved by using novel YOLO algorithm	Accuracy and confidence intervals	Accuracy/precision was improved using mobile net.
[13]	Accuracy in object detection was achieved using Flip-Mosaic algorithm to enhance the network’s perception of small targets in YOLOV5.	Accuracy and false rate detection	Flip-Mosaic data augmentation technique can lower the false detection rate and increase vehicle detection accuracy.
[14]	The traditional YOLO v5 model is enhanced with a prediction extension box that takes into account the coverage area and redundancy of actual targets, ensuring the safety of image perception.	suggested object detection algorithm has been shown to improve the range of detected targets, which considerably improves perception safety throughout the autonomous driving procedure.	Prediction extension box is added to YOLO v5 which improves the range of detected targets which improves the perception safety.
[15]	This study introduces an extensive dataset containing 8,388 annotated images of various vehicles, with a total of 27,766 labels in 11 different categories. The dataset was gathered under various weather conditions in the province of Quebec, Canada. The study demonstrates the effective use of deep neural networks for detecting road-related items in the context of smart cities and communities	Precision, Recall, and mAP.	Improved precision, recall, robustness and mAP.
[16]	offers a thorough review of object detection approaches under challenging weather conditions. It begins by exploring the difficulties caused by different weather effects such as rain, snow, fog, and varying lighting, all of which can greatly degrade detection system performance	classifies current object detection methods into three groups: direct detection models, distributed models that perform image restoration (enhancement) before detection, and end-to-end models that combine image restoration and object detection in a single framework
[17]	This paper introduces the SCOPE (Stacked Classifiers for Object Prediction and Estimation) algorithm, designed to improve object detection capabilities in autonomous vehicles. The primary goal is to create a robust, high-performance model that accurately identifies various objects—including cars, trucks, pedestrians, bicyclists, and traffic lights—while addressing challenges related to detection accuracy and efficiency.	accuracy, precision, and recall.	The SCOPE algorithm demonstrated strong performance on self-driving cars ensuring a reduced false positive rate.

Table 2. Comparison of Object Detection Approaches in Autonomous Vehicles

Paper	Approach	Performance Metrics	Outcome
[8]	19 short videos obtained from automated driving lab were tested	Performance compared with Aggregate channel feature detector in terms of reliability and effectiveness	The model outperformed ACF detector. It was recommended to use sensor fusion approach to enhance reliability and effectiveness of collision warning system.
[12]	The object detection accuracy was improved by using novel YOLO algorithm	Accuracy and confidence intervals	Accuracy/precision was improved using mobile net.
[13]	Accuracy in object detection was achieved using Flip-Mosaic algorithm to enhance the network’s perception of small targets in YOLOV5.	Accuracy and false rate detection	Flip-Mosaic data augmentation technique can lower the false detection rate and increase vehicle detection accuracy.
[14]	The traditional YOLO v5 model is enhanced with a prediction extension box that takes into account the coverage area and redundancy of actual targets, ensuring the safety of image perception.	suggested object detection algorithm has been shown to improve the range of detected targets, which considerably improves perception safety throughout the autonomous driving procedure.	Prediction extension box is added to YOLO v5 which improves the range of detected targets which improves the perception safety.
[15]	This study introduces an extensive dataset containing 8,388 annotated images of various vehicles, with a total of 27,766 labels in 11 different categories. The dataset was gathered under various weather conditions in the province of Quebec, Canada. The study demonstrates the effective use of deep neural networks for detecting road-related items in the context of smart cities and communities	Precision, Recall, and mAP.	Improved precision, recall, robustness and mAP.
[16]	offers a thorough review of object detection approaches under challenging weather conditions. It begins by exploring the difficulties caused by different weather effects such as rain, snow, fog, and varying lighting, all of which can greatly degrade detection system performance	classifies current object detection methods into three groups: direct detection models, distributed models that perform image restoration (enhancement) before detection, and end-to-end models that combine image restoration and object detection in a single framework
[17]	This paper introduces the SCOPE (Stacked Classifiers for Object Prediction and Estimation) algorithm, designed to improve object detection capabilities in autonomous vehicles. The primary goal is to create a robust, high-performance model that accurately identifies various objects—including cars, trucks, pedestrians, bicyclists, and traffic lights—while addressing challenges related to detection accuracy and efficiency.	accuracy, precision, and recall.	The SCOPE algorithm demonstrated strong performance on self-driving cars ensuring a reduced false positive rate.

Table 3. Comparison of Object Detection Approaches in Autonomous Vehicles

Paper	Approach	Performance Metrics	Outcome
[21]	This research paper [21] aims to enhance object detection performance using YOLO versions 5, 7, and 9 in AWCs for autonomous driving. Due to the large number and broad range of hyperparameters, manually finding the best values through trial and error is difficult. To address this, the study applies three optimization algorithms—Gray Wolf Optimizer (GWO), Artificial Rabbit Optimizer (ARO), and Chimpanzee Leader Selection Optimization (CLEO)—to independently fine-tune the hyperparameters of YOLOv5, YOLOv7, and YOLOv9. Results demonstrate that these optimization methods substantially boost the performance of the YOLO models in object detection under AWCs, with improvements of 6.146% overall, 6.277% for YOLOv7 combined with CLEO, and 6.764% for YOLOv9 combined with GWO.	Hyperparameters.	The findings indicate that these optimization techniques significantly enhance the performance of the YOLO models for object detection in AWCs, achieving an overall improvement of 6.146%, a 6.277% enhancement for YOLOv7 when combined with CLEO, and a 6.764% increase for YOLOv9 when paired with GWO.
[18]	This paper [18] focuses on developing robust detection and classification algorithms tailored to the unique challenges found in India, including diverse traffic patterns, unpredictable driving behaviors, and varying weather conditions	Average accuracy and peak accuracy.	Precision and accuracy was improved using YOLO v8.

Table 3. Comparison of Object Detection Approaches in Autonomous Vehicles

Paper	Approach	Performance Metrics	Outcome
[21]	This research paper [21] aims to enhance object detection performance using YOLO versions 5, 7, and 9 in AWCs for autonomous driving. Due to the large number and broad range of hyperparameters, manually finding the best values through trial and error is difficult. To address this, the study applies three optimization algorithms—Gray Wolf Optimizer (GWO), Artificial Rabbit Optimizer (ARO), and Chimpanzee Leader Selection Optimization (CLEO)—to independently fine-tune the hyperparameters of YOLOv5, YOLOv7, and YOLOv9. Results demonstrate that these optimization methods substantially boost the performance of the YOLO models in object detection under AWCs, with improvements of 6.146% overall, 6.277% for YOLOv7 combined with CLEO, and 6.764% for YOLOv9 combined with GWO.	Hyperparameters.	The findings indicate that these optimization techniques significantly enhance the performance of the YOLO models for object detection in AWCs, achieving an overall improvement of 6.146%, a 6.277% enhancement for YOLOv7 when combined with CLEO, and a 6.764% increase for YOLOv9 when paired with GWO.
[18]	This paper [18] focuses on developing robust detection and classification algorithms tailored to the unique challenges found in India, including diverse traffic patterns, unpredictable driving behaviors, and varying weather conditions	Average accuracy and peak accuracy.	Precision and accuracy was improved using YOLO v8.

The main objective of research paper [12] is to enhance object detection accuracy in autonomous vehicles by using the Novel YOLO algorithm compared to the Mobile Net algorithm for image classification. Precision in object recognition was improved by integrating the Novel YOLO and Mobile Net methods. The study involved two groups of 40 participants each, with each group undergoing 20 testing cycles. Statistical analysis was conducted using a 95% confidence interval and an independent sample T-test, with a power of 0.8. The alpha and beta values were set at 0.05 and 0.2, respectively. Sample sizes for both groups were calculated using the Clinical tool, with Group 1 applying the Novel YOLO method and Group 2 utilizing Mobile Net. The dataset was sourced from Kaggle. Results demonstrated a notable accuracy difference, with the Novel YOLO algorithm achieving an impressive accuracy of 88.6040%, surpassing Mobile Net’s 72.5320%. To confirm the significance of this difference, an independent sample test was conducted, yielding a p-value of 0.002. Since this p-value is below the conventional threshold of 0.01 (and 0.05), the difference is statistically significant. These results underscore the superiority of the Novel YOLO algorithm in enhancing object detection precision in autonomous vehicles compared to Mobile Net. The Novel YOLO algorithm significantly improved object detection accuracy, outperforming Mobile Net’s 72.5320% with an impressive 88.6040% accuracy rate.

An enhanced technique for vehicle detection in various traffic situations based on an enhanced YOLO v5 network is suggested in order to lower the false detection rate of vehicle targets brought on by occlusion. The Flip-Mosaic algorithm is used in the suggested technique to improve the network’s ability to perceive small targets. A multi-type vehicle target dataset was created and gathered under various conditions. The dataset was used to train the detection model. According to the experimental findings, the Flip-Mosaic data augmentation technique can lower the false detection rate and increase vehicle detection accuracy [13].

When perceptual results are not always achieved in autonomous driving applications, the lack of interpretability of neural networks can result in unanswered Safety of the Intended Functionality (SOTIF) questions. This research [14] offers an object detection algorithm to improve the accuracy of the perception system’s detection in order to address the safety flaws in the current object detection procedure. Authors have used the well-known YOLO v5 one-stage object detection method as the baseline and assess our suggested model. The traditional YOLO v5 model is enhanced with a prediction extension box that takes into account the coverage area and redundancy of actual targets, ensuring the safety of image perception. The suggested object detection algorithm has been shown to improve the range of detected targets, which considerably improves perception safety throughout the autonomous driving procedure [14].

The main obstacle in the development of autonomous vehicles is latency. The delay between processing camera input data and the machine learning algorithm’s decision to steer and move the vehicle on a safe road is known as the latency. Authors suggested a MobileNet SSD framework to solve these problems by connecting MobileNet and SSD, which makes it enough for real-time applications. The separable convolutions used in the model are stepwise sparable convolutions and spatial separable convolutions. The outcome indicates how effectively our suggested MobileNet SSD model lowers latency and computational expenses.

3.1. Object Detection and Classification Based on Weather Condition

The weather significantly impacts mobility and traffic in various negative ways. On average, precipitation is recorded approximately 11.0% of the time worldwide. Research has clearly demonstrated that when compared to normal weather, rainfall can lead to a 70% increase in the probability of accidents. Additionally, snowfall occurs in 77% of countries around the globe. For instance, US national statistics reveal that yearly, 24% of weather-related vehicle accidents occur on roads that are slippery, snowy, slushy, or icy, while 15% happen during active snowfall or a mix with sleet, underscoring the genuine dangers associated with snowy conditions. Environmental factors such as fog, haze, sandstorms, and intense sunlight significantly reduce visibility, which poses real challenges for drivers.

The rapid advancement of autonomous vehicles necessitates the integration of advanced sensing systems to effectively navigate the various challenges posed by road traffic. Although multiple datasets exist to aid object detection for self-driving cars, it is essential to assess their effectiveness under diverse global weather conditions. To address this need, a new dataset called the Canadian Vehicle Datasets (CVD), along with deep learning models that leverage this data [15]. The CVD consists of street-level videos captured by Thales, Canada, using high-quality cameras mounted on vehicles operating in Quebec. These recordings span both daytime and nighttime and encompass various weather scenarios such as haze, snow, rain, gloom, night, and clear sunny days. From the videos, 10,000 images of vehicles and other road elements were extracted, with 8,388 images annotated, yielding a total of 27,766 labeled objects across 11 distinct categories. We evaluated the YOLOv8 model trained on the existing RoboFlow dataset and compared its performance against a version trained on an expanded dataset combining RoboFlow with CVD. The inclusion of the proposed dataset resulted in improved accuracy metrics, with Precision, Recall, and mAP reaching 73.26%, 72.84%, and 73.47%, respectively. Ultimately, the model trained on this diverse dataset demonstrated increased robustness and offers significant benefits for both autonomous and traditional vehicle applications, making it suitable not only for Canadian conditions but also for other regions with similar climates.

Harsh weather conditions and dynamic traffic scenarios create significant obstacles to accurately capture perception data, hindering advances in object detection technologies. This paper [16] offers a comprehensive review of object detection approaches under challenging weather conditions. It begins by exploring the difficulties caused by different weather effects such as rain, snow, fog, and varying lighting, all of which can greatly degrade detection system performance. Additionally, the study classifies current object detection methods into three groups: direct detection models, distributed models that perform image restoration (enhancement) before detection, and end-to-end models that combine image restoration and object detection in a single framework. A comprehensive evaluation of the advantages and limitations of each category is provided. Lastly, the study highlights the major challenges and potential directions for future research in this field.

The integration of artificial intelligence (AI) into autonomous vehicles has revolutionized object detection, enhancing safety in complex driving environments. However, achieving accurate and efficient real-time object detection remains a critical challenge due to the diversity of objects and the need for rapid decision-making. This paper introduces the SCOPE (Stacked Classifiers for Object Prediction and Estimation) algorithm, designed to improve object detection capabilities in autonomous vehicles. The primary goal is to create a robust, high-performance model that accurately identifies various objects—including cars, trucks, pedestrians, bicyclists, and traffic lights—while addressing challenges related to detection accuracy and efficiency.

Traditional object detection methods often struggle with inconsistent performance in dynamic, real-world conditions. These challenges call for advanced approaches to boost both the speed and accuracy of object classification in autonomous driving contexts. The SCOPE algorithm begins by prepossessing the Self-Driving Cars dataset, resizing images to 224 × 224 pixels, and normalizing the bounding box coordinates. The dataset is then split into training and testing sets, where two base classifiers—MLP and DL4jMLPClassifier—are trained. Their outputs are combined through a Random Forest meta-classifier in a stacked ensemble framework. The model’s effectiveness is evaluated using standard metrics such as accuracy, precision, and recall.

The SCOPE algorithm demonstrated strong performance on the Self-Driving Cars dataset, achieving 94 % accuracy, 91% precision, and 92% recall. Additionally, it exhibited low inference latency and a reduced false positive rate, ensuring reliable and consistent performance for real-time autonomous driving applications [17].

Object detection and classification are vital for the safe and efficient navigation of autonomous vehicles, especially in the complex environments typical of Indian roads. This paper [18] focuses on developing robust detection and classification algorithms tailored to the unique challenges found in India, including diverse traffic patterns, unpredictable driving behaviors, and varying weather conditions. Although considerable advancements have been made in object detection for autonomous systems, many current methods struggle to adapt effectively to the conditions prevalent on Indian roads. To address this, we propose a novel solution based on the YOLOv8 deep learning model, which is lightweight, scalable, and optimized for real-time deployment using onboard cameras. Our experimental evaluation involved real-world scenarios featuring a wide range of weather and traffic situations. Videos captured from different environments were used to test the model’s performance, specifically measuring accuracy and precision across 35 object categories. Results showed a precision score of 0.65 for detecting multiple classes, underscoring the model’s capability to identify a broad spectrum of objects. Additionally, real-time testing achieved an average accuracy above 70% in all scenarios, with peak accuracy reaching 95% under ideal conditions. The evaluation criteria extended beyond standard metrics to include factors particularly relevant to Indian roads, such as poor lighting, occlusions, and erratic traffic behavior. Overall, the proposed approach outperforms existing methods by striking an effective balance between model complexity and detection performance.

The study in [20] tackles the problem of accurate object detection in foggy conditions, which is a significant challenge in computer vision. We introduce a new method utilizing a real-world dataset gathered from various foggy weather scenarios, emphasizing different levels of fog density. By annotating the Real-Time Traffic Surveillance (RTTS) dataset and employing the YOLOv8x model, we thoroughly examine how fog density affects detection accuracy. Our results show that the YOLOv8x model attains a mean average precision (mAP) of 78.6% across different fog densities, surpassing existing state-of-the-art techniques by 4.2% on the enhanced dataset. Furthermore, we demonstrate that increasing dataset diversity substantially improves the model’s ability to detect objects in difficult foggy conditions. This work advances object detection technology specifically designed for foggy environments, with important applications in areas such as autonomous driving and surveillance, enhancing both safety and operational efficiency.

This research paper [21] focuses on improving the performance of object detection by utilizing YOLO versions 5, 7, and 9 in adverse weather conditions (AWCs) for autonomous driving. Although the standard hyperparameter settings are effective for clear images, fine-tuning is essential for AWCs. Given the vast number and diverse range of hyperparameters, determining the optimal values through trial and error is challenging. To tackle this issue, the study employs three optimization algorithms—Gray Wolf Optimizer (GWO), Artificial Rabbit Optimizer (ARO), and Chimpanzee Leader Selection Optimization (CLEO)—to independently refine the hyperparameters of YOLOv5, YOLOv7, and YOLOv9. The findings indicate that these optimization techniques significantly enhance the performance of the YOLO models for object detection in AWCs, achieving an overall improvement of 6.146%, a 6.277% enhancement for YOLOv7 when combined with CLEO, and a 6.764% increase for YOLOv9 when paired with GWO.

This paper [24] presents a comprehensive examination of object detection strategies designed for self-driving cars, covering both conventional and deep learning-based strategies as well as new developments. In order to illustrate their shortcomings in managing intricate real-world situations, we start by looking at traditional methods like Haar cascades and Histogram of Oriented Gradients (HOG). Authors then examine cutting-edge deep learning models, such as Convolutional Neural Networks (CNNs), Region-based CNNs (R-CNNs), You Only Look Once (YOLO), and Single Shot Detectors (SSDs), assessing their precision, speed, and resilience under various driving scenarios. Additionally, the study investigates the use of sensor fusion techniques, which combine information from radar, LiDAR, and cameras to improve detection reliability.

Potential solutions are discussed, along with difficulties including occlusions, bad weather, and real-time processing limitations. We also examine how evaluation criteria, annotation techniques, and dataset quality affect model performance. The study concludes by outlining potential future paths, such as the use of edge computing, transformer-based designs, and continuous learning to increase adaptability. In order to satisfy the changing requirements of autonomous driving systems, this thorough overview attempts to assist researchers and practitioners in choosing and developing object detection techniques.

Autonomous vehicles (AVs) must be able to identify and recognize things in their environment, even in inclement weather, in order to achieve the highest level of automation. Sand weather makes it extremely difficult to detect objects because of environmental factors such opacity, low visibility, and shifting lighting. The performance of various activation functions in sandy weather was assessed in [27] using the You Only Look Once (YOLO) version 5 and version 7 architectures. Their performance was assessed using mean average precision (mAP), recall, and precision. Although authors used only sandy photos, they used the Detection in Adverse Weather Nature (DAWN) dataset, which includes a variety of weather situations. Additionally, utilizing a variety of augmentation techniques, including blur, saturation, brightness, darkness, noise, exposer, hue, and grayscale, we expanded the DAWN dataset and produced an augmented version of it. According to their findings, YOLOv5 with the LeakyReLU activation function outperformed other designs in the original DAWN dataset in terms of the documented research findings in sandy conditions, achieving 88% mAP. YOLOv7 with SiLU obtained 94% mAP on the expanded DAWN dataset that we created [27]. Table 1, Table 2 and Table 3 show the object detection approaches in AV along with performance metrics and outcome of the paper.

4. Analysis on the Current Work

Although enough experimental and theoretical research has been done on object detection techniques. But we need to do object detection and classification in suburban and rural areas. The object detection and classification in AWC has been performed in [21]. To commercialize this we need to work on rural areas data set and train the data in AWC.

Further predictive maintenance should be considered in object detection and classification. The real-time object detection and classification need to be performed very efficiently and accurately. We need to consider the parameters which can cause missed detection or wrong detection in object detection algorithms.

In [8] although authors have recommended using a sensor fusion unit. Predictive maintenance can be adopted to continuously monitor proposed’s system performance as prediction extension box is added in Yolo v5 [14].

Relative humidity also affects system’s reliability [26]. In addition to considering the foggy conditions in [16,20], relative humidity factor must be examined in object detection techniques.

In [27] the performance of YOLO has been adopted in sandy weather conditions using YOLO v5. The prediction box used in [14] in YOLO V5 can be adopted in the proposed model.

5. Proposed Methodology for Object Detection in AV in Generic Way

The methodology of the proposed system is displayed in Figure 4. The proposed system comprises of the following steps.

• Data collection: Data of vehicles will be collected from different car companies using script.
• Data pre-processing: A dataset will be created comprising of different cars.
• Parameters identification: YOLO algorithm will be used to detect objects. Different performance parameters will be identified. The data comprises of different objects (vehicles) will be collected from car companies. Object are considered as vehicles in this work. A script will be implemented to collect data of vehicles. Then You Only Look Once (YOLO) algorithm will be used to detect vehicles and classify them. Afterwards the performance metrics will be identified. Following are the performance metrics which will be considered to evaluate the performance and also train the Long-Short Term Memory (LSTM) network to predict any anomaly behaviour in detection.
  • Intersection over Union (IoU): It provides a numerical value that reflects how closely the model’s prediction matches the actual object position. A higher IoU score indicates a better match between predicted and ground truth bounding boxes, implying superior localisation accuracy. Its mathematical formula is presented below.

    $$IoU={\displaystyle \frac{AoO}{AoU}}$$

    (1)
    In equation (1.1) AoO corresponds to area of overlap and AoU represents area of union.
    In addition, IoU has a wide range of applications. For example, in self-driving cars, it helps with object recognition and tracking, resulting in safer and more efficient driving. Security systems use IoU to recognise and identify objects or individuals in surveillance footage. In medical imaging, it helps to accurately identify anatomical structures and anomalies.
  • Precision, Recall, and F1-Score are fundamental metrics used to assess the performance of object detection models. These metrics provide valuable insights into the model’s ability to identify objects of interest within images. These metrics are based on the following concepts

    (a)

    True Positive TP: This is the case when the model performs perfectly. The IoU is above the threshold.

    (b)

    False Positive FP: The model identifies the object which does not exits. In this case the value of IoU is below threshold.

    (c)

    False Negative FN: The models does not identify the object despite of its presence.

    (d)

    True Negative TN: Not applicable in object detection.

  • Precision: It is used to quantify the accuracy of the positive predictions made be the model. It is calculated using equation below,

    $$Precision={\displaystyle \frac{TP}{TP+FP}}$$

    (2)
  • Recall: Recall evaluates the model’s completeness in detecting objects of interest. A high recall score shows that the model correctly recognises the majority of the relevant objects in the dataset. Equation (1.3) is used to compute it.

    $$Recall={\displaystyle \frac{TP}{TP+FN}}$$

    (3)

5.1. Loss Function

By measuring and balancing three components YOLO make sure that the model accurately predicts the bounding box locations, confidence scores and class probabilities. These three components are: Classification loss, Localisation loss and Confidence loss. These values are balance so that the model performs efficiently.

• Deep learning algorithms: Deep learning techniques are the brain of PM systems. These algorithms are used to analyze data to identify the most important features which can cause possible errors or failures. They learn from historical data to predict equipment failures, anomalies. In this work, after calculating these values, a LSTM network will be trained on these values. The probability of model’s failure will be the output.
• Decision Making:The insights and predictions generated by the LSTM network are processed by decision-making modules. These modules are responsible to determine what maintenance actions are needed. They can recommend preventive or corrective maintenance tasks.

6. Conclusions

In this paper, a detailed analysis on object detection techniques in autonomous vehicles is presented. We also included techniques in adverse weather conditions. According to our analysis, object detection techniques can provide more accurate results by incorporating predictive maintenance. Some of the YOLO versions have incorporated the predictive maintenance toolbox. This paper also covers the aspects of predictive maintenance in context of autonomous cars. Then we have included the proposed framework including predictive maintenance in generic way. In the proposed famework, the performance metrics for evaluating the performance are also shown.