Optimizing Intrusion Detection for IoT: A Systematic Review of Machine Learning & Deep Learning Approaches with Feature Selection & Data Balancing

1. Introduction

The Internet of Things (IoT) refers to a network of physical objects or "things" embedded with sensors, software, and other technologies to connect and exchange data with other devices and systems over the Internet. These "things" can range from ordinary household objects to sophisticated industrial tools. IoT aims to enable these objects to collect and share data, monitor and control physical environments, and perform specific actions through actuators, ultimately improving efficiency, productivity, and decision-making across various domains.

There are 15.9 billion connected IoT devices worldwide as of 2023, accounting for approximately 75% of all internet-connected devices. As illustrated in Figure 1, this number is expected to continue rising significantly in the coming years [1,2].

The scope of the IoT spans across various sectors, with significant contributions from device manufacturing (30%), consumer IoT (24%), healthcare (18%), transportation (12%), and utilities (9%) [3]. According to a market analysis report by Markets and Markets [4], IoT is anticipated to generate revenue of over 650 billion USD by 2026.

The rapid expansion of the IoT presents numerous challenges that impact its potential. A significant challenge is the need for global cooperation and standardization. Diverse and fragmented IoT ecosystems, evident in the different approaches taken by various regions, obstruct the development of a unified global framework. This lack of standardization undermines interoperability, seamless integration, and the security of IoT devices and systems across the world [5].

Additionally, the vast number of devices expected to connect within IoT networks poses substantial scalability and energy efficiency challenges. Managing these devices and their immense data demands robust architectures and protocols. Batteries power many IoT devices, and the impracticality of replacing batteries in large-scale deployments raises concerns about energy sustainability. The absence of unified architecture and standard protocols complicates interoperability and exacerbates security and integration issues [6,7]. Moreover, the impact of IoT on privacy and ethics, including issues like surveillance and data-driven life, calls for the development of comprehensive ethical frameworks and protective technologies to safeguard user rights and data integrity [5,7].

1.1. Threats and Security Measures in IoT

Security and privacy represent some of the most significant and persistent challenges in the IoT ecosystem. As IoT devices expand, the landscape of potential cyber threats broadens, exposing these interconnected systems to various sophisticated cyber-attacks. The number of malware attacks targeting IoT networks rose dramatically, from 32 million incidents in 2018 to 112 million in 2022. During this period, the finance sector experienced a Y-o-Y growth of 252%, retail saw an increase of 159%, and education faced a growth of 146% in cyber incidents, highlighting a pressing need for robust security measures [8,9,10].

The Zscaler 2023 Enterprise IoT and OT Threat Report [11] indicated a staggering 400% increase in attacks in 2023, with manufacturing and education sectors being the prime targets. Geographically, the United States and Mexico were the most affected. The report underscores the predominance of botnets and Distributed Denial of Service (DDoS) attacks, which are becoming increasingly complex and challenging to mitigate.

Major industry reports have focused on emerging security dimensions in response to the escalating threat landscape. For instance, the Globalstar satellite IoT initiative aims to enhance efficiency, reduce costs, and introduce complex security implications due to the expanded attack surface that such enhanced connectivity entails [12]. Furthermore, the integration of Artificial Intelligence (AI) in IoT, as highlighted by Telenor, offers advanced operational intelligence and automation capabilities. However, these advancements also introduce additional vulnerabilities, especially in data privacy and network security [13].

In 2024, IoT Analytics [14] emphasized ongoing concerns about the inadequacy of current security measures to address evolving threats effectively. This was echoed by the Palo Alto Networks 2023 Benchmark Report on IoT Security, which called for more robust, integrated security solutions that can scale with the increasing number of connected devices [15]. A recent Bitdefender study highlighted that specific malware threats have adapted to exploit IoT vulnerabilities, signaling an urgent need for continuous updates to security protocols and practices [16].

To counter these threats, the IoT security strategy has evolved to focus on both cryptographic measures and detection-based approaches. Cryptographic techniques aim to secure data transmission across IoT devices with lightweight, efficient encryption methods suitable for the constrained nature of many IoT devices [17,18]. On the detection front, IDS has increasingly incorporated ML and DL to enhance their effectiveness, providing advanced capabilities to identify and mitigate potential threats dynamically [19,20].

Despite these efforts, the continuous advancement in IoT technology and the corresponding evolution of cyber threats pose ongoing challenges. It is imperative for research and industry practices to keep pace with these developments, ensuring that security solutions not only address current vulnerabilities but are also adaptable to future threats. This proactive approach to IoT security is essential to safeguard the vast networks of interconnected devices that are increasingly integral to personal and professional environments.

1.2. Limitations and Challenges Associated with IoT Security and Attack Taxonomy

Understanding the limitations and challenges associated with IoT systems is crucial for providing adequate security. Traditional network security measures are not directly suitable for the IoT environment due to its distinct characteristics and the complex nature of its network interactions [21,22,23]. The principal challenges include:

• Hardware Limitations: IoT devices often have constrained computational power, memory, and energy resources, limiting complex security measures’ implementation. These inherent limitations make it challenging to deploy robust cryptographic and intrusion detection systems that are resource-intensive.
• Deployment and Scalability: The large-scale deployment of IoT devices complicates the management of security measures. A significant challenge is ensuring consistent security protocols across many devices without centralized control.
• Connectivity: IoT devices typically connect through heterogeneous networks that can be insecure or unreliable. This variability in connectivity exposes IoT systems to increased risks of network-based attacks.
• Heterogeneity: The diversity of IoT devices in terms of operating systems, hardware capabilities, and communication protocols complicates the implementation of uniform security strategies, thus hampering interoperability and seamless integration of security solutions.
• Embedded and Outdated Software: Many IoT devices run on embedded software that may not be regularly updated, leading to vulnerabilities. The lack of regular updates and patches increases the risk of security breaches.
• Big Data: The vast amount of data generated by IoT devices necessitates effective protection measures to ensure privacy and security. Managing the security of such large-scale data without compromising system performance is daunting.
• Dynamic Topology: The dynamic nature of IoT networks, where devices can frequently join and leave, makes it challenging to maintain a stable security infrastructure and manage the integrity of the network.

With these limitations in mind, securing IoT ecosystems becomes increasingly challenging, making them vulnerable targets for various attacks [23]. Understanding the attack taxonomy is crucial for developing effective security strategies. Figure 2 illustrates the different types of attacks typically performed on IoT networks, highlighting the necessity for tailored security approaches that address the diverse threats encountered in such environments.

The IoT attack taxonomy outlined in Figure 2 categorizes the attacks into five main categories: Physical Attacks, Network Attacks, Software Attacks, Authentication and Access Attacks, and Privacy and Confidentiality Breaches. Physical attacks, such as jamming and tampering, often require physical access to devices and can be mitigated through robust physical security measures and tamper-evident designs. Network attacks, including eavesdropping, MITM attacks, and botnets, exploit vulnerabilities in the communication channels and can be addressed through secure communication practices and IDS. Software attacks, such as malware and zero-day exploits, target the software stack of IoT devices and necessitate regular updates, patch management, and IDS. Authentication and access attacks focus on compromising user credentials and session integrity, requiring robust authentication mechanisms and continuous monitoring. Privacy breaches, including data leakage and inference attacks, highlight the need for data encryption, access controls, and privacy-preserving techniques.

The majority of attacks performed on IoT networks are software and network-based attacks [11]. Given that IDSs are built to thwart most of these attacks, it seems sensible to explore the development of lightweight IDS for IoT. This study focuses on driving future research towards building such lightweight IDS for IoT systems using ML and DL algorithms.

1.3. Trends in IDS Research: Focus on IoT and Feature Selection

To underline the significance of these developments, Figure 3 showcases the trends in IDS research related to IoT and feature selection extracted from IEEE publications from 2015 to 2024. This analysis illustrates how the research community increasingly focuses on refining IDS capabilities specifically for IoT environments through advanced methodologies like feature selection.

These trends highlight the rising importance of tailored security solutions for IoT and emphasize the critical role of feature selection in enhancing IDS operational efficiency. Such insights are instrumental in driving future research and technological advancements in IoT security.

1.4. Importance of Dataset Optimization in ML/DL-Based IDS Development

The rapid expansion of IoT has introduced unique challenges in data management, especially in the domain of IDS that utilize advanced ML and DL technologies. These systems depend heavily on comprehensive and accurately curated datasets to perform effectively. A thorough understanding of ML and DL methodologies and their application in cybersecurity is crucial for developing robust IDS. Feature extraction and preparation ensure that only the most relevant and impactful data is processed, reducing computational overhead and enhancing real-time analytical performance. This is particularly crucial in IoT contexts, where devices often have limited processing power and memory [24].

Feature selection plays a pivotal role in the optimization of IDS datasets. Identifying and retaining only the most significant features that influence detection accuracy can significantly reduce the computational burden on IoT devices. This step enhances ML and DL models’ performance and conserves energy—a critical consideration in sustained IoT operations. Efficient feature selection methods have been proven to improve model training times and enhance the generalizability of the IDS [25,26].

Furthermore, the balance of datasets is essential for the efficacy of ML/DL-based IDS. Unbalanced datasets can lead to biased models that fail to generalize well, particularly in detecting less frequent, albeit more harmful, attacks. Techniques such as synthetic data generation, under-sampling, and over-sampling are vital in addressing these imbalances. Such methods help ensure that IDS is well-equipped to recognize and respond to various intrusion attempts, thereby maintaining system integrity across different operational scenarios [27].

In conclusion, the development of lightweight and efficient IDS for IoT, leveraging ML and DL, hinges significantly on the quality of the datasets utilized. Optimizing these datasets through strategic feature selection and ensuring their balance is indispensable. Such efforts are fundamental to enhancing the accuracy and responsiveness of IDS solutions, thereby securing IoT networks against an increasingly sophisticated landscape of cyber threats. This focus on dataset optimization lays the groundwork for the subsequent sections, which delve into specific ML/DL methodologies and the technical implementation of IDS in diverse IoT settings.

1.5. Contributions and Comparative Analysis

This systematic literature review (SLR) offers significant contributions to the field of IoT security, emphasizing the integration of ML and DL in IDS. First, it provides a comprehensive synthesis of the existing research on IoT security threats and various IDS solutions, giving readers a clear overview of current challenges and the state of the art. It underscores the crucial role of dataset optimization, mainly through effective feature selection and data balancing, to enhance the performance of ML/DL algorithms in IDS.

Secondly, the paper thoroughly reviews various ML and DL techniques applicable to IDS, especially in IoT ecosystems, considering the typical constraints such as limited computational power and energy resources. This review does not evaluate these techniques but instead categorizes and discusses their potential based on existing research, highlighting effective strategies for implementing lightweight and efficient IDS solutions.

In addition, this review extensively covers the available datasets most commonly used for training and testing IDS models. A detailed discussion of these datasets highlights their characteristics. This examination is crucial for understanding current IDS approaches’ strengths and limitations and identifying gaps where future research could contribute more robust datasets.

Table 1 compares our systematic review against notable survey papers published from 2021 onwards on IDS. This underscores the comprehensive nature of our approach, highlighting our unique emphasis on both ML and DL techniques, as well as critical aspects like feature selection and data imbalance.

Our review is notable for its rigorous approach. It utilizes a comprehensive methodology that assesses ML and DL applications in IDS and emphasizes the importance of optimized feature selection and data balance. This makes our study particularly invaluable for researchers and practitioners aiming to implement efficient IDS in IoT settings where performance and accuracy are crucial.

1.6. Structure of the Study

The study is meticulously organized to guide the reader through the complexities of applying ML and DL in IDS for IoT environments:

• Section 1: Introduction - Outlines the IoT landscape, security challenges, and the role of ML/DL in addressing these issues.
• Section 2: Research Methodology - Describes the methods used to gather and analyze relevant data systematically.
• Section 3: IDS Architectures and Advancements in IoT - Provides an in-depth exploration of various IDS architectures.
• Section 4: ML/DL Use in IDS - Examines how different ML and DL models enhance detection capabilities.
• Section 5: Optimizing IDS with Feature Selection - Discusses the critical role of feature selection in enhancing IDS efficiency.
• Section 6: Datasets and Data Balancing in IDS - Examins a range of datasets specific to IoT and broader network environments and discusses various techniques to ensure data balance, which is crucial for the effectiveness of IDS models in handling real-world uneven data distributions.
• Section 7: Observations, Challenges, and Future Directions - Reflects on the findings and outlines potential areas for future research.
• Section 8: Conclusion - Summarizes the essential findings and contributions of the study, emphasizing their implications for IoT security.

2. Research Methodology

This SLR investigates the application of ML and DL in IDS, encompassing both general systems and those tailored for the IoT. Spanning research published from 2018 to 2023, the study methodically collects, reviews, and synthesizes findings from a broad range of sources to establish a comprehensive understanding of the field and its evolution towards lightweight IDS solutions for IoT.

An SLR is characterized by a structured and replicable approach that minimizes bias and provides comprehensive coverage of relevant literature. SLRs involve clearly defined questions, predetermined methodology, and specified study eligibility criteria. This process includes comprehensive literature searches, systematic data extraction, and the rigorous evaluation of the quality of each identified study. This structured approach aims to summarize the evidence on a topic and identify and analyze trends and gaps to guide future studies [41]. The methodological rigor ensures the reliability and validity of the review’s conclusions, which is essential for advancing the field of IDS, particularly in the context of IoT.

2.1. Objectives and Research Questions

This subsection outlines the objectives and research questions guiding the systematic literature review. These questions are designed to address critical areas in the development of IDS using ML and DL methodologies, particularly within the IoT domain.

Table 2 presents the formulated research questions and the corresponding objectives. Each research question and objective is strategically designed to build upon the insights gathered from the literature, ensuring a comprehensive understanding of the current landscape and future prospects of IDS in IoT environments. This systematic approach helps pinpoint gaps in the existing research and suggests pathways for future inquiries.

2.2. Research Study Selection

The research study selection process was meticulously structured to ensure the inclusion of relevant and high-quality articles on IDS using ML and DL within the IoT environment. This section details each step of the systematic approach used to identify and evaluate the literature, as visually summarized in the accompanying flowchart (Figure 4).

Stage 1: Formulation of Problem Statement

The problem statement was clearly defined to guide the systematic review, focusing on the application of ML and DL in enhancing IDS in IoT settings. It centers on the vulnerabilities and scalability challenges that IoT environments face due to high-dimensional and imbalanced data, as well as the limitations of existing IDS solutions. The review aims to explore and critically analyze how advancements in ML and DL, especially through innovative feature selection and data balancing techniques, can lead to more efficient and effective IDS frameworks specifically tailored for the IoT context.

Stage 2: Identify the Keywords and Deciding the Duration of the Study

Keywords were carefully selected based on their relevance to the core topics of IDS, IoT, and ML/DL technologies. The primary keywords included those listed in Table 3. Various combinations of these and equivalent keywords were used to ensure comprehensive topic coverage. The duration of the study was set from 2019 to 2024 to focus on the most recent developments in the field.

Stage 3: Identification of Search Engine and Necessary Filters

The search utilized several renowned academic databases with suitable filters to ensure a broad and diverse source of literature, as listed in Table 4.

Stage 4: Perform Preliminary Search and Generate Temporary List of Articles

An initial search was conducted using the specified keywords and filters, resulting in a temporary list of potentially relevant articles.

Stage 5: Extract Information from the Temporary List and Update the Keywords

Information was extracted from the temporary list, and keywords were refined to enhance the specificity and relevance of subsequent searches.

Stage 6: Perform the Initial Search and Generate Initial List of Articles

The refined keywords generated a more focused search and an initial list of articles that closely matched the review’s objectives.

Stage 7: Apply Article Selection Criteria and Generate the Final List

The final stage of the article selection process involved meticulously applying predefined selection criteria to the initial list of articles. Each article underwent a detailed review to ascertain whether it comprehensively described its methodology and presented practical comparisons or empirical data to substantiate its conclusions. Articles that did not meet these stringent standards were excluded from further analysis. This rigorous vetting process ensured the inclusion of only the most relevant and methodologically robust articles in the review.

Ultimately, this process led to the selection of 98 IDS-related articles, of which 71 specifically addressed IDS implementations in IoT environments. The remaining 27 articles explored ML or DL techniques in IDS implementations and discussed the application of either feature selection or data-balancing techniques.

3. IDS Architectures and Advancements in IoT

IDS is an essential tool in cybersecurity, tasked with monitoring network or system activities to detect unauthorized access, misuse, or policy violations. Their primary function is to alert system administrators or security personnel to potential threats, safeguarding data integrity and operational continuity [42]. As network complexities increase, the role of IDS becomes increasingly crucial, particularly in detecting sophisticated cyber threats that continually evolve to exploit network vulnerabilities [43].

The evolution of IDS from traditional rule-based systems, which rely on known signatures of malicious activities, to more sophisticated systems has been significantly influenced by advancements in computational technologies [44]. This shift has been necessitated by the dynamic nature of modern network environments, where static detection methodologies are insufficient. Today’s IDS can incorporate ML and AI to dynamically learn from network behavior and adapt to new, previously unseen threats as they emerge [45].

Modern IDS are designed to be highly adaptive, utilizing advanced computational technologies to handle the increased scalability and diversity of network environments. These systems are particularly vital in settings like the IoT, where the number of connected devices presents unique security challenges. By employing advanced ML and DL techniques, IDS can detect complex patterns and anomalies indicative of new or evolving threats, ensuring robust defense against a broad spectrum of cyber attacks [46,47].

3.1. Types of IDS and Their Applicability to IoT

IDS can be categorized based on their monitoring strategies and deployment as:

Network-based IDS (NIDS)

NIDS monitors network traffic for signs of intrusion and is strategically placed to oversee all traffic within the network. In IoT environments, NIDS must handle diverse devices and high data volumes, making advanced ML algorithms essential for effective traffic analysis and anomaly detection. Recent studies highlight the effectiveness of deep learning models in improving the accuracy and efficiency of NIDS in IoT settings [48].

Host-based IDS (HIDS)

HIDS is installed directly on devices, monitoring all system interactions and network traffic to detect potential threats. This placement particularly benefits devices that manage sensitive data or critical operations. Optimizing HIDS for IoT involves balancing detection capabilities with the device’s computational limits to minimize performance impacts while maintaining robust security [49].

Anomaly-based IDS (AIDS)

AIDS is designed to detect unusual patterns that may signify a threat by comparing activities to a baseline of "normal" network or system behavior. This type of IDS is highly effective in identifying novel attacks that have not been previously encountered or defined in threat databases. The strength of anomaly-based IDS lies in their ability to adapt to network security’s dynamic and evolving landscapes, making them especially suited for IoT environments where device behaviors and interactions can be highly variable and unpredictable. These systems employ a range of machine learning and deep learning techniques to continuously refine their understanding of what is considered normal, improving their accuracy over time [50].

Integrating any of these IDS types into IoT architectures requires careful planning to address IoT environments’ specific security needs and resource constraints. Each type offers unique benefits: NIDS provides broad network coverage, HIDS offers detailed scrutiny directly on the devices, and AIDS identifies unusual activities through behavior analysis. Together, they form a comprehensive defense strategy against various cyber threats, ensuring robust security across the IoT landscape.

3.2. Advances Through ML and DL

The integration of ML and DL has significantly transformed IDS’s capabilities, particularly in addressing the scalability and complexity of IoT environments.

ML in IDS

ML techniques, such as decision tree (DT), Support Vector Machine (SVM), and clustering, have been crucial in improving the anomaly detection capabilities of IDS. DT, for example, effectively categorizes network behaviors, making them suitable for the dynamic nature of IoT networks. SVM, known for its robustness in high-dimensional spaces, is adept at distinguishing between normal and malicious activities with high accuracy. These ML techniques adapt to evolving threat landscapes, continuously improving their predictive accuracy without human intervention [51].

DL in IDS

DL, utilizing architectures like Artificial Neural Networks (ANN), Deep Neural Networks (DNN), and Convolutional Neural Networks (CNN), enhances IDS by detecting complex patterns and anomalies that are difficult for traditional methods to capture. In IoT, where devices generate large volumes of data, neural networks can analyze this data to detect sophisticated attack patterns. The strength of DL techniques like CNN is their ability to learn feature representations from vast amounts of data, automating feature extraction and significantly boosting detection rates. This capability is particularly beneficial in IoT environments, where the diversity of devices and interactions complicates the security landscape [52].

These advancements in ML and DL improve IDS systems’ detection capabilities and ensure they can scale effectively with the expanding IoT infrastructure. The deep integration of these technologies into IDS frameworks enhances their ability to preemptively identify and mitigate threats, thereby fortifying IoT networks against a wide array of cyber threats.

4. ML/DL Use in IDS

Building on the advances discussed in Section 3.2, this section delves deeper into the application of ML and DL in IDS. We will explore the full spectrum of the ML/DL process from data collection to model deployment, detailed in Figure 5, which outlines each critical phase of developing and deploying an ML/DL-based IDS.

Data Ingestion

Data ingestion in IoT IDS involves collecting a broad spectrum of network traffic data, which is crucial for training robust detection systems. This foundational step ensures that the IDS is equipped with comprehensive data reflective of both normal operations and potential security threats.

Data Pre-Processing

Data pre-processing in IoT IDS involves transforming raw data into a structured format that is ready for analysis. This includes using tools like the CICFlowMeter to convert raw network data into analyzable formats [53]. In recent studies, deep learning techniques such as CNN [54,55] and Long Short-term Memory (LSTM) [56,57] are employed for efficient feature extraction. Data balancing is another crucial aspect of this stage, ensuring the model is not biased towards the majority class, which will be discussed more comprehensively in Section 6.

Feature Reduction

Feature reduction streamlines the data further, enhancing model efficiency and focus using techniques like Feature Selection [58] and Principle Component Analysis (PCA) [59]. More details on feature selection methods and their impact on IDS performance will be explored in Section 5. Post-reduction, the dataset is split into training and testing sets.

Algorithm Selection

Choosing a suitable algorithm is pivotal. This decision is based on the nature of the data and the specific requirements of the IDS. The selection process considers factors like data complexity, the volume of data, and the expected types of intrusion behaviors to be detected. Subsections 4.1 and 4.2 have more detailed discussions focusing on the specific ML and DL algorithms used in IDS.

Training

The training phase involves feeding the prepared and reduced training dataset into the selected ML or DL model. Based on the dataset, the model learns to identify and categorize behavior as usual or potentially malicious.

Testing

Post-training, the model is rigorously tested using a separate testing set to evaluate its performance. This phase is critical for assessing the model’s performance and ability to generalize to new data without overfitting the training set.

Parameter Tuning

Parameter tuning is often necessary if the initial testing phase does not yield satisfactory results. This may involve manual adjustments or the application of advanced optimization techniques like the Slime Mold Algorithm (SMA) [60], Modified Red Fox Optimizer (MRFO) [61], Tunicate Swarm Algorithm (TSA) [62], Chaotic Butterfly Optimization Algorithm (CBOA)[63], Genetic Algorithm (GA) [64] and others. The process iterates—training, testing, and tuning—until the model meets the predefined performance criteria.

Deploy Model

Once optimized, the model is deployed within an IDS framework. It operates in real-time, monitors network traffic, filters, and alerts security personnel to potential threats.

The Table 5 summarizes the articles reviewed that discuss various ML and DL techniques used in IDS in IoT environments, providing a snapshot of how these technologies are applied in the field.

4.1. ML Application in IDS

In this subsection, we explore using ML algorithms in IDS, focusing on traditional and ensemble learning approaches. ML techniques are pivotal in IDS because they can efficiently process and classify vast data, identifying patterns that signify potential security threats.

4.1.1. Traditional Machine Learning Algorithms

Traditional ML algorithms are the foundation of many predictive modeling systems in IDS due to their effectiveness and simplicity in handling various data types. The following are some of the key algorithms used:

• Logistic Regression (LR): Often used for binary classification tasks, LR models the probabilities for classification problems, such as distinguishing between normal and malicious activities [97].
• Support Vector Machine (SVM): SVM is robust in high-dimensional spaces, making it suitable for IDS where it constructs a hyperplane in a multidimensional space to separate different classes [70].
• Decision Trees (DT): DTs are popular for their simplicity and interpretability. They use a tree-like model of decisions and their possible consequences [68].
• Naïve Bayes (NB): This algorithm applies Bayes’ Theorem with the assumption of independence between features. It’s particularly effective in IDS for its speed and performance with large datasets [81].
• k-Nearest Neighbors (kNN): A non-parametric method for classifying data based on the closest training examples in the feature space [84].

4.1.2. Ensemble Learning Algorithms

Ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone [114]. Some of the notable ensemble algorithms include:

• Random Forest (RF): An ensemble of Decision Trees, typically trained with the “bagging” method, RF is very effective in IDS due to its ability to reduce overfitting and improve model accuracy [66].
• AdaBoost: Works by combining multiple weak classifiers to form a robust classifier. AdaBoost is used in IDS to enhance classification accuracy by focusing more on complex cases [124].
• Gradient Boosting Machine (GBM): GBM builds an additive model in a forward stage-wise fashion; it allows for the optimization of arbitrary differentiable loss functions, making it versatile for various IDS tasks [75].
• Extreme Gradient Boosting (XGB): Known for its efficiency, flexibility, and portability, XGB delivers high performance and speed when processing large volumes of data [75].
• CatBoost: This algorithm excels in handling categorical features and is less prone to overfitting than other methods [98].

4.1.3. Innovative ML Implementations in IDS:

This section outlines various innovative implementations of ML across different studies, illustrating their contribution to enhancing security capabilities in IDS systems.

• RF in Smart Cities: The Anomaly Detection IoT (AD-IoT) system [66] utilizes the RF algorithm to detect compromised IoT devices within smart city networks. This system is designed for distributed fog networks, aiming to enhance the responsiveness and scalability of attack detection.
• Multi-Algorithm IDS for Smart Homes: Anthi et al. [67] developed a novel three-layer IDS for IoT networks within smart homes. This system employs multiple algorithms such as NB, Bayesian Networks (BN), DT, LR, SVM, and RF to classify device behaviors and detect malicious activities.
• DDoS Attack Detection: Chaudhary & Gupta [68] addressed significant security vulnerabilities associated with DDoS attacks through compromised IoT devices. The authors propose a new ML approach for efficient DDoS traffic detection within local network routers. Upon careful testing and evaluation, RF outperformed LR, SVM, and DT.
• Comprehensive Algorithm Evaluation: A study by Hasan et al. [69] evaluated the effectiveness of various ML algorithms like LR, SVM, DT, and RF in detecting attacks within IoT systems.
• Edge Computing in Smart Homes: The implementation discussed in [128] leverages an ML classifier, specifically a Radial Basis Function (RBF)- SVM, within a smart home system simulation developed on Alibaba ECS, focusing on enhancing network security detection using edge computing technology.
• DoS Protection with ML: Verma & Ranga [75] explored the application of various ML classifiers to protect IoT systems against DoS attacks, assessing their performance on prominent IoT datasets and implementing them on IoT-specific hardware.
• ML Classifier Evaluation Framework: The framework introduced by Shafiq et al. [81] incorporates a hybrid algorithm based on a bijective soft set approach to evaluate and select the most effective ML algorithm from several available options for identifying cyber-attacks in IoT networks.
• Network Intrusion Detection: The study by Kocher and Kumar [84] applied multiple ML classifiers to the recent UNSW-NB15 dataset to evaluate their performance in network intrusion detection.

4.2. DL Application in IDS

DL has significantly enhanced the capabilities of IDS through its ability to perform complex feature extraction and pattern recognition tasks. DL models excel in identifying subtle patterns in high-dimensional data, often indicative of cybersecurity threats.

4.2.1. Traditional DL Algorithms

• Multi-Layer Perceptron (MLP): MLPs are fundamental neural networks with one or more hidden layers between input and output layers. They are effective for pattern recognition tasks due to their ability to learn non-linear decision boundaries [115].
• Artificial Neural Networks (ANN) and Deep Neural Networks (DNN): ANNs are the backbone of many DL approaches [65], with DNNs representing an extension of ANNs that contain multiple hidden layers [95]. These structures are adept at processing complex datasets commonly found in IDS. DNNs, in particular, enhance the capability to capture deeper levels of data abstraction.
• Convolutional Neural Networks (CNN): CNNs are particularly beneficial for feature extraction because they can process data in a grid-like topology, such as images or time series. They use convolutional layers to detect essential features automatically without any human supervision. CNNs are utilized in both one-dimensional (1D) and two-dimensional (2D) forms for analyzing network traffic and log data [79].
• Recurrent Neural Networks (RNN): RNNs are designed to handle sequential data, making them suitable for time-series analysis in IDS [77]. LSTM units [121] and Gated Recurrent Unit (GRU) [126] are enhancements over traditional RNNs, providing solutions to the vanishing gradient problem and improving the memory capacity of the model.
• Autoencoders (AE): AEs are used for unsupervised learning of efficient coding. They learn to compress (encode) the input into a more miniature representation and then reconstruct (decode) the output from this representation. Convolutional Variational Auto Encoders (CVAE) [72] is a type of AE that provides a probabilistic manner for describing observations in latent space.
• Generative Adversarial Networks (GAN): GANs involve two neural networks, a generator and a discriminator, which compete against each other. This structure is highly effective for generating new data samples. Conditional Generative Adversarial Networks (CGANs) extend GANs by adding a condition to the generation process, enhancing their applicability in IDS for generating realistic attack scenarios to test systems [87].
• Deep Belief Networks (DBN): DBNs are generative models that consist of multiple layers of stochastic, latent variables. They are effective in feature extraction and classification tasks [59].

4.2.2. Select Deep Learning Implementations in IDS:

This subsection highlights diverse research efforts that have effectively integrated DL techniques to advance the capabilities of IDS, focusing on feature extraction and comprehensive modeling.

• DDoS Attack Detection Using ANN: Soe et al. [65] developed a detection system tailored for IoT environments to identify DDoS attacks caused by malware like Mirai efficiently. This implementation leverages an ANN.
• Hybrid IDS Using DBN: In the work presented by Yang et al. [59], a hybrid IDS was proposed using DBN combined with a modified density peak clustering algorithm (MDPCA). This system segments the training data into manageable clusters, thereby reducing data imbalance and enhancing the detection of minority class attacks, while DBNs are used for high-level feature extraction.
• Three-Layer IDS for Smart Homes: The research by Latif et al. [64] introduced a novel three-layer IDS designed explicitly for Industrial Internet of Things (IIoT) networks. Using deep transfer learning (DTL) and a tri-layer architecture combining CNN, GA, and bootstrap aggregation.
• Innovative IDS with DNN and Transfer Learning: The study outlined by Qureshi et al. [72] utilized a DNN combined with ASTL. This approach integrates features extracted from a pre-trained network with original data features, enhancing the IDS’s capability to detect network security breaches.
• Hybrid CNN-LSTM Model for IoT Security: Roopak et al. [74] explored a hybrid CNN-LSTM model focused on combating DDoS attacks in IoT networks. This model efficiently handles spatial and temporal data, significantly improving the system’s capability to detect and respond to cybersecurity threats.
• Advanced RNN for Fog Computing: The implementation by Almiani et al. [77] detailed an advanced IDS tailored for the security needs of Fog computing environments essential for IoT. This system employed a multi-layered RNN to detect various cyber threats effectively.

4.3. Performance Measures in IDS

Various metrics are utilized to evaluate the performance of IDS, which employs ML and DL techniques effectively. These metrics help quantify the effectiveness of the IDS in identifying legitimate threats while minimizing false alarms. Below, we explore these measures in detail.

4.3.1. Confusion Matrix

The confusion matrix not only gives a count of correct and incorrect classifications, but it also provides a clear visual representation of the model’s performance, especially in terms of distinguishing between classes. Here’s a deeper look at each element and its significance:

• True Positives (TP): Correctly predicted positive observations.
• True Negatives (TN): Correctly predicted negative observations.
• False Positives (FP): Incorrectly predicted as positive, also known as Type I error.
• False Negatives (FN): Incorrectly predicted as negative, also known as Type II error.

$$\begin{array}{ccc}& \mathbf{Predicted}\mathbf{Positive}& \mathbf{Predicted}\mathbf{Negative}\\ \mathbf{Actual}\mathbf{Positive}& \mathrm{TP}& \mathrm{FN}\\ \mathbf{Actual}\mathbf{Negative}& \mathrm{FP}& \mathrm{TN}\\ \end{array}$$

4.3.2. Key Performance Metrics

Accuracy: Represents the ratio of correctly predicted observations to the total observations.

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

Precision (Positive Predictive Value): Indicates the proportion of identifications that were actually correct.

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

Recall/Sensitivity (True Positive Rate): Reflects the proportion of actual positives correctly identified.

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

Specificity (True Negative Rate): Shows the proportion of actual negatives that were correctly identified.

$$\mathrm{Specificity}={\displaystyle \frac{TN}{TN+FP}}$$

F1-Score: Combines precision and recall into a single metric using their harmonic mean, useful for unbalanced classes.

$$\mathrm{F}1-\mathrm{Score}=2\times {\displaystyle \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

Area Under the ROC Curve (AUC): The Receiver Operating Characteristic (ROC) curve is a graphical representation of a classifier’s performance. As shown in Figure 6 it plots the True Positive Rate (TPR) against the False Positive Rate (FPR) at various threshold settings. The AUC provides a single scalar value to summarize the classifier’s ability to discriminate between positive and negative classes. A higher AUC value indicates better classifier performance. The plot in Figure 6 illustrate the ROC curves for different classifiers as follows:

• Perfect Classifier: This classifier reaches the top left corner of the plot, indicating a TPR of 1 with a FPR of 0. It has an AUC of 1.00.
• High-Performance Classifier: This classifier performs well with a high TPR and a low FPR, achieving an AUC of 0.88.
• Low-Performance Classifier: This classifier performs modestly, with an AUC of 0.65. It shows a higher FPR for a given TPR than the high-performance classifier.
• Random Classifier: This classifier performs no better than random guessing, resulting in an AUC of 0.50, represented by a diagonal line from (0,0) to (1,1).

The ROC curve and AUC are essential tools for evaluating the performance of binary classifiers in various applications, including medical diagnosis, fraud detection, and machine learning model assessment.

Detection Rate (DR): Similar to recall, indicates how well the system identifies positive instances.

$$\mathrm{DR}={\displaystyle \frac{TP}{TP+FN}}$$

False Alarm Rate (FAR): Measures the probability of falsely classifying a negative observation as positive.

$$\mathrm{False}\mathrm{Alarm}\mathrm{Rate}={\displaystyle \frac{FP}{FP+TN}}$$

Matthews Correlation Coefficient (MCC): Provides a balanced measure that considers both the size of positive and negative elements in the dataset.

$$\mathrm{MCC}={\displaystyle \frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}$$

Kappa: Assesses the agreement between observed and expected accuracy, providing insight into the accuracy obtained by chance.

$$\mathrm{Kappa}={\displaystyle \frac{P_o-P_e}{1-P_e}}$$

where $P_o$ is the observed agreement, and $P_e$ is the expected agreement by chance.

These metrics provide a comprehensive toolkit for analyzing the performance of IDS models, offering insights into their strengths and weaknesses in real-world scenarios.

To provide a comprehensive overview, a detailed summary located in Appendix 1 extends beyond the discussions in Section 4. It presents an in-depth study of various implementations of ML and DL algorithms for IDS, collating research efforts from IoT ecosystems and traditional network environments. This summary illustrates the wide-ranging applications and effectiveness of these advanced computational techniques in enhancing the capabilities of IDSs. It serves as a valuable resource for understanding diverse methodologies and their impacts across different security contexts.

5. Optimizing IDS with Feature Selection

As discussed in Section 1.4 on the importance of dataset optimization for ML/DL-based IDS development, feature selection emerges as a critical strategy for creating lightweight IDS for IoT. By eliminating unnecessary features in attack classification, feature selection enhances the IDS’s attack detection capabilities and significantly reduces the computational load on IoT devices. This step is crucial in improving the performance of ML and DL models and conserving energy, which is vital for the sustainable operation of IoT systems.

This section will delve deeper into various feature selection techniques and their application in IDS, mainly focusing on how they contribute to developing lightweight and efficient IDS solutions for IoT environments. We will explore different approaches to feature selection, including filter, wrapper, and embedded methods, each offering unique advantages and suited for various scenarios within the IDS framework. q

5.1. Feature Selection & Types

Feature selection is a critical process in machine learning that involves selecting a subset of relevant features for model construction. It helps enhance the performance of machine learning models by eliminating irrelevant or redundant data, reducing dimensionality, and speeding up the learning process [129]. Feature selection techniques were broadly classified into three categories.

• Filter Methods: These methods apply a statistical measure to assign a scoring to each feature. The measure ranks the features and is either selected to keep or removed from the dataset. Filter methods are generally faster and less computationally expensive than other feature selection methods because they do not involve training models. A common filter method is the correlation-based feature selection, which measures the correlation between each feature and the target variable [25].
• Wrapper Methods: Wrapper methods consider selecting a set of features as a search problem, where different combinations are prepared, evaluated, and compared to other combinations. A predictive model evaluates a combination of features and assigns a score based on model accuracy. Wrapper methods can be very computationally intensive and usually involve training a new model for each feature (or combination of features) added or removed [25].
• Embedded Methods: Embedded methods perform feature selection during the model training process and are specific to given learning algorithms. These methods integrate feature selection as part of the training process and are more efficient than wrapper methods since they include feature selection as part of the model construction process. Techniques like Lasso and Ridge regression are embedded methods that regularize coefficients to zero to reduce the number of features [130].

5.2. General Approaches for Feature Selection in IDS

5.2.1. Filter-Based Feature Selection in IDS

Filter-based methods assess the importance of features using statistical tests. These methods are favored for their efficiency and are especially suitable for high-dimensional data in IDS. Here’s a detailed look at several statistical measures used in filter-based feature selection, alongside their practical implementations in the field of IDS:

Correlation:

Measures the linear relationship between two variables, focusing on how closely changes in one variable are associated with changes in another. In IDS, correlation is used to select features that strongly relate to the detection of threats. For instance, the research by Woo et al. [131] and Dat-Thinh et al.[92] utilize correlation-based filters to streamline feature sets for enhanced intrusion detection accuracy.

$$\mathrm{Correlation}={\displaystyle \frac{\sum (x-\overline{x})(y-\overline{y})}{\sqrt{\sum {(x-\overline{x})}^2\sum {(y-\overline{y})}^2}}}$$

Information Gain (IG):

Assesses how much information a feature provides about the class, calculating the reduction in entropy. IG has been effectively applied in IDS to filter out less informative features, enhancing model performance as shown in studies by Kasongo & Sun [132] and by Wirawan et al.[133].

$$\mathrm{IG}(T,f)=H\left(T\right)-H\left(T\right|f)$$

where $H\left(T\right)$ is the entropy of the target variable, and $H\left(T\right|f)$ is the conditional entropy of the target given feature f.

Mutual Information (MI):

A measure that captures the amount of information obtained about one random variable through another. It is instrumental in IDS for identifying features that share MI with the class attribute. The implementation of MI in IDS is evident in works like [134] and [58], where it helps select features that contribute significantly to classification accuracy.

$$\mathrm{MI}(X,Y)=\sum _{y\in Y}\sum _{x\in X}p(x,y)log\left({\displaystyle \frac{p(x,y)}{p\left(x\right)p\left(y\right)}}\right)$$

where $p(x,y)$ is the joint probability distribution function of X and Y, and $p\left(x\right)$ and $p\left(y\right)$ are the marginal probability distribution functions of X and Y, respectively.

Chi-Square:

This test measures the lack of independence between a feature and the class. It is widely used in IDS to determine which features are statistically significant to the class outcomes. Chi-Square tests are highlighted in the works of Kocher & Kumar [84] and Gad et al. [109], helping to identify features strongly associated with the presence or absence of intrusions.

$${\chi }^2=\sum {\displaystyle \frac{{(O_i-E_i)}^2}{E_i}}$$

where $O_i$ is the observed frequency, and $E_i$ is the expected frequency under the null hypothesis.

Gain Ratio:

An extension of IG that normalizes its values to reduce bias towards multi-valued features. This method is particularly effective in environments where features vary in the number of levels. Studies by Kumar & Gupta [85] and Albulayhi et al. [88] have employed the Gain Ratio to ensure a balanced consideration of feature importance across different feature types.

$$\mathrm{Gain}\mathrm{Ratio}={\displaystyle \frac{\mathrm{IG}(T,f)}{H\left(f\right)}}$$

where $H\left(f\right)$ is the entropy of feature f.

5.2.2. Wrapper and Embedded Feature Selection in IDS

Wrapper and embedded methods represent sophisticated approaches to feature selection in IDS. Wrapper methods evaluate feature subsets using the performance of predictive models, while embedded methods incorporate feature selection directly into the learning algorithm.

5.2.2.1. Wrapper Methods in IDS

Wrapper methods assess feature subsets based on the predictive accuracy of a model. Although computationally intensive, they often yield higher-performing feature sets than filter methods. These methods utilize a specific ML algorithm to evaluate the effectiveness of feature combinations iteratively:

• SVM-Based Wrapper: A study by Taher et al. [135] uses an SVM to evaluate feature subsets, aiming to maximize classification accuracy directly.
• Extra Tree (ET) Wrapper: The ET classifier serves as a wrapper in the study by Kasongo & Sun [136], assessing feature sets and leveraging its ensemble nature to evaluate feature importance effectively.
• Non-dominated Sorting Genetic Algorithm (NSGA) with LR: Khammassi & Krichen [137] combined the NSGA with LR in a wrapper approach to optimize feature combinations, balancing IDS complexity and performance.
• Rf Wrapper: Research by Kumar & Gupta [85] employs RF to evaluate features, utilizing its inherent feature importance measures to guide the selection process.
• Multiple Classifier Wrappers: The work by Rahman et al. [86] uses various classifiers, including SVM, DT, and NB, within a wrapper framework to identify the most compelling features for detecting network intrusions.

5.2.2.2. Embedded Methods in IDS

Embedded methods integrate the feature selection process within the training phase of the model, making them particularly efficient as they include feature selection as part of the algorithm’s objective function:

• DT Embedded Method: DT inherently evaluates feature importance during model training, effectively reducing the feature space without separate validation, as shown in the work of Sarker et al. [138].
• DL Embedding: Yu & Bian [139] employed CNN and DNN to automatically select features through their training process, adjusting weights in network layers that correspond directly to feature relevance.
• RF Regressor Embedded Method: As detailed in the work by Malathi and Padmaja [53], embedding feature selection within an RF Regressor allows for simultaneous learning and feature evaluation, enhancing the model’s focus on the most predictive features.

5.3. Nature-Inspired Optimization Algorithms for Feature Selection in IDS

Nature-inspired optimization algorithms mimic biological processes and natural phenomena to solve complex optimization problems. These algorithms are particularly effective for feature selection in IDS, optimizing the feature set to enhance detection accuracy while reducing computational complexity.

5.3.1. Filter Implementation in IDS

In the filter approach, nature-inspired algorithms assess feature relevance independently of any learning algorithm, focusing on the data’s intrinsic properties.

• Reptile Search Algorithm (RSA): The authors Dahou et al. [56] utilized RSA for its efficiency in exploring and exploiting the search space, optimizing feature selection by mimicking reptilian motion, which significantly improves IDS performance.
• Spider Monkey Optimization (SMO): The study by Otoum et al. [96] employs SMO, inspired by the foraging behavior of spider monkeys, to optimize the selection of features, thereby enhancing the detection capabilities of IDS systems.
• Particle Swarm Optimization (PSO): PSO’s mechanism of social information sharing is utilized to guide the search towards optimal feature sets, significantly improving computational efficiency and detection rates in IDS [124].
• Improved Dynamic Sticky Binary PSO (IDSBPSO): This variant [106] introduces dynamic adjustments to particles’ behavior, sticking to promising areas of the search space to find the best feature subsets.
• African Vulture Optimization (AVO): Implemented by Alsirhani et al. [140], AVO draws inspiration from the scavenging behavior of vultures to efficiently scan and select optimal feature sets for IDS.

5.3.2. Wrapper Implementation in IDS

Wrapper methods utilize nature-inspired algorithms to select features based on the performance improvements they offer to a specific predictive model used within the IDS.

• Discrete Variant of the Cuttlefish Algorithm (DF-CFA): Al-Daweri et al. [141] apply this algorithm, which mimics the adaptive coloration of cuttlefish, to optimize the feature space and enhance model accuracy specifically tailored for intrusion detection.
• Firefly Optimization (FFO): Saraeian & Golchi [142] and Karthikeyan et al. [122] have implemented FFO algorithm for feature selection. It simulates the behavior of fireflies, which is particularly effective in finding global optima in the search space, thereby selecting the most relevant features for IDS.
• Genetic Algorithm (GA): GA’s evolutionary strategies are leveraged by Zhang et al. [143] to evaluate and select features that maximize the performance of the IDS.
• Flamingo Search Algorithm (FSA): This algorithm optimizes feature selection by mimicking the foraging behavior of flamingos, focusing on efficiently identifying relevant features to improve detection rates [60].
• Modified Equilibrium Optimization Algorithm (MEOA): This algorithm enhances traditional equilibrium optimization by introducing modifications that adapt better to the IDS’s requirements, selecting features that significantly boost system performance [62].
• Modified FFO: The work by Almuqren et al. [63] enhances the standard FFO algorithm to suit the complex feature spaces in IDS better, improving both the efficiency and accuracy of the intrusion detection system.
• Chaotic Binary Pelican Optimization Algorithm (CBPOA): As used by Alrowais et al. [111], this algorithm introduces a variation of POA using chaotic sequences, enhancing the exploration capabilities and ensuring a more diverse search for optimal features in IDS.

6. Datasets and Data Balancing in IDS

In IDS, the quality and balance of datasets are crucial for training effective ML/DL models. Data balancing addresses the common class imbalance issue within IDS datasets, where specific network threats or behaviors are significantly underrepresented. This imbalance can lead to biased models that perform well on overrepresented classes but poorly on underrepresented ones, ultimately compromising the IDS’s ability to detect and respond to various threats accurately.

This section explores the datasets used in IDS and the importance of balancing these datasets. Effective data balancing ensures that the trained IDS models can generalize well across all types of network behavior and attacks, improving the sensitivity and specificity of threat detection. Subsequent discussions will delve into various techniques implemented to achieve data balance, which is vital for enhancing IDS’ overall performance and reliability in real-world scenarios.

6.1. IDS Datasets

Table 6 lists 9 IoT-specific and 12 general IDS datasets used in research, crucial for training and testing IDS solutions across various network scenarios.

6.1.1. IoT IDS Datasets

6.1.1.1. CICIoT2023 Dataset

The CICIoT2023 dataset ([144]) is a comprehensive collection designed to reflect the complexities of IoT environments and facilitate advanced IDS testing. This dataset encompasses a variety of attack types, making it particularly valuable for evaluating IDS capabilities across different IoT scenarios. Notable for its diversity, CICIoT2023 includes numerous categories of attacks, from common DDoS attacks to sophisticated IoT-specific threats like Mirai and Bashlite. As visualized in Figure 7, the distribution of attack types highlights a predominant focus on DDoS attacks across various protocols and techniques, ranging from SYN floods to UDP-based attacks. This array provides researchers with a realistic spectrum of challenges, simulating a variety of intrusion attempts that an IoT network might face in real-world deployments. The dataset’s detailed classification of attack types aids in fine-tuning IDS algorithms to detect subtle nuances between different malicious activities and benign traffic, ensuring a robust defense mechanism for IoT networks.

6.1.1.2. Edge-IIoT Dataset

The Edge-IIoT Dataset dataset, compiled in 2022, is tailored for intrusion detection within edge computing environments integral to IIoT architectures. The data composition, shown in Figure 8, illustrates a balanced mixture of attack types and normal activities, crucial for developing robust IDS models. This dataset includes a diverse range of attack types, such as DDoS over different protocols (UDP, ICMP), ransomware, SQL injections, and less frequent but critical threats like XSS and MITM attacks. Notably, the dataset’s variety supports the development of models that recognize and differentiate between nuanced attack vectors and normal network behavior, which is essential for maintaining security in dynamic edge computing scenarios.

6.1.1.3. WUSTL-IIoT-2021 Dataset

The WUSTL-IIoT-2021 dataset focuses on the security of IIoT environments and is designed to address complex security challenges in these settings. It features a rich collection of attack scenarios relevant to industrial networks, such as Mirai and Bashlite botnet attacks, demonstrating how compromised IoT devices can be used to conduct massive DDoS attacks. The dataset provides a diverse array of attack types, including various forms of DDoS, reconnaissance, and infiltration activities, offering researchers a comprehensive resource to develop and validate IDS solutions for the IIoT. The attack distribution within the dataset, illustrated in Figure 9, highlights the prevalence of DDoS attacks, underscoring their significance in IIoT security challenges.

6.1.1.4. IoTID20 Dataset

The IoTID20 dataset, developed in 2020, explicitly addresses security in IoT networks and provides a detailed view of various attack vectors prevalent in IoT environments. This dataset is highly regarded for its comprehensive collection of IoT-specific attack types, including DDoS, command injection, and several others tailored to IoT scenarios. The distribution of attack types in the IoTID20 dataset, as shown in Figure 10, illustrates various attack categories, highlighting the dataset’s utility in training and testing IDS designed to safeguard IoT ecosystems. This extensive range of attacks helps fine-tune IDS capabilities to effectively detect and mitigate IoT-specific threats.

6.1.1.5. MQTT-IoT-IDS2020 Dataset

The MQTT-IoT-IDS2020 dataset, designed to mirror the behavior of IoT networks communicating via the MQTT protocol, is characterized by a prominent imbalance in data classes, showcasing a high prevalence of normal activities followed by aggressive scans, UDP scans, and SSH brute-force attacks, among others. The dataset is a fundamental resource for developing and testing IDS solutions that focus on typical IoT-specific threats, such as MQTT brute force attacks, reflecting highly relevant scenarios in real-world applications. This dataset’s distribution of attack types, which ranges from the most frequent normal activities to several forms of network probing and intrusion attempts, highlights the variety of attack vectors that an effective IoT IDS must handle. This broad spectrum of attack types is visualized in Figure 11, visually representing the dataset’s composition and anomaly detection and classification challenges.

6.1.1.6. TON-IoT Dataset

The TON-IoT dataset, created in 2020, is a comprehensive data source for testing IDS, specifically within IoT and IIoT environments. This dataset captures a variety of network traffic scenarios, including both normal activities and a spectrum of attack types such as Backdoor, DDoS, Injection, Normal activities, Password, Ransomware, Scanning, and XSS attacks. The bar plot in Figure 12 displays the distribution of these activities, showing a significant prevalence of normal traffic followed by DDoS and Injection attacks, highlighting the dataset’s versatility in representing realistic network behaviors for rigorous IDS testing.

6.1.1.7. IoT-23 Dataset Overview

The IoT-23 dataset features a rich collection of IoT-specific attack scenarios and benign samples, making it a critical tool for developing intrusion detection systems tailored to the IoT environment. The dataset primarily consists of labeled network flows, capturing various attack types from various IoT devices. Notable for its volume and variety, IoT-23 encompasses common threats like Mirai and Bashlite alongside less frequent but equally concerning attacks such as partOfAHorizontalPortScan and Okiru. The distribution of these attack types, as illustrated in Figure 13, reveals the predominant focus on probing and malware dissemination, which are pivotal for developing robust detection algorithms. The varied nature of the dataset helps in testing the resilience and effectiveness of IDS solutions across different attack vectors and intensities, providing a comprehensive platform for empirical research and development in IoT security.

6.1.1.8. BOT-IoT Dataset Overview

The BoT-IoT dataset, created by Koroniotis et al., is a comprehensive dataset for IoT and network security research, extensively used to evaluate IDS tailored for IoT networks. It notably includes various simulated IoT environments to model attacks, such as DDoS, DoS, reconnaissance, and more exotic threats like theft and backdoor attacks. The distribution of attack types in the BoT-IoT dataset, as shown in Figure 14, illustrates a significant prevalence of DDoS attacks, followed by DoS, which highlights the dataset’s focus on these types of network disruptions that are critically impactful in IoT contexts. The dataset also includes examples of reconnaissance and theft, providing a nuanced view into less frequent but equally critical threats and enabling comprehensive testing of IDS capabilities.

6.1.1.9. N-BAIOT Dataset Overview

The N-BAIOT dataset is specifically designed to address the intricacies of network traffic within IoT environments. It features various attacks, including Mirai UDP, Mirai SYN, Mirai ACK, and several Gafgyt attack types, such as UDP, TCP, and Combo, reflecting a broad spectrum of attack vectors pertinent to IoT security. This diversity aids in developing robust detection models that can handle various intrusion scenarios prevalent in IoT settings. The dataset’s comprehensive nature is illustrated in Figure 15, which details the distribution of attack types, offering a visual breakdown of normal versus malicious activities and showcasing the dataset’s suitability for testing anomaly detection algorithms.

6.1.2. Regular IDS Datasets

The range of regular IDS datasets developed over the years is critical in advancing intrusion detection research. Beginning with the well-recognized KDDCup99 dataset [164] from 1999, which has been fundamental in shaping initial intrusion detection systems, the evolution of datasets has seen significant refinement and diversification. The KDDCup99 dataset, despite its age, continues to be a benchmark for new detection algorithms due to its comprehensive set of features and attack types.

Progressing to more contemporary datasets, the CSE-CICDIS2018 [152], Kitsune Network Attack Dataset [154], and CIC-IDS2017 [158] provide more affluent, more modern environments that reflect current network configurations and advanced attack scenarios. For instance, the CSE-CICDIS2018 and CIC-IDS2017 datasets include a variety of attack simulations that help researchers develop and evaluate the efficacy of their IDS models against a spectrum of sophisticated threats. These datasets, enhanced by detailed traffic profiles and diverse attack representations, support the development of systems capable of distinguishing between benign and malicious activities with higher accuracy.

Lastly, the transition into more specialized datasets such as CTU-13 [160] and ISCXIDS2012 [161], which include botnet traffic and more granular attack details, underscores the progression towards datasets that not only challenge the robustness of intrusion detection models but also push the boundaries of what these systems can recognize and mitigate in evolving network environments.

6.2. Balancing IDS Datasets

Balancing techniques in IDS datasets is crucial for addressing the skewed distribution of attack classes as shown by almost all datasets in section 6.1, which can heavily influence the performance of IDS. Various approaches have been developed to ensure balanced datasets, enhancing the predictive accuracy and reliability of IDS models. These techniques are generally categorized into resampling techniques, algorithmic approaches, and data augmentation methods.

6.2.1. Resampling Techniques

Resampling techniques adjust the class distribution in a dataset by either undersampling the majority class or oversampling the minority class. A popular method for oversampling is the Synthetic Minority Over-sampling Technique (SMOTE), which has been applied to various datasets like BoT-IoT [65], ToN-IoT [109], and several others, including UNSW BoT IoT [116] and NSL-KDD [113]. These studies have demonstrated that SMOTE can significantly improve models’ performance metrics by creating synthetic examples rather than over-replicating existing ones.

For undersampling, Random Under Sampling for the Majority Class has been used in diverse datasets like IoTID20 and CIC-IDS-2017 to reduce the number of majority class samples, thereby balancing the dataset more effectively [92]. Hybrid techniques combining both oversampling and undersampling, such as those described in studies using NSL-KDD and CICIDS2017 datasets [165], aim to optimize the balance between class distributions.

6.2.2. Algorithmic Approaches

Algorithmic balancing involves modifying existing data clustering algorithms to manage imbalanced datasets better. The MDPCA utilized in conjunction with datasets like NSL-KDD and UNSW-NB15 [59] enhances the discrimination power of the algorithm by focusing on the density-based clustering of minority class examples. This method helps identify core samples of minority classes crucial for training robust detection models.

6.2.3. Data Augmentation

Data augmentation involves artificially increasing the size and diversity of training datasets by creating modified versions of existing data points. GANs have been used for data augmentation, as they can generate new, synthetic data samples that are realistic yet diverse. Studies utilizing GANs for datasets like UNSW-NB15 and CICIDS2017 [94] have shown that these networks can produce new samples that help train models more effectively distinguish between different network behaviors.

7. Observations, Challenges, and Future Directions

The development of IDS for IoT and broader network environments continues to be a dynamic research area driven by the rapid evolution of technology and emerging cyber threats. This section consolidates critical findings from recent studies, identifies significant challenges current systems face, and outlines promising directions for future research to enhance IDS effectiveness and adaptability.

7.1. Observations

• Advances in ML and DL: Integrating ML and DL in IDS has significantly improved detection capabilities, particularly in recognizing complex patterns and anomalies that traditional methods might miss. These approaches enhance the ability to detect sophisticated attacks, including zero-day vulnerabilities, by learning from large datasets.
• Shift from ML to DL Techniques: There is a noticeable trend toward employing DL techniques, such as CNN, for both feature extraction and attack detection. With their ability to process raw data and automatically extract features, DL models have shown superior performance in IDS tasks compared to traditional ML methods.
• Effectiveness of Feature Selection and Data Balancing: Techniques like feature selection and data balancing have been critical in optimizing IDS performance. Feature selection helps reduce model complexity and computational overhead, which is crucial for deployment in resource-constrained environments like IoT. Data balancing techniques such as SMOTE and GANs address the class imbalance issue, ensuring the models are trained on diverse attack scenarios.
• Increased Adoption of Hybrid and Ensemble Approaches: There has been a noticeable shift towards hybrid and ensemble IDS that combine multiple detection techniques or integrate various data handling strategies. This approach enhances the system’s robustness by leveraging the strengths of different methods, improving detection rates and accuracy.
• Emphasis on Real-Time Detection: The necessity for real-time threat detection has driven the development of IDS to analyze data streams in real-time, reducing the latency between threat identification and response. This capability is critical for mitigating cyber-attacks impact on IoT systems and networks.

7.2. Challenges

• Dynamic and Sophisticated Threats: Cyber threats are becoming more sophisticated, with attackers constantly developing new strategies to bypass security measures. IDS must evolve to detect zero-day attacks and advanced persistent threats, often exhibiting subtle and low-frequency characteristics.
• Big Data and Scalability Issues: The exponential growth in IoT devices results in massive data streams that IDS must process, often in real time. Scalability remains a formidable challenge, requiring efficient data management and processing capabilities to handle such volumes without degradation in performance.
• Resource Consumption and Lightweight IDS: Developing IDS that are both lightweight and efficient in resource consumption is critical, particularly for IoT environments where devices have limited processing power and battery life. Ensuring that IDS solutions do not overburden the host devices remains a significant challenge.
• Integration and Standardization: Integrating IDS seamlessly across diverse platforms and ensuring compatibility with new IoT protocols is increasingly challenging. There is also a lack of standardization in security protocols across devices and networks, complicating the deployment of universal solutions.
• Maintaining High Accuracy with Imbalanced Data: Imbalanced datasets, where some attack types are underrepresented, can lead to biased models that perform well on frequent classes but poorly on rare yet critical attack types. This remains a significant hurdle in developing robust IDS.

7.3. Future Directions

• Development of Adaptive Systems: Future IDS should focus on adaptability, utilizing online learning or transfer learning to continuously adjust to new data or attack patterns without full retraining cycles. This would help maintain high detection rates over time.
• Cross-Domain Security Solutions: There is a critical need for solutions operating across different network layers and environments—from edge devices to the cloud—providing end-to-end security. This includes developing IDS that can handle diverse IoT devices and communication protocols.
• Leveraging Emerging Technologies: Integrating innovative technologies like blockchain for decentralized data sharing and AI for predictive threat intelligence could significantly enhance the detection capabilities and reliability of IDS. Blockchain can ensure data integrity and traceability, while AI can predict and preemptively counter emerging threats.
• Enhanced Anomaly Detection Techniques: Investing in research that improves anomaly detection algorithms to reduce false positives and adapt to network traffic’s evolving behavior can help preemptively identify potential threats. This includes developing more sophisticated behavioral analysis techniques and anomaly detection models.
• Low-Cost, Energy-Efficient Solutions: Future research should also focus on developing low-cost, energy-efficient IDS solutions that can be widely adopted, especially in developing regions and applications with significant cost constraints. This includes optimizing algorithms for minimal resource consumption and developing hardware-accelerated IDS.
• Real-Time Adaptability and Response: Future IDS systems should incorporate mechanisms for real-time adaptability and automated response to detected threats. This includes integrating ML models that can dynamically update themselves based on new attack patterns and developing automated response strategies to mitigate threats as they are detected.

Expanding on these areas, further research should also explore the development of IDS-specific benchmarks and datasets that reflect real-world environments to evaluate better and compare the efficacy of different IDS approaches. Additionally, more focus on developing low-cost, energy-efficient solutions would facilitate broader adoption, especially in developing regions and applications with significant cost constraints.

8. Conclusion

This paper explores a systematic review of IDS within IoT and general network environments, highlighting the pivotal role that ML and DL play in enhancing cybersecurity measures. As technology evolves and the digital landscape expands, the complexity of cyber threats grows, necessitating more sophisticated and robust IDS solutions. This review has synthesized the recent advancements in IDS, mainly focusing on integrating advanced computational techniques such as feature selection, data balancing, and innovative algorithms to address network threats’ dynamic and increasingly sophisticated nature.

Our discussions underscored several critical trends: the shift from traditional ML to more complex DL models, the adoption of hybrid approaches combining various techniques, and the emphasis on developing lightweight and efficient systems suitable for resource-constrained IoT devices. Moreover, the challenges identified through this review, including scalability and the need for real-time threat detection, reflect the ongoing need for research and development in this field. These challenges are not only technological but also extend into the realms of standardization and regulatory compliance, which are crucial for the widespread adoption of these systems.

Looking forward, the future of IDS appears promising yet demanding, with numerous opportunities for breakthroughs in algorithmic efficiency, cross-domain applications, and the use of emerging technologies such as blockchain and AI for enhanced security measures. The insights gathered here should serve as a foundation for ongoing and future research efforts, aiming to develop IDS solutions that are effective, efficient, and adaptable to the ever-changing landscape of cybersecurity threats. Integrating these advanced systems into real-world applications will significantly secure our digital infrastructures against increasingly sophisticated cyber threats.

This paper provides an overview of optimized intrusion detection strategies for IoT environments underpinned by advanced ML and DL techniques. For further details on the specific systems discussed and a complete list of acronyms, readers are directed to Appendix 1 and Appendix 2, respectively.