Performance Study on the Use of Genetic Algorithm for Reducing Feature Dimensionality on an Embedded Intrusion Detection System

1. Introduction

In order to keep network systems operational and secure from Internet attacks, Cybersecurity defensive systems are responsible for producing and processing a sizable amount of data from firewalls, Intrusion Detection Systems (IDS), or Intrusion Prevention Systems (IPS) in the form of network traffic metadata and system logs.

IDS are crucial in the network infrastructure as they provide a security layer that can detect a potential attack going on by analyzing the network traffic in real-time that violates the security policy and compromises its Confidentiality, Integrity, and Availability [1,2].

It began development in the 1980s and since then, it has become a fundamental research and development area within computer security and data protection, and they still growing with enhanced features and improved safety [3].

IDS are divided into three categories according to Jamalipour and Mourali [4]: IDS based on Intrusion Data Sources, IDS based on Detection Techniques, and IDS based on Placement Strategies.

Host-based IDSs (HIDS) and Network-based IDSs (NIDS) are two types of IDS based on intrusion data sources. The HIDS are helpful in identifying assaults on systems including databases, operating systems, and software logs that don’t involve network traffic. In the network environment, NIDS is used to analyze network traffic and find attacks.

Four categories of IDS based on detection techniques using four different techniques can be identified. They can be signature-based IDS, which compares network traffic with a known pattern or signature. Anomaly-based IDS is a different technique that uses pre-defined standards to determine what is malicious or benign based on network protocols. It is quite effective at identifying malicious activity but is unable to identify the specific sort of attack. Similar to anomaly-based IDS, specification-based IDS are based on rules and thresholds specified by experts, but in this case, the rules are implemented based on someone’s competence. Hybrid-based intrusion detection systems (IDS) incorporate both signature-based and anomaly-based approaches, offering the best of "two worlds" in terms of storage cost and computing cost.

There are three types of IDS based on placement strategies: centralized IDS, distributed IDS, and hybrid IDS. To find assaults, centralized IDS examines the data traffic that is going through, either at the root node or a strategic node. Every node in the network will be equipped with a comprehensive IDS, taking advantage of a thorough analysis of each node to prevent attacks from both inside and outside the network. This is known as distributed IDS placement. A central IDS that is better able to analyze huge amounts of data is combined with a variety of lightweight nodes dispersed around the network that have a lower detection ratio but are nonetheless sufficient for the purpose of a hybrid IDS placement.

The ability to distinguish between benign and malicious (attack) packets is crucial when designing a Network Intrusion Detection System (NIDS) based on Machine Learning (ML) algorithms. This success depends heavily on the availability of historical data on attack types and their characteristics.

In the case of the NIDS, which is the current studied topic, there are already various well-known datasets that cover a wide range of attack types, including CICIDS-2017 [5], KDD [6], NSL-KDD [7], UNSW-NB15 [8], and the most recent, HIKARI-2021 [9].

The network traffic-related datasets from the NIDS contain a lot of entries and features. The metadata from a single packet moving across the network can be quite large, and the majority of the datasets described start with 50 features or more; some even have more than 500,000 entries.

The problem and in some cases the benefits of having large datasets is the high number of features. Feature analysis is part of the Exploratory Data Analysis (EDA) step when researchers must understand what is the impact of each feature and decided if it must be used to train the model or if it must be changed in some way, like normalization, or even discard them [10]. Choosing the best features to use to train the model is key for getting the results to address a specific problem. The number and the types of features will make a significant impact in different aspects: the time needed to train and to infer, the results of the model when predicting, and its explainability. Feature selection has many benefits for the models such as the prevention of over-fitting, it prevents noise resistance and it improves predictive performance and some studies show that a lower number of features can increase the detection rate [11].

R. Antunes et al [12] divide the feature selection into 3 types: filter, wrapper, and embedded methods. The filter method is based on the preprocessing of the data which is based on the findings from Exploratory Data Analysis where we use algorithms to rank the features based on statistics such as the correlation, chi-square, and others. The wrapper method consists of using an ML algorithm itself to train a model, and return the most significant features from it, and use them to train a new model with those identified before [13]. The embedded method consists of selecting the features during the training phase without the need to split the dataset into train and test. This reduces the computation time of the training since it optimizes the number of features to use faster than the other two approaches mentioned before.

Genetic algorithms have garnered significant attention from the intrusion detection research community due to their close alignment with the requirements for building efficient intrusion detection systems. The high re-trainability of Genetic Algorithms enhances the adaptability of intrusion detection systems, while their ability to evolve over time through crossover and mutation makes them a suitable choice for dynamic rule generation [14].

Due to the rapidly evolving of Cybersecurity threats, IDS can quickly become outdated as new exploits and attacks emerge. This ongoing challenge require systems that are adaptive and capable of continuous learning. Consequently, Machine Learning (ML) has become essential in this field. ML-based Intrusion Detection Systems have been developed and are showing promising results in classifying network traffic effectively [15,16].

In this research, we studied the feature selection step by reducing the feature dimensionality using a Genetic Algorithm in the HIKARI-2021 dataset. The main goal is to retain only the most relevant and important features, without sacrificing the accuracy of the machine learning model trained on the full dataset. To accomplish this, an XGBoost ML model is deployed, which is known for its ability to handle large datasets and identify complex relationships between features.

The XGBoost algorithm can be described as an ML algorithm that uses the decision-tree-based predictive model through gradient boosting frameworks [17]. This algorithm, designed for speed and efficiency, maximizes memory and hardware efficiency in tree-boosting algorithms and supports deployment across various computing environments [18].

The combination of the Genetic Algorithm and XGBoost model will also be compared to other dimensionality reduction techniques, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and the Chi Squared-test results from a previous study [19].

The end goal of this research is the deployment of the ML-based IDS in an embedded system to behave as an IDS. Embedded systems are creating a huge impact on the way people live nowadays enhancing applications in the areas of health, military, IoT, industry, and automation, among others [20].

For this, a Nvidia Jetson Nano is used as the hardware to support the AI-based IDS running with the subset of features chosen by our Genetic Algorithm and predicting with the trained model XGboost and a comparison will be made regarding the time needed to predict between the jetson nano and the server used to run the Genetic Algorithms.

The paper is organized as follows: the next section presents the Related Work discussing similar approaches from previous research, the Exploratory Data Analysis shows an analysis of the HIKARI-2021 dataset, the Feature Dimensionality Reduction Techniques explain how we design our tests and parameters of the feature selection algorithms used and the Results and Discussion section shows a comparison between the results obtained and the last section concludes our work.

2. Related Work

Different algorithms for supervised learning were already tested in the HIKARI-2021 dataset such as gradient boosting machine (GBM), Light GBM, CatBoost, and XGBoost in the research made by Maya Louk [21], a Multi-layer Perception [22], Graph Neural Networks (GNNs) [23] and other traditional ML algorithms such as KNN, Random Forest and SVM [19]. Although some of them achieve great results and interesting findings, most of the studies mention the importance of further research in the dataset by using different models. Different feature selection techniques and Genetic Algorithms have already proved some value for this case, as in IDS is very common to have datasets with a large number of features.

For the same purpose of this research, building an AI-based ID, Ajeet Rai studied the impact and optimization of Ensemble Methods and Deep Neural Networks by implementing feature selection using Genetic Algorithms, reducing from 122 to 43 features, [24] and their results show that three in a total of four algorithms have better results when using feature selection. Those results were made in a small sub-dataset of the original KDD which probably indicates that by using more data, the results could be even better.

On a different dataset, NSL-KDD, the researchers [25] tested three known algorithms for feature selection - Correlation-based Feature Selection (CFS), Information Gain (IG) and Correlation Attribute Evaluator (CAE) with a Genetic Algorithm - to compare the performance of a Naive Bayes and J48 model. They also concluded that the Genetic Algorithm outperforms the others in the score while reducing the time complexity.

Researchers in [26] used the UNSW-NB15 dataset to compare different dimensionality reduction techniques: the particle swarm optimization (PSO), grey wolf optimizer (GWO), firefly optimization (FFA) and Genetic Algorithm (GA). Their results showed that the PSO achieved better accuracy and precision than the others for the algorithms J48 and SVM. Another interesting finding in this study was that it created new subsets from each subset of all individual algorithms and tested different combinations of features based on joining (inner/out) features. This achieved even better results than using just one algorithm for feature reduction.

Embedded systems are becoming popular in the field of IDS due its their ability to be placed anywhere, the low power consumption, and the cheap cost. Research shows that even an Arduino Uno can behave as an IDS using AI [27]. Another extensive study was made by comparing other low-power embedded systems with different models trained in different conditions proposing a lightweight optimized deep learning IDS [28]. Although these result shows great potential when the model starts to get bigger and the amount of data increase it is necessary for more computational power. Our approach of using the Nvidia Jetson Nano is not so common in the IDS context but it is already used in other cases such as Home intrusions using computer vision and other techniques [29]

3. Materials and Methods

This research uses as materials the most recent IDS dataset - HIKARI 2021 [9] which is publically available. The methods used in this research consist of an Exploratory Data Analysis using statistics algorithms, data preprocessing using Python libraries, and then machine learning algorithms to train a model and then infer data on it.

The system architecture that supports the materials and methods used in this research, Figure 1, consists of five phases: Exploratory Data Analysis where it is studied the dataset (HIKARI) regarding its content - features and data, then Data Preprocessing that applies dimensionality reduction and feature selection that are the different methods studied, then the Model Training using the XGBoost algorithm and the best subset of features chosen before and lastly the deployment of the model to the embedded system.

In the embedded system the same performance evaluation is made as well as in the server that ran the other experiments. This test consists of running the different models trained with different numbers of features and measuring the time. The goal is to keep the classification performance and verify which runs faster and if the embedded system could be a better approach to this use case.

3.1. Exploratory Data Analysis (EDA) to the HIKARI-2021 Dataset

The HIKARI-2021 [9] dataset is currently the most up-to-date, containing encrypted synthetic attacks and benign traffic. The main advantage of this dataset compared to the others is that the network logs are only anonymous in specific logs of benign traffic, the dataset can be reproducible, has ground-through data, and the main characteristic that distinguishes it from the others - having encrypted traffic.

This dataset has a total of 555 278 records, and 88 features, where 2 of them are labels (one binary and other string) where one corresponds to 1 if the traffic is malicious and 0 means benign or background, and another label that classifies the traffic into the different categories: Benign, Background, Probing, Bruteforce, Bruteforce-XML and XMRIGCC-CryptoMiner.

In Table 1 it is possible to see the names of the features as well as their types. Most of them are integers or floats which are related to the traffic analysis and some are strings related to the identification of the source/destiny and label.

One problem of this dataset, which is normal in datasets of IDS, is the fact of being unbalanced. Table 2 shows well how unbalanced the dataset is, having the most quantity of traffic in the benign and background traffic which is normal traffic.

Based on a HIKARI-related study [10] the authors conducted a feature analysis that concluded that some of the features might be removed to accomplish better results. The same approach was followed here, removing the columns: "Unnamed: 0.1", "Unnamed: 0.2", "uid", "originh" and "originp" which are not relevant for a generic classification model but contribute to bias the training step of the classification by learning specific id tags that would otherwise not appear in a real traffic dataset.

3.2. Data Preprocessing

Regarding the data, the preprocessing methods are endless and they must be used according to the data type and the problem we are dealing with. This research, as mentioned previously, addresses the problem of IDS which have a large number of features. Hence techniques related to Dimensionality Reduction and Feature Selection were used.

3.2.1. Dimensionality Reduction

Feature reduction techniques are methods used to reduce the dimensionality of a dataset by selecting or transforming the most important features while retaining as much relevant information as possible.

These techniques are particularly useful in machine learning and data analysis to address issues related to high dimensionality, multicollinearity, and overfitting. The high dimensionality of IDS datasets makes them an excellent use case of these methods since we can get a better understanding of the importance of each feature and we can also reduce the training and inference times of the models used for intrusion detection tasks.

There are two main groups for feature reduction techniques: Feature Extraction and Feature Selection [30].

In Feature Extraction, techniques reduce the dimensionality of the dataset by transforming the original features into a new set of features, that are a mapping of the original ones, while trying to preserve the essential information and patterns present in the data. One disadvantage is that when the original features have an obvious physical meaning, the new features may lose meaning. Methods like Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) fall into this category.

PCA reduces the complexity of high-dimensional data by identifying principal components — orthonormal eigenvector and eigenvalue pairs — where data variance is maximized. It projects the data onto these components, focusing on those with the highest eigenvalues, and compressing the data while preserving essential information [31].

LDA projects the n-dimensional dataset into a smaller k-dimensional dataset (k < n), while at the same time maintaining all the relevant class discrimination information [32].

Feature Selection, on the other hand, focuses on selecting a subset of the original features while discarding the rest. The Selected Features are considered the most relevant for the specific task, and irrelevant or redundant features are removed. Chi-squared (Chi-2) and Genetic Algorithms (GA) are both feature selection methods.

Chi-squared is a numerical test that measures the deviation from the expected distribution considering the feature event is independent of the class value [33]. Its use can be as simple as importing the method from the scikit-learn and defining a K number of top features that have more impact on the label.

3.2.2. Genetic Algorithm for Feature Selection

A Genetic Algorithm (GA) is an optimization strategy that operates on a population of potential solutions, often represented as individuals or candidate solutions, and use genetic operators such as selection, crossover, and mutation to evolve and improve the population over generations. This type of algorithm has been around for many years and has proven itself as a a useful approach to finding solutions to real-world complex problems.

It can also be used to perform dimensionality reduction of the dataset. In a Genetic Algorithm approach, the genomes represent a subset of features in the dataset, and the fitness function is used to evaluate the quality of the dimensionality reduction. The fitness function measures the accuracy of machine learning model trained on the reduced dataset, with higher accuracy indicating a better dimensionality reduction.

3.2.3. Genetic Encoding and Initialization

When implementing a GA, the first step is to identify a way to express our solution as a genome. In this case, the genome is a binary one dimensional array with the same length as the number of features in the dataset ( Figure 2), where each bit represents removal or retention of a feature in our dataset (Table 3).

The subsequent step is to randomly initialize a population of arrays with some of the features set to be removed. The number of features removed needs to be chosen beforehand, so that the algorithm will generate arrays with the correct number of features. Each individual is applied to the dataset as a sort of mask, where if a feature is set as ’0’ in the array, it will be removed from the dataset. After going through the entire array, and removing or maintaining features, the resulting dataset is ready to the evaluated.

This population is evaluated through a fitness score. In this case, the score of each individual can be the accuracy of the machine learning model on a new dataset with only the selected features. The model is trained and tested on the new dataset, and the score is stored so it can be later compared to the other individuals.

The algorithm will evaluate the fitness of each individual by testing the corresponding feature subset’s performance using a ML model. Through successive generations, the Genetic Algorithm refines the population, favoring individuals that lead to better model performance. This iterative process mimics the principles of natural selection, allowing the algorithm to converge towards an optimal solution that balances predictive accuracy and feature number for the given dataset and model.

3.2.4. Sparsity Check

Removing features from the dataset, most of the time, results in a loss in accuracy, especially when random features are dropped. This indicates that a genome composed of a lot of features will probably have a better accuracy than a genome with almost no features so the GA will converge on solutions where almost no features are removed, but since we are aiming to reduce the dataset’s dimensions, there must be a mechanism to control the sparsity levels of genome.

To address this issue, a sparsity check function was built to choose the pruning percentage which one wants to apply to the model, as well as to ensure that the generated offspring meet this criteria; if they do not, an algorithm randomly selects values in the mask to modify, in order to meet the sparsity criteria.

3.2.5. Selection Method

Only a few genomes from this initial generation will be chosen to pass on their genes to the next generation. The selection of pairs of individuals (parents) is solely reliant on their fitness score (float values from 0 to 1). Every individual has a chance of becoming a parent with a probability proportional to its fitness. The fitter individuals have a better chance of breeding and passing on their genes to the next generation. The selection method was a memory- optimized version of the roulette wheel approach [34].

Inside the selection function, an individual’s fitness is compared to a randomly generated float value between 0 and 1; if the fitness is high, there is a larger probability that the random number will be less than the fitness, resulting in the individual being chosen as a parent.

3.2.6. Crossover and Mutation

A Genetic Algorithm’s most important phase is crossover. A crossover point is picked at random from within the genetic code, then the offspring will randomly inherit one side of the crossover point from each parent. When a new generation of offspring is born, some of their genes may be susceptible to a low-probability mutation. This means that some of the bits in the genetic code can be switched around. Mutations are an important step of the GA because it guarantees diversity in the population and prevents premature convergence. Individuals with a high fitness score would soon dominate the population in the absence of mutations, limiting the GA to a local optimum and hindering the GA’s ability to produce superior offspring.

3.3. Model Training

Our model was based on the XGBoost Classifier library and training was done with the SciKit-Learn Python API for XGBoost using all parameters set to default [35].

A crucial way to be confident when saying a model is ready to be used and fits the expectations of the job needed is by evaluating the model. Different metrics can be used, each problem will require its own metrics regarding speed, performance, results, outcomes, and others. For this specific topic and because the focus was the performance of classification it was decided to use the most popular metrics in IDS: accuracy and f1 score.

Accuracy is calculated by dividing the number of correctly identified predictions, true positives (TP) and true negatives (TN), by the total number of predictions (TP+FN+TN+FP) which means that by having a high accuracy the algorithm can be more trusted to classify the traffic but in IDS field accuracy is not the most important because of the existent datasets are imbalanced and the model can label everything as “benign” and achieve high accuracy due the fact the label with the highest frequency is “benign”, balanced accuracy is a better option for this cases.

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(1)

Precision is calculated by dividing the number of correctly identified malicious predictions, true positives (TP), by the total number of predictions that the algorithm classified as malicious, true positives (TP) + false positive (FP), which means that high precision is valuable to say that we can correctly identify the malicious traffic with a low percentage of false positives.

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(2)

Recall, also known as Sensitivity or true positive (TP) rate, is a metric that focuses on the ability of a model to correctly identify all positive instances in the dataset. It is calculated by dividing the true positives (TP) by the sum of true positives (TP) and false negatives (FN). In the IDS context recall is a crucial metric since it represents the ability of a model to catch as much malicious traffic as possible. High recall means that the model is able to identify the majority of true threats, however it may lead to more false positives (FP).

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(3)

The F1 score is a metric that combines both precision and recall providing a balanced measure of a model’s performance. It is useful when it’s wanted to find a balance between precision and recall, ensuring that false positives and false negatives are both minimized. F1 score is the metric that is more trustworthy in IDS studies because the higher it is it ensures that the model not only catches malign traffic but also minimizes false positives.

$$F1score=2·\frac{\mathrm{Precision}·\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

3.4. Model Inferencing and IDS

Regarding the inference, the idea of building a fully working IDS using AI the different parts are combined into a single system. For that, all of the previous work is now stored in one file - a ".pkl" file containing the model trained. An existing traditional dataset is used as a packet sniffer - ciclfowmeter which is publicly available and for that research, it is used to capture network traffic that is loaded into our model by selecting the features needed for our prediction.

All of these pieces fit into an embedded system - Nvidia Jetson Nano - running a Linux Ubuntu distribution being able to run both cicflowmeter and our model for real-time predictions.

4. Results and Discussion

In this final chapter, we dive into a comparative analysis of dimensionality reduction techniques applied to our dataset in conjunction with the XGBoost model. Specifically, we explore the impact of Feature Reduction using Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), as well as Feature Selection through with Chi-squared (Chi-2) and the Genetic Algorithm.

Our goal is to understand the effectiveness of each technique in reducing the dimensions of the HIKARI Dataset while striking a balance with the XGBoost model’s performance.

The server used was a single machine with a dual Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz 10 core CPU, with 120 GB of RAM with no powerful background apps running simultaneously.

The embedded system, NVIDIA Jetson Nano Developer Kit, is described by NVIDIA as a small, powerful computer designed to run different Neural Networks in parallel for different use cases related to computer vision, speech processing robotics, and others [36]. Nvidia Jetson Nano has a Quad-core ARM A57 @ 1.43 GHz CPU, 128-core Maxwell GPU, and 4 GB LPDDR4 as memory.

4.1. Dimensionality Reduction

As previously mentioned, the two dimensionality reduction techniques utilized were PCA and LDA. With PCA, it was possible to choose between 70 to 1 features. But with LDA it was limited to a maximum of 5 feature dimensions (Table 4).

LDA is a supervised method that aims to find the directions (linear discriminants) that maximize the separation between different classes. The number of linear discriminants is at most c-1, where c is the number of classes. This is because LDA projects data points onto a lower-dimensional space with the goal of maximizing class separability, and the maximum number of discriminative directions is determined by the number of classes [37].

On the other hand, PCA is an unsupervised method that does not take into account class labels. It finds the directions (principal components) that maximize the variance of the data. Therefore, PCA can generate as many principal components as the original dimension of the feature space or the number of samples, whichever is smaller.

Regarding LDA’s effectiveness, from the experimental results, we can see that it achieved lower Accuracy and a lower F1-Score for the same dimensions as PCA. This shows that PCA can strike a better balance between feature dimension and performance.

On both approaches, there is a noticeable decrease in training time as the feature dimension also is reduced, but there isn’t a difference on the inference times on the Test Set, this is due to the high computational power of the Computer used, where the number of data on the Test set is so small that it doesn’t make too much of an impact.

Table 5. PCA.

Feature Dimensions	Train time(s)	Test time(s)	Accuracy (%)	F1-Score Weighted (%)
70	217.12	0.073	76.53	75.23
60	210.09	0.073	76.43	75.13
50	192.97	0.068	76.32	74.98
40	120.18	0.073	75.54	74.20
30	116.52	0.068	75.25	73.88
20	111.14	0.064	74.22	72.66
10	94.85	0.066	73.71	71.93
5	83.03	0.063	73.11	71.00
4	77.33	0.072	72.43	70.17
3	77.98	0.071	71.88	69.49
2	60.25	0.073	71.45	69.54
1	46.99	0.071	69.79	65.75

Table 5. PCA.

Feature Dimensions	Train time(s)	Test time(s)	Accuracy (%)	F1-Score Weighted (%)
70	217.12	0.073	76.53	75.23
60	210.09	0.073	76.43	75.13
50	192.97	0.068	76.32	74.98
40	120.18	0.073	75.54	74.20
30	116.52	0.068	75.25	73.88
20	111.14	0.064	74.22	72.66
10	94.85	0.066	73.71	71.93
5	83.03	0.063	73.11	71.00
4	77.33	0.072	72.43	70.17
3	77.98	0.071	71.88	69.49
2	60.25	0.073	71.45	69.54
1	46.99	0.071	69.79	65.75

4.2. Feature Selection

In regards to the Feature Selection techniques, we can see that both Chi-2 and the GA achieved better results than the Feature Reduction of LDA, both Chi-2 ( Table 6) and the GA (Table 7) maintained a higher Accuracy and F1-Score as the feature dimensions lowered. The GA was the one with the better results, it achieved 73.59% F1-Score with only a dimension of 3, beating the PCA and the Chi-2.

The lower results of the Chi-2 compared to the GA can be explained by its approach to feature selection, since It relies on statistical measures to identify the relationship between each feature and the target variable. However, it may not explore the entire feature space comprehensively, and its effectiveness can be limited if the relationship between features. The GA, on the other hand, performs a more exhaustive search of feature combinations through its evolutionary process, making it more likely to discover relevant feature interactions and dependencies.

One of the main drawbacks of the GA is its runtime, as in the total time that the Genetic Algorithm took to go through the various possible candidates in each generation. In Table 7, we can see that it takes a considerable time to find the optimal solution for the different feature dimensions that are selected, this time decreases as the number of dimensions also become smaller but its an important thing to consider, since the other approaches to reduce the dimensions are almost instantaneous.

With these two methods, we can see which features are chosen as the most impactful to the accuracy of the model. On PCA and LDA this was not possible since the features were transformed. In Table 8 and Table 9 the five most important features are shown.

When comparing the results of all the techniques used, we can visualize in Figure 3 that the GA emerged as the most effective approach in maintaining optimal performance even as the number of features was decreased. In contrast, the Chi-2 approach demonstrated a poorer performance when compared to its counterparts.

4.3. Performance Evaluation between the Jetson Nano and the Server

One of the goals of this paper contributions is to assess the feasibility of using an embedded system to implement the IDS functionality. With that in mind, we evaluate all the different models, trained in the Server machine, with the same reference test set in the Jetson device and compare the time needed to infer the complete test set. The results in Table 10 show that the Nvidia Jetson takes 10 times more than the Server to classify the traffic in the test set - about 111,000 entries.

From the results table, it is noticed that although Chi Square is the algorithm that has a higher f1 score, it is also the one that takes more time, both in the Jetson and in the server, to predict the test set. Another relevant aspect is that LDA takes more time to predict the complete test set when configured to use one component than when using five components.

The results can be graphically explained in Figure 4 where the xx axis identifies the number of features considered and the yy axis (which is in log scale) represents the time used in seconds for running the inference on the test set. We can observe that the time needed to infer in the Jetson Nano is 10 times larger than the time needed by the server. It is also visible, both for the server and for the Jetson, that the time taken for the inference increases with the number of features, which is explainable by the complexity of having more features to process.

The Figure 5, also with the xx axis representing the number of features and the yy axis representing the inference time in seconds in log scale, represents the time of the 4 models on the Jetson device. It is easily noticed that the increase on the number of features also increases the time to infer. The exception is the LDA algorithm, where running one component is slower than running with five components.

5. Conclusions

In this work, we study the importance of feature selection when using an IDS dataset. First, an exploratory data analysis was done in the HIKARI-2021 dataset to characterise the features. In order to reduce the model complexity, we propose to reduce the number of features considered, with the goal of achieving the same performance metrics (f1 score) but with a faster time to run the model.

We applied four different algorithms for feature selection on the HIKARI-2021 dataset and performed an evaluation on the performance metrics and running time.

Our study showed that the Genetic Algorithm outperformed the most traditional algorithms such as PCA, LDA or chi-square test. Although the GA presents better results regarding the F1-score, it needs more time than the others to select the number of features more relevant.

We also studied the use of an embedded system, the Nvidia Jetson Nano, to implement the IDS functionality as an edge device. To evaluate this system, a comparison was made between this device and a typical server, showing that the Jetson can behave as an IDS but it takes 10 factor more time than the Server for inferencing a data set.