Defense Against Information Integrity Attacks in Federated IoT Systems Using Inertial Momentum Aware IALM-RPCA

1. Introduction

The era of the Internet of Things (IoT) has seen an increase in the connectivity of everyday devices, establishing a network that spans personal, public, and industrial domains. This interconnection has generated a massive flow of data, with devices continuously communicating, collecting, and transmitting information. While the increase of IoT devices represents an advancement in technology into the future, it has also presented a number of new challenges, particularly in the areas of data processing and learning methodologies [1].

These devices, despite their limited processing power, have strengthened the demand for advanced computational models capable of handling large datasets with minimal latency. Federated Learning (FL) has emerged as an advanced paradigm that takes advantage of the distributed nature of IoT networks. By enabling collaborative model training while maintaining data localized, FL enhances privacy and optimizes the computational load on individual devices [2].

However, the decentralization that defines FL also introduces new vulnerabilities. The integrity of data is of utmost importance, as it is the fundamental basis that ensures the reliability and success of Federated Learning. Attacks that involve the injection of corrupted data by malicious actors represent a significant danger to the integrity of Federated Learning systems [3]. These attacks can hamper the learning process, lead to incorrect model updates, and ultimately compromise the entire learning infrastructure [4].

In light of the ever-increasing number of IoT devices and the ever-increasing dependence on FL, it is vital not only to address the issues that lie ahead but also to come up with effective countermeasures [5]. Recognizing the growing deployment of IoT devices and the reliance on FL for data processing and machine learning tasks, our study seeks to strengthen the FL framework against such threats. We propose to explore and refine methodologies that can enhance the robustness of FL against data uncertainties and information integrity attacks, particularly those that originate from the client side.

The goals of this work are designed to enhance the stability and security of the Federated Learning approach. By doing so, we aim to protect the cooperative nature of the learning process, particularly in the context of the rapidly expanding realm of IoT technologies. This effort is crucial to ensure that as more devices connect and communicate, they do so on a platform that’s resistant to disruptions and compromises.

The primary objectives of our research are threefold: firstly, to tackle the immediate challenges arising within the rapidly evolving domains of the IoT and Federated Learning; secondly, proposing a novel momentum aware Inexact Augmented Lagrange Multiplier Robust PCA algorithm which is more efficient than the existing one and thirdly, to establish a benchmark for future innovation in secure and dependable decentralized learning ecosystems.

The key contributions of this research are as follows:

1.

We developed a multi-stage federated learning model in order to strengthen the robustness and efficiency of decentralized learning against client-side data poisoning attacks.

2.

As a countermeasure against information integrity attacks, specifically client-side data poisoning attacks, we developed a sophisticated defence strategy using the Augmented Lagrange Multiplication method. Our proposed method not only ensures the successful recovery of data after being corrupted but also detects client-side data integrity attacks in an efficient and effective manner.

3.

We have proposed a novel approach to the Inexact Augmented Lagrange Multiplier (IALM)-RPCA[6] method by integrating the inertial momentum with an average constant momentum factor ($\beta$ = 0.5) without any tuning of the $\beta$ value persisting the trait of traditional IALM -RPCA algorithm.

4.

The proposed model and the novel approach of IALM-RPCA has been validated through a set of extensive experiments considering a benchmark dataset known as N-BaIoT [7]. Novel approach of IALM RPCA exhibits a positive outcome over existing IALM RPCA method for reconstruction.

The remainder of the paper is organized as follows: Section 2 provides a review of related work, focusing on foundational studies on Federated Learning and the security vulnerabilities it faces. Section 3 outlines our methodology, including the proposed multi-stage model and defense mechanisms against poisoning attacks and a novel approach by introducing inertial momentum aware IALM-RPCA method with constant momentum factor. Section 4 presents the results, offering insights into the dataset, confusion matrix analysis, impact of adversarial attacks, accuracy trajectories over epochs, performance metrics under varying scenarios, comparative analysis between conventional and novel process and overall observations. Section 5 discusses the relevance of Federated Learning, the significance of data reconstruction, comparison with previous works, and future research directions. Section 6 concludes the paper.

2. Related Work

This section is focused on federated learning, more specifically highlighting the inherent vulnerabilities of federated learning as well as its potential defense mechanisms against threats, with a particular emphasis on poisoning attacks. The covered area can be roughly divided into two categories: the preliminary studies on federated learning and the security challenges that are associated with it.

2.1. Foundational Work on Federated Learning

In recent years, federated learning has experienced considerable developments, which has resulted in the opening up of a wide variety of strategies and approaches that can be utilized to take advantage of this decentralized training paradigm. In the meantime, it has been leading the way in advancing the field of machine learning. McMahan et al. [8] established a benchmark for future research when they proposed a revolutionary method for training machine learning models without the use of centralized data repositories. This mechanism emphasized effective communication and paved the way for further investigations. This study was further developed by Konečný et al. [9], who investigated the communication challenges associated with federated learning and developed architectures that reduce those difficulties.

Yang et al. [10] explored the challenges and future directions in the area of federated learning, presenting a more comprehensive perspective on its potential. Also, Smith et al. [11] have presented a multi-task learning technique in federated settings, addressing the problems and potential solutions connected to ensuring privacy in such decentralized systems. Additionally, recent work by Lim et al. [12] on federated learning in mobile and edge computing environments has shed light on new challenges and opportunities in this domain. Bonawitz et al. [13] have made significant contributions to scalable federated learning, focusing on practical applications in large-scale systems. By sharing an additional model with each client, Tanmoy et al.[14] show an increase in the accuracy and fast convergence of the global model in FL setting in some cases compared to the existing FedAVG method. Later on, Tanmoy et al.[15] have evaluated accuracy and weight divergence of global model in FL setting for both model sharing and data sharing process.

The decentralization of data, despite its obvious challenge, comes with complications. Predicting data patterns across multiple nodes, each of which has its unique characteristics, has been a primary focus of the investigation. Discussion on the challenges faced by poisoning attacks inside federated environments is provided by Bhagoji et al. [16], which is helpful. Further insights into optimizing federated learning algorithms are offered by Li et al. [17], focusing on enhancing model performance and efficiency.

2.2. Security Vulnerabilities in Federated Learning

The escalating use of federated learning has notably increased security concerns, particularly in the face of poisoning attacks. These attacks, aimed at disrupting the learning process through data manipulation, have been extensively analyzed by Biggio et al. [18]. Their work provides a deep dive into the systemic vulnerabilities of federated learning systems and the potential consequences of malicious activities. Recent developments in poisoning attacks, especially in IoT, have introduced new challenges, as discussed by Liu et al. [19]. They emphasized the complexity of security in IoT implementations of federated learning.

Recent advancements in the landscape of poisoning attacks, especially the emergence of “bi-level poisoning attacks,” have added a new layer of complexity, especially in IoT domains. Billah et al. [20] conducted an in-depth exploration of the implications of these sophisticated attacks on regression models, explaining their significant impact. Moreover, Zhang et al. [21] have brought to light various threats within the federated learning framework, emphasizing the urgent need for effective and robust security measures to counteract these challenges.

In response to the evolving security threats in federated learning, particularly in the context of IoT, researchers have proposed advanced defensive strategies. Wang et al. [22] and Wei et al. [23] have introduced comprehensive approaches to safeguard against data poisoning and model tampering. These strategies are crucial in maintaining the integrity and reliability of federated learning systems in increasingly complex and interconnected environments.

Moreover, to tackle the problem of corrupted matrices, the first paper that paves the way to use of ALM RPCA instead of Robust PCA to mitigate the corrupted matrix was authored by lin et al.[6]. Later, various optimization methods have made ALM RPCA more precise. Zhu et al.[24] proposed a momentum-enhanced proximal ADMM framework for RPCA that improves convergence and robustness in nonconvex image and video processing applications. The paper authored by Wang et al.[25]introduced an inertial momentum-based optimization algorithm for SVD-free RPCA to improve computational efficiency and convergence for large-scale noisy data problems. Wu et al. [26] acquainted inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems. Xia et al.[27] showed an inertial constraint term into RPCA optimization accelerates convergence and improves matrix recovery performance under mixed-noise environments by using fractional function to approximate the $L_0$ norm and dynamic, non-linear thresholding.

Despite considerable academic research dedicated to unraveling the complexities of federated learning, a pronounced knowledge gap persists, especially concerning its inherent security vulnerabilities, like client-side poisoning attacks. This research endeavor is committed to addressing these vulnerabilities, aiming to illuminate the inherent weaknesses of federated learning systems. By doing so, it lays a foundational framework for future advancements designed to secure these systems against the nature of evolving threats.

This work highlights the critical need for ongoing vigilance and proactive innovation in security strategies. It underscores the importance of developing robust and adaptable defenses to safeguard federated learning models, particularly against sophisticated and dynamically changing threats. In doing so, this research not only seeks to bridge the current knowledge gap but also to pave the way for the creation of more robust and secure federated learning architectures, ensuring their efficacy and reliability in diverse and challenging environments.

3. Methodology

During this study, we have employed a wide range of distinct approaches to the implementation of investigations, all of which are described in further depth in the accompanying breakdown for this section. Detailed information regarding the procedures for multi-stage federated learning is supplied to protect the clients’ data from attacks on their information integrity. Clarification is then provided on the countermeasure technique we utilized to recover the poisoned data. This was done to remove any ambiguity that may have been caused. Discussion is also taking on regarding the amount of data uncertainty for the training data.

3.1. Overview of the Workflow

The methodology that we provide is founded on several essential steps, including data collection, the incorporation of federated learning, adding sparse noise, the processing of the data, the training of a federated model following the processing, and the subsequent strategies for data evaluation. Every one of these primary steps is capable of being broken down even further into deeper components. Figure 1 depicts the core process that we followed consistently to provide a greater understanding of our methodology. The next parts will go into greater depth regarding the complexities of each component.

A paradigm in the field of machine learning known as federated learning makes it possible to train algorithms across a large number of decentralized devices or servers, each of which stores its local data samples. When the goal is to construct machine learning models using data scattered across various devices, such as smartphones, without centralizing the data on a single server, it is crucially important that this strategy ignores the need to transfer data samples directly. This is essential when the purpose is to develop these models.

3.2. Operation of Federated Learning

The typical operation of the Federated Learning algorithm is described below. Here in Figure 2 , a visual representation of FL process has been added.

1.

The initial model is trained on a central server, which may be initialized randomly or based on some previous knowledge.

2.

The most recent model is communicated to all of the devices (clients) that are actively participating in the federation.

3.

Every device generates an updated model by using the data from its local environment.

4.

The changes that were made locally are sent to the server. Users’ privacy can be protected by making these changes anonymous, random, or otherwise hard to understand.

5.

The server aggregates all of these modifications and creates a new global model, often by taking an average of them. To balance the contribution of each device, the updates can be weighted, for example by the number of local samples on each device.

6.

Steps 2 through 5 are carried out multiple times throughout several rounds until the performance of the model satisfies the specified requirements.

Federated Learning is a very active research area and there are many different variants of the basic Federated Learning algorithm that are designed to address its various challenges. Some examples include the Federated Averaging (FedAvg) model that was proposed by McMahan et al., 2016 [9] and the SCAFFOLD model [28] that was proposed by Karimireddy et al., 2020.

It is important to highlight that even though Federated Learning can help protect privacy because it does not exchange raw data, it does not automatically assure full privacy or security. This is something that should be kept in mind. There is a chance that additional security measures, such as differential privacy[29] or secure multiparty computation, might be implemented to provide more robust assurances regarding the confidentiality of user information.

3.3. Poisoning Attack on Federated Learning and Recovery

Particularly in federated learning environments, poisoning attacks provide significant threats to machine learning models. They introduce inaccuracies into the training data, leading to incorrect pattern learning and inaccurate performance from the model. The nature of these attacks might either be harmful. A successful recovery from such attacks requires attack detection, separation of affected data, data reconstruction, model re-training, and post-recovery analysis. In addition, an effective understanding of the potential strategies for attacks and the system architecture to enhance robustness is required.

3.3.1. Sparse Noise Introduction

During our research, we looked into a particular form of poisoning attack known as sparse noise, which is distinguished by the fact that it corrupts only a portion of the data and the magnitude is significantly high or low compared with the normal data distribution, and instances are sparse in nature. This allowed the structure of the dataset to be preserved while allowing the integration of the intended disruptions. This effectively simulated potential attacks in the real world on IoT client data.

The provided algorithm, termed as Sparse Noise Introduction - Poisoning Attack, is described in Algorithm 1.

The algorithm begins with an initialization phase where a sparsity level is set, typically at 0.05, indicating the proportion of the dataset’s entries that will be targeted with noise. Based on this sparsity level, it calculates the number of noisy entries by multiplying the dataset’s total size by the sparsity factor. Next, it randomly selects indices from the data set equal to the number of noisy entries identified.

For each of these indices, Gaussian noise is generated, following a normal distribution with a mean of zero and a standard deviation of 5. Here in Figure 3 the probability density function is shown. The algorithm then applies this noise to the dataset, altering the original entries at the chosen indices to simulate the effect of an information integrity attack.

The outcome is the data set with the the noise introduced in a sparse manner, indicating a poisoning attack. To persist the effect of attack irrespective of different type of feature magnitude so that for later comparison of two algorithms the real insight could be reflected , we have used the standard calling method after mixing raw noise with the data set. So, keeping the shape noisy dataset similar , all features equally weighted so that later on RPCA could treat the value of all features fairly.

Figure 4. Primary Data Distribution of Some Selected feature

Figure 4. Primary Data Distribution of Some Selected feature

Figure 5 presents the distributions of several representative features. Although Gaussian noise was added randomly to entries throughout the entire feature matrix, rather than to each features individually, the resulting distributions of the selected features become bell-shaped and approximately Gaussian. This demonstrates the impact of the corruption process on the underlying data distribution.

Now, To enable effective low-rank and sparse decomposition by RPCA, we have scaled the data, where in standardization the usefulness of scaling is evident as it ensures equal feature contribution to the nuclear norm minimization and allows the regularization parameter $\lambda$ to work uniformly across all features, leading to unbiased recovery of the low-rank structure and consistent sparse anomaly detection with efficient IALM convergence.

Here in Figure 6 only the feature values have been rescaled without changing the shape and noise structure. The standardization merely shifts each feature to zero mean and unit variance by a linear transformation, preserving the underlying data distribution and the additive Gaussian noise pattern. This is important for the noise characteristics to remain consistent and the low-rank plus sparse structure required by RPCA to be preserved for faithful reconstruction and anomaly detection.

It is essential to recognize that this method alters quite a few of the entries in the dataset with noise based on the Gaussian distribution. The subsequent analyses or performances of machine learning models may be dramatically impacted as a result of this.

3.3.2. Reconstruction of Noisy Data Using RPCA

The following section will provide a more in-depth breakdown of the procedures that are utilized for robust principal component analysis (RPCA) [30] and matrix completion. The large variety of RPCA approaches includes methods like Accelerated Proximal Gradient, Dual Method, Singular Value Thresholding, Alternating Direction Method, and ALM (Augmented Lagrange Multiplier), amongst others. The Inexact ALM [30] approach has emerged as the leading choice among them, demonstrating speed and accuracy that are superior to any other method. As a result, its position as the favored alternative for data reconstruction in our research has been enhanced as a result of this development. As can be deduced from the names of the procedures, certain strategies, such as the Augmented Lagrange Multiplier and the Singular Value Thresholding, have been shown to be effective at completing matrices.

3.3.3. Inexact ALM RPCA Algorithm

Consider a matrix $M\in {\mathbb{R}}^{m\times n}$. The Inexact ALM RPCA algorithm addresses the optimization problem:

$$\underset{L,S}{min}{\parallel L\parallel }_{*}+\lambda {\parallel S\parallel }_1\mathrm{subject}\mathrm{to}L+S=M,$$

(1)

where ${\parallel ·\parallel }_{*}$ represents the nuclear norm (i.e., the sum of singular values) and ${\parallel ·\parallel }_1$ denotes the ${\ell }_1$ norm (i.e., the sum of absolute values). Here, L and S are low-rank and sparse matrices, respectively.

The algorithm procedure is:

1.

Initialize: $Y=0$, $S=0$, and $L=0$. Set $\mu ={\displaystyle \frac{1.25}{{\parallel M\parallel }_2}}$, $\rho =1.5$, and ${max}_{\mu }={10}^{10}$.

2.

Repeat until convergence:

(a)

Update S using the shrinkage operator: $S=\mathrm{shrink}(M-L+Y/\mu ,\lambda /\mu )$.

(b)

Update L with the Singular Value Thresholding (SVT) operator:

$$\begin{array}{cc}\hfill L& =\mathrm{SVT}(M-S+Y/\mu ,1/\mu ),\hfill \\ \hfill \mathrm{where}U\Sigma V^T& =\mathrm{svd}\left(X\right),\hfill \\ \hfill \Sigma & =\mathrm{diag}\left(\mathrm{shrink}\right(\sigma ,\tau \left)\right).\hfill \end{array}$$

(c)

Update Y: $Y=Y+\mu (M-L-S)$.

(d)

Update $\mu$: $\mu =min(\rho \mu ,{max}_{\mu })$.

3.

Convergence is achieved when ${\parallel M-L-S\parallel }_F/{\parallel M\parallel }_F<\mathrm{tol}$, with ’tol’ as a pre-defined threshold.

3.3.4. Inertial Momentum Aware Inexact ALM RPCA Algorithm

In case of inertial momentum added IALM RPCA all steps of traditional IALM RPCA are consistent except addition of constant momentum factor ($\beta$=0.5) and extrapolation step of finding $L_k$ using $L_{k-1}$ and $L_{k-2}$ of previous step for each k’th iteration until convergence. We have taken into consideration that beta should be an average value as if $\beta$=1 , the full difference affect the current value of denoised and sparse matrix which will turn the current result to backward most and in case of negative momentum parameter $\beta$<0 results in opposite of momentum which is of no use. And if $\beta$=0 that leads to traditional IALM RPCA.

Here is the algorithm procedure:

1.

Initialize: $Y=0$, $S=0$, and $L=0$. Set $\mu ={\displaystyle \frac{1.25}{{\parallel M\parallel }_2}}$, $\rho =1.5$, ${max}_{\mu }={10}^{10}$ and $\beta$=0.5.

2.

Repeat until convergence:

(a)

$\overline{L}$ &= $L_{k-1}$ + $\beta$*($L_{k-1}$ - $L_{k-2}$) $\leftarrow \mathrm{addition}\mathrm{of}\mathrm{inertial}\mathrm{momentum}$

(b)

Update $S_k$ using the shrinkage operator: $S_k=\mathrm{shrink}(M-\overline{L}+Y/\mu ,\lambda /\mu )$.

(c)

Update $L_k$ with the Singular Value Thresholding (SVT) operator:

$$\begin{array}{cc}\hfill L_k& =\mathrm{SVT}(M-S_k+Y/\mu ,1/\mu ),\hfill \\ \hfill \mathrm{where}U\Sigma V^T& =\mathrm{svd}\left(X\right),\hfill \\ \hfill \Sigma & =\mathrm{diag}\left(\mathrm{shrink}\right(\sigma ,\tau \left)\right).\hfill \end{array}$$

(d)

Update Y: $Y=Y+\mu (M-L_k-S_k)$.

(e)

Update $\mu$: $\mu =min(\rho \mu ,{max}_{\mu })$.

3.

Convergence is achieved when $\parallel M-L_k-S_k{\parallel }_F/{\parallel M\parallel }_F<\mathrm{tol}$, with ’tol’ as a pre-defined threshold.

We have used traditional IALM RPCA algorithm with inertial momentum factor. We have not changed L1 norm and thresholding process rather we preserve the norm type. Keeping all traits of IALM RPCA unaltered we have used extrapolation method to introduce inertial momentum factor. Moreover, we have assumed an average momentum factor $\beta$ = 0.5 whereas a paper authored by Xia et al[27]. have used grid search to find beta and other parameters. Grid search can become very time-consuming, especially when there are many parameters exist, datasets are large and each training or optimization run is expensive.

3.4. Overview of Used Multi-Stage Federated Learning Model

Within the scope of this study, the application of the Federated Averaging (FedAvg) algorithm was applied. The FedAvg algorithm [9], which was proposed by McMahan et al. in 2016, has now developed into a standard method in federated learning.

The following is an outline of the primary processes that make up the FedAvg algorithm:

1.

Initialization: The server initializes a global model $w_0$.

$$w_0\leftarrow \mathrm{Initialization}\mathrm{of}\mathrm{the}\mathrm{global}\mathrm{model}$$

2.

Model Distribution: The server sends the global model $w_t$ to each of the K clients.

$$\mathrm{Send}{w}_t\mathrm{to}\mathrm{each}\mathrm{of}\mathrm{the}\mathrm{K}\mathrm{clients}$$

3.

Local Training: Each client k computes an updated model based on its own local data $D_k$. Each client performs E epochs of SGD with batch size B on its local dataset to compute the update.

$$w_{t+1}^k\leftarrow \mathrm{SGD}(w_t;D_k)$$

4.

Local Model Upload: Each client sends its model updates back to the server.

$$\mathrm{Send}{w}_{t+1}^k\mathrm{to}\mathrm{the}\mathrm{server}$$

5.

Global Aggregation: The server aggregates these updates to form a new global model. This is done by taking a weighted average of the clients’ updates, where the weights could be proportional to the number of data points each client has.

$$w_{t+1}\leftarrow {\displaystyle \frac{1}{{\sum }_{k=1}^K{n}_k}}{\displaystyle \sum _{k=1}^K}{n}_k{w}_{t+1}^k$$

6.

Repeat Steps 2-5: This process is repeated for several rounds until the model performance meets the desired criteria.

In this part, a detailed description of the research methodologies that were used in this investigation was provided. The mathematical foundations of the robust variation of Principal Component Analysis (PCA), which is referred to as Robust PCA, has been investigated and evaluated by this study. The principal component analysis (PCA) is subject to severe erroneous or outliers in the data, which is why many people choose the robust form of the principal component analysis (PCA)[6]. In order to solve this problem, a number of different procedures for doing robust PCA were outlined [6]. In terms of the rank of the estimate, the relative error, and the amount of processing time that was necessary, each of these algorithms had its own unique set of benefits and drawbacks.

A lot of effort and consideration has been already given to various kinds of RPCA algorithm to make them more precise and robust. We have tried to make the IALM RPCA method more robust and efficient without changing it’s core methodology which displayed high level of efficacy and performance over the traditional one. Following the breakdown of the algorithm into its component parts and subsequent expression using mathematical notation, the readability of the algorithm was improved significantly.

In order to correctly reconstruct the data, both traditional IALM RPCA and inertial momentum aware IALM RPCA were applied and that results in revealing the robustness of novel approach of IALM RPCA , and after that, the research endeavored to investigate Federated Learning. This cutting-edge method of machine learning allowed for the protection of users’ privacy concerns while simultaneously facilitating the creation of models that were founded on reconstructed data. Due to the fact that it enables the construction of models without the prerequisite of having to provide raw data, this method is significant when dealing with sensitive data.

The methodologies and algorithms that have been discussed up to this point in this section served as the basis for the tests and results that will be discussed in the section that will immediately follow this one. The performance of each method on the dataset under consideration will offer us insights into the method of dealing with outliers in PCA that is the most effective and efficient based on those findings. In conjunction with federated learning, the objective of this research is to develop a method of data analysis that is robust, efficient, and protective of users’ privacy even when dealing with erroneous or infected data. This will be accomplished by developing a technique for data analysis that is robust, efficient, and protective of users’ privacy.

3.5. Global Model Architecture

The global model, a hybrid Inception–Transformer architecture intended for network traffic categorization, was cooperatively trained by multiple clients without exchanging raw data. In Figure 7, the pipeline is demonstrated. The goal is to learn discriminative patterns from network flow data in order to differentiate benign traffic and various attack types.

In order to capture long-range dependencies and contextual linkages among traffic characteristics, the model starts with a Transformer encoder block that contains a Multi-Head Self-Attention (MHSA) layer with three attention heads and a head dimension of 64. To maintain the original feature representations and stabilize training, a residual connection and layer normalization are used. Instead of the conventional feed-forward network used in standard Transformers, the architecture incorporates three stacked Inception blocks. Four parallel branches make up each Inception block: (i) a convolutional branch with kernel size 10, (ii) two-stage convolutional branch with kernel sizes 10 and 30, (iii) a two-stage convolutional branch with kernel sizes 10 and 50, and (iv) a max-pooling branch that is followed by a convolutional layer.

To extract multi-scale patterns from network traffic data, the outputs of these branches are concatenated. After concatenation, batch normalization and ReLU activation are used to enhance feature representation and convergence. A 1x1 convolutional layer is used to project the output of the stacked Inception blocks back to the original dimension after it has been regularized using dropout. The projected features are then added to the attention output to create a residual connection, and layer normalization comes next. The resultant feature maps are combined using Global Average Pooling to provide a fixed-length representation for classification. After a fully connected layer with 32 neurons and ReLU activation, dropout regularization is applied to this representation.

Lastly, class probabilities for both benign traffic and several attack classes are produced via a softmax output layer. This architecture effectively learns both contextual and local discriminative patterns by fusing the multi-scale extraction capability of Inception modules with the global dependency modeling capability of Transformer attention. This makes it appropriate for multi-class attack classification and federated network intrusion detection.

4. Results

The findings of our research shed insight into the discover of novel approach of IALM RPCA adding momentum term using extrapolation method and in combine with Federated Learning, it shows the efficacy as well as the robustness across a wide variety of circumstances. These findings demonstrate both the limitations and strengths that are inherent in Federated Learning models, and they also indicate the critical role that data reconstruction plays in reducing the vulnerabilities that are emphasized. In addition, these findings highlight the crucial role that data reconstruction plays in reducing the vulnerabilities that are highlighted using less computation and time.

4.1. Datasets

In our research, we utilized the ’N-BaIoT Dataset’ [7] available in the UCI Machine Learning Repository. This dataset was developed in response to the notable absence of publicly accessible botnet datasets, specifically tailored for the IoT domain. It encompasses real traffic data extracted from 9 commercial IoT devices that were genuinely compromised by renowned malware, such as Mirai and BASHLITE. Figure 8 illustrates the dataset’s classes along with their respective value counts, including ten ’attack’ types and one ’benign’ class.

The Table 1 provides a detailed overview of the key attributes of the dataset employed in our study. It is structured into two columns: `Characteristic` and `Detail`, offering a clear distinction between the attributes and their specifics. The dataset is categorized as `Multivariate, Sequential`, suggesting its complexity and sequential nature. It encompasses an extensive collection of 7,062,606 instances, each described by 115 real-number attributes. This dataset is versatile, as indicated by its applicability in both `Classification` and `Clustering` tasks. The inclusion of the `Date Donated` offers insight into the dataset with a donation date of March 19, 2018.

Although the primary purpose of this dataset was to differentiate between benign and malicious traffic through the use of anomaly detection techniques, the multifaceted nature of the malicious data enables it to also be used for multi-class classification. This is due to the fact that the malicious data consists of 10 different attacks that were initiated by two different botnets.

For further insights and dataset access, consult the UCI Machine Learning Repository at the following URL: UCI Machine Learning Repository - N-BaIoT Dataset. A deeper exploration of the dataset and related research can be undertaken on Kaggle at: N-BaIoT on Kaggle.

4.2. Confusion Matrix Insights

Figure 9 visualizes the classification capabilities of our model. A noticeable trend is the high true positives under a “No Attack” context. However, adversarial interventions significantly lowered true classifications. A marked improvement in true positives was observed post-reconstruction.

4.3. Impact of Adversarial Attacks

Different levels of poisoning were tested in order to determine how well our Federated Learning strategy performed in a variety of information integrity attacks. The following Table 2 presents a comprehensive summary of the performance of the federated learning model when subjected to a variety of information integrity attack scenarios.

The Table 2 provides a comprehensive evaluation of the Federated Learning approach under varying information integrity attack scenarios. The scenarios are determined based on different poisoning rates ranging from 0%, signifying no attack, to 15%, indicating that 15% clients’ data were subjected to poisoning.

Each scenario comprises:

• The number of clients, uniformly represented as K = 10.
• The accuracies observed during ten distinct training rounds: the initial (1^st round) to the last stage (10^th round).

For each of these rounds, the performance metrics are split into three categories:

• B (Best Client Accuracy): This depicts the highest accuracy achieved by any single client.
• W (Worst Client Accuracy): This represents the lowest accuracy across all clients.
• G (Global Model Accuracy): This metric illustrates the accuracy of the federated global model after aggregating updates from all clients.

One may see how the integrity attacks affect the performance of the Federated Learning model by looking at the Table 2. For instance, when there is no client attack present (the poisoning rate is set to 0%), the global model reaches an accuracy of 73.11% in the first round and significantly improves to 82.74% by the tenth round. It is important to note the model’s robustness and its flexibility to adversarial strategies as the poisoning rate increases. This is clear from the variances in accuracy across different poisoning rates.

According to the Table 2, it is clear that the Federated Learning model maintains a noteworthy performance even when adversarial attacks are taken into consideration, and this is especially true when measured against the global accuracy metric.

4.4. Accuracy Trajectory over Epochs

As clear in Figure 10, the model’s learning trajectory is central to comprehending its adaptability. Notably, even in the context of adversarial challenges, the rate of convergence remained noteworthy, particularly after the introduction of the data reconstruction mechanism. In the above Figure 10, It is clearly seen that the dotted line representing the IALM RPCA with inertial momentum approach outperforms the traditional IALM RPCA approach in all level of attack and the with no attack the model performance is undoubtedly the best one.

4.5. Performance Metrics Under Varying Scenarios

The model’s performance metrics across different scenarios: No Attack, Post-Attack, and Post-Reconstruction by two approaches are consolidated in Figure 11. It’s clear that adversarial attacks notably impaired the performance, but the data reconstruction process played a pivotal role in increasing the model’s efficacy and the inertial momentum aware IALM RPCA with constant momentum factor is giving a better performance than existing IALM RPCA.

4.6. Comparative Analysis of Performance Metrics

The comparison of different evaluation metrics and the time required for noisy data reconstruction are demonstrated here. Here, different types of attack scenario have been taken into consideration to measure the robustness of each algorithm. The improvement column of each table states all that.

Table 3. Performance metrics of the FL model for 25% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	20.00%	76.73%	79.54%	56.73%	59.54%
Precision	80.89%	21.10%	73.37%	76.81%	52.27%	55.71%
Recall	86.16%	21.45%	79.51%	82.19%	58.06%	60.74%
F1-score	81.83%	12.65%	74.97%	77.68%	62.32%	65.03%

Table 3. Performance metrics of the FL model for 25% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	20.00%	76.73%	79.54%	56.73%	59.54%
Precision	80.89%	21.10%	73.37%	76.81%	52.27%	55.71%
Recall	86.16%	21.45%	79.51%	82.19%	58.06%	60.74%
F1-score	81.83%	12.65%	74.97%	77.68%	62.32%	65.03%

Table 4. Performance metrics of the FL model for 50% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	21.40%	77.58%	79.20%	56.18%	57.80%
Precision	80.89%	16.63%	74.55%	76.11%	57.92%	59.48%
Recall	86.16%	23.09%	80.41%	82.00%	57.32%	58.91%
F1-score	81.83%	12.74%	75.73%	77.45%	62.99%	64.71%

Table 4. Performance metrics of the FL model for 50% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	21.40%	77.58%	79.20%	56.18%	57.80%
Precision	80.89%	16.63%	74.55%	76.11%	57.92%	59.48%
Recall	86.16%	23.09%	80.41%	82.00%	57.32%	58.91%
F1-score	81.83%	12.74%	75.73%	77.45%	62.99%	64.71%

Table 5. Performance metrics of the FL model for 75% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	18.84%	77.63%	79.27%	58.79%	60.43%
Precision	80.89%	11.62%	74.05%	76.79%	62.43%	65.17%
Recall	86.16%	17.41%	80.23%	81.77%	62.82%	64.36%
F1-score	81.83%	8.04%	75.90%	77.27%	67.05%	69.23%

Table 5. Performance metrics of the FL model for 75% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	18.84%	77.63%	79.27%	58.79%	60.43%
Precision	80.89%	11.62%	74.05%	76.79%	62.43%	65.17%
Recall	86.16%	17.41%	80.23%	81.77%	62.82%	64.36%
F1-score	81.83%	8.04%	75.90%	77.27%	67.05%	69.23%

Table 6. Performance metrics of the FL model for 100% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	23.70%	74.82%	75.19%	51.82%	51.49%
Precision	80.89%	11.72%	72.78%	72.97%	61.06%	61.25%
Recall	86.16%	26.13%	77.55%	76.53%	51.42%	50.4%
F1-score	81.83%	15.00%	73.25%	72.64%	58.25%	57.64%

Table 6. Performance metrics of the FL model for 100% attack

Metric	No Attack	Post Attack	Post-Reconstruction		Improvement
			IALM	IALM (Mom.)	IALM	IALM (Mom.)
Accuracy	83.34%	23.70%	74.82%	75.19%	51.82%	51.49%
Precision	80.89%	11.72%	72.78%	72.97%	61.06%	61.25%
Recall	86.16%	26.13%	77.55%	76.53%	51.42%	50.4%
F1-score	81.83%	15.00%	73.25%	72.64%	58.25%	57.64%

This comparative evaluation shows that the post-reconstruction performance came close to approaching the no-attack baseline, which highlights the efficiency of the data reconstruction method that we employed. Surprisingly, even in the presence of a threat condition, the reconstructed model displayed a strong recovery, reaching an accuracy of 83.62 percent, which was only a bit behind the 86.02 percent that was seen in the case where there was no attack.

4.7. Comparative Analysis of Reconstruction Time

To compare time , reconstruction has been done under different attack scenarios 20 times and average , median , max time and min time are recorded.

The IALM with Momentum methodology consistently needs more computing time than the normal IALM method across all attack percentages, as shown by the reconstruction time analysis in Table 7. For example, the average reconstruction time went from 524.35 seconds for IALM to 557.34 seconds for IALM with Momentum under the 25% attack scenario. The momentum-based variation exhibits increased minimum, maximum, average, and median reconstruction durations for 50%, 75%, and 100% attack settings.

Table 7. Comparative Analysis of Reconstruction Time(sec) Under Different Attack Percentages

Attack (%)	Min Time		Max Time		Average Time		Median Time
	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)
25%	508.08	538.47	573.27	654.05	524.35	557.34	519.13	548.285
50%	452.50	491.08	475.82	514.78	460.31	499.12	459.89	496.27
75%	465.23	506.76	512.35	568.31	489.59	533.00	494.06	537.00
100%	464.90	506.06	521.25	560.12	491.50	535.72	498.82	546.14

Table 7. Comparative Analysis of Reconstruction Time(sec) Under Different Attack Percentages

Attack (%)	Min Time		Max Time		Average Time		Median Time
	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)	IALM	IALM (Mom.)
25%	508.08	538.47	573.27	654.05	524.35	557.34	519.13	548.285
50%	452.50	491.08	475.82	514.78	460.31	499.12	459.89	496.27
75%	465.23	506.76	512.35	568.31	489.59	533.00	494.06	537.00
100%	464.90	506.06	521.25	560.12	491.50	535.72	498.82	546.14

The momentum mechanism, which adds an extra update component during optimization, is responsible for this rise in computing cost. This approach helps to increase optimization stability and refine low-rank reconstruction even if it slows down the reconstruction process. As a result, although though IALM with Momentum takes longer to execute, it achieves better reconstruction quality, which improves classification performance over normal IALM in terms of accuracy, precision, recall, and F1-score.

As a result, there is a trade-off between reconstruction efficacy and computational efficiency. While IALM with Momentum compromises more computing time to obtain higher overall prediction performance and robustness against attacks, standard IALM delivers quicker execution. But in case of good Cyber-Physical System (CPS) attack removing is much more important than time efficacy as if attack remains it can results in rigorous loss in the long run.

4.8. Performance Assessment of Various $\beta$ Values at Different Poisoning Attack Intensities

For each attack level, comparison of the different $\beta$ values and identification the best performer has been performed here.

Table 8. Performance under 25% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	78.01	74.30	79.98	75.85
0.3	78.06	75.36	81.50	76.45
0.5	79.54	76.81	82.19	77.68
0.7	79.62	76.78	82.14	77.72
0.9	78.01	75.70	80.55	76.25

Table 8. Performance under 25% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	78.01	74.30	79.98	75.85
0.3	78.06	75.36	81.50	76.45
0.5	79.54	76.81	82.19	77.68
0.7	79.62	76.78	82.14	77.72
0.9	78.01	75.70	80.55	76.25

Table 9. Performance under 50% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	78.18	75.87	81.53	76.67
0.3	79.28	76.58	82.10	77.75
0.5	79.20	76.11	82.00	77.45
0.7	78.24	75.69	81.66	76.68
0.9	77.55	74.52	80.52	75.78

Table 9. Performance under 50% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	78.18	75.87	81.53	76.67
0.3	79.28	76.58	82.10	77.75
0.5	79.20	76.11	82.00	77.45
0.7	78.24	75.69	81.66	76.68
0.9	77.55	74.52	80.52	75.78

Table 10. Performance under 75% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	79.28	76.34	81.58	77.15
0.3	78.66	75.75	81.75	76.97
0.5	79.27	76.79	81.77	77.28
0.7	76.89	73.89	79.81	74.96
0.9	78.57	76.68	81.25	77.03

Table 10. Performance under 75% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	79.28	76.34	81.58	77.15
0.3	78.66	75.75	81.75	76.97
0.5	79.27	76.79	81.77	77.28
0.7	76.89	73.89	79.81	74.96
0.9	78.57	76.68	81.25	77.03

Table 11. Performance under 100% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	74.54	71.23	76.32	72.09
0.3	71.55	68.08	71.49	67.65
0.5	75.19	72.97	76.53	72.64
0.7	79.18	75.81	81.72	77.22
0.9	77.64	76.09	80.34	75.69

Table 11. Performance under 100% Poisoning Attack

$\mathit{\beta }$	Accuracy	Precision	Recall	F1-score
0.1	74.54	71.23	76.32	72.09
0.3	71.55	68.08	71.49	67.65
0.5	75.19	72.97	76.53	72.64
0.7	79.18	75.81	81.72	77.22
0.9	77.64	76.09	80.34	75.69

Regarding the impact of the inertial momentum parameter $\beta$, a distinct pattern becomes apparent. The optimal balance between robustness and convergence speed is consistently achieved with moderate values of $\beta$ (0.5 – 0.7). In particular, $\beta$ = 0.7 exhibits higher resistance in the most severe 100% assault scenario, achieving an F1-score of 77.22%, whereas $\beta$ = 0.3 performs best under a 50% poisoning attack. Although $\beta$ = 0.5 does not always achieve the highest score, it maintains stable performance across all attack intensities, indicating strong generalization. In general, extremely tiny momentum ($\beta$ = 0.1) and very large momentum ($\beta$ = 0.9) result in worse performance, indicating that the quality of RPCA reconstruction may be adversely affected by either insufficient or excessive inertial effect. Overall, the findings show that the strongest resistance against data poisoning attempts is offered by $\beta$ values between 0.5 and 0.7.

5. Discussion

The results of our study highlight the inherent potential of federated learning (FL) in assuring model robustness, particularly when it is set to the test in adversarial situations. The adaptability of the FL approach is further shown by the introduction and subsequent effectiveness of data reconstruction techniques, which is highlighted by the findings.

5.1. Relevance of Federated Learning

The findings of our study provide strong evidence on the efficacy of Federated Learning, demonstrating its robustness not only in typical scenarios but also in the face of adversarial conditions. This may be observed from the model’s persistent effort to maintain an outstanding level of performance, as illustrated in Table 2. The model demonstrated its flexibility and adaptation in the face of increasing rates of adverse poisoning. This was obvious from the relatively steady global accuracy metrics, even in situations where the integrity of client data was corrupted.

5.2. Significance of Data Reconstruction

The impact of adversarial attacks on the Federated Learning paradigm, specifically in relation to model accuracy and classification abilities, is unquestionable. This is demonstrated in Figure 9. Nevertheless, a major turn occurs with the implementation of data reconstruction approaches. The observed increase in the number of true positive results after the reconstruction process indicates that the implementation of strategic data restoration techniques has the potential to mitigate the harmful impacts of adversarial attacks. The robustness of the model, including its ability to recover from losses as shown in Figure 10, emphasizes the significant impact of effective data reconstruction methods.

5.3. Comparison with Previous Work

The findings of our study can be situated within the broader framework of the N-BaIoT dataset study, which focused on the detection of IoT botnet attacks [7]. The dataset was aimed to enable the distinction between benign and malicious network traffic. However, our research expanded its applicability by investigating its potential inside a Federated Learning framework with the novel IALM RPCA approach, particularly in the presence of adversarial circumstances. The expansion of the dataset’s application scope is not only enhanced, but it also highlights the practicality of employing Federated Learning models in real-world situations that may involve limitations.

5.4. Future Research Directions

Although our work provides a full overview of the Federated Learning landscape with momentum based IALM RPCA in the presence of adversarial situations, there are still some areas that have not been explored:

• This study aims to conduct a comprehensive examination of advanced data reconstruction strategies, with the potential to enhance the recovery capabilities of compromised models to a greater extent.
• The expansion of this research to include other publically accessible datasets would serve to enhance the validation breadth, hence broadening the applicability of the findings.
• An investigation on the extent to which these findings can be scaled, particularly in cases where the number of clients (K) is significantly increased.
• This study is designed to investigate and evaluate advanced adversarial techniques and their corresponding responses in the context of Federated Learning.

In summary, our study highlights the undeniable potential of Federated Learning in addressing adversarial challenging situations. When coupled with effective data reconstruction techniques, these models have the ability to demonstrate durability, therefore guaranteeing their potential usability in various real-world situations where data integrity may be compromised. The findings, at their essence, highlight the inherent robustness of the Federated Learning paradigm, which is especially evident when the paradigm is further strengthened by capable data reconstruction approaches. Even while threatening activities can in fact affect performance, our research has shown that strategic countermeasures with novel approach can guarantee model robustness and capabilities.

6. Conclusions

In this paper, we present an in-depth investigation of the complexities of a multi-stage Federated Learning model, with an emphasis on the model’s defense against Information Integrity Attacks in IoT contexts. The core focus of the study focuses on the integration of attack detection and data recovery, which constitutes a novel technique. This is demonstrated by the comprehensive examination of poisoning attacks, namely those arising from the presence of sparse noise inside the environment of the IoT. The results, which were emphasized by the utilization of the IALM RPCA Algorithm, depict a promising outlook on the robustness of Federated Learning in handling reconstructed data following an attack. The results obtained from this research provide a significant basis for the evolving IoT environment and the increasing significance of decentralized learning approaches like Federated Learning. Moreover, a novel approach has been detected which outperforms traditional IALM RPCA in case of reconstruction. Nevertheless, the fact that this journey has been informative, it is evident that the continuously developing domain of IoT security and training needs further investigation, hence demonstrating the potential for ongoing advancements and enhancements in this domain.