Optimization, Communication, and Personalization in Federated Learning for Massive Networks

1. Introduction

The rapid proliferation of edge devices such as smartphones, wearables, and IoT sensors has created unprecedented opportunities for distributed machine learning paradigms. Among these, Federated Learning (FL) has emerged as a promising approach for collaboratively training machine learning models across decentralized data sources while preserving user privacy and minimizing raw data exchange. Unlike traditional centralized learning, where all training data is aggregated on a central server, FL enables participants (or clients) to train local models on their private data and share only model updates with a central server for aggregation. As FL matures from academic exploration to real-world deployment, it faces a myriad of challenges that threaten its scalability, efficiency, and robustness. Chief among these is the issue of communication bottlenecks, particularly in large-scale federated learning environments involving millions of devices with heterogeneous computational capabilities, network bandwidths, and availability patterns. In such settings, communication between clients and the central server quickly becomes the primary performance bottleneck, overshadowing computation time and severely impacting convergence rates. The fundamental federated learning algorithm, Federated Averaging (FedAvg), introduced by McMahan et al., mitigates communication costs by performing multiple local updates before communicating with the server. However, even with this improvement, the algorithm still suffers from high communication overhead due to the iterative nature of model training and the high dimensionality of modern deep learning models. When scaled to tens or hundreds of millions of clients, these costs become prohibitive [1]. Moreover, the communication issue is exacerbated by the inherent heterogeneity in client devices. Network conditions can vary significantly across clients, especially when training spans different geographic regions or network infrastructures. Clients may have asymmetric uplink and downlink speeds, intermittent connectivity, or restrictive data plans, all of which contribute to unreliable and expensive communication [2]. As a result, the efficiency of the FL process becomes closely tied to the slowest or least reliable clients, a phenomenon often referred to as the straggler effect [3]. Various strategies have been proposed to tackle communication challenges in FL, including model compression and sparsification, client selection and participation scheduling, asynchronous updates, and novel aggregation protocols. While each of these methods provides partial solutions, none singularly resolves the tension between communication efficiency, convergence accuracy, and system robustness. Furthermore, there is a fundamental trade-off between reducing communication and maintaining model performance [4]. Aggressive compression or quantization techniques may reduce communication payloads but can degrade model accuracy or cause instability in convergence. Similarly, selective client participation reduces total bandwidth use but may introduce biases if the selected clients are not representative of the overall data distribution. This survey aims to provide a comprehensive overview of the communication bottleneck in large-scale federated learning, examining its root causes, associated trade-offs, and state-of-the-art solutions. We categorize the landscape of proposed methods into several major strategies, analyze their theoretical and empirical effectiveness, and discuss their practical implications. Additionally, we highlight open research questions and outline potential directions for future work in communication-efficient federated learning systems. The rest of this paper is organized as follows. Section 2 provides background on the federated learning paradigm and formulates the communication problem. Section 3 reviews model compression and quantization techniques. Section 4 discusses adaptive communication protocols [5]. Section ?? explores strategies for client selection and scheduling. Section 6 presents asynchronous and decentralized communication frameworks. Section ?? outlines current challenges and future research directions [6]. Finally, Section 13 concludes the survey.

2. Background and Problem Formulation

Federated learning (FL) is a distributed optimization paradigm in which a central server coordinates training across a large number of clients, each holding a local dataset that is never directly shared with the server or other clients. The central objective in FL is to minimize a global empirical risk defined over the union of all local data distributions, while maintaining data locality. Formally, let $\mathcal{K}=\{1,2,\dots ,K\}$ denote the set of participating clients, and let ${\mathcal{D}}_k$ represent the local dataset held by client $k\in \mathcal{K}$, such that the total number of samples across all clients is $n={\sum }_{k=1}^K{n}_k$, where $n_k=\left|{\mathcal{D}}_k\right|$. We define the global objective function as the following empirical risk minimization problem:

$$\underset{\mathbf{w}\in {\mathbb{R}}^d}{min}F\left(\mathbf{w}\right)=\sum _{k=1}^K{\displaystyle \frac{n_k}{n}}{F}_k\left(\mathbf{w}\right)\mathrm{where}{F}_k\left(\mathbf{w}\right)={\displaystyle \frac{1}{n_k}}\sum _{{\xi }_i\in {\mathcal{D}}_k}\ell (\mathbf{w};{\xi }_i),$$

(1)

where $\mathbf{w}\in {\mathbb{R}}^d$ is the model parameter vector, $\ell (\mathbf{w};{\xi }_i)$ is the loss function evaluated on sample ${\xi }_i$ from client k, and $F_k\left(\mathbf{w}\right)$ is the local empirical loss function specific to client k [7]. The central goal of federated optimization is to obtain a model ${\mathbf{w}}^{*}$ that minimizes $F\left(\mathbf{w}\right)$, while enforcing that data remains decentralized and communication between clients and the server is minimized [8]. The canonical optimization strategy in federated learning is the Federated Averaging (FedAvg) algorithm, which proceeds in rounds. At each communication round t, the server broadcasts the current global model ${\mathbf{w}}^t$ to a subset of clients ${\mathcal{S}}^t\subseteq \mathcal{K}$ [9]. Each participating client $k\in {\mathcal{S}}^t$ performs E steps of local stochastic gradient descent (SGD) on its own dataset to compute an updated model ${\mathbf{w}}_k^{t+1}$:

$${\mathbf{w}}_k^{t+1}={\mathbf{w}}^t-\eta \sum _{e=1}^E\nabla F_k\left({\mathbf{w}}_k^{t,e}\right),$$

(2)

where $\eta >0$ is the local learning rate and ${\mathbf{w}}_k^{t,e}$ is the model parameter at local epoch e on client k. The server then aggregates the updates via a weighted average:

$${\mathbf{w}}^{t+1}=\sum _{k\in {\mathcal{S}}^t}{\displaystyle \frac{n_k}{{\sum }_{j\in {\mathcal{S}}^t}{n}_j}}{\mathbf{w}}_k^{t+1}\text{[10]}$$

(3)

While FedAvg reduces communication frequency by allowing multiple local updates before synchronization, the communication burden still remains a significant bottleneck. Each participating client must transmit a model vector ${\mathbf{w}}_k^{t+1}\in {\mathbb{R}}^d$ back to the server after each round, leading to $O\left(d\right)$ communication cost per client per round. In deep neural networks, where d often ranges from millions to billions of parameters, such communication can become prohibitively expensive, particularly when extended to large-scale settings involving thousands or millions of clients. Moreover, the number of communication rounds T required for convergence may increase under non-iid data distributions and client heterogeneity, further compounding the communication overhead. Additionally, the stochastic nature of client availability introduces further complexity. Let ${\mathcal{A}}_t\subseteq \mathcal{K}$ denote the set of clients available at round t, and ${\mathcal{S}}^t\subseteq {\mathcal{A}}_t$ be the subset selected for participation [11]. The randomness in ${\mathcal{A}}_t$ introduces uncertainty in the optimization trajectory. If the availability is highly skewed or correlated with data characteristics, it may result in biased updates that affect convergence stability and generalization performance. Communication cost can be mathematically characterized in terms of total bits transmitted per round [12]. Let C denote the number of participating clients per round, d be the number of model parameters, and b be the number of bits used to encode each parameter (e.g., 32 bits for standard float32) [13]. Then, the communication cost per round is approximately $C·d·b$. Aggregated over T rounds, the total communication cost is $O\left(CdbT\right)$, which scales linearly with the number of rounds, model size, and clients per round. In practice, this scaling renders vanilla FL methods impractical for massive-scale deployments unless specialized communication-efficient strategies are employed. The above formulation also assumes a synchronous update protocol, where the server waits for all selected clients in ${\mathcal{S}}^t$ to complete their local updates and send back their model parameters. However, in realistic scenarios, clients may have heterogeneous computational capabilities and communication bandwidths, leading to stragglers—clients that significantly delay the round. To address this, asynchronous FL variants have been proposed, where clients communicate with the server independently, and the server performs updates based on stale or partial information [14]. Let ${\tau }_k^t$ denote the staleness of client k’s update at round t, defined as the number of rounds that have elapsed since the client received the global model ${\mathbf{w}}^{t-{\tau }_k^t}$ [15]. The presence of such stale gradients complicates convergence analysis and often necessitates additional mechanisms such as temporal weighting or delay compensation to mitigate performance degradation. Furthermore, in environments with millions of clients, a fundamental challenge arises in selecting a subset of clients at each round that is both representative of the global data distribution and capable of contributing timely updates [16]. Random selection may yield suboptimal coverage, while uniform stratification is difficult without centralized access to client data. Let ${\mathbf{p}}^t=[p_1^t,p_2^t,\dots ,p_K^t]$ be the probability vector for client selection at round t [17]. Designing ${\mathbf{p}}^t$ to maximize utility while satisfying fairness, availability, and efficiency constraints is an open optimization problem with significant implications on both communication efficiency and model performance [18]. In summary, the communication bottleneck in large-scale federated learning is a complex, multi-faceted problem governed by the interaction between model dimensionality, number of clients, data heterogeneity, availability dynamics, and system-level constraints. Mathematical formulations such as those in equations (1)–(3) provide a foundational lens for understanding the optimization landscape, yet they abstract away critical systems-level details that must be accounted for in real-world implementations [19]. As the scale of federated deployments continues to grow, a deeper theoretical and practical understanding of communication-efficient FL becomes not only desirable but essential [20].

3. Model Compression and Quantization Techniques

To alleviate the communication bottleneck in large-scale federated learning systems, one of the most widely adopted strategies is model compression, which aims to reduce the volume of data transmitted between clients and the central server. Compression techniques achieve this by reducing the size of the model updates (or gradients) communicated during each training round, typically through quantization, sparsification, or a combination of both. In this section, we provide a comprehensive exploration of these techniques, emphasizing their mathematical underpinnings, operational trade-offs, and implications for convergence behavior in federated optimization.

3.1. Gradient and Model Quantization

Quantization techniques reduce communication costs by representing high-precision floating-point values with lower-precision approximations [21]. Let $\mathbf{g}\in {\mathbb{R}}^d$ denote the gradient or model update vector produced by a client after local training. Instead of transmitting $\mathbf{g}$ directly, the client transmits a quantized version $\mathcal{Q}\left(\mathbf{g}\right)$, where $\mathcal{Q}:{\mathbb{R}}^d\to {\mathcal{C}}^d$ is a quantization operator mapping vectors to a finite codebook $\mathcal{C}\subset \mathbb{R}$. The expected communication savings depend on the cardinality of $\mathcal{C}$ and the encoding scheme used. For instance, if $\mathcal{C}$ contains $2^b$ symbols, then each coordinate can be represented using b bits, yielding a communication cost of $d·b$ bits per transmission [22]. A commonly used stochastic quantization method is the s-level unbiased quantizer ${\mathcal{Q}}_s$, introduced in [23]. For a scalar value $g_i\in \mathbb{R}$, the quantized output is:

$${\mathcal{Q}}_s\left(g_i\right)={\parallel \mathbf{g}\parallel }_2·\mathrm{sign}\left(g_i\right)·{\xi }_i,\mathrm{where}{\xi }_i\in \left\{{\displaystyle \frac{j}{s}}:j\in \left[s\right]\right\},$$

(4)

and ${\xi }_i$ is sampled such that $\mathbb{E}\left[{\mathcal{Q}}_s\left(g_i\right)\right]=g_i$, thereby ensuring unbiasedness. The quantization level s controls the trade-off between communication cost and quantization error, with higher values of s offering better fidelity at the expense of increased bit complexity. The quantization error can be bounded as:

$$\mathbb{E}\left[\parallel {\mathcal{Q}}_s{\left(\mathbf{g}\right)-\mathbf{g}\parallel }_2^2\right]\le min\left\{{\displaystyle \frac{d}{s^2}},{\displaystyle \frac{1}{s}}\right\}{\parallel \mathbf{g}\parallel }_2^2,$$

(5)

implying that lower quantization levels incur larger approximation errors that may slow convergence or destabilize training. Another popular quantization technique is ternary quantization, where gradients are restricted to the set $\{-1,0,+1\}$ with scalar scaling, allowing efficient encoding using just 2 bits per coordinate.

3.2. Sparsification and Top-k Compression

Another prominent compression technique involves sparsification, wherein only the most informative components of the gradient vector are communicated. In Top-k sparsification, a client transmits only the k largest-magnitude components of its update $\mathbf{g}$, with the rest set to zero. Let ${\mathcal{S}}_k\left(\mathbf{g}\right)$ be the sparsification operator that retains the k largest elements in magnitude:

$${\mathcal{S}}_k{\left(\mathbf{g}\right)}_i=\left\{\begin{array}{cc}{g}_i\hfill & \mathrm{if}i\in \mathrm{Top}-k\left(\mathbf{g}\right),\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(6)

To ensure information is not permanently lost during sparsification, an error feedback mechanism is typically employed. Each client maintains a residual vector ${\mathbf{r}}_t$ representing the accumulated quantization or sparsification error. Before computing the update at round t, the client adds ${\mathbf{r}}_t$ back into the local update:

$${\tilde{\mathbf{g}}}_t={\mathbf{g}}_t+{\mathbf{r}}_t,{\mathbf{r}}_{t+1}={\tilde{\mathbf{g}}}_t-{\mathcal{S}}_k\left({\tilde{\mathbf{g}}}_t\right).$$

(7)

This error feedback loop ensures that untransmitted components are eventually communicated, enabling convergence under certain conditions [24]. The convergence rate of sparsified SGD with error feedback has been analyzed in, showing that sparsified methods can retain convergence guarantees comparable to vanilla SGD, provided the sparsification budget k is sufficiently large [25].

3.3. Low-Rank and Sketching Methods

In addition to quantization and sparsification, recent work has explored low-rank approximation techniques to compress model updates. These methods approximate the gradient matrix $\mathbf{G}\in {\mathbb{R}}^{d\times B}$, where B is the mini-batch size, with a low-rank matrix $\mathbf{U}{\mathbf{V}}^{\top }$ where $\mathbf{U}\in {\mathbb{R}}^{d\times r}$ and $\mathbf{V}\in {\mathbb{R}}^{B\times r}$ for rank $r\ll min(d,B)$. The compressed update is transmitted in the form of the factors $\mathbf{U}$ and $\mathbf{V}$, reducing communication to $O\left(r\right(d+B\left)\right)$ [26]. Sketching techniques such as CountSketch or Random Projection further reduce dimensionality by projecting high-dimensional updates into lower-dimensional subspaces using randomized linear maps. Let $\mathbf{S}\in {\mathbb{R}}^{m\times d}$ be a sketching matrix, where $m\ll d$, then the compressed update is:

$$\tilde{\mathbf{g}}=\mathbf{S}\mathbf{g},\mathrm{and}\mathrm{recovered}\left(\mathrm{approximately}\right)\mathrm{as}\widehat{\mathbf{g}}={\mathbf{S}}^{†}\tilde{\mathbf{g}},$$

(8)

where ${\mathbf{S}}^{†}$ is the pseudo-inverse of $\mathbf{S}$. While these techniques provide strong compression ratios, they often require centralized coordination or shared randomness between clients and server, complicating decentralized implementations.

3.4. Implications for Convergence and System Design

The use of compression in federated learning introduces a fundamental trade-off between communication efficiency and optimization fidelity. Let ${\mathbf{w}}^t$ denote the global model at round t, and ${\widehat{\mathbf{g}}}^t$ be the aggregated compressed update. The global model update becomes:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t-{\eta }_t{\widehat{\mathbf{g}}}^t.$$

(9)

When ${\widehat{\mathbf{g}}}^t$ is a biased or noisy approximation of the true gradient ${\mathbf{g}}^t$, the convergence rate and final accuracy can be adversely affected [27]. Several works have shown that with careful design—e.g., unbiased quantization, error feedback, or adaptive compression levels—it is possible to retain convergence guarantees with bounded error. Let $\mathbb{E}\left[\parallel {\widehat{\mathbf{g}}}^t-{\mathbf{g}}^t{\parallel }^2\right]\le {\delta }_t$, then under smoothness assumptions on the objective $F\left(\mathbf{w}\right)$, we can bound the suboptimality gap after T rounds as:

$$\mathbb{E}\left[F\left({\mathbf{w}}^T\right)-F\left({\mathbf{w}}^{*}\right)\right]=O\left({\displaystyle \frac{1}{T}}+{\displaystyle \frac{1}{T}}\sum _{t=1}^T{\delta }_t\right).$$

(10)

This result illustrates that as long as the average compression error ${\delta }_t$ remains small, compressed FL methods can converge to a neighborhood of the global optimum [28]. However, in practice, the choice of compression scheme, its implementation efficiency, and its interaction with client heterogeneity must all be carefully calibrated to avoid performance degradation.

3.5. Summary

Model compression and quantization constitute a powerful class of techniques to mitigate the communication bottleneck in federated learning [29]. Through quantized updates, sparsified gradients, low-rank factorization, and random projections, these methods aim to reduce the size of transmitted messages without sacrificing model performance. However, compression inevitably introduces approximation errors, necessitating the use of error correction mechanisms such as residual accumulation and adaptive encoding. While the theoretical underpinnings of these methods are increasingly well-understood, practical deployment remains challenging due to system heterogeneity, variable client bandwidths, and compatibility with secure aggregation protocols [30]. In the next section, we examine adaptive communication strategies that dynamically adjust communication frequency and client participation to further enhance the efficiency of large-scale federated learning systems.

4. Adaptive Communication Protocols

In large-scale federated learning, the communication overhead not only depends on the size of the transmitted model updates but also on the frequency and timing of communication rounds between the clients and the central server [31]. Adaptive communication protocols seek to optimize the trade-off between communication efficiency and convergence speed by dynamically adjusting when and how clients exchange information. This section delves into the mathematical frameworks, algorithmic strategies, and theoretical guarantees underlying adaptive communication in federated optimization.

4.1. Communication Frequency Adaptation

A fundamental aspect of adaptive communication is controlling the communication interval, i.e., how many local computation steps a client performs before communicating with the server. Standard FedAvg uses a fixed number of local epochs E per communication round, but recent research suggests that dynamically tuning E based on system or optimization metrics can significantly improve efficiency. Consider the local update at client k after $E_k^t$ local epochs during round t:

$${\mathbf{w}}_k^{t+1}={\mathbf{w}}^t-\eta \sum _{e=1}^{E_k^t}\nabla F_k\left({\mathbf{w}}_k^{t,e}\right),$$

(11)

where $E_k^t$ is now client- and round-dependent. Increasing $E_k^t$ reduces communication frequency but may lead to local model drift, especially in heterogeneous data environments, which can slow down convergence or degrade final accuracy [32]. Conversely, smaller $E_k^t$ values increase communication overhead but ensure more frequent global synchronization. Adaptive strategies model this trade-off explicitly. For example, adaptive local update control approaches estimate an optimal $E_k^t$ by minimizing an objective that balances local computation cost $C_{\mathrm{comp}}$ and communication cost $C_{\mathrm{comm}}$, while constraining convergence error. Formally, one may solve:

$$\underset{E_k^t\in {\mathbb{Z}}_{+}}{min}\alpha C_{\mathrm{comp}}\left(E_k^t\right)+\beta C_{\mathrm{comm}}\left(E_k^t\right)\mathrm{s}.\mathrm{t}.ϵ\left(E_k^t\right)\le {ϵ}_{max},$$

(12)

where $\alpha ,\beta >0$ are weighting factors and $ϵ\left(E_k^t\right)$ is the expected convergence error as a function of local update length. Analytical approximations of $ϵ\left(E_k^t\right)$ depend on problem smoothness and heterogeneity and can be used to derive principled adaptation policies.

4.2. Event-Triggered Communication

Rather than communicating at fixed intervals, event-triggered communication protocols transmit updates only when local model changes exceed a specified threshold. This approach leverages the observation that not all local updates contribute equally to global optimization progress. Formally, let ${\mathbf{w}}_k^{t+1}$ be the local model after local computation, and ${\mathbf{w}}_k^{t,\mathrm{last}}$ be the model last communicated by client k. The client transmits the update only if:

$$\parallel {\mathbf{w}}_k^{t+1}-{\mathbf{w}}_k^{t,\mathrm{last}}{\parallel }_p\ge {\tau }_k^t,$$

(13)

where ${\parallel ·\parallel }_p$ denotes the p-norm (typically $p=2$ or $p=\infty$) and ${\tau }_k^t>0$ is a dynamically adjusted threshold [33]. Choosing ${\tau }_k^t$ involves a trade-off: larger thresholds reduce communication but may delay convergence, while smaller thresholds increase communication load [34]. Some protocols adapt thresholds ${\tau }_k^t$ based on network conditions, client importance, or model dynamics [35]. Event-triggered schemes can be combined with compression techniques to amplify communication savings while preserving accuracy.

4.3. Adaptive Client Participation

Another dimension of adaptivity addresses which clients participate in each communication round [36]. Large-scale federated systems cannot afford to involve all clients simultaneously due to bandwidth constraints and client availability variability. Adaptive client selection policies aim to identify subsets ${\mathcal{S}}^t\subseteq \mathcal{K}$ that maximize training utility while minimizing communication. Let ${\mathbf{p}}^t=[p_1^t,\dots ,p_K^t]$ be the vector of client selection probabilities at round t. Adaptive methods adjust ${\mathbf{p}}^t$ based on criteria such as:

• Client reliability: Favoring clients with stable connectivity and timely responses to reduce straggler effects [37].
• Data representativeness: Selecting clients whose data distributions complement existing updates to reduce bias and improve generalization.
• Update magnitude: Prioritizing clients with larger local model changes to accelerate learning progress.

Formally, selection probabilities can be optimized by solving:

$$\underset{{\mathbf{p}}^t\in {\Delta }^K}{max}U\left({\mathbf{p}}^t\right)-\lambda R\left({\mathbf{p}}^t\right),$$

(14)

where ${\Delta }^K$ is the probability simplex, $U\left({\mathbf{p}}^t\right)$ measures expected utility of the selected clients, $R\left({\mathbf{p}}^t\right)$ quantifies communication or fairness costs, and $\lambda$ balances the trade-off. Heuristic algorithms such as importance sampling or multi-armed bandits have been proposed to solve this problem efficiently.

4.4. Theoretical Guarantees

Adaptive communication protocols often introduce non-uniform communication intervals and variable client participation, complicating convergence analysis. Nonetheless, under assumptions such as bounded gradient variance and smoothness of local objectives, recent theoretical work has established convergence guarantees for adaptive FL methods [38]. For example, when clients perform $E_k^t$ local SGD steps before communication, the expected suboptimality after T rounds satisfies:

$$\mathbb{E}[F\left({\mathbf{w}}^T\right)-F\left({\mathbf{w}}^{*}\right)]\le O\left({\displaystyle \frac{1}{{\sum }_{t=1}^T{\sum }_{k\in {\mathcal{S}}^t}{n}_k{E}_k^t}}+{\displaystyle \frac{{\sum }_{t=1}^T{\sum }_{k\in {\mathcal{S}}^t}{\sigma }_k^2}{{\left({\sum }_{t=1}^T{\sum }_{k\in {\mathcal{S}}^t}{n}_k{E}_k^t\right)}^2}}\right),$$

(15)

where ${\sigma }_k^2$ denotes the variance of local gradients on client k. This bound highlights the importance of balancing communication frequency and local computation to optimize convergence speed [39].

4.5. Practical Considerations and Challenges

Adaptive communication protocols require mechanisms to monitor client states, estimate update significance, and coordinate participation, all of which introduce overhead and system complexity. Further, fairness concerns arise since clients with poor connectivity or slower updates may be excluded more frequently, potentially biasing the model and reducing inclusivity. Designing adaptive schemes that are robust, privacy-preserving, and scalable remains an open challenge. Combining adaptive communication with compression, asynchronous updates, and secure aggregation protocols presents promising avenues for research and development.

4.6. Summary

Adaptive communication protocols dynamically tune communication frequency, event triggers, and client participation to mitigate bandwidth constraints in large-scale federated learning. These approaches leverage system and optimization feedback to optimize the communication-computation trade-off, leading to improved efficiency without sacrificing convergence guarantees [40]. However, practical deployment necessitates addressing challenges related to client heterogeneity, fairness, and implementation overhead. The next section explores client selection strategies in greater detail, a critical component of adaptive communication frameworks.

5. Client Selection and Scheduling

In federated learning systems encompassing potentially millions of devices, selecting which subset of clients participates in each communication round is a crucial factor affecting both system efficiency and model performance [41]. Client selection and scheduling techniques aim to optimize the use of limited communication resources, reduce latency caused by slow or unavailable clients (stragglers), and improve convergence rates by judiciously leveraging data diversity and client capabilities [42]. This section provides an in-depth examination of the mathematical formulations, algorithmic strategies, and theoretical insights underpinning client selection in large-scale federated learning [43].

5.1. Problem Formulation

Let $\mathcal{K}=\{1,2,\dots ,K\}$ denote the entire set of clients, where K can be very large [44]. At each communication round t, a subset ${\mathcal{S}}^t\subseteq \mathcal{K}$ of size $S\ll K$ is selected to participate. The selection process can be modeled as sampling from a probability distribution ${\mathbf{p}}^t=(p_1^t,p_2^t,\dots ,p_K^t)$ over clients, with constraints:

$$\sum _{k=1}^K{p}_k^t=1,p_k^t\ge 0,\forall k,t\text{[45]}.$$

Each client k possesses local data distribution ${\mathcal{D}}_k$ and may have associated costs or capabilities such as communication bandwidth $b_k$, computational speed $c_k$, and availability indicator $a_k^t\in \{0,1\}$ [46]. The goal of client selection is to maximize a utility function $U\left({\mathcal{S}}^t\right)$ that quantifies the contribution of selected clients towards global model improvement, subject to resource constraints and fairness criteria. A common optimization formulation is:

$$\underset{{\mathcal{S}}^t\subseteq \mathcal{K},\left|{\mathcal{S}}^t\right|=S}{max}{\mathbb{E}}_{{\mathbf{p}}^t}\left[U\left({\mathcal{S}}^t\right)\right]\mathrm{s}.\mathrm{t}.\sum _{k\in {\mathcal{S}}^t}{C}_k\le C_{max},\mathrm{and}\mathrm{fairness}\mathrm{constraints},$$

(16)

where $C_k$ denotes cost metrics (e.g., communication or computation costs) and $C_{max}$ is a system budget.

5.2. Importance Sampling for Variance Reduction

One theoretically grounded client selection strategy is importance sampling, which adjusts the selection probabilities $p_k^t$ based on the expected informativeness or gradient norm magnitude from each client. The intuition is to prioritize clients whose updates have higher variance or contribute more significantly to reducing the global objective [47]. Formally, let ${\mathbf{g}}_k^t$ be the stochastic gradient computed by client k at round t. The variance of the aggregated gradient estimator

$${\widehat{\mathbf{g}}}^t={\displaystyle \frac{1}{S}}\sum _{k\in {\mathcal{S}}^t}{\displaystyle \frac{{\mathbf{g}}_k^t}{p_k^t}}$$

can be minimized by setting

$$p_k^t\propto {\parallel {\mathbf{g}}_k^t\parallel }_2,$$

which allocates higher sampling probability to clients with larger gradient magnitudes, thereby reducing estimator variance. However, computing exact gradient norms for all clients prior to selection is impractical, so approximations or historical statistics are often used [48].

5.3. Straggler Mitigation via Deadline-Aware Scheduling

Straggler clients — those with slow computation or poor network conditions — can substantially delay synchronous federated learning rounds. Scheduling techniques that mitigate straggler effects improve training throughput and stability [49]. One approach is deadline-aware scheduling, where the server imposes a time constraint $T_{max}$ per round [50]. Clients whose expected runtime $r_k^t$ exceeds $T_{max}$ are excluded from participation:

$${\mathcal{S}}^t=\{k\in \mathcal{K}:r_k^t\le T_{max},a_k^t=1\},$$

and the server aggregates updates from this subset [51]. This heuristic prioritizes faster clients, ensuring timely model aggregation at the cost of potentially discarding updates from slower clients with useful data [52]. Balancing deadline selection $T_{max}$ is critical to avoid biasing the model towards fast clients only.

5.4. Fairness and Diversity Considerations

Unbalanced client selection risks marginalizing clients with limited connectivity or less frequent availability, exacerbating bias and degrading model generalization. Incorporating fairness into scheduling ensures equitable representation across clients and data distributions. Mathematically, fairness can be enforced by constraining selection probabilities to satisfy:

$${\displaystyle \frac{1}{T}}\sum _{t=1}^T\mathbb{P}(k\in {\mathcal{S}}^t)\ge {\varphi }_k,$$

where ${\varphi }_k$ is the minimum participation frequency for client k. Alternatively, fairness-aware objective functions incorporate penalty terms or regularizers that promote selection diversity:

$$\underset{{\mathbf{p}}^t}{max}U\left({\mathbf{p}}^t\right)-\lambda \sum _{k=1}^K{\left(p_k^t-{\displaystyle \frac{1}{K}}\right)}^2,$$

where $\lambda >0$ controls the trade-off between utility and uniform client selection. Diverse client sampling can also mitigate the impact of non-IID data distributions, enhancing convergence and robustness.

5.5. Scheduling Algorithms

Several algorithmic frameworks have been proposed for client scheduling:

• Greedy Selection: Select clients maximizing incremental utility until resource constraints are met. This approach is simple but can be myopic.
• Multi-Armed Bandits (MAB): Model client selection as an MAB problem where the server learns client utility over time, balancing exploration and exploitation [53].
• Optimization-Based Selection: Solve a constrained optimization problem using techniques such as projected gradient descent or integer programming to find selection probabilities [54].

5.6. Convergence Analysis with Partial Client Participation

Partial client participation complicates convergence guarantees, especially under heterogeneous data distributions. Recent theoretical results establish that under bounded gradient variance and smoothness assumptions, federated averaging with random partial participation converges at a rate of:

$$\mathbb{E}[F\left({\mathbf{w}}^T\right)-F\left({\mathbf{w}}^{*}\right)]=O\left({\displaystyle \frac{1}{\sqrt{ST}}}+{\displaystyle \frac{\Delta }{T}}\right),$$

where S is the number of clients sampled per round, T is the total number of rounds, and $\Delta$ quantifies data heterogeneity. These results highlight that increasing S improves convergence but raises communication cost, underscoring the need for optimal client selection.

5.7. Summary

Client selection and scheduling play pivotal roles in managing communication constraints, addressing system heterogeneity, and enhancing model quality in federated learning. By leveraging importance sampling, deadline-aware heuristics, and fairness-aware algorithms, federated systems can balance efficiency, robustness, and equity. Future research directions include scalable scheduling for ultra-large client pools, integration with secure aggregation, and adaptive strategies responsive to dynamic client behavior.

6. Asynchronous Federated Learning

Traditional federated learning algorithms, such as FedAvg, assume synchronous communication rounds where the server waits for all selected clients to complete their local computations and send updates before performing aggregation. However, in large-scale deployments, synchronous operation is often impractical due to client heterogeneity in computation power, communication bandwidth, and availability [55]. Asynchronous federated learning (AFL) relaxes this synchronization constraint, allowing the server to incorporate client updates as they arrive, thereby improving system efficiency and reducing idle time. This section thoroughly examines the mathematical foundations, algorithmic designs, and convergence properties of asynchronous federated learning.

6.1. Modeling Asynchrony

In AFL, each client k independently computes local updates based on potentially stale global models, and the server continuously integrates these updates without waiting for all clients. Denote by ${\mathbf{w}}^t$ the global model at the server at iteration t, and by ${\mathbf{w}}_k^{t-{\tau }_k}$ the version of the global model that client k received at some past iteration $t-{\tau }_k$, where ${\tau }_k\ge 0$ is the staleness or delay associated with client k’s update. Client k performs local computation starting from ${\mathbf{w}}_k^{t-{\tau }_k}$, producing an update $\Delta {\mathbf{w}}_k^t$, which is sent asynchronously to the server [56]. The server then updates the global model as follows:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\eta }_t\sum _{k\in {\mathcal{A}}^t}\Delta {\mathbf{w}}_k^t,$$

(17)

where ${\mathcal{A}}^t$ is the set of client updates received at iteration t, and ${\eta }_t$ is the learning rate schedule. The asynchronous nature means that $\Delta {\mathbf{w}}_k^t$ may be computed using stale parameters, introducing challenges in convergence and stability.

6.2. Impact of Staleness

The delay ${\tau }_k$ induces a discrepancy between the model parameters used to compute local updates and the current global model. This discrepancy can be formally analyzed by decomposing the update as:

$$\Delta {\mathbf{w}}_k^t=-\eta \nabla F_k\left({\mathbf{w}}^{t-{\tau }_k}\right)+{\xi }_k^t,$$

(18)

where $\nabla F_k\left({\mathbf{w}}^{t-{\tau }_k}\right)$ is the gradient of the local loss function $F_k$ evaluated at the stale global model and ${\xi }_k^t$ is a stochastic noise term due to local sampling. Because ${\mathbf{w}}^{t-{\tau }_k}$ may differ significantly from ${\mathbf{w}}^t$, the aggregated update may no longer be a descent direction for the current global objective, potentially slowing convergence or causing divergence [57]. Under standard assumptions of L-smoothness and $\mu$-strong convexity of the global objective $F\left(\mathbf{w}\right)={\sum }_{k=1}^K{p}_k{F}_k\left(\mathbf{w}\right)$, the convergence rate of AFL algorithms can be characterized by bounding the error introduced by staleness. For example, the expected optimization error satisfies:

$$\mathbb{E}[F\left({\mathbf{w}}^t\right)-F\left({\mathbf{w}}^{*}\right)]\le \mathcal{O}\left({\displaystyle \frac{1}{t}}\right)+\mathcal{O}\left(\eta L^2\underset{k}{max}{\tau }_k^2\right),$$

(19)

indicating that the staleness term ${max}_k{\tau }_k$ critically affects convergence. Careful design of learning rates ${\eta }_t$ and staleness-aware aggregation strategies is necessary to mitigate these effects.

6.3. Algorithmic Strategies for AFL

Several algorithms have been proposed to accommodate asynchronous updates while ensuring convergence and system efficiency:

• Staleness-weighted aggregation: Assign weights to client updates inversely proportional to their staleness, e.g.,

  $${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\eta }_t\sum _{k\in {\mathcal{A}}^t}{\displaystyle \frac{1}{1+{\tau }_k}}\Delta {\mathbf{w}}_k^t,$$

  which downweights outdated updates to reduce their negative impact.
• Adaptive learning rates: Employ time-decaying or delay-aware learning rates ${\eta }_t\left({\tau }_k\right)$ that reduce step sizes for stale updates, balancing convergence speed and stability [58].
• Bounded staleness protocols: Enforce a maximum allowable delay ${\tau }_{max}$ by discarding excessively stale updates, trading off system responsiveness with convergence guarantees [59].
• Asynchronous variance reduction: Integrate variance reduction techniques such as SVRG or SAGA adapted to asynchronous settings to reduce gradient noise amplified by staleness [60].

6.4. Theoretical Convergence Guarantees

The convergence analysis of AFL typically relies on bounded delay assumptions and Lipschitz continuity of gradients. Under these assumptions, it can be shown that asynchronous federated algorithms converge to stationary points with rates similar to their synchronous counterparts, up to additional error terms that depend on staleness and system heterogeneity. Formally, assuming bounded gradient variance ${\sigma }^2$, bounded delay ${\tau }_{max}$, and step size ${\eta }_t={\displaystyle \frac{{\eta }_0}{\sqrt{t}}}$, the expected convergence rate satisfies:

$$\underset{t\in \left[T\right]}{min}\mathbb{E}\left[\parallel \nabla F\left({\mathbf{w}}^t\right){\parallel }^2\right]\le \mathcal{O}\left({\displaystyle \frac{1}{\sqrt{T}}}\right)+\mathcal{O}\left({\displaystyle \frac{{\tau }_{max}^2}{T}}\right),$$

(20)

demonstrating that as the number of iterations T increases, the negative impact of staleness diminishes [61].

6.5. Practical Challenges and System Considerations

Deploying AFL in real-world systems requires addressing issues such as:

• Client heterogeneity: Diverse computational speeds and network conditions yield variable staleness distributions, complicating system design [62].
• Model consistency: Ensuring that asynchronous updates do not cause model parameter inconsistencies or conflicts, especially when using non-convex models.
• Privacy preservation: Integrating asynchronous updates with privacy-preserving mechanisms like secure aggregation or differential privacy is non-trivial due to the dynamic and out-of-sync nature of updates [63].
• Fault tolerance: Handling client dropouts and unreliable connections without stalling the global training process.

Strategies combining asynchronous communication with robust aggregation, adaptive staleness control, and secure protocols are active research areas.

6.6. Summary

Asynchronous federated learning offers a promising paradigm to overcome the scalability and latency challenges inherent in synchronous protocols, particularly in large-scale, heterogeneous environments. By incorporating updates as they arrive and mitigating staleness-induced errors, AFL enhances resource utilization and reduces training time [64]. Nevertheless, careful algorithmic and system design is essential to preserve convergence guarantees, maintain fairness, and uphold privacy. The following section will explore advanced model compression techniques that can be integrated with asynchronous and synchronous FL to further alleviate communication bottlenecks [65].

7. Model Compression and Quantization Techniques

Communication overhead is one of the fundamental bottlenecks in large-scale federated learning systems, especially as models grow in size and the number of participating clients increases dramatically. Since federated learning requires frequent transmission of model updates or gradients between clients and the central server, reducing the communication payload without compromising model accuracy is critical [66]. Model compression and quantization techniques have thus emerged as indispensable tools to alleviate bandwidth constraints, lower energy consumption on edge devices, and accelerate convergence. This section delves into the mathematical principles, algorithmic frameworks, and trade-offs involved in model compression and quantization within federated learning [67].

7.1. Problem Setup and Objectives

Consider a global model parameter vector $\mathbf{w}\in {\mathbb{R}}^d$ of dimensionality d, typically in the order of millions or more for modern deep learning architectures. Each client k computes a local update $\Delta {\mathbf{w}}_k$, which must be communicated to the server. Direct transmission of $\Delta {\mathbf{w}}_k$ incurs a communication cost proportional to d, which becomes prohibitive for resource-constrained clients or limited bandwidth networks. The goal of compression is to find a function $Q:{\mathbb{R}}^d\to \mathcal{C}$ mapping the update vector to a compressed representation in a code space $\mathcal{C}$ with reduced bit-length [68]. Formally, the compression function Q aims to minimize communication bits while controlling the introduced approximation error:

$$\underset{Q}{min}\mathbb{E}\left[\parallel Q(\Delta {\mathbf{w}}_k)-\Delta {\mathbf{w}}_k{\parallel }^2\right],\mathrm{s}.\mathrm{t}.\mathrm{bit}-\mathrm{length}\left(Q(\Delta {\mathbf{w}}_k)\right)\le B,$$

where $B\ll d\times \mathrm{bit}-\mathrm{width}$ of the original update, and the expectation is over randomness in the compression operator (if randomized).

7.2. Quantization Methods

Quantization discretizes continuous-valued parameters into a finite set of levels, thereby enabling compact encoding [69].

Uniform Quantization

The simplest quantization scheme partitions the value range into q equally spaced levels. For each coordinate ${[\Delta {\mathbf{w}}_k]}_i$, the quantized value is:

$$Q_i(\Delta {\mathbf{w}}_k)=\mathrm{clip}\left(\Delta {\mathbf{w}}_k,a,b\right)\approx {\displaystyle \frac{b-a}{q-1}}·⌊{\displaystyle \frac{{(\Delta {\mathbf{w}}_k)}_i-a}{(b-a)/(q-1)}}+{\xi }_i⌋+a,$$

where a and b are bounds (e.g., min and max), and ${\xi }_i\sim \mathrm{Uniform}(0,1)$ is a random dithering variable to ensure unbiasedness [70]. Uniform quantization achieves a compression ratio of approximately ${\displaystyle \frac{32}{{log}_{2}q}}$ per coordinate assuming 32-bit floats originally.

Stochastic Quantization

To maintain unbiasedness, stochastic quantization selects adjacent quantization levels probabilistically such that

$$\mathbb{E}\left[Q_i(\Delta {\mathbf{w}}_k)\right]={(\Delta {\mathbf{w}}_k)}_i,$$

which reduces bias-induced convergence degradation. Formally,

$$Q_i(\Delta {\mathbf{w}}_k)=\left\{\begin{array}{cc}{q}_j,\hfill & \mathrm{with}\mathrm{probability}p={\displaystyle \frac{q_{j+1}-{(\Delta {\mathbf{w}}_k)}_i}{q_{j+1}-q_j}},\hfill \\ q_{j+1},\hfill & \mathrm{with}\mathrm{probability}1-p,\hfill \end{array}\right.$$

where $q_j$ and $q_{j+1}$ are the two closest quantization levels surrounding ${(\Delta {\mathbf{w}}_k)}_i$ [71].

7.3. Sparsification Techniques

Sparsification reduces communication by sending only a small subset of update coordinates with the largest magnitudes, zeroing out others. Formally, define a sparsification operator $S_k$ such that:

$${\left[S_k(\Delta {\mathbf{w}}_k)\right]}_i=\left\{\begin{array}{cc}{(\Delta {\mathbf{w}}_k)}_i,\hfill & i\in {\mathcal{I}}_k,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where ${\mathcal{I}}_k\subseteq \{1,\dots ,d\}$ is a subset with $|{\mathcal{I}}_k|=s\ll d$. The selection can be deterministic (top-s largest coordinates) or randomized with sampling probabilities proportional to magnitude. Sparsification introduces bias if the zeroed coordinates are consistently ignored; hence, error-feedback mechanisms store residuals locally to be added in subsequent rounds:

$${\mathbf{r}}_k^{t+1}={\mathbf{r}}_k^t+\Delta {\mathbf{w}}_k^t-S_k(\Delta {\mathbf{w}}_k^t),$$

and compress $S_k\left({\mathbf{r}}_k^{t+1}\right)$ in the next iteration [72]. This technique provably preserves convergence guarantees despite aggressive sparsification.

7.4. Low-Rank and Structured Compression

Another approach exploits redundancy in parameter updates by approximating $\Delta {\mathbf{w}}_k$ with low-rank or structured representations [73]. For example, matrix factorization techniques approximate updates as:

$$\Delta {\mathbf{W}}_k\approx {\mathbf{U}}_k{\mathbf{V}}_k^{\top },$$

where $\Delta {\mathbf{W}}_k\in {\mathbb{R}}^{m\times n}$ reshapes the update vector, and ${\mathbf{U}}_k\in {\mathbb{R}}^{m\times r}$, ${\mathbf{V}}_k\in {\mathbb{R}}^{n\times r}$ with rank $r\ll min(m,n)$. Communicating ${\mathbf{U}}_k$ and ${\mathbf{V}}_k$ reduces bits substantially, though computational overhead for decomposition must be accounted for. Other structural compression includes quantizing convolutional filters or pruning network weights dynamically during training to reduce communication [74,75].

7.5. Theoretical Trade-Offs

Compression inherently introduces noise or bias, which affects convergence [76]. Let the compressed update be $Q(\Delta {\mathbf{w}}_k)=\Delta {\mathbf{w}}_k+{\mathbf{e}}_k$, where ${\mathbf{e}}_k$ is the compression error with zero mean and bounded variance:

$$\mathbb{E}\left[{\mathbf{e}}_k\right]=0,\mathbb{E}[\parallel {\mathbf{e}}_k{\parallel }^2]\le \delta \parallel \Delta {\mathbf{w}}_k{\parallel }^2,$$

for some $\delta \in (0,1)$. Under such assumptions, the convergence rate of federated averaging with compressed updates satisfies:

$$\mathbb{E}[F\left({\mathbf{w}}^T\right)-F\left({\mathbf{w}}^{*}\right)]=\mathcal{O}\left({\displaystyle \frac{1}{\sqrt{T}}}+\delta \right),$$

indicating a trade-off between compression ratio and final accuracy [77].

7.6. Integration with Federated Learning Protocols

Compression techniques can be applied both on client-to-server uplink updates and server-to-client downlink broadcasts. In practice, uplink compression is prioritized due to constrained client bandwidth [78]. Adaptive compression strategies, which dynamically adjust quantization levels or sparsity ratios based on network conditions or model training stage, further enhance performance. Moreover, compression must be carefully designed to maintain privacy guarantees. For instance, combining quantization with secure aggregation protocols requires compatible encoding schemes to allow aggregation of compressed values without revealing individual updates [79].

7.7. Summary

Model compression and quantization techniques are vital for scalable federated learning by significantly reducing communication overhead. The choice among uniform or stochastic quantization, sparsification with error feedback, and low-rank approximation depends on system constraints and model characteristics [80]. Understanding the interplay between compression-induced noise and convergence behavior guides the design of efficient federated algorithms. Future directions include learning adaptive compression policies and combining compression with privacy and robustness mechanisms [81].

8. Privacy and Security Challenges in Large-Scale Federated Learning

Federated learning’s fundamental promise is to enable collaborative model training without directly sharing raw user data, thereby enhancing privacy [82]. However, the distributed nature of FL introduces unique privacy and security vulnerabilities that are exacerbated in large-scale deployments with thousands or millions of heterogeneous clients. This section explores the mathematical foundations and practical implications of privacy-preserving techniques, threat models, and defenses in federated learning, emphasizing the complex interplay between privacy guarantees, communication efficiency, and learning utility.

8.1. Threat Models and Privacy Risks

Despite never transmitting raw data, FL is vulnerable to a range of attacks that exploit model updates. Let $D_k$ denote the private dataset of client k, and let ${\mathbf{w}}^t$ be the global model at round t [83]. Clients compute local updates $\Delta {\mathbf{w}}_k^t$ based on $D_k$ and send them to the server. An adversary can be either the server or other clients aiming to infer sensitive information about $D_k$ from $\Delta {\mathbf{w}}_k^t$.

Inference Attacks

Model updates leak information through gradients or weight changes. Gradient inversion attacks reconstruct input data $x\in D_k$ by optimizing an auxiliary input $\widehat{x}$ to minimize

$$\underset{\widehat{x}}{min}{∥{\nabla }_{\mathbf{w}}\ell ({\mathbf{w}}^t;\widehat{x})-\Delta {\mathbf{w}}_k^t∥}^2,$$

where ℓ is the loss function. Such attacks exploit the functional dependency of gradients on specific training samples.

Membership Inference Attacks

Adversaries infer whether a particular data point $x^{*}$ was part of $D_k$ by analyzing statistical differences in updates [84]. Formally, given access to updates $\Delta {\mathbf{w}}_k^t$, the attacker learns a classifier

$$\mathcal{A}:\Delta {\mathbf{w}}_k^t\mapsto \{0,1\},$$

indicating presence or absence of $x^{*}$ in the local dataset, compromising membership privacy [85].

Poisoning and Backdoor Attacks

Malicious clients can send carefully crafted updates $\Delta {\mathbf{w}}_m^t$ to corrupt the global model or insert backdoors [86]. The objective is to maximize an attack loss $L_{\mathrm{attack}}\left({\mathbf{w}}^{t+1}\right)$ by manipulating the aggregation step:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\eta }_t\left(\sum _{k\ne m}\Delta {\mathbf{w}}_k^t+\Delta {\mathbf{w}}_m^t\right)\text{[87]}.$$

Such adversarial behavior threatens model integrity and reliability [88].

8.2. Differential Privacy in Federated Learning

Differential privacy (DP) offers rigorous quantification of privacy leakage by ensuring the output of a mechanism is statistically indistinguishable when any single data point is added or removed. Formally, a randomized mechanism $\mathcal{M}$ satisfies $(\epsilon ,\delta )$-DP if for all neighboring datasets D and $D^{\prime }$ differing in one record,

$$Pr[\mathcal{M}\left(D\right)\in S]\le e^{\epsilon }Pr[\mathcal{M}\left(D^{\prime }\right)\in S]+\delta ,\forall S\subseteq \mathrm{Range}\left(\mathcal{M}\right)\text{[89]}.$$

Applying DP to FL typically involves perturbing local updates before transmission. Each client adds noise ${\mathbf{z}}_k^t$ sampled from a Gaussian distribution $\mathcal{N}(0,{\sigma }^2\mathbf{I})$:

$${\widetilde{\Delta \mathbf{w}}}_k^t=\Delta {\mathbf{w}}_k^t+{\mathbf{z}}_k^t,$$

where the noise scale $\sigma$ is calibrated to achieve a desired privacy budget $(\epsilon ,\delta )$. The global model update becomes

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\eta }_t\sum _{k=1}^K{\widetilde{\Delta \mathbf{w}}}_k^t.$$

While DP provides provable privacy guarantees, it also injects bias and variance into the optimization process, degrading convergence speed and model accuracy. Balancing privacy parameters with utility is thus a core challenge [90].

8.3. Secure Aggregation Protocols

To prevent the server from inspecting individual updates, secure aggregation protocols enable the server to only observe the aggregate sum of encrypted client updates. Clients employ cryptographic primitives such as additive homomorphic encryption or secret sharing to encode updates:

$$\mathrm{Enc}(\Delta {\mathbf{w}}_k^t)\underset{\mathrm{aggregation}}{\to }\mathrm{Enc}\left(\sum _{k=1}^K\Delta {\mathbf{w}}_k^t\right),$$

which the server decrypts to update the global model without learning any individual contribution. Such protocols incur communication and computational overhead, especially in large-scale FL with many clients [91]. Designing scalable secure aggregation methods that tolerate client dropout and asynchronous participation remains an open problem [92].

8.4. Robust Aggregation Methods

Robust aggregation algorithms aim to defend against adversarial updates by mitigating the influence of outliers or malicious clients. Techniques include:

• Median and Trimmed Mean Aggregation: Replacing the average with coordinate-wise median or trimmed mean reduces sensitivity to extreme values.
• Krum and Multi-Krum: Selecting client updates closest to the majority by minimizing the sum of distances to other updates to exclude outliers.
• Norm Bounding and Clipping: Limiting the $L_2$ norm of updates to control the impact of large malicious updates [93].

These defenses improve resilience but often rely on assumptions about the fraction of adversarial clients and can degrade performance in highly heterogeneous or non-IID settings [94].

8.5. Trade-Offs and Open Challenges

Ensuring privacy and security in large-scale FL involves fundamental trade-offs:

• Privacy vs. Utility: Adding noise for DP or restricting updates reduces model accuracy.
• Security vs. Scalability: Cryptographic protocols add overhead and complexity, challenging deployment at scale.
• Robustness vs [95]. Fairness: Filtering or down-weighting suspicious updates may unintentionally marginalize honest but statistically distinct clients.

Future research directions include designing privacy-preserving algorithms that adaptively balance these trade-offs, developing scalable cryptographic schemes tailored for heterogeneous client populations, and integrating anomaly detection with robust aggregation for dynamic adversary mitigation.

8.6. Summary

Privacy and security are paramount in federated learning but remain challenging to guarantee simultaneously at scale. Differential privacy and secure aggregation provide formal protections against information leakage, while robust aggregation methods defend against poisoning attacks. However, the large scale, heterogeneity, and asynchronous participation in federated learning exacerbate these challenges, necessitating novel approaches that holistically address privacy, security, efficiency, and learning performance.

9. System Heterogeneity and Scalability Challenges

Large-scale federated learning systems operate across a vast and diverse array of client devices, each characterized by distinct computational capabilities, network conditions, power constraints, and data distributions [96]. This heterogeneity introduces significant challenges to both the efficiency and the effectiveness of federated training algorithms. Scalability concerns arise not only from the sheer number of clients but also from the complex interactions between system-level variability and algorithmic convergence [97]. This section rigorously analyzes the modeling of system heterogeneity, its impact on federated learning, and algorithmic strategies to mitigate scalability bottlenecks [98].

9.1. Modeling Device and Network Heterogeneity

Let $\mathcal{K}=\{1,2,\dots ,K\}$ denote the set of participating clients, where K can range from thousands to millions [99]. Each client $k\in \mathcal{K}$ is associated with a set of system parameters:

$${\Theta }_k=\{{\tau }_k,c_k,b_k,{\nu }_k\},$$

where:

• ${\tau }_k$ is the local computation speed (e.g., FLOPS),
• $c_k$ is the communication bandwidth available to client k,
• $b_k$ is the battery or power budget,
• ${\nu }_k$ quantifies data heterogeneity or non-IID degree in client k’s local dataset $D_k$ [100].

These parameters evolve over time and directly influence the client’s participation frequency, update latency, and reliability [101]. The variation in ${\tau }_k$ leads to stragglers—clients that require significantly more time to perform local computation—causing synchronization delays in synchronous federated protocols.

9.2. Impact on Federated Optimization

Consider the standard Federated Averaging (FedAvg) update at round t:

$${\mathbf{w}}^{t+1}={\mathbf{w}}^t+{\eta }_t\sum _{k\in {\mathcal{S}}^t}{\displaystyle \frac{n_k}{n_{{\mathcal{S}}^t}}}\Delta {\mathbf{w}}_k^t,$$

where ${\mathcal{S}}^t\subseteq \mathcal{K}$ is the subset of clients selected at round t, $n_k=\left|D_k\right|$, and $n_{{\mathcal{S}}^t}={\sum }_{k\in {\mathcal{S}}^t}{n}_k$ [102]. Due to heterogeneous resources, the latency $L_k^t$ for client k can be approximated by:

$$L_k^t=\underset{\mathrm{Computation}\mathrm{time}}{\underbrace{{\displaystyle \frac{C_{\mathrm{comp}}}{{\tau }_k}}}}+\underset{\mathrm{Communication}\mathrm{time}}{\underbrace{{\displaystyle \frac{S_{\mathrm{comm}}}{c_k}}}}+{\delta }_k^t,$$

where $C_{\mathrm{comp}}$ is the number of local computation operations, $S_{\mathrm{comm}}$ the size of communicated data, and ${\delta }_k^t$ captures stochastic delays (e.g., network jitter). In synchronous aggregation, the server waits for all clients in ${\mathcal{S}}^t$ to finish, resulting in overall latency:

$$L_{\mathrm{sync}}^t=\underset{k\in {\mathcal{S}}^t}{max}{L}_k^t,$$

which can be dominated by stragglers. Conversely, asynchronous federated learning avoids waiting but complicates the convergence analysis due to stale updates.

9.3. Client Sampling and Scheduling Strategies

Scalability to massive K necessitates partial client participation via stochastic sampling [103]. Let $p_k^t$ denote the probability that client k is selected at round t [104]. Optimizing $p_k^t$ to balance training speed, fairness, and resource constraints is nontrivial. Adaptive sampling strategies weight clients by resource availability or data quality metrics. For example, sampling proportional to

$$p_k^t\propto {\displaystyle \frac{{\tau }_k{c}_k}{{\nu }_k+ϵ}},$$

prioritizes clients with faster computation and communication while penalizing highly heterogeneous data, thus accelerating convergence [105].

9.4. Mitigating Stragglers and Asynchrony

Several algorithmic solutions address straggler effects:

Partial Participation with Deadlines

Set a maximum wait time $T_{max}$ for client responses. Updates arriving after $T_{max}$ are discarded, reducing iteration latency at the expense of fewer updates per round.

Asynchronous Federated Optimization

Clients communicate updates independently and the server maintains a global model updated incrementally [106]. Formally, for asynchronous update j received at time $t_j$ from client $k_j$:

$${\mathbf{w}}^{t_j}={\mathbf{w}}^{t_j-1}+{\eta }_{t_j}\Delta {\mathbf{w}}_{k_j}^{t_j-{\tau }_{k_j}},$$

where ${\tau }_{k_j}$ accounts for staleness [107]. Convergence proofs require bounding staleness and controlling update variance [108].

9.5. Energy Efficiency and Power Constraints

Battery limitations $b_k$ impose additional constraints, often requiring clients to adjust participation frequency or local computation intensity dynamically [109]. Modeling energy consumption $E_k^t$ as

$$E_k^t=\alpha C_{\mathrm{comp}}/{\tau }_k+\beta S_{\mathrm{comm}}/c_k,$$

with constants $\alpha ,\beta$ representing power costs per computation and communication unit, clients may adopt sleep schedules or early stopping to conserve energy, affecting global training dynamics.

9.6. Scalability in Model and Data Heterogeneity

Heterogeneous data distributions across clients induce statistical challenges impacting convergence rates. Define the degree of non-IID-ness ${\nu }_k$ as

$${\nu }_k=\parallel \nabla F_k\left(\mathbf{w}\right)-\nabla F\left(\mathbf{w}\right)\parallel ,$$

where $F_k$ is client k’s local objective, and F is the global objective [110]. High variance in ${\nu }_k$ leads to slower convergence and potential divergence in federated averaging. Techniques such as control variates and personalized federated learning frameworks mitigate data heterogeneity effects by correcting client drift or learning client-specific models [111].

9.7. Summary

System heterogeneity, encompassing computational, communication, power, and data diversity, presents formidable obstacles to scalable federated learning. Modeling client-specific parameters allows for informed scheduling and adaptive algorithms that mitigate stragglers and balance fairness with efficiency [112]. Asynchronous methods, deadline-based partial participation, and energy-aware protocols form key approaches to address latency and resource constraints [113]. The confluence of system and statistical heterogeneity necessitates holistic optimization to realize the promise of federated learning at scale.

10. Communication Efficiency and Compression Techniques

Communication bottlenecks constitute a critical challenge in large-scale federated learning (FL) due to limited bandwidth, high latency, and the enormous scale of participating devices [114]. Since each communication round involves transmitting model updates—often high-dimensional parameter vectors—efficient communication protocols and compression schemes are essential to reduce overhead without significantly degrading model convergence or accuracy [115]. This section presents a rigorous analysis of communication costs, quantization, sparsification, and adaptive compression strategies, including their theoretical underpinnings and practical trade-offs.

10.1. Communication Cost Formulation

Consider the global model parameter vector at round t as ${\mathbf{w}}^t\in {\mathbb{R}}^d$, where d is the model dimension, typically ranging from millions to billions in deep learning models. Each client k computes a local update $\Delta {\mathbf{w}}_k^t\in {\mathbb{R}}^d$, which must be communicated to the server [116]. The raw communication cost per round is

$$C_{\mathrm{raw}}=K\times d\times b,$$

where K is the number of participating clients and b is the number of bits per parameter (e.g., 32 bits for full-precision floating-point) [117]. At large scales, $C_{\mathrm{raw}}$ becomes prohibitively expensive.

10.2. Quantization Techniques

Quantization reduces b by representing model updates with fewer bits. Let $\mathcal{Q}:{\mathbb{R}}^d\to {\mathcal{C}}^d$ be a stochastic quantizer mapping continuous-valued updates to a discrete codebook $\mathcal{C}$. A popular approach is stochastic s-level quantization:

$$\mathcal{Q}(\Delta w_i)={\parallel \Delta \mathbf{w}\parallel }_2·sign(\Delta w_i)·{\xi }_i,$$

where

$${\xi }_i=\left\{\begin{array}{cc}{\displaystyle \frac{l}{s}},\hfill & \mathrm{with}\mathrm{probability}1-p,\hfill \\ {\displaystyle \frac{l+1}{s}},\hfill & \mathrm{with}\mathrm{probability}p,\hfill \end{array}\right.l=\lfloor s|\Delta w_i{|/\parallel \Delta \mathbf{w}\parallel }_2\rfloor ,p=s|\Delta w_i{|/\parallel \Delta \mathbf{w}\parallel }_2-l.$$

This unbiased quantizer satisfies

$$\mathbb{E}\left[\mathcal{Q}\right(\Delta \mathbf{w}\left)\right]=\Delta \mathbf{w},$$

and has bounded variance proportional to ${\displaystyle \frac{d}{s^2}}$, trading off compression ratio against gradient noise.

10.3. Sparsification Methods

Gradient sparsification reduces communication by transmitting only a subset of significant update coordinates. Define the sparsification operator ${\mathcal{S}}_k^t:{\mathbb{R}}^d\to {\mathbb{R}}^d$ that retains the top-m coordinates (in magnitude) of $\Delta {\mathbf{w}}_k^t$ and zeros out the rest:

$${\mathcal{S}}_k^t(\Delta {\mathbf{w}}_k^t)=\Delta {\mathbf{w}}_k^t\odot {\mathbf{m}}_k^t,$$

where ${\mathbf{m}}_k^t\in {\{0,1\}}^d$ is a mask vector with exactly m ones corresponding to the selected coordinates [118]. To ensure unbiasedness and reduce error accumulation, error-feedback mechanisms maintain a local residual vector ${\mathbf{r}}_k^t$ updated as:

$${\mathbf{r}}_k^{t+1}={\mathbf{r}}_k^t+\Delta {\mathbf{w}}_k^t-{\mathcal{S}}_k^t(\Delta {\mathbf{w}}_k^t),$$

which is incorporated into the next round’s update. Error compensation improves convergence guarantees [119].

10.4. Adaptive Compression and Communication Frequency

Adaptive compression techniques dynamically adjust quantization levels or sparsification ratios based on model training progress or network conditions. For instance, early training stages tolerate higher compression noise, allowing aggressive reduction, whereas later stages reduce compression to refine convergence. Communication frequency control strategies skip communication rounds or perform multiple local updates before aggregation. Let E be the number of local epochs between communications; then communication rounds occur at intervals of length E, reducing communication cost by approximately a factor of E:

$$C_{\mathrm{comm}}\approx {\displaystyle \frac{K\times d\times b}{E}}text\text{[120]}.$$

This FedAvg-style local updating trades off communication for increased local computation, but large E can cause client drift and degrade convergence in non-IID data settings.

10.5. Theoretical Trade-offs and Convergence Bounds

Combining compression and local updates introduces noise and bias [121]. Let the compression operator be $\mathcal{C}$ with contraction parameter $\delta$ satisfying

$${\mathbb{E}\parallel \mathcal{C}\left(x\right)-x\parallel }^2\le (1-\delta ){\parallel x\parallel }^2,$$

for all $x\in {\mathbb{R}}^d$. Then, under smoothness and convexity assumptions, the expected optimization error after T rounds obeys bounds of the form:

$$\mathbb{E}[F\left({\mathbf{w}}^T\right)-F\left({\mathbf{w}}^{*}\right)]\le \mathcal{O}\left({\displaystyle \frac{1}{\eta T}}+{\displaystyle \frac{{\sigma }^2}{K}}+{\displaystyle \frac{1-\delta }{\delta }}\right),$$

where ${\sigma }^2$ is gradient variance and $\eta$ is learning rate. Higher compression (smaller $\delta$) increases the optimization error floor, necessitating a careful balance.

10.6. Practical Considerations

Communication compression schemes must also consider:

• Encoding and Decoding Overhead: Computational cost on resource-constrained devices.
• Robustness to Packet Loss and Asynchrony: Ensuring stability in unreliable networks [122].
• Compatibility with Privacy Mechanisms: Noise added by differential privacy can interact nontrivially with compression noise [123].

10.7. Summary

Communication efficiency is a critical enabler for scaling federated learning. Quantization and sparsification substantially reduce transmitted data sizes, while adaptive compression and communication frequency adjustment provide flexible trade-offs between communication cost and convergence speed [124]. However, these techniques introduce additional noise and bias, requiring rigorous theoretical analysis and practical tuning to ensure robust, efficient, and accurate federated training at scale.

11. Personalization in Large-Scale Federated Learning

Traditional federated learning aims to learn a single global model ${\mathbf{w}}^{*}$ that performs well on the aggregated data distribution across all clients [125]. However, in large-scale scenarios, client data distributions are typically heterogeneous and non-identically distributed (non-IID), leading to suboptimal performance of a single global model on individual clients. Personalization techniques seek to tailor models to each client’s unique data characteristics while leveraging shared knowledge from the federation. This section investigates mathematical formulations of personalization, algorithmic approaches, and challenges in balancing global generalization and local specialization.

11.1. Problem Formulation

Let $F_k\left(\mathbf{w}\right)$ denote the local objective function of client k defined as

$$F_k\left(\mathbf{w}\right)={\mathbb{E}}_{\mathbf{x}\sim D_k}\left[\ell (\mathbf{w};\mathbf{x})\right],$$

where $\ell (\mathbf{w};\mathbf{x})$ is the loss of model parameters $\mathbf{w}$ on data point $\mathbf{x}$ [126]. The classical federated learning objective is the weighted average:

$$\underset{\mathbf{w}}{min}F\left(\mathbf{w}\right)=\sum _{k=1}^K{\displaystyle \frac{n_k}{n}}{F}_k\left(\mathbf{w}\right),$$

with $n_k=\left|D_k\right|$ and $n={\sum }_{k=1}^K{n}_k$. In personalization, the goal is to learn client-specific models ${\mathbf{w}}_k$ by solving

$$\underset{{\left\{{\mathbf{w}}_k\right\}}_{k=1}^K}{min}\sum _{k=1}^K{\displaystyle \frac{n_k}{n}}\left(F_k\left({\mathbf{w}}_k\right)+\lambda \mathcal{R}({\mathbf{w}}_k,{\mathbf{w}}_G)\right),$$

where ${\mathbf{w}}_G$ is a global reference model, $\lambda >0$ is a regularization parameter, and $\mathcal{R}$ is a proximity function encouraging similarity between personalized and global models, typically

$$\mathcal{R}({\mathbf{w}}_k,{\mathbf{w}}_G)={\displaystyle \frac{1}{2}}{\parallel {\mathbf{w}}_k-{\mathbf{w}}_G\parallel }^2\text{[127]}.$$

This formulation balances local fitting and global knowledge transfer [128].

11.2. Meta-Learning Approaches

Meta-learning optimizes the global model such that it can be efficiently adapted to any client with a few gradient steps. A prototypical meta-learning formulation is:

$$\underset{\mathbf{w}}{min}\sum _{k=1}^K{\displaystyle \frac{n_k}{n}}{F}_k\left(\mathbf{w}-\alpha \nabla F_k\left(\mathbf{w}\right)\right),$$

where $\alpha$ is the inner learning rate [129]. This bilevel optimization enables fast adaptation and has been instantiated in algorithms like Model-Agnostic Meta-Learning (MAML) [130].

11.3. Multi-Task and Clustered Federated Learning

When clients form latent clusters with similar data distributions, multi-task FL frameworks learn a set of models ${\left\{{\mathbf{w}}_c\right\}}_{c=1}^C$ indexed by clusters:

$$\underset{\left\{{\mathbf{w}}_c\right\}}{min}\sum _{k=1}^K\sum _{c=1}^C{\pi }_{k,c}{F}_k\left({\mathbf{w}}_c\right),$$

where ${\pi }_{k,c}\in [0,1]$ indicates the membership probability of client k in cluster c [131]. Cluster assignments may be learned jointly with model parameters to exploit statistical similarity.

11.4. Personalized Model Architectures

Personalization can also be achieved via model architectures decomposing parameters into shared and private components:

$${\mathbf{w}}_k={\mathbf{w}}_G+{\mathbf{v}}_k,$$

where ${\mathbf{w}}_G$ is the global shared model and ${\mathbf{v}}_k$ is the client-specific perturbation. Training optimizes jointly over ${\mathbf{w}}_G$ and $\left\{{\mathbf{v}}_k\right\}$, often with regularization on ${\mathbf{v}}_k$ to control complexity.

11.5. Challenges and Open Questions

Key challenges in personalization include:

• Communication Overhead: Personalized models require additional communication and storage, especially with large ${\mathbf{v}}_k$ vectors.
• Privacy Risks: Personalization may leak client-specific information; privacy-preserving personalization remains an active research area.
• Fairness and Robustness: Balancing performance across diverse clients to avoid disadvantaging minority groups.
• Theoretical Guarantees: Establishing convergence and generalization bounds for personalized federated algorithms under heterogeneous data.

11.6. Summary

Personalization enriches federated learning by addressing client heterogeneity through local adaptation, meta-learning, clustering, and architectural decomposition. These methods improve client-level performance and user satisfaction but introduce new trade-offs in communication, privacy, and algorithmic complexity. Designing scalable, privacy-aware, and theoretically sound personalization algorithms remains an essential direction for advancing federated learning at scale.

12. Incentive Mechanisms and Client Participation

In large-scale federated learning (FL) systems, the participation of heterogeneous clients—often voluntary and resource-constrained—is a critical factor affecting both training efficiency and model quality [132]. Designing effective incentive mechanisms is essential to motivate clients to contribute their data and computational resources while ensuring fairness and sustainability of the learning ecosystem. This section provides a comprehensive survey of incentive models, participation strategies, and their theoretical foundations within federated learning frameworks [133].

12.1. Rationale and Challenges

Unlike centralized training, FL relies on distributed devices owned by independent agents (e.g., users, organizations), each with private costs and utilities. Participation incurs costs such as energy consumption, bandwidth usage, and privacy risks. Therefore, without proper incentives, clients may behave selfishly, reduce participation frequency, or supply low-quality updates, leading to degraded system performance. Key challenges include:

• Asymmetric Information: The server often lacks knowledge of individual client costs and data quality.
• Strategic Behavior: Clients may misreport resources or withhold updates to maximize personal utility.
• Fairness: Incentives must balance rewarding contribution and preventing exploitation of disadvantaged clients.
• Scalability: Mechanisms must be computationally and communication efficient for massive client populations.

12.2. Mathematical Modeling of Client Utilities

Let client k’s utility at round t be defined as

$$U_k^t=R_k^t-C_k^t,$$

where

• $R_k^t$ is the reward (monetary, reputation, or other incentives) offered by the server,
• $C_k^t$ represents the client’s cost, encompassing computational, communication, and privacy expenses.

Clients participate if $U_k^t\ge 0$ [134]. The server aims to design reward schemes $R_k^t$ that maximize global objectives (e.g., model accuracy, fairness) subject to budget constraints

$$\sum _{k=1}^K{R}_k^t\le B_t,$$

where $B_t$ is the total available incentive budget at round t.

12.3. Game-Theoretic Incentive Mechanisms

Federated learning with self-interested clients can be modeled as a game $\mathcal{G}=(\mathcal{K},\left\{A_k\right\},\left\{U_k\right\})$, where:

• $\mathcal{K}$ is the set of clients,
• $A_k$ is the action space for client k, typically participation level or quality of update,
• $U_k$ is the utility function as defined above.

The server seeks a mechanism that induces a Nash equilibrium maximizing collective welfare [135]. Mechanisms include:

Auction-Based Approaches

Clients bid their costs and the server selects winners maximizing model improvement per unit cost [136]. Vickrey–Clarke–Groves (VCG) auctions and reverse auctions guarantee truthfulness and efficiency.

Contract Theory

The server offers contracts $(R_k,{\theta }_k)$ specifying rewards $R_k$ contingent on observed quality ${\theta }_k$. Clients self-select contracts matching their type, ensuring incentive compatibility and individual rationality [137].

12.4. Reputation and Trust Systems

Reputation scores $r_k^t$ quantify client reliability and contribution history. The server uses $r_k^t$ to modulate rewards or participation priority, promoting long-term engagement and discouraging malicious behavior. Formally, let the updated reward be

$$R_k^t=f(r_k^t,q_k^t),$$

where $q_k^t$ measures update quality. Reputation evolves as

$$r_k^{t+1}=\gamma r_k^t+(1-\gamma )q_k^t,$$

with forgetting factor $\gamma \in (0,1)$.

12.5. Participation and Scheduling Policies

To manage limited communication budgets and system heterogeneity, the server implements client selection policies ${\pi }_t:\mathcal{K}\to \{0,1\}$ indicating client participation at round t. Optimizing ${\pi }_t$ balances:

$$\underset{{\pi }_t}{max}\mathbb{E}\left[\Delta F\left({\mathbf{w}}^t\right)\right]-\eta \sum _{k=1}^K{\pi }_t\left(k\right)C_k^t,$$

where $\Delta F\left({\mathbf{w}}^t\right)$ is the expected model improvement and $\eta$ trades off accuracy and cost [138]. Scheduling may prioritize clients with:

• High-quality data,
• Low latency and resource costs,
• Good reputation scores,
• Diverse data to reduce bias [139].

12.6. Privacy-Aware Incentives

Clients face privacy risks quantified by a leakage function ${\mathcal{L}}_k^t$ [140]. Privacy-preserving mechanisms (e.g., differential privacy) degrade update utility, increasing $C_k^t$ [141]. Incentive schemes must compensate privacy costs or adjust noise parameters to balance privacy and participation.

12.7. Summary

Incentive mechanisms are pivotal in sustaining large-scale federated learning ecosystems by aligning client motivations with global learning objectives. Game-theoretic, contract-based, and reputation-driven designs provide theoretical frameworks to encourage truthful and sustained participation while addressing privacy, fairness, and scalability [142]. Effective participation policies and budget-aware rewards ensure efficient resource utilization and robust model convergence across diverse client populations.

13. Conclusion

In this survey, we have provided an extensive overview of the critical challenges and recent advances in large-scale federated learning (FL). As a paradigm that enables collaborative model training across massive numbers of decentralized clients without sharing raw data, FL presents unique opportunities and formidable obstacles in scalability, communication efficiency, system heterogeneity, personalization, incentive design, and privacy preservation.

We began by outlining the foundational principles of federated learning and highlighting the scalability challenges posed by the sheer volume of participating devices and the high dimensionality of modern models. We then explored optimization algorithms tailored to FL settings, emphasizing the trade-offs between local computation and global communication. The necessity of communication-efficient strategies was examined through various compression and quantization techniques that reduce bandwidth consumption while maintaining convergence guarantees.

Addressing client heterogeneity, we surveyed personalization methods that adapt global models to individual clients’ data distributions via meta-learning, clustering, and architectural decomposition. These approaches enhance model accuracy and user experience but introduce additional complexities in privacy, communication, and theoretical analysis.

Recognizing that client participation in FL is inherently voluntary and resource-sensitive, we discussed incentive mechanisms grounded in game theory, contract theory, and reputation systems. These mechanisms play a pivotal role in encouraging truthful and sustained client engagement, balancing fairness, privacy, and system efficiency.

Throughout the survey, we underscored the interplay between theoretical guarantees and practical system considerations, advocating for holistic approaches that integrate algorithmic innovation with system design and economic incentives. Despite substantial progress, numerous open problems remain, including robust handling of extreme data heterogeneity, scalable and privacy-preserving personalization, dynamic incentive models under evolving network conditions, and securing FL systems against adversarial threats.

In conclusion, advancing large-scale federated learning demands multidisciplinary efforts spanning machine learning, optimization, communication theory, economics, and security. By addressing the intertwined challenges outlined herein, future research will unlock FL’s full potential to enable privacy-aware, efficient, and personalized intelligence across ubiquitous edge devices, fundamentally transforming the landscape of collaborative artificial intelligence.