FAFedZO: Faster Zero-order Adaptive Federated Learning Algorithm

1. Introduction

With the rapid development of big data and artificial intelligence technologies, machine learning has become one of the key technologies in various fields. However, in practical applications, issues of data privacy and security have become a non-negligible challenge. In traditional machine learning, data usually needs to be sent to data centers for model training, but in the process of transmission, it faces the problems of data leakage and privacy protection. To address this issue, federated learning(FL) [1], an emerging distributed machine learning technology, has emerged. It achieves collaborative learning among multiple data sources, and while protecting data privacy by exchanging model parameters instead of raw data. Currently, FL is utilized across diverse domains such as autonomous driving [2], personalized recommendation systems [3], and medical informatics [4], among others.

However, existing federated learning methods also have some obvious shortcomings. For instance, some federated learning methods are unable to effectively prevent privacy leaks when faced with complex attack techniques, and the convergence speed of current federated learning algorithms remains a bottleneck. Additionally, designing more efficient optimization algorithms tailored to different types of models and tasks is also an urgent issue that needs to be addressed. Researching how to enhance the privacy protection capabilities of federated learning is of great significance in an environment where data security and privacy protection are increasingly critical. This not only safeguards users’ personal information but also promotes the widespread application of federated learning technology across various domains. In practical applications, federated learning needs to meet the demands of various scenarios. By investigating the issue of slow convergence speed, it can better satisfy these practical application needs and enhance the practicality and value of federated learning.

In recent years, the federated optimization for model training has increasingly captured the interest of both academic researchers and industry professionals. To achieve fast convergence rates and reduce communication loads, a wide range of algorithms have been proposed, including first-order methods (such as FedAvg [1], FedFOR [5], FedFomo [6]) and second-order methods (such as Fed-Sophia [7]). Most of these algorithms rely on gradient or Hessian information, and these methods are often only applicable to differentiable functions. However, when the objective function is difficult to compute gradients or the gradient computation cost is excessively high, these algorithms will not be able to solve such problems, such as in the case of black-box functions and in federated hyperparameter tuning [8], so we must look for other solutions. To alleviate the reliance on gradient or Hessian information, the federated zeroth-order optimization (FedZO) algorithm is proposed in [9] through the integration of zeroth-order optimization into the federated learning. The characteristic of derivative-free zeroth-order optimization is that it only queries the objective function values to tackle the problems of federated optimization. It approximates the full gradient and stochastic gradient of the function using function values, without using gradient and Hessian information. In each round of communication, several local updates are performed utilizes a two-point stochastic gradient estimator. Some recent studies such as [10] and [11] have also employed the zeroth-order idea. [10] proposed FedDisco algorithm, leveraging zeroth-order optimization techniques reduces communication overhead significantly. [11] designed a federated zeroth-order algorithm FedZeN, it focused on convex optimization and estimate the curvature of the global objective.

However, existing zeroth-order methods rely solely on function value information, lacking the utilization of gradient and other relevant information. So in complex federated learning scenarios, this method may result in the model failing to fully leverage the feature information of the data, leading to the need for more iterations to achieve convergence during the optimization process. This significantly increases computational time and affects model performance and convergence speed. If the convergence speed of zeroth-order optimization can be accelerated, it can play a better role in scenarios with limited resources such as large-scale distributed optimization and edge computing, improving resource utilization efficiency and thereby promoting the widespread deployment of optimization technologies in resource-sensitive applications.

The emergence of adaptive methods has effectively alleviated the problem of slow convergence. In deep learning, an excessively large learning rate may lead to unstable model training, resulting in overfitting or underfitting. Besides, traditional fixed learning rate methods often struggle to adapt to complex and diverse training data and model structures, potentially leading to slow convergence speed of the model. The adaptive method can dynamically adjust the learning rate based on feedback from the training process, thereby accelerating the convergence of the model. In addition, compared with stochastic gradient descent, this method can also escape saddle points more quickly [12]. Therefore, introducing it in practice represents a crucial approach to enhancing the performance of federated learning algorithms.

However, improper design of adaptive federated learning methods may lead to convergence issues [13]. Reddi et al. [14] first proposed a federated version of the adaptive optimizer, among them are FedAdagrad and FedYogi. However, their analysis is only valid under the condition that ${\beta }_1=0$ (where ${\beta }_1$ is the decay parameter), which cannot leverage the advantages of momentum. In the algorithm FedCAMS [15], it provides a complete proof, but it does not improve the convergence speed. Moreover, they usually require a global learning rate for initialization and adjustment, and the selection of this global learning rate may affect the performance of the algorithm. Existing adaptive methods still have shortcomings such as lagging response to complex environmental changes and high consumption of computational resources. Our goal is to design adaptive algorithms tailored to various real-world scenarios, characterized by efficiency, precision, and flexibility, so as to rapidly adapt to environmental changes, optimize resource allocation, and provide robust support for development across various fields.

Based on the above, we combine the gradient-free optimization with adaptive methods, aiming to leverage the advantages of both to solve the problem of unavailable gradient information of the objective function in a finite-sum optimization problem, while achieving fast convergence and effectively improving the training efficiency and performance of the model.

Contributions The contributions of this paper are summarized as follows:

• By combining the zero-order optimization and adaptive gradient method, we have proposed a novel faster zero-order adaptive federated learning algorithm, called FAFedZO, it can eliminate the reliance on gradient information and accelerate convergence at the same time.
• We conducted a theoretical analysis of the proposed zeroth-order adaptive algorithm and provided a convergence analysis framework under some mild assumptions, demonstrating its convergence. Additionally, we have analyzed the computational complexity and convergence rate of the algorithm.
• Extensive comparative experimental results under both IID and non-IID settings have confirmed the efficacy of our proposed FAFedZO algorithm on the MNIST and CIFAR-10 datasets. when gradient information is not available, it can achieve faster convergence speed.

The structure of the rest of this paper is outlined below. Section 2 provides a summary of the related work. In Section 3, we present the formulation of the federated optimization problem and the algorithm framework of FAFedZO. Section 4 provides the convergence analysis of FAFedZO. Section 5 presents the outcomes of our experiments. Section 6 discusses the beneficial applications of our method in the real world. The paper concludes with Section 7.

2. Related Work

Federated Learning

There have been numerous studies on federated learning, and the pioneering work on federated learning began with [1], where proposed the algorithm called FedAvg. After FedAvg, numerous additional first-order schemes emerged, such as FedNova [16], FedProx [17], SCAFFOLD [18], FedSplit [19], and FedPD [20]. Among them, SCAFFOLD employs control variates to rectify "client drift" while maintaining the same sampling and communication complexity as FedAvg. FedProx introduced a penalty-based approach, which can reduced communication complexity to $O\left({ϵ}^{-2}\right)$. To further minimize communication costs, various second-order optimization methods have been introduced, including GIANT [21] and FedDANE [22]. Additionally, there are also some FL algorithms based on momentum, including [23,24,25]. The work presented in [23] proposes a momentum fusion approach for synchronizing the server and local momentum buffers, however, it does not aim to decrease complexity. [24] proposed a momentum-based global update algorithm, Fed-GLOMO, which reduces variance on the server side using variance reduction techniques. [25] proposed the STEM algorithm, it employs momentum-assisted stochastic gradient directions for updates at both worker nodes and the central server.

Recent research [26,27] has also focused on the application of federated learning in satellite–terrestrial integrated networks. [26] proposes a federated split learning framework, which aims to leverage the computing capabilities of satellite and terrestrial networks for collaborative processing of sequential data while protecting data privacy. [27] presents an intrusion detection method that combines federated learning with conditional generative adversarial networks to address the challenges of data privacy and imbalanced datasets. However, there are still numerous federated optimization problems in reality that are challenging to solve, for instance when gradient information is unavailable or costly to acquire. Therefore, the study of gradient-free zeroth-order optimization is essential.

Zero-order Optimization

Early literatures that began to use the zero-order idea for estimation include [28,29,30]. Specifically, In [28], the authors developed a distributed zero-order algorithm utilizing gradient tracking techniques. In [29], the author provides the first generalization error analysis for black-box learning via derivative-free optimization, and demonstrates that under the assumption of Lipschitz and smoothing (unknown) losses, the ZoSS method attains a comparable generalization error boundary to Stochastic Gradient Descent (SGD). [30] proposed and analyzed zero-order stochastic approximation algorithms for non-convex and convex objective function, focusing on solving the problems of constrained optimization and high-dimensional settings. Recent key research achievements have also applied zero-order methods to various fields, such as [9,10,11] and [31]. In [9], a derivative-free algorithm FedZO is proposed. It is proven that this algorithm achieves a linear increase in speed in relation to the number of devices involved and local iteration times under non-convex settings. Under the framework of cross-device federated learning, [31] introduces a dual-communication zeroth-order method, which is the first technique to incorporate wireless channels into the algorithm. It employs a single-point gradient estimator, replacing long vectors with scalar values for communication while leveraging the characteristics of wireless channels. This represents a significant new achievement in the field of zeroth-order optimization, achieving a convergence rate of $O\left({\displaystyle \frac{1}{\sqrt[3]{K}}}\right)$ in non-convex settings. However, due to the nature of zeroth-order optimization, it can lead to slower convergence speeds. Therefore, an adaptive method is introduced below.

Adaptive Methods

Early proposed adaptive algorithms include [32,33]. The Adam method in [32] introduces decay coefficients and combines momentum optimization to better adapt to non-stationary data and large-scale datasets, thereby accelerating model convergence. The AdaGrad method in [33] can adaptively adjust the learning rate of each parameter that can makes sparse features to obtain larger learning rates, while frequently occurring features obtain smaller learning rates. Then researchers have extended these algorithms to the context of federated learning, like recent studies such as [32-35]. In [34], the authors designed an Adaptive Local Iteration Differential Privacy Federated Learning algorithm (ALI-DPFL) and demonstrated its superiority in resource-constrained scenarios. The adaptive methods in [35] and [36] effectively mitigate the issue of non-IID data, achieving critical progress in enabling better performance when dealing with non-IID data. In [35] proposed an adaptive fairness federated aggregation algorithm Ada-FFL, which dynamically adjusts fairness coefficients based on local model updates to ensure both convergence performance of the global model and fairness among clients. [36] introduced a novel framework called FedARF, aimed at enhancing federated learning performance through adaptively reconstructing local features during the training process. It adopts an adaptive feature fusion strategy, enabling the model to better adapt to the data distribution of each client, thereby accelerating the convergence speed on non-IID data. [37] introduced a novel federated learning algorithm named AFedAvg, which significantly reduced communication costs and accelerated convergence speed by combining adaptive communication frequency and gradient sparsity techniques. This algorithm dynamically selects the communication frequency for the next round based on the number of sparse parameters. The results show that it can achieve a communication compression ratio ranging from 2.4 times to 23.1 times, which marks a significant step forward in mitigating the obstacles posed by high communication volume to artificial intelligence. However, none of these studies have investigated the integration of adaptive methods within the context of zero-order optimization. Hence, our article addresses this by combining zero-order optimization with adaptive approaches to conduct our discussion.

3. Problem Formulation and Algorithm Design

In this part, we will introduce the federated optimization problem and the design of the zero-order adaptive federated optimization algorithm FAFedZO.

3.1. Federated Optimization Problem Formulation

We consider a federated learning task involving a central server and Q edge devices with an index of $\{1,2,...Q\}$. The central server aims to facilitate collaboration among these devices to address a specific optimization problem

$$\begin{array}{c}\hfill \underset{\Xi \in R^d}{min}f(\Xi )\triangleq {\displaystyle \frac{1}{Q}}\sum _{i=1}^Q{f}_i(\Xi ),\end{array}$$

(1)

and

$$\begin{array}{c}\hfill f_i(\Xi )\triangleq E_{{\vartheta }_i\sim D_i}\left[F_i(\Xi ,{\vartheta }_i)\right],\end{array}$$

(2)

here, $\Xi \in R^d$ represents a d-dimensional model parameter. For each edge device i, $f_i(\Xi )$ denotes its local loss function, while $f(\Xi )$ stands for the global loss function. In formula (1), $f_i(\Xi )$ evaluates the anticipated risk on the data distribution $D_i$ on the edge device i, which is presented in formula (2). ${\vartheta }_i\sim D_i$ denotes the random variable ${\vartheta }_i$ is uniformly drawn from the distribution $D_i$, and $F_i(\Xi ,{\vartheta }_i)$ denotes the loss function of ${\vartheta }_i$ at the parameter $\Xi$.

3.2. Algorithm Design of FAFedZO

We explored how to design a method that combines adaptive gradient with zero-order optimization in federated learning.

Firstly, we expand our algorithm description within the context of the FedAvg framework. We are focused here on solving problem (1) through zeroth-order optimization methods and propose a novel zero-order fast adaptive FL method (FAFedZO) employs a shared adaptive learning rate to address the issues. In particular, Algorithm 1 outlines the specifics of our FAFedZO method.

At the beginning, input the parameters and perform initialize. For all i, compute ${\Xi }_{1,i}={\Xi }_{0,i}-{\eta }_0{n}_{0,i}$, which represents the first model update based on the initial parameters and the initial gradient estimates.

Then, for each edge device i, perform the following steps: First, extract a mini-batch ${\mathfrak{B}}_{t,i}$ of size b from the local dataset $D_i$. Next, compute the stochastic gradient estimates ${\widehat{g}}_{t,i}$ and ${\widehat{g}}_{t-1,i}$ for the current model parameters ${\Xi }_{t,i}$ based on the mini-batch ${\mathfrak{B}}_{t,i}$. Here, we elaborate on the specific method for estimating the gradients.

To address the issue of unavailable gradient information and reduce the frequency of model exchange, we achieve this by using gradient estimators and performing stochastic zero-order updates in each communication round. In particular, at the t-th round, edge device i calculates a two-point stochastic gradient estimator [28] as outlined below

$$\begin{array}{c}\hfill {\tilde{\nabla }}_v^{\mu }{F}_i({\Xi }_{t,i},{\vartheta }_{t,i})={\displaystyle \frac{d{\mathbf{v}}_t}{{\mu }_t}}\left(F_i({\Xi }_{t,i}+{\mu }_t{\mathbf{v}}_t,{\vartheta }_{t,i})-F_i({\Xi }_{t,i},{\vartheta }_{t,i})\right),\end{array}$$

(3)

where ${\Xi }_{t,i}$ denote the local model of edge device i, while ${\vartheta }_{t,i}$ signifies a random variable drawn by edge device i according to its local data distribution $D_i$ during the t-th round. ${\mathbf{v}}_t$ represents a randomly chosen d-dimensional direction, uniformly sampled from the unit sphere ${\mathbb{S}}^d$, while ${\mu }_t$ stands for a positive step size.

When computing gradient estimates, we do not directly use precise gradient information but instead approximate the gradient through random sampling and estimation. This zero-order optimization approach does not require the exact calculation of function derivatives, rather, it optimizes based on sampling and estimation of function values.

Afterward, we update local models according to the following stochastic zeroth-order update method

$$\begin{array}{c}\hfill {\Xi }_{t+1,i}={\Xi }_{t,i}-{\eta }_t{\tilde{\nabla }}_v^{\mu }{F}_i({\Xi }_{t,i},{\vartheta }_{t,i}),t=1,2,\cdots ,T\end{array}$$

where ${\eta }_t$ denotes the learning rate.

In step 9 of Algorithm 1, we use $n_{t,i}$ to update the model, where $n_{t,i}$ is the momentum-based variance reduced gradient estimator, is the weighted sum of the current gradient estimate and previous gradient estimates, with the weights determined by ${\chi }_t$. Its definition is as follows

$$\begin{array}{c}\hfill n_{t,i}={\tilde{\nabla }}_v^{\mu }{F}_i({\Xi }_{t,i};{\vartheta }_{t,i})+(1-{\chi }_t)(n_{t-1,i}-{\tilde{\nabla }}_v^{\mu }{F}_i({\Xi }_{t-1,i};{\vartheta }_{t,i})),\end{array}$$

(4)

where the hyperparameter ${\chi }_t\in (0,1)$, representing the decay factor.

In step 10 of Algorithm 1, we incorporate the coordinate-wise adaptive learning rate approach, akin to that utilized in Adam [32], which is defined as follows

$$\begin{array}{c}\hfill {\iota }_{t,i}=\varrho {\iota }_{t-1,i}+(1-\varrho ){\left({\tilde{\nabla }}_v^{\mu }{F}_i({\Xi }_{t,i};{\vartheta }_{t,i})\right)}^2,\end{array}$$

(5)

where the hyperparameter $\varrho \in (0,1)$, representing the decay factor.

Then, when the current iteration number t is an integer multiple of the local update number p (i.e., $mod(t,p)=0$), we perform step 12.

In step 12 of Algorithm 1, for ${\iota }_{t,i}$, we perform averaging and aggregation step and resulting in ${\overline{\iota }}_t$. Subsequently, we utilize the ${\overline{\iota }}_t$ to create an adaptive matrix $K_t=diag(\sqrt{{\overline{\iota }}_t}+\rho )$(where $diag$ denotes diagonal matrix), and where $\rho >0$ ($\rho$ represent tuning parameter).

At the step 13 in algorithm 1, perform averaging and aggregation on all nodes of $n_{t,i}$.

In step 14, the global model is updated based on the obtained $K_t$ and ${\overline{n}}_t$.

Otherwise, proceed to steps 15 and 16 of the algorithm. Keep $K_t$ as its previous value $K_{t-1}$ and update the global model in the same manner as before.

In step 16 of algorithm 1, the identical $K_t$ is employed for the local updates across various edge devices.

Here, model parameters, $n_{t,i}$, ${\iota }_{t,i}$ are aggregated by the global server every interval of p steps.

At the step 17 in algorithm 1, all edge devices calculate the model difference of their local models in the current round, that is, ${\Delta }_{t,i}={\Xi }_{T,i}-{\Xi }_{0,i}$ ($i\in \left[Q\right]$, $\left[Q\right]$ represents all Q edge devices from 1 to Q), uploaded to the central server.

At the step 18, 19 in Algorithm 1, after receiving the local model difference, central server summarizes it as ${\Delta }_t={\displaystyle \frac{1}{Q}}{\sum }_{i\in \left[Q\right]}{\Delta }_{t,i}$, and proceeds to update the global model, that is ${\Xi }_{t+1}={\Xi }_t+{\Delta }_t$.

Finally, after all iterations are completed, the algorithm outputs a model $\overline{\Xi }$, which is uniformly randomly selected from all the global models ${\left\{{\overline{\Xi }}_t\right\}}_{t=1}^T$ obtained during the iterations, as the final result.

In the algorithm, $K_t$ is a diagonal matrix whose diagonal elements are related to ${\overline{\iota }}_t$, which is computed based on certain statistics from the local model update process. Specifically, ${\iota }_{t,i}$ is related to the square of the gradient estimates, and ${\overline{\iota }}_t$ is obtained by averaging ${\iota }_{t,i}$ across all devices. Subsequently, $K_t$ is updated accordingly. This approach allows $K_t$ to adaptively adjust based on the variations in gradient information during the local model update process.

During the model training process, the data distribution on different devices may vary, leading to differing updates in model parameters. By adaptively updating $K_t$, it is possible to influence the aggregation and update methods of the global model. For instance, when the gradient variation on a particular device is significant (i.e., ${\overline{\iota }}_t$ is large), the value of $K_t$ will also change accordingly, thereby assigning different weights to the amount of updates from this device when aggregating local updates. This enables the algorithm to better adapt to the data distribution and model training situations on different devices, accelerating the convergence speed of the model and enhancing the final model performance.

4. Convergence Analysis of FAFedZO Method

For this part, the convergence of FAFedZO method will be discussed. To facilitate the theoretical analysis of the proposed algorithm, we need to make some assumptions as follows.

Assumption A1. 

The global loss function $f(\Xi )$ in (1) has a lower bound, that is, there is a fixed value $f^{*}$ that exists $f(\Xi )\ge f^{*}>-\infty .$

The Assumption 1 means that regardless of the value of the optimization variable $\Xi$, the global loss function $f(\Xi )$ will not decrease indefinitely, and there exists a minimum value $f^{*}$, ensuring that the optimization problem is meaningful.

Assumption A2. 

We suppose that function $F_i(\Xi ,{\vartheta }_i),f_i(\Xi ),f(\Xi )$ are all L-smooth, that is, for all ${\Xi }_1\in R^d,{\Xi }_2\in R^d$, we can conclude that

$$\begin{array}{cc}\hfill \parallel \nabla f_i\left({\Xi }_1\right)-\nabla f_i\left({\Xi }_2\right)\parallel & \le L\parallel {\Xi }_1-{\Xi }_2\parallel ,\hfill \\ \hfill f\left({\Xi }_1\right)\le f\left({\Xi }_2\right)+<\nabla f\left({\Xi }_2\right)& ,{\Xi }_1-{\Xi }_2>+{\displaystyle \frac{L}{2}}{\parallel {\Xi }_1-{\Xi }_2\parallel }^2.\hfill \end{array}$$

The Assumption 2 is common in the theoretical analysis of non-convex optimization, indicating that the gradient changes of $f_i(\Xi )$, $f(\Xi )$ are smooth and do not change suddenly, which is crucial for proving the convergence of algorithms.

Assumption A3. 

$\nabla F_i(\Xi ,{\vartheta }_i)$ serves as a precise approximation of $\nabla f_i(\Xi )$ without bias, namely

$$E_{{\vartheta }_i}[\nabla F_i(\Xi ,{\vartheta }_i)]=\nabla f_i(\Xi ),\forall \Xi \in R^d.$$

The Assumption 3 allows us to use sample gradients to approximate the true gradient.

Assumption A4. 

The variance of the stochastic gradient is limited within a certain range, that is, there have a fixed value ${\sigma }_g$ can satisfies

$$E_{{\vartheta }_i}{\parallel \nabla F_i(\Xi ,{\vartheta }_i)-\nabla f_i(\Xi )\parallel }^2\le {\sigma }_g^2,\forall \Xi \in R^d.$$

The Assumption 4 indicates that the variance of stochastic gradients will not be infinite, which is important for controlling noise and uncertainty in the optimization process.

Assumption A5. 

The difference between each local loss function and the global loss function remains within a certain limit, that is, there have a fixed value ${\sigma }_h$ can satisfies

$$\parallel \nabla f(\Xi )-\nabla f_i{(\Xi )\parallel }^2\le {\sigma }_h^2,\forall \Xi \in R^d.$$

The Assumption 5 ensures that local optimization does not lead to a significant decrease in global performance.

Assumption A6. 

The inter-node variance is bounded, namely

$$\parallel \nabla f_i(\Xi )-\nabla f_j{(\Xi )\parallel }^2\le {\zeta }^2,\forall \Xi \in R^d.$$

The Assumption 6 indicates that the differences between loss functions on different nodes are limited, ensuring that the optimization processes on different nodes do not diverge too much.

Assumption A7. 

In our algorithms, for all $t\ge 1$, the adaptive matrices $K_t$ satisfies the condition that

$${\lambda }_{min}\left(K_t\right)\ge \rho >0,$$

there ρ represents an appropriate positive value.

The Assumption 7 guarantees the step size in the optimization process does not become too small, thereby guaranteeing the convergence of the algorithm.

Assumption A8. 

The gradients of function $f_i(\Xi )$ is G-bounded, that is, for all i, $\Xi \in R^d$, we have

$$\parallel \nabla f_i(\Xi )\parallel \le G.$$

The Assumption 8 means that the gradients of the loss functions on each node are not infinite, ensuring stability and controllability in the optimization process.

We define the $ϵ$-stationary point as follows:

Definition 1. 

A point Ξ is called ϵ-stationary point if $\parallel \nabla f(\Xi )\parallel \le ϵ$. Generally, a stochastic algorithm is defined to achieve an ϵ-stationary point in T iterations if $\mathbb{E}\parallel \nabla f\left({\Xi }_T\right)\parallel \le ϵ$.

Then, we investigate the convergence characteristics of our novel method based on Assumptions 1-8. The Appendix A contains the comprehensive proofs of our results.

About the convergence of the FAFedZO algorithm, we present the main results of this paper as follows:

Theorem 1. 

Assuming that the sequence ${\left\{{\overline{\Xi }}_t\right\}}_{t=1}^T$ is produced by Algorithm 1. Based on the aforementioned Assumptions 1-8, given that $\forall t\ge 0$, ${\chi }_{t+1}=c·{\eta }_t^2$, $c={\displaystyle \frac{1}{12Lp{\overline{h}}^3{\rho }^2}}+{\displaystyle \frac{30L^2}{{\rho }^2}}\le {\displaystyle \frac{60L^2}{{\rho }^2}}$, $w_t=max({\displaystyle \frac{3}{2}},1728L^3{p}^3{\overline{h}}^3-t)$, $\overline{h}={\displaystyle \frac{1}{L}}$ and set

$$\begin{array}{c}\hfill {\eta }_t={\displaystyle \frac{\rho \overline{h}}{{\left(w_t+t\right)}^{1/3}}}\end{array}$$

(6)

we can conclude that

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{T}}\sum _{t=1}^T\mathbb{E}\parallel \nabla f\left({\overline{\Xi }}_t\right)\parallel & \le P\sqrt{{\displaystyle \frac{1}{T}}{\sum }_{t=0}^{T-1}\mathbb{E}\left[{\mathcal{Q}}_t\right]}\hfill \\ \hfill & \le P[\left[{\displaystyle \frac{12Lp}{{\rho }^{2}T}}+{\displaystyle \frac{L}{{\rho }^2{T}^{2/3}}}\right]\mathbb{E}\left[f\left({\overline{\Xi }}_0\right)-f^{*}\right]+{\displaystyle \frac{6L^2{\mu }^2{p}^2}{{\rho }^{2}QT}}\hfill \\ \hfill & +\left[{\displaystyle \frac{{12}^2\times 75p}{{\rho }^{2}T}}+{\displaystyle \frac{900}{{\rho }^2{T}^{2/3}}}\right]\left[{\displaystyle \frac{5L^2{\mu }^2}{3}}+3{\zeta }^2\right](lnT+1)\hfill \\ \hfill & +{\displaystyle \frac{L^2{\mu }^{2}p}{2{\rho }^{2}Q{T}^{2/3}}}{]}^{1/2}\hfill \end{array}$$

(7)

where $P=5\sqrt{2d(G^2+{\sigma }_g^2)+2{\rho }^2+{\displaystyle \frac{1}{2}}{d}^2{L}^2{\mu }^2}$

Remark 1. 

We utilize ζ as an indicator of data heterogeneity. The final results demonstrate that an increase in ζ (indicating greater data heterogeneity) leads to a slowdown in the training process. In addition, as the parameters L and μ increase, the boundary in the theorem conclusion will also increase, which will also result in a decrease in the training speed.

Remark 2. 

A suitable value of ρ ensures a balanced incorporation of adaptive information in the learning rate. In practice, we commonly select ρ to be within the order of $O\left(1\right)$, steering clear of excessively small or large values.

Analysis of computational complexity and convergence speed

(computational complexity) For ${\mathcal{Q}}_t={\displaystyle \frac{1}{12{\eta }_t^2}}{∥{\overline{\Xi }}_{t+1}-{\overline{\Xi }}_t∥}^2+{\displaystyle \frac{1}{4{\rho }^2}}{∥\nabla f\left({\overline{\Xi }}_t\right)-{\overline{n}}_t∥}^2$, we have

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{T}}\sum _{t=0}^{T-1}\mathbb{E}\left[{\mathcal{Q}}_t\right]& ={\displaystyle \frac{1}{T}}\sum _{t=0}^{T-1}\mathbb{E}\left[{\displaystyle \frac{1}{12{\eta }_t^2}}{∥{\overline{\Xi }}_{t+1}-{\overline{\Xi }}_t∥}^2+{\displaystyle \frac{1}{4{\rho }^2}}{∥\nabla f\left({\overline{\Xi }}_t\right)-{\overline{n}}_t∥}^2\right]\hfill \\ \hfill & \le \left[{\displaystyle \frac{12Lp}{{\rho }^{2}T}}+{\displaystyle \frac{L}{{\rho }^2{T}^{2/3}}}\right]\mathbb{E}\left[f\left({\overline{\Xi }}_0\right)-f^{*}\right]+{\displaystyle \frac{6L^2{\mu }^2{p}^2}{{\rho }^{2}QT}}+{\displaystyle \frac{L^2{\mu }^{2}p}{2{\rho }^{2}Q{T}^{2/3}}}\hfill \\ \hfill & +\left[{\displaystyle \frac{{12}^2\times 75p}{{\rho }^{2}T}}+{\displaystyle \frac{900}{{\rho }^2{T}^{2/3}}}\right]\left[{\displaystyle \frac{5L^2{\mu }^2}{3}}+3{\zeta }^2\right](lnT+1)\hfill \end{array}$$

without loss of generality, we let ${\displaystyle \frac{p}{Q}}=O\left(1\right)$, and choose $p=T^{{\displaystyle \frac{1}{3}}}$. To make the right side of the inequality less than ${ϵ}^2$, we can get $T=O\left({ϵ}^{-3}\right)$ and ${\displaystyle \frac{T}{p}}=T^{{\displaystyle \frac{2}{3}}}={ϵ}^{-2}$. Therefore, to satisfy the definition of an $ϵ$-stationary point, which is $\mathbb{E}\parallel \nabla f\left({\Xi }_T\right)\parallel \le ϵ$ and $\mathbb{E}\left[{\mathcal{Q}}_t\right]\le {ϵ}^2$, we obtain the total sample cost as $O\left({ϵ}^{-3}\right)$ and the communication round as $O\left({ϵ}^{-2}\right)$.

(convergence speed) In the FedZO algorithm, when full devices participate, it can satisfy

$$\begin{array}{cc}\hfill \underset{t\in \left[T\right]}{min}\mathbb{E}{∥\nabla f\left({\mathbf{x}}^t\right)∥}^2\le & 4{\displaystyle \frac{f\left({\mathbf{x}}^0\right)-f_{*}}{HT\eta }}+\eta {\displaystyle \frac{24dL}{N}}({\sigma }_g^2+3{\sigma }_h^2)\hfill \\ \hfill & +{\displaystyle \frac{d{L}^2{\mu }^2}{12}}+4L^2{\mu }^2,\hfill \end{array}$$

when partial devices participate, it can satisfy

$$\begin{array}{cc}\hfill \underset{t\in \left[T\right]}{min}\mathbb{E}{∥\nabla f\left({\mathbf{x}}^t\right)∥}^2& \le 4{\displaystyle \frac{f\left({\mathbf{x}}^0\right)-f_{*}}{HT\eta }}+\eta {\displaystyle \frac{32dL}{M}}({\sigma }_g^2+3{\sigma }_h^2)\hfill \\ \hfill & +\eta {\displaystyle \frac{36HL{\sigma }_h^2}{M}}+24\eta H{L}^3{\mu }^2+{\displaystyle \frac{d{L}^2{\mu }^2}{12}}+4L^2{\mu }^2.\hfill \end{array}$$

Regardless of the situation, we can observe that the FedZO algorithm achieves the convergence rate of $O\left({\displaystyle \frac{1}{T}}\right)$. Furthermore, as indicated by Theorem 1, our algorithm can achieve the convergence rate of $O\left({\displaystyle \frac{1}{T^{2/3}}}\right)$. Therefore, our FAFedZO method theoretically possesses a faster convergence rate compared to general zero-order methods. This further validates the advantages of our approach.

5. Experimental Results

For this part, we are conducting comparative experiments on the MNIST and CIFAR-10 datasets, present some experimental outcomes to assess the performance of the proposed FAFedZO method in federated black-box attacks, thereby confirming the advantages of this algorithm.

Experimental environment and datasets

Experimental environment configuration: This study utilizes the FedZO framework as the experimental baseline to conduct research on federated learning algorithms. This framework is renowned for its exceptional scalability and operational convenience, supporting numerous prominent federated learning algorithms. For this study, Python version 3.10.13 is adopted as the programming language, and PyTorch serves as the development platform. All tests are conducted on a Windows 10 platform equipped with an NV GTX1650 GPU and CUDA version 12.4.99.

Dataset: By conducting comparative experimental studies on the MNIST and CIFAR-10 datasets respectively, we have validated the practicality of the FAFedZO algorithm while significantly enhancing the performance of the model.

MNIST (Modified National Institute of Standards and Technology) is a widely used benchmark dataset for evaluating and comparing the performance of handwritten digit recognition algorithms. It was adapted by Professor Yann LeCun and his colleagues at New York University in 1998 from an original dataset of the National Institute of Standards and Technology (NIST). This dataset comprises a collection of grayscale images of handwritten digits (0-9) with 60,000 training samples and 10,000 test samples, each image measuring 28$\times 28$pixels. Each image has undergone a preprocessing pipeline, including centering and normalization, aimed at enhancing the accuracy of classification algorithms. The MNIST dataset is extensively utilized for training and testing image processing and machine learning algorithms, particularly as a classic dataset for introductory deep learning and neural network research. An example of this dataset is shown in Figure 1.

CIFAR-10 (Canadian Institute For Advanced Research-10) is a commonly used dataset in computer vision, particularly for image classification tasks. It is consist of 60,000 32x32 color images across 10 categories and each category contains 6,000 images, and the dataset is divided into 50,000 training and 10,000 test images. Due to its smaller image size and fewer categories, it is often used for rapid verification, prototype development, as well as learning and understanding the basics of various computer vision algorithms. It is also employed as a benchmark dataset for deep learning models to assess their performance and generalization ability on image classification tasks.

Experimental result analysis

Here, we present the results of simulation experiments to assess the performance of the FAFedZO method in the context of black-box attack strategies.

Given the characteristics of black-box scenarios, optimizing black-box attacks falls into the realm of zero-order optimization. We investigate black-box attacks on a trained deep neural network(DNN) classification model. The purpose of our experiment is to train an interference image with the same size as the image in the dataset, which makes it difficult for the human eyes to recognize the difference between the original image and the adversarial image after adding the interference image, but it can induce the classification model to make a wrong judgment. We want to achieve a higher attack success rate with as little disturbance as possible, so we consider the loss function as shown below:

$$\begin{array}{cc}\hfill & \psi (\Xi ,\vartheta )\hfill \\ \hfill =& \underset{I}{\underbrace{max\left\{{\phi }_{y_{\vartheta }}\left({\displaystyle \frac{1}{2}}tanh\left(tan{h}^{-1}2\vartheta +\Xi \right)\right)-{max}_{j\ne y_{\vartheta }}\left\{{\phi }_j\left({\displaystyle \frac{1}{2}}tanh\left(tan{h}^{-1}2\vartheta +\Xi \right)\right)\right\},0\right\}}}\hfill \\ \hfill +& c\underset{II}{\underbrace{{∥{\displaystyle \frac{1}{2}}tanh\left(tan{h}^{-1}2\vartheta +\Xi \right)-\vartheta ∥}^2}}\hfill \end{array}$$

(8)

Here $\Xi$ represents the interference image to be trained, $\vartheta$ represents the original image in the dataset, and $y_{\vartheta }$ represents the label corresponding to the image $\vartheta$ (for example, the label of the image "deer" is 4) , ${\phi }_j\left(\vartheta \right)$ represents the confidence that the classification model recognizes image $\vartheta$ as label j. I in (8) formula measures the probability of attack failure (marked as attack loss), $II$ represents the image distortion caused by disturbance, and c is the balance coefficient. In this way our goal can be achieved by minimizing $\psi (\Xi ,\vartheta )$. Next we use the (8) formula to construct the local loss function.

We divide the samples in the dataset into Q groups randomly and unevenly without repetition, and then distribute them to each edge device, where Q is the total count of edge devices we preset. Then for all edge devices we define its local loss function as:

$$\begin{array}{c}\hfill f_i(\Xi )\triangleq E_{{\vartheta }_i\sim D_i}\left[\psi (\Xi ,{\vartheta }_i)\right],i\in \{1,2,...,Q\}\end{array}$$

(9)

Where $D_i$ represents the dataset at the ith edge device. In this way, the federal black-box attack problem on the DNN classification model can be formulated as a federated optimization problem: $f(\Xi )\triangleq {\displaystyle \frac{1}{Q}}{\sum }_{i=1}^Q{f}_i(\Xi )$. Next, we use the FAFedZO algorithm proposed in this paper to solve this problem.

We select balance parameter $c=1$. And for the remaining parameters, we select them as $(b_1,b_2,\eta ,\mu )=(25,20,0.1,0.001)$.

In Figure 2, we demonstrate the change in the accuracy of the federated black-box attack as the number of communication rounds increases. Figure (a) illustrates the impact of the account of local updates $E\in \{3,15,60\}$ on attack accuracy of the proposed FAFedZO algorithm when the total number of edge devices $Q=50$ and the number of participating edge devices $M=30$. We also compare it with the FedZO algorithm. It can be observed that the larger the E value, the higher the attack accuracy of the algorithm. Furthermore, the attack accuracy of FAFedZO is significantly higher than that of FedZO. When $E=60$, they achieve comparable accuracy, proving that the algorithm proposed in this paper outperforms the original FedZO algorithm. Figure (b) shows how the convergence performance of the FAFedZO method is affected by the number of participating edge devices $M\in \{10,50,100\}$ when $Q=100$ and $E=60$. It can be seen that by adjusting the M value, the FAFedZO algorithm can effectively enhance the attack accuracy.

In Figure 3, we present the relationships between attack loss and communication rounds as well as attack accuracy and communication rounds, respectively. The influence of the number of local updates $E\in \{1,5,20\}$ on the convergence performance of the algorithm is studied under the condition of $Q=50$ and $M=50$, i.e., all devices participate. It is evident that the FAFedZO scheme is capable of significantly decreasing the attack loss and improve attack accuracy compared to the FedZO algorithm under different E values. Moreover, as the value of E increases, the convergence speed of the FAFedZO algorithm also accelerates, with lower attack loss and higher attack accuracy, both of which tend to stabilize faster.

Figure 4 also shows the relationship between attack loss, attack accuracy, and communication rounds. However, we study how the number of participating edge devices $M\in \{10,20,50\}$ influences the convergence performance of the scheme under the condition of $Q=50$ and $E=1$. It is easy to see that the FAFedZO algorithm is far superior to the FedZO algorithm in terms of both attack loss and accuracy. Even when the FedZO algorithm takes the optimal value of M among 10, 20, and 50, its performance is still weaker than the worst value of the FAFedZO algorithm. In addition, as M increases, the attack loss value decreases, and the accuracy increases, which is in line with our expectations.

The aforementioned experiments were all conducted on the MNIST dataset, and similar conclusions were also drawn from experiments performed on the CIFAR-10 dataset. As shown in Figure 5, under the conditions of $Q=50$ and $M=50$, the impact curves of the number of local updates $E\in \{1,5,20\}$ on attack loss and accuracy are presented. It is evident that by adjusting the E value, the performance of the FAFedZO algorithm can also be significantly enhanced.

In addition to investigating the impact of the number of local updates and the number of participating edge devices on the algorithm’s performance, we also studied the influence of varying the number of random directions on the proposed algorithm. As shown in Figure 6, on the MNIST dataset, under the conditions of $Q=50$ and $M=10$, we present the curves illustrating the impact of the number of directions $H\in \{3,15,60\}$ on attack loss and attack accuracy. It can be observed that we have similar conclusions to the above figures, which further confirms the superiority of the FAFedZO algorithm.

In addition, we also discuss the performance of our algorithm under the setting of non-independent and identically distributed (non-iid) data.

Figure 7 presents the impact of the number of local updates on the attack loss and accuracy of the algorithm under the non-iid setting. It can be observed that, compared with Figure 2, the attack accuracy in this case is lower than that under the iid setting, which is attributed to the influence of the non-iid setting and aligns with our expectations. Furthermore, the results also indicate that under the non-iid setting, the performance of the FAFedZO algorithm remains superior to that of the FedZO algorithm, and both achieve comparable performance levels when $E=60$.

The conclusions we previously mentioned are also supported by Figure 8 and Figure 9. Therefore, in summary, the FAFedZO algorithm demonstrates superior performance in both iid and non-iid environments.

6. Discussion

Our approach combines zeroth-order optimization with adaptive techniques, yielding significant benefits in various real-world applications. For instance, in healthcare, medical data contains sensitive patient information and varies across different regions and hospitals. Utilizing this method allows the model to automatically adapt to these variations during training. Additionally, zeroth-order optimization can be leveraged to explore better solutions, enhancing the model’s generalization capability across diverse medical scenarios and improving its assistance in disease diagnosis. In the financial sector, financial institutions need to accurately identify risks such as fraudulent transactions. By employing this method, risk characteristics can be adaptively captured from vast amounts of financial transaction data, and zeroth-order optimization can be used for efficient optimization, thereby improving the accuracy of risk identification and preventing financial risks. In the realm of IoT devices, where resources are limited, zeroth-order optimization eliminates the need for gradient computation, reducing computational load. Adaptive methods can adjust training strategies based on device resources and data characteristics, enabling federated learning to operate efficiently on IoT devices and enhancing their data processing capabilities.

7. Conclusion

In this paper, we proposed FAFedZO, a zero-order federated optimization method based on an adaptive approach with a shared adaptive learning rate. In the non-convex setting, when gradient information is not available, FAFedZO can estimates the first-order gradient information through function queries to optimize the objective function. We have combined the advantages of zero-order optimization and adaptability. Zero-order estimation can solve the problem that the objective function is not available, thus the gradient information is not usable. Adaptability can accelerate the performance of the algorithm. Furthermore, our theoretical analysis of the algorithm demonstrates the convergence of FAFedZO. Finally, we conducted extensive experiments to verify the effectiveness of the proposed algorithm again on the MNIST and CIFAR-10 datasets, respectively.