A Model-Based Stochastic Augmented Lagrangian Method for Online Stochastic Optimization

1. Introduction

In recent years, online optimization problems have garnered widespread attention because of their ability to make real-time decisions with partial information. In the online optimization process, decisions are made sequentially according to feedback from the environment. The general form of this problem can be written as follows:

$$min{f}_t\left(x\right),t\in \{1,2,\dots ,T\}$$

where T is the total number of decision rounds. The meaning of online optimization is clearly defined in several classic review articles[1,2,3]. These articles define regret to measure the online optimization algorithm.

$$Regre{t}_T:=\sum _{t=1}^T{f}_t\left(x_t\right)-\underset{x\in X}{min}\sum _{t=1}^T{f}_t\left(x\right)$$

Considering online optimization with constraint, many algorithms are proposed to solve constrained online optimization like in [4,5]. Constraint is added in the problem framework.

$$s.t.g_t\left(x\right)\le 0$$

Constraint violation is defined as:

$$Violatio{n}_T:=\sum _{t=1}^T{\left[g_t\left(x\right)\right]}_{+}$$

In practice, each problem encountered during the online process is closer to being generated randomly. Therefore, studying online stochastic optimization problems is more suitable for meeting real-world demands, where objective and constraint contain random parameters. Taira et al. [6] study an online stochastic optimization algorithm, in which at each round the parameters are sampled independently from a fixed distribution. To more closely approximate the randomly generated nature of the problem, Cao et al.[7] study the following online stochastic optimization problem with time-varying distributions:

$$\begin{array}{cc}\hfill min& F_t\left(x\right):=E_{\theta \sim P_t}\left[f(x,\theta )\right]\hfill \\ \hfill \mathit{s}.\mathit{t}.& G_t\left(x\right):=E_{\xi \sim Q_t}\left[g(x,\xi )\right]\le 0,t\in \{1,2,\dots ,T\}\hfill \end{array}$$

(1)

In their work, they employ the projected stochastic gradient method (PSGD), which is a deterministic offline version of online gradient descent (OGD) [1], to solve the problem. However, PSGD uses a fixed step size and lacks an adaptive mechanism, which may lead to slower convergence in online settings. Research on this kind of issue is still very limited at present. Thus, we aim to design a new algorithm to efficiently solve online stochastic optimization problem with time-varying distributions.

For stochastic optimization problems, existing stochastic gradient-based algorithms such as SGD [8], SVRG [9], and SPDAM [10] give us some insights into analyzing and handling stochastic problems. The Lagrangian method [11] is widely used in constrained optimization problems. Several works use this method in online optimization, for example, [12,13]. Liu et al. proposed a model-based augmented Lagrangian method to solve online constrained optimization [14]. This method performs well for specific structured problems. It simplifies the computation by approximating the objective and constraint functions at each round. Together with the corresponding primal-dual update formula, this method guarantees a sublinear upper bound. Especially for online optimization, [15] shows that the augmented Lagrangian method converges super-linearly asymptotically.

The regret of online algorithms has a dynamic version, which is defined in [1]. Dynamic regret measures the ability of algorithm to track the optimal solution at each round. [7] defines dynamic regret and constraint violation to measure its performance and adapts to stochasticity:

$$\begin{array}{cc}\hfill & Regret\left(T\right):=E[\sum _{t=1}^T{F}_t\left(x_t\right)]-\sum _{t=1}^T{F}_t\left(x_t^{*}\right)\hfill \\ \hfill & Violatio{n}_i\left(T\right):=E[\sum _{t=1}^T{G}_{t,i}\left(x_t\right)]\hfill \end{array}$$

(2)

Recent studies on distributed online optimization with coupled constraints [16] and online composite optimization with time-varying regularizers [17] also provide valuable insights into handling dynamic and structured online optimization problems, further motivating our approach.

Inspired by these ideas, we solve the online stochastic optimization problem with time-varying distribution by using the model-based augmented Lagrangian method in [14]. At the same time, we incorporate time-varying distribution approximations and a dynamic step size. Stochastic dynamic regret and constraint violation are used to evaluate the performance of our algorithm [7].

The following are the main contributions of our work:

1.

We propose a model-based stochastic augmented Lagrangian method (MSALM) for online stochastic optimization. In each round, we construct model functions to approximate the stochastic objective and constraint functions, which are sampled from time-varying distributions. This construction reduces computational complexity. The step size is designed in a dynamic form and decreases as t increases to accelerate convergence.

2.

We adopt dynamic regret and constraint violation as our performance metrics. These measures are particularly suited for online stochastic optimization with time-varying distributions. Under the assumptions, we prove that the algorithm’s regret and constraint violation have sublinear bounds in terms of the total number of rounds T.

3.

We demonstrate the practical efficacy of our proposed algorithm through a series of simulation experiments. In the contexts of adaptive filtering and online logistic regression, we compare our method with PSGD. The results show that MSALM attains lower regret and constraint violation bounds than PSGD, indicating that MSALM converges more rapidly toward the theoretical optimum while maintaining stricter adherence to constraints. In addition, results from the time-varying smart grid energy dispatch, online network resource allocation, and path planning problems collectively confirm that regret and constraint violation have bounds $O\left(\sqrt{T}\right)$.

2. MSALM for Online Stochastic Optimization

This section presents the online stochastic optimization problem and the details of the model-based stochastic augmented Lagrangian method (MSALM). Then we describe the update strategies of our algorithm.

2.1. The Online Stochastic Optimization Problem

The online optimization problem is a process of making decisions sequentially with partial information. It generates a sequence of decisions through continuous interaction with the environment. The environment refers to the optimization objective (loss function) and its constraints at each round. If the generation process of the objectives and constraints is stochastic, then this problem becomes an online stochastic problem, as shown in 1. The random parameters $\theta$ and $\xi$ represent the samples, which are drawn from the time-varying distributions defined in 1. The problem determines the online decision process. At round t, the decision $x_{t-1}$ is selected based on previous information. Then the distributions $P_{t-1}$ and $Q_{t-1}$ are updated to $P_t$ and $Q_t$. Parameters ${\theta }_t\sim P_t$ and ${\xi }_t\sim Q_t$ are drawn from the current distribution. After that, the $f_t$ and $g_t$ are obtained as the samples of $F_t$ and $G_t$ as:

$$f_t\left(x\right)=f(x,{\theta }_t),g_t\left(x\right)=g(x,{\xi }_t)$$

where $x\in X\subset R^n$. New decision $x_t$ is selected by solving this optimization problem.

Based on the above setup, we make the following standard assumptions.

Assumption 1.

X is a bounded set, and there exists a constant $R>0$ for any $x,y\in X$ satisfying

$$∥x-y∥\le R$$

Assumption 2.

$f(x,\theta )$ and $g(x,\xi )$ are convex and differentiable for any $\theta \in \mathrm{\Theta }$ and $\xi \in \mathrm{\Xi }$.

Assumption 3

(Slater’s condition). At each round t, there exists a decision $x_t\in X$ for all $i=1,2,\dots ,m$ satisfying

$$g_t^{\left(i\right)}\left(x_t\right)\le 0$$

2.2. MSALM Algorithm

To efficiently solve the online stochastic optimization problem with time-varying distributions defined in 1, we extend the model-based augmented Lagrangian method (MALM) proposed by Liu et al. [14] to the stochastic setting. In the MALM framework [14], model-based means that we conservatively approximate the objective and constraint based on the properties of functions. The approximations ${\widehat{f}}_t\left(x\right)$ and ${\widehat{g}}_t\left(x\right)$ are the model functions to objective and constraint at $x_t$. Model functions satisfy the following conditions [14]:

Assumption 4.

$1.$For any $x\in X$,

$${\widehat{f}}_t\left(x\right)\le f_t\left(x\right),{\widehat{f}}_t\left(x_t\right)=f_t\left(x_t\right)$$

and ${\widehat{g}}_t\left(x\right)\le g_t\left(x\right)$, with equality at $x=x_t$

2.For any $x\in X$ and any $i=1,2,\dots ,m$,

$${\widehat{g}}_t^{\left(i\right)}\left(x\right)\le g_t^{\left(i\right)}\left(x\right),{\widehat{g}}_t^{\left(i\right)}\left(x_t\right)=g_t^{\left(i\right)}\left(x_t\right)$$

$3.{\widehat{g}}_t(·)=[{\widehat{g}}_t^{\left(1\right)}(·),{\widehat{g}}_t^{\left(2\right)}(·),\dots ,{\widehat{g}}_t^{\left(m\right)}(·)]$ is a bounded mapping on X, and there exists a constant $D>0$ for any $x\in X$ satisfying

$$∥{\widehat{g}}_t\left(x\right)∥\le D$$

In many approximations, we need gradient information of $f_t\left(x\right)$ and $g_t\left(x\right)$. However, in our stochastic setting, the exact gradients $\nabla F_t\left(x\right)$ and $\nabla G_t\left(x\right)$ are not directly accessible. Instead, we use unbiased stochastic gradient estimates $\nabla f_t\left(x\right)$ and $\nabla g_t\left(x\right)$ based on the sampled functions.

$$\nabla f_t\left(x\right)=E_{{\theta }_t\sim P_t}[\nabla F_t\left(x\right)],\nabla g_t\left(x\right)=E_{{\xi }_t\sim Q_t}[\nabla G_t\left(x\right)]$$

Efficient models that satisfy Assumption4 can be designed.Depending on the properties of the $f_t\left(x\right)$ and $g_t\left(x\right)$, different models are selected for approximation. Several model functions are presented in the MALM algorithm [14] :

• Linearized model:

  $$\begin{array}{cc}\hfill & {\widehat{f}}_t\left(x\right):=f_t\left(x\right)+\langle \nabla f_t\left(x\right),x-x_t\rangle \hfill \\ \hfill & {\widehat{g}}_t^{\left(i\right)}\left(x\right):=g_t^{\left(i\right)}\left(x\right)+\langle \nabla g_t^{\left(i\right)}\left(x\right),x-x_t\rangle ,i=1,\dots ,m\hfill \end{array}$$
• Quadratic model:

  $${\widehat{f}}_t\left(x\right):=f_t\left(x\right)+\langle \nabla f_t\left(x\right),x-x_t\rangle +{\displaystyle \frac{\iota }{2}}{\parallel x-x_t\parallel }^2$$
• Truncated model:

  $${\widehat{f}}_t\left(x\right):={\left[f_t\left(x\right)+\langle \nabla f_t\left(x\right),x-x_t\rangle \right]}_{+}$$
• Plain model:

  $$\begin{array}{cc}\hfill & {\widehat{f}}_t\left(x\right):=f_t\left(x\right)\hfill \\ \hfill & {\widehat{g}}_t^{\left(i\right)}\left(x\right):=g_t^{\left(i\right)}\left(x\right),i=1,\dots ,m\hfill \end{array}$$

Then we define the model-based stochastic augmented Lagrangian of the problem as:

$${\widehat{L}}_{t,\sigma }(x,\lambda ):={\widehat{f}}_t\left(x\right)+{\displaystyle \frac{1}{2\sigma }}[{∥{[\lambda +\sigma {\widehat{g}}_t\left(x\right)]}_{+}∥}^2-{∥\lambda ∥}^2]$$

where the operator ${[·]}_{+}$ means $max\{·,0\}$.

In the MALM algorithm [14], the primal variable is updated by solving the following proximal augmented Lagrangian subproblem:

$$x_{t+1}=arg\underset{x\in X}{min}[{\widehat{L}}_{t,\sigma }(x,{\lambda }_t)+{\displaystyle \frac{\alpha }{2}}{∥x-x_t∥}^2]$$

where $\alpha >0$ is the parameter of the proximal term. The optimality condition of the problem can be transformed as:

$$x_{t+1}=x_t-{\displaystyle \frac{1}{\alpha }}{\nabla }_x{\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)$$

The${\displaystyle \frac{1}{\alpha }}$ plays the role of the step size in a gradient descent step.

We design the step size as ${\alpha }_t={\alpha }_0\sqrt{t}$ which decreases as t increases, to accelerate convergence. At the beginning of the iteration, the algorithm has a larger step size. With the iteration of the algorithm, the step size continues to decrease. This design meets the requirements of different stages of the iterative process. In the early stage, a larger step size can accelerate algorithm iteration. In the late stage, a smaller step size can control the update amplitude and pursue precision. The proposed update for MSALM is:

$$x_{t+1}=arg\underset{x\in X}{min}[{\widehat{L}}_{t,\sigma }(x,{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}{∥x-x_t∥}^2]$$

(3)

The multiplier $\lambda$ is updated by,

$${\lambda }_{t+1}={[{\lambda }_t+\sigma {\widehat{g}}_t\left(x_{t+1}\right)]}_{+}$$

(4)

The algorithm based on model-based augmented Lagrangian method is as follows,

Algorithm 1 MSALM

Require: Choose an initial point $x_0\in X$ arbitrarily. Set parameters ${\alpha }_0>0$, $\sigma >0$.Set the initial multiplier ${\lambda }_0=0$.  for t=1, 2, ..., T do     Submit the decision $x_t$.     Update distributions $P_t$ and $Q_t$ to determine $F_t$ and $G_t$.     Generate $f_t$ and $g_t$ by sampling ${\theta }_t\sim P_t$ and ${\xi }_t\sim Q_t$.     Approximate $f_t\left(x\right)$ and $g_t\left(x\right)$ as ${\widehat{f}}_t\left(x\right)$ and ${\widehat{g}}_t\left(x\right)$.     Update $x_{t+1}$ and ${\lambda }_{t+1}$ by 3 and 4 end for

We measure the algorithm performance by stochastic dynamic regret and constraint violation 2. $x_t^{*}$ is the theoretical optimal decision at each round.

3. Convergence Analysis

In this section, we analyze the performance of MSALM in online stochastic optimization by stochastic dynamic regret and constraint violation.

For proving stochastic dynamic regret, we adopt the definition of drift and the assumption on the drift from [7]:

$$\Delta \left(T\right):=\sum _{t=2}^T∥x_{t-1}^{*}-x_t^{*}∥$$

Assumption 5.

Before the start of the algorithm, there is a $\overline{\Delta }\left(T\right)$ for any T satisfying

$$\Delta \left(T\right)\le \overline{\Delta }\left(T\right)$$

The drift and assumption 5 ensure that although the problem with time-varying distributions retains the characteristic of sharing a common decision set, as in standard online optimization.

Assumption 6.

The gradient of $f_t\left(x\right)$ is bounded, i.e., there is a constant $G_f>0$ satisfying

$$∥\nabla f_t∥\le G_f$$

Lemma 1.

Under the Assumption1-6, we have,

$$E[\sum _{t=1}^T{\alpha }_t({∥x_t^{*}-x_t∥}^2-{∥x_t^{*}-x_{t+1}∥}^2)]\le {\alpha }_{0}R\sqrt{T}(2\overline{\Delta }\left(T\right)+3R)$$

Proof.

$$\begin{array}{c}\hfill \parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2\le \parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}^{*}{\parallel }^2-{\parallel x_{t+1}^{*}-x_{t+1}\parallel }^2\\ \hfill +2\parallel x_t^{*}-x_{t+1}^{*}\parallel \parallel x_{t+1}^{*}-x_{t+1}\parallel \end{array}$$

Let $b_t=x_t^{*}-x_{t+1}^{*}$,

$$\parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2\le \parallel x_t^{*}-x_t{\parallel }^2-\parallel b_t{\parallel }^2-\parallel x_{t+1}^{*}-x_{t+1}{\parallel }^2+2\parallel b_t\parallel ·\parallel x_{t+1}^{*}-x_{t+1}\parallel$$

(5)

Let $A_t=E\left[{∥x_t^{*}-x_t∥}^2\right]$. By Assumption 1, we have $E[\parallel x_{t+1}^{*}-x_{t+1}\parallel ]\le R$ and $A_t\le R^2$, then we take the expectation of 5,

$$E[\parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2]\le A_t-A_{t+1}-\parallel b_t{\parallel }^2+2R\parallel b_t\parallel$$

Multiplying by ${\alpha }_t$ and summing,

$$E[\sum _{t=1}^T{\alpha }_t({∥x_t^{*}-x_t∥}^2-{∥x_t^{*}-x_{t+1}∥}^2)]\le \sum _{t=1}^T{\alpha }_t(A_t-A_{t+1})+\sum _{t=1}^T{\alpha }_t(2R\parallel b_t\parallel -\parallel b_t{\parallel }^2)$$

(6)

We examine the two terms on the right-hand side of the above inequality separately.

$$\begin{array}{cc}\hfill \sum _{t=1}^T{\alpha }_t(A_t-A_{t+1})& ={\alpha }_1{A}_1-{\alpha }_T{A}_{T+1}+\sum _{t=2}^T({\alpha }_t-{\alpha }_{t-1})A_t\hfill \\ & \le {\alpha }_0{R}^2+R^2({\alpha }_0\sqrt{T}-{\alpha }_0)={\alpha }_0{R}^2\sqrt{T}\hfill \end{array}$$

$$\begin{array}{cc}\hfill \sum _{t=1}^T{\alpha }_t(2R\parallel b_t\parallel -\parallel b_t{\parallel }^2)& \le 2{\alpha }_{0}R\sqrt{T}\sum _{t=1}^T\parallel b_t\parallel \hfill \\ & \le 2{\alpha }_{0}R\sqrt{T}\Delta (T+1)\hfill \\ & \le 2{\alpha }_{0}R\sqrt{T}(\overline{\Delta }\left(T\right)+R)\hfill \end{array}$$

6 can be written as:

$$\begin{array}{cc}\hfill E[\sum _{t=1}^T{\alpha }_t({∥x_t^{*}-x_t∥}^2-{∥x_t^{*}-x_{t+1}∥}^2)]& \le {\alpha }_0{R}^2\sqrt{T}+2{\alpha }_{0}R\sqrt{T}(\overline{\Delta }\left(T\right)+R)\hfill \\ \hfill & ={\alpha }_{0}R\sqrt{T}(2\overline{\Delta }\left(T\right)+3R)\hfill \end{array}$$

Theorem 1.

Suppose Assumption 1-6 hold. The stochastic dynamic regret of MSALM has an sublinear upper bound, when the parameters are set ${\alpha }_0={\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}$ and $\sigma ={\displaystyle \frac{1}{\sqrt{T}}}$.

$$\begin{array}{c}\hfill Regret\left(T\right)\le {\displaystyle \frac{D^2\sqrt{T}}{2}}+(G_f^2+R)\sqrt{T\overline{\Delta }\left(T\right)}+{\displaystyle \frac{3R^2}{2}}\sqrt{{\displaystyle \frac{T}{\overline{\Delta }\left(T\right)}}}=O\left(\sqrt{T\overline{\Delta }\left(T\right)}\right)\end{array}$$

Proof. We construct an auxiliary optimization problem:

$$\underset{x\in X}{min}{\widehat{L}}_{t,\sigma }(x,{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}({∥x-x_t∥}^2-{∥x-x_{t+1}∥}^2)$$

The optimal solution of the auxiliary problem satisfies:

$${\nabla }_x{\widehat{L}}_{t,\sigma }(x,{\lambda }_t)+{\alpha }_t(x_{t+1}-x_t)=0$$

(7)

Comparing 7 with the optimality condition of ??, $x_{t+1}$ is the optimal point of the auxiliary problem.

We have the inequality:

$${\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}{∥x_{t+1}-x_t∥}^2\le {\widehat{L}}_{t,\sigma }(x_t^{*},{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}({∥x_t^{*}-x_t∥}^2-{∥x_t^{*}-x_{t+1}∥}^2)$$

(8)

We analyze the two sides of the inequality separately. According to , the left-hand side becomes:

$${\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)={\widehat{f}}_t\left(x_{t+1}\right)+{\displaystyle \frac{1}{2\sigma }}[{∥{\lambda }_{t+1}∥}^2-{∥{\lambda }_t∥}^2]$$

Combining ${\widehat{f}}_t$ is convex and the assumption4,

$${\widehat{f}}_t\left(x_{t+1}\right)\ge {\widehat{f}}_t\left(x_t\right)+\langle G_f,x_{t+1}-x_t\rangle \ge f_t\left(x_t\right)-G_f∥x_{t+1}-x_t∥$$

Th inequality 8 becomes:

$${\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)\ge f_t\left(x_t\right)-G_f∥x_{t+1}-x_t∥+{\displaystyle \frac{1}{2\sigma }}[{∥[{\lambda }_{t+1}∥}^2-{∥{\lambda }_t∥}^2]$$

Meanwhile, the right-hand side of the inequality 3 is bounded as:

$${\widehat{L}}_{t,\sigma }(x_t^{*},{\lambda }_t)\le f_t\left(x_t^{*}\right)+\langle {\lambda }_t,{\widehat{g}}_t\left(x_t^{*}\right)\rangle +{\displaystyle \frac{\sigma }{2}}{\parallel {\widehat{g}}_t\left(x_t^{*}\right)\parallel }^2$$

Since $x_t^{*}$ is a feasible solution, we have $\langle {\lambda }_t,{\widehat{g}}_t\left(x_t^{*}\right)\rangle \le 0$. Considering the assumptionA4, we have:

$${\widehat{L}}_{t,\sigma }(x_t^{*},{\lambda }_t)\le f_t\left(x_t^{*}\right)+{\displaystyle \frac{\sigma }{2}}{D}^2$$

From the above two parts, the inequality8 becomes:

$$\begin{array}{c}\hfill f_t\left(x_t\right)-G_f∥x_{t+1}-x_t∥+{\displaystyle \frac{1}{2\sigma }}[{∥{\lambda }_{t+1}∥}^2-{∥{\lambda }_t∥}^2]+{\displaystyle \frac{{\alpha }_t}{2}}{∥x_{t+1}-x_t∥}^2\\ \hfill \le f_t\left(x_t^{*}\right)+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}({∥x_t^{*}-x_t∥}^2-{∥x_t^{*}-x_{t+1}∥}^2)\end{array}$$

We rearrange and bound the above inequality as:

$$\begin{array}{c}\hfill f_t\left(x_t\right)-f_t\left(x_t^{*}\right)\le {\displaystyle \frac{G_f^2}{2{\alpha }_t}}-{\displaystyle \frac{1}{2\sigma }}[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}(\parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2)\end{array}$$

Summing from $t=1$ to T, we have:

$$\begin{array}{c}\hfill \sum _{t=1}^T(f_t\left(x_t\right)-f_t\left(x_t^{*}\right))\le \sum _{t=1}^T{\displaystyle \frac{G_f^2}{2{\alpha }_t}}-{\displaystyle \frac{1}{2\sigma }}\sum _{t=1}^T[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]+{\displaystyle \frac{\sigma D^{2}T}{2}}\\ \hfill +{\displaystyle \frac{1}{2}}\sum _{t=1}^T{\alpha }_t(\parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2)\end{array}$$

(9)

Taking expectation of the inequality 9 and substituting ${\lambda }_1=0$:

$$\begin{array}{c}\hfill E\left[\sum _{t=1}^T(f_t\left(x_t\right)-f_t\left(x_t^{*}\right))\right]\le {\displaystyle \frac{1}{2}}E\left[\sum _{t=1}^T{\alpha }_t(\parallel x_t^{*}-x_t{\parallel }^2-\parallel x_t^{*}-x_{t+1}{\parallel }^2)\right]\\ \hfill +\sum _{t=1}^T{\displaystyle \frac{G_f^2}{2{\alpha }_t}}-{\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{T+1}{\parallel }^2]+{\displaystyle \frac{\sigma D^{2}T}{2}}\end{array}$$

where for the term ${\sum }_{t=1}^T{\displaystyle \frac{G_f^2}{2{\alpha }_t}}$, we have:

$$\sum _{t=1}^T{\displaystyle \frac{G_f^2}{2{\alpha }_t}}\le {\displaystyle \frac{G_f^2\sqrt{T}}{{\alpha }_0}}$$

When the parameters are set ${\alpha }_0={\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}$ and $\sigma ={\displaystyle \frac{1}{\sqrt{T}}}$, we substitute the result of Lemma1:

$$\begin{array}{cc}\hfill Reg\left(T\right)=E\left[\sum _{t=1}^T(f_t\left(x_t\right)-f_t\left(x_t^{*}\right))\right]& \le {\displaystyle \frac{D^2\sqrt{T}}{2}}+(G_f^2+R)\sqrt{T\overline{\Delta }\left(T\right)}+{\displaystyle \frac{3R^2}{2}}\sqrt{{\displaystyle \frac{T}{\overline{\Delta }\left(T\right)}}}\hfill \\ \hfill & =O\left(\sqrt{T\overline{\Delta }\left(T\right)}\right)\hfill \end{array}$$

Assumption 7.

The gradient of $g_t\left(x\right)$ is bounded, i.e., there is a constant $G_g>0$ satisfying

$$∥\nabla g_t∥\le G_g$$

Lemma 2.

$${\displaystyle \frac{1}{2\sigma }}E[{∥{\lambda }_{t+1}∥}^2-{∥{\lambda }_t∥}^2]\le 2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}E[{∥x^s-x_t∥}^2-{∥x^s-x_{t+1}∥}^2]-{\epsilon }_{0}E\left[∥{\lambda }_t∥\right]$$

where $x^s$ is the Slater’s point.

Proof. According to inequality8, we consider a point $x^s$ which satisfy the Slater’s condition.

$${\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}\parallel x_{t+1}-x_t{\parallel }^2\le {\widehat{L}}_{t,\sigma }(x^s,{\lambda }_t)+{\displaystyle \frac{{\alpha }_t}{2}}(\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2)$$

(10)

From the definition of the augmented Lagrangian function,

$${\widehat{L}}_{t,\sigma }(x^s,{\lambda }_t)={\widehat{f}}_t\left(x^s\right)+{\displaystyle \frac{1}{2\sigma }}[\parallel {[{\lambda }_t+\sigma {\widehat{g}}_t\left(x^s\right)]}_{+}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]$$

Using the non-expansiveness property of the projection operator,

$$\parallel {[{\lambda }_t+\sigma {\widehat{g}}_t\left(x^s\right)]}_{+}{\parallel }^2\le \parallel {\lambda }_t{\parallel }^2+2\sigma \langle {\lambda }_t,{\widehat{g}}_t\left(x^s\right)\rangle +{\sigma }^2{\parallel {\widehat{g}}_t\left(x^s\right)\parallel }^2$$

Therefore,

$${\widehat{L}}_{t,\sigma }(x^s,{\lambda }_t)\le {\widehat{f}}_t\left(x^s\right)+\langle {\lambda }_t,{\widehat{g}}_t\left(x^s\right)\rangle +{\displaystyle \frac{\sigma }{2}}{\parallel {\widehat{g}}_t\left(x^s\right)\parallel }^2$$

By assumption 3 and 4, we have

$${\widehat{L}}_{t,\sigma }(x^s,{\lambda }_t)\le f_t\left(x^s\right)-{\epsilon }_0\parallel {\lambda }_t\parallel +{\displaystyle \frac{\sigma }{2}}{D}^2$$

The inequality 10 becomes:

$$\begin{array}{c}\hfill f_t\left(x_t\right)-G_f\parallel x_{t+1}-x_t\parallel +{\displaystyle \frac{1}{2\sigma }}[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]+{\displaystyle \frac{{\alpha }_t}{2}}{\parallel x_{t+1}-x_t\parallel }^2\\ \hfill \le f_t\left(x^s\right)-{\epsilon }_0\parallel {\lambda }_t\parallel +{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}(\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2)\end{array}$$

We rearrange terms and take expectation,

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le E[f_t\left(x^s\right)-f_t\left(x_t\right)]+G_{f}E[\parallel x_{t+1}-x_t\parallel ]-{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x_{t+1}-x_t{\parallel }^2]\\ \hfill -{\epsilon }_{0}E[\parallel {\lambda }_t\parallel ]+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2]\end{array}$$

We bound each term.

$$\begin{array}{c}\hfill E[f_t\left(x^s\right)-f_t\left(x_t\right)]\le G_{f}E[\parallel x^s-x_t\parallel ]\le G_{f}R\\ \hfill G_{f}E[\parallel x_{t+1}-x_t\parallel ]-{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x_{t+1}-x_t{\parallel }^2]\le G_{f}R\end{array}$$

The inequality becomes:

$${\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le 2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2]-{\epsilon }_{0}E[\parallel {\lambda }_t\parallel ]$$

(11)

Lemma 3.

There exist constants $C_1,C_2,C_3,C_4>0$ for any $t\ge 0$ and positive integer s satisfying

$$E[\parallel {\lambda }_t\parallel ]\le \psi (\sigma ,{\alpha }_0,s):=C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s$$

where $C_1={\displaystyle \frac{4G_{f}R}{{\epsilon }_0}}$, $C_2={\displaystyle \frac{{\alpha }_0{R}^2\sqrt{s}}{{\epsilon }_0}}$, $C_3={\displaystyle \frac{D^2}{{\epsilon }_0}}-D$, $C_4=2D+{\displaystyle \frac{{\epsilon }_0}{2}}+s{\displaystyle \frac{8\sigma D^2}{{\epsilon }_0}}log{\displaystyle \frac{32D^2}{{\epsilon }_0^2}}$.

Proof. For any $t\ge 0$, the inequality 11 can be:

$${\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le 2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2]-{\epsilon }_{0}E[\parallel {\lambda }_t\parallel ]$$

Summing from t to $t+s-1$,

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2\sigma }}\sum _{l=0}^{s-1}E[\parallel {\lambda }_{t+l+1}{\parallel }^2-\parallel {\lambda }_{t+l}{\parallel }^2]\le \sum _{l=0}^{s-1}{\displaystyle \frac{{\alpha }_{t+l}}{2}}E[\parallel x^s-x_{t+l}{\parallel }^2-\parallel x^s-x_{t+l+1}{\parallel }^2]\\ \hfill -{\epsilon }_0\sum _{l=0}^{s-1}E[\parallel {\lambda }_{t+l}\parallel ]+s\left(2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2\right)\end{array}$$

(12)

We have,

$$\begin{array}{cc}\hfill \sum _{l=0}^{s-1}{\displaystyle \frac{{\alpha }_{t+l}}{2}}E[\parallel x^s-x_{t+l}{\parallel }^2-\parallel x^s-x_{t+l+1}{\parallel }^2]& \le {\displaystyle \frac{{\alpha }_0}{2}}\sqrt{t+s-1}\sum _{l=0}^{s-1}E[\parallel x^s-x_{t+l}{\parallel }^2-\parallel x^s-x_{t+l+1}{\parallel }^2]\hfill \\ & \le {\displaystyle \frac{1}{2}}{\alpha }_0\sqrt{t+s-1}{R}^2\hfill \end{array}$$

Since the projection operator has the non-expansiveness:

$$\parallel {\lambda }_{t+1}-{\lambda }_t\parallel =\parallel {[{\lambda }_t+\sigma {\widehat{g}}_t\left(x_{t+1}\right)]}_{+}-{\lambda }_t\parallel \le \parallel \sigma {\widehat{g}}_t\left(x_{t+1}\right)\parallel \le \sigma D$$

For any $l\ge 0$,

$$\parallel {\lambda }_{t+l}\parallel \ge \parallel {\lambda }_t\parallel -\sigma Dl$$

Therefore,

$$\sum _{l=0}^{s-1}E[\parallel {\lambda }_{t+l}\parallel ]\ge \sum _{l=0}^{s-1}\left(E\right[\parallel {\lambda }_t\parallel ]-\sigma Dl)=sE[\parallel {\lambda }_t\parallel ]-\sigma D{\displaystyle \frac{s(s-1)}{2}}$$

Substituting the above results into the inequality 12,

$${\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+s}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le s\left(2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2\right)+{\displaystyle \frac{1}{2}}{\alpha }_0\sqrt{t+s-1}{R}^2-{\epsilon }_0\left(sE[\parallel {\lambda }_t\parallel ]-\sigma D{\displaystyle \frac{s(s-1)}{2}}\right)$$

Since $\parallel {\lambda }_{t+s}{\parallel }^2\ge 0$, we have,

$${\epsilon }_{0}sE[\parallel {\lambda }_t\parallel ]\le s\left(2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2\right)+{\displaystyle \frac{1}{2}}{\alpha }_0\sqrt{t+s-1}{R}^2+{\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_t{\parallel }^2]+{\epsilon }_0\sigma D{\displaystyle \frac{s(s-1)}{2}}$$

From the update rule,

$$|\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2|\le 2\parallel {\lambda }_t\parallel \parallel {\lambda }_{t+1}-{\lambda }_t\parallel +\parallel {\lambda }_{t+1}-{\lambda }_t{\parallel }^2\le 2\sigma D\parallel {\lambda }_t\parallel +{\sigma }^2{D}^2$$

We define:

$$\theta =C_1+C_2{\alpha }_0+C_3\sigma$$

where $C_1,C_2,C_3$ are appropriate constants.

Specifically, from the rearranged inequality,

$${\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+s}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le K_s-{\epsilon }_{0}sE[\parallel {\lambda }_t\parallel ]$$

where $K_s$ collects all the positive bounded terms.

From earlier, we have:

$$|\parallel {\lambda }_{t+1}\parallel -\parallel {\lambda }_t\parallel |\le \sigma D$$

Now, from Lemma2, we have:

$${\displaystyle \frac{1}{2\sigma }}E[\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2]\le K_1-{\epsilon }_{0}E[\parallel {\lambda }_t\parallel ]$$

where $K_1=2G_{f}R+{\displaystyle \frac{\sigma }{2}}{D}^2+{\displaystyle \frac{{\alpha }_t}{2}}E[\parallel x^s-x_t{\parallel }^2-\parallel x^s-x_{t+1}{\parallel }^2]$.

Note that,

$$\parallel {\lambda }_{t+1}{\parallel }^2-\parallel {\lambda }_t{\parallel }^2=(\parallel {\lambda }_{t+1}\parallel -\parallel {\lambda }_t\parallel \left)\right(\parallel {\lambda }_{t+1}\parallel +\parallel {\lambda }_t\parallel )$$

When $\parallel {\lambda }_t\parallel$ is large, say $\parallel {\lambda }_t\parallel \ge \theta$, then,

$$\parallel {\lambda }_{t+1}{\parallel }^2-{\parallel {\lambda }_t\parallel }^2\le -2\sigma {\epsilon }_0\theta +2\sigma K_1$$

Based on this condition, we can obtain the inequality below by applying Lemma from[14],

$$\parallel {\lambda }_t\parallel \le \theta +\sigma D+{\displaystyle \frac{4{\sigma }^2{D}^2}{\frac{{\epsilon }_0}{2}}}log\left[{\displaystyle \frac{8{\sigma }^2{D}^2}{{\left(\frac{{\epsilon }_0}{2}\right)}^2}}\right]$$

where $\theta ={\displaystyle \frac{2K_1}{{\epsilon }_0}}$.

Taking expectation and substituting back,

$$\begin{array}{cc}\hfill E[\parallel {\lambda }_t\parallel ]\le & {\displaystyle \frac{2G_{f}R}{{\epsilon }_0}}+{\displaystyle \frac{\sigma D^2}{{\epsilon }_0}}+{\displaystyle \frac{{\alpha }_0{R}^2}{2{\epsilon }_0}}+\sigma D\hfill \\ \hfill & +\sigma D·{\displaystyle \frac{8\sigma D}{{\epsilon }_0}}log\left[{\displaystyle \frac{32{\sigma }^2{D}^2}{{\epsilon }_0^2}}\right]\hfill \end{array}$$

Simplifying and grouping terms, we obtain the desired form,

$$E[\parallel {\lambda }_t\parallel ]\le \psi (\sigma ,{\alpha }_0,s):=C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s$$

where,

$$\begin{array}{cc}\hfill C_1& ={\displaystyle \frac{2G_{f}R}{{\epsilon }_0}}\hfill \\ \hfill C_2& ={\displaystyle \frac{R^2}{2{\epsilon }_0}}\hfill \\ \hfill C_3& ={\displaystyle \frac{D^2}{{\epsilon }_0}}+D\hfill \\ \hfill C_4& ={\displaystyle \frac{8D^2}{{\epsilon }_0}}log\left[{\displaystyle \frac{32D^2}{{\epsilon }_0^2}}\right]\hfill \end{array}$$

Theorem 2.

Suppose Assumption 1-7 hold. The constraint violation of MSALM has an sublinear upper bound, when the parameters are set ${\alpha }_0={\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}$ and $\sigma ={\displaystyle \frac{1}{\sqrt{T}}}$

$$Violatio{n}_i\left(T\right):=E\left[\sum _{t=1}^T{G}_{t,i}\left(x_t\right)\right]\le \mathcal{O}\left(\sqrt{T}\right)$$

Proof. According to the Lagrange multiplier update rule, assumption 4 and assumption 7, we have:

$$g_{t,i}\left(x_t\right)\le {\displaystyle \frac{1}{\sigma }}({\lambda }_{t+1,i}-{\lambda }_{t,i})+G_g\parallel x_{t+1}-x_t\parallel$$

Summing from $t=1$ to T,

$$\sum _{t=1}^T{g}_{t,i}\left(x_t\right)\le {\displaystyle \frac{1}{\sigma }}\sum _{t=1}^T({\lambda }_{t+1,i}-{\lambda }_{t,i})+G_g\sum _{t=1}^T\parallel x_{t+1}-x_t\parallel \le {\displaystyle \frac{1}{\sigma }}{\lambda }_{T+1,i}+G_g\sum _{t=1}^T\parallel x_{t+1}-x_t\parallel$$

Taking expectation,

$$E\left[\sum _{t=1}^T{g}_{t,i}\left(x_t\right)\right]\le {\displaystyle \frac{1}{\sigma }}E\left[{\lambda }_{T+1,i}\right]+G_g\sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]\le {\displaystyle \frac{1}{\sigma }}E[\parallel {\lambda }_{T+1}\parallel ]+G_g\sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]$$

According to the Lemma3, for any $t\ge 0$ and positive integer s,

$$E[\parallel {\lambda }_t\parallel ]\le C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s$$

Specifically, for $t=T+1$, choosing $s=\sqrt{T}$,

$$E[\parallel {\lambda }_{T+1}\parallel ]\le C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma \sqrt{T}$$

Substituting the parameter choices ${\alpha }_0={\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}$ and $\sigma ={\displaystyle \frac{1}{\sqrt{T}}}$,

$$\begin{array}{cc}\hfill E[\parallel {\lambda }_{T+1}\parallel ]\le & C_1+C_2{\displaystyle \frac{1}{\sqrt{\tilde{\Delta }\left(T\right)}}}+C_3{\displaystyle \frac{1}{\sqrt{T}}}+C_4{\displaystyle \frac{1}{\sqrt{T}}}\sqrt{T}\hfill \\ \hfill =& C_1+C_2{\displaystyle \frac{1}{\sqrt{\tilde{\Delta }\left(T\right)}}}+C_3{\displaystyle \frac{1}{\sqrt{T}}}+C_4\hfill \end{array}$$

Since $\overline{\Delta }\left(T\right)$ is an upper bound of $\Delta \left(T\right)$, under reasonable assumptions $\overline{\Delta }\left(T\right)$ does not tend to zero, therefore:

$$E[\parallel {\lambda }_{T+1}\parallel ]\le \mathcal{O}\left(1\right)$$

Computing the gradient,

$${\nabla }_x{\widehat{L}}_{t,\sigma }(x,{\lambda }_t)=\nabla {\widehat{f}}_t\left(x\right)+{[{\lambda }_t+\sigma {\widehat{g}}_t\left(x\right)]}_{+}·\nabla {\widehat{g}}_t\left(x\right)$$

The gradients are bounded:

$$\parallel \nabla {\widehat{f}}_t\left(x\right)\parallel \le G_f,\parallel \nabla {\widehat{g}}_t\left(x\right)\parallel \le G_g$$

Meanwhile, according to Lemma 3, $\parallel {\lambda }_t\parallel$ is bounded, therefore,

$$\parallel {\nabla }_x{\widehat{L}}_{t,\sigma }(x_{t+1},{\lambda }_t)\parallel \le G_f+G_g(\parallel {\lambda }_t\parallel +\sigma D)$$

Thus,

$$\parallel x_{t+1}-x_t\parallel \le {\displaystyle \frac{1}{{\alpha }_t}}(G_f+G_g(\parallel {\lambda }_t\parallel +\sigma D\left)\right)$$

Taking expectation,

$$E[\parallel x_{t+1}-x_t\parallel ]\le {\displaystyle \frac{1}{{\alpha }_t}}(G_f+G_g\left(E\right[\parallel {\lambda }_t\parallel ]+\sigma D\left)\right)$$

Substituting ${\alpha }_t={\alpha }_0\sqrt{t}$ and the bound from Lemma3,

$$E[\parallel x_{t+1}-x_t\parallel ]\le {\displaystyle \frac{1}{{\alpha }_0\sqrt{t}}}(G_f+G_g(C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s+\sigma D))$$

Summing from $t=1$ to T,

$$\sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]\le {\displaystyle \frac{1}{{\alpha }_0}}\sum _{t=1}^T{\displaystyle \frac{1}{\sqrt{t}}}(G_f+G_g(C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s+\sigma D))$$

Using the integral bound,

$$\sum _{t=1}^T{\displaystyle \frac{1}{\sqrt{t}}}\le 1+{\int }_1^T{\displaystyle \frac{1}{\sqrt{t}}}dt=1+2(\sqrt{T}-1)\le 2\sqrt{T}$$

Therefore,

$$\sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]\le {\displaystyle \frac{2\sqrt{T}}{{\alpha }_0}}(G_f+G_g(C_1+C_2{\alpha }_0+C_3\sigma +C_4\sigma s+\sigma D))$$

Substituting the parameter choices ${\alpha }_0={\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}$ and $\sigma ={\displaystyle \frac{1}{\sqrt{T}}}$, and choosing $s=\sqrt{T}$,

$$\begin{array}{cc}\hfill \sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]\le & 2\sqrt{T\overline{\Delta }\left(T\right)}(G_f+G_g(C_1+C_2{\displaystyle \frac{1}{\sqrt{\overline{\Delta }\left(T\right)}}}\hfill \\ \hfill & +C_3{\displaystyle \frac{1}{\sqrt{T}}}+C_4+D{\displaystyle \frac{1}{\sqrt{T}}}\left)\right)\hfill \end{array}$$

Under reasonable assumptions (that $\overline{\Delta }\left(T\right)$ does not grow too fast), we have,

$$\sum _{t=1}^{T}E[\parallel x_{t+1}-x_t\parallel ]\le \mathcal{O}\left(\sqrt{T}\right)$$

So we have,

$$\begin{array}{cc}\hfill Vi{o}_i\left(T\right)=& E\left[\sum _{t=1}^T{g}_{t,i}\left(x_t\right)\right]\hfill \\ \hfill \le & {\displaystyle \frac{1}{\sigma }}E[\parallel {\lambda }_{T+1}\parallel ]+G_g\sum _{t=1}^{T}E\parallel x_{t+1}-x_t\parallel \hfill \\ \hfill \le & {\displaystyle \frac{1}{\sigma }}·\mathcal{O}\left(1\right)+G_g·\mathcal{O}\left(\sqrt{T}\right)\hfill \\ \hfill =& \sqrt{T}·\mathcal{O}\left(1\right)+\mathcal{O}\left(\sqrt{T}\right)\hfill \\ \hfill =& \mathcal{O}\left(\sqrt{T}\right)\hfill \end{array}$$

4. Numerical Experiments

In this section, our algorithm is demonstrated to be capable of solving many real problems. Firstly, we explored the influence of some initial parameter values on our algorithm on different mathematical models. The best parameter combinations for each model are provided. Then we compare our algorithm with the PSGD algorithm in the same simulation environment. Secondly, we created a simulation experiment to test our algorithm. The performance of our algorithm when solving different real problems is observed. Then the results of the experiments are presented. Finally, we combined our algorithm with supervised learning for path planning. The results show that regret and constraint violation also have convergent bounds. This proves that our algorithm has the ability to solve online path planning problems.

In addition, in this paper, the model training was conducted on Python 3. 12. 6 and all the experiments were conducted on Matlab 2025b on a laptop with Windows 11 installed for fairness. The CPU of this laptop is AMD Ryzen AI 9 H 465 w/ Radeon 880M (2.00 GHz) and 32GB of RAM.

4.1. Comparative Experiment with the Existing Algorithm

We compared our algorithm with the PSGD algorithm using adaptive filtering and online logistic regression problems.

Adaptive filtering is a core recursive estimation technique in modern signal processing, system identification, and control. In many applications, the impulse response exhibits sparsity in the time domain. Meanwhile, we consider that physically realizable systems are all stable, so their impulse response energy must be finite. There are two constraints, based on the actual context of the problem. The mathematical model of adaptive filtering is as follows:

$$\begin{array}{cc}\hfill & \underset{\mathbf{x}\in {\mathbb{R}}^n}{min}a·x+b\hfill \\ \hfill {\mathrm{s}.\mathrm{t}.\parallel x\parallel }_1& \le {\gamma }_s,{\parallel x\parallel }_2^2\le {\gamma }_e\hfill \end{array}$$

At each round, a and b are drawn from two independent normal distributions, whose means and standard deviations both vary smoothly over time in a sinusoidal or cosinusoidal manner.

Online logistic regression is a classic binary classification method in machine learning. It has significant application value in dynamic data stream environments. This problem is mathematically formulated as a constrained regularized empirical risk minimization problem. The objective function controls the model’s complexity and prevents overfitting. The mathematical model of online logistic regression is as follows:

$$\begin{array}{cc}\hfill \underset{\theta \in {\mathbb{R}}^n}{min}{\displaystyle \frac{1}{m}}\sum _{i=1}^m& [-y_{i}log\left(\sigma \left(x_i^{\top }\theta \right)\right)-(1-y_i)log(1-\sigma \left(x_i^{\top }\theta \right))]+{\displaystyle \frac{\lambda }{2}}{\parallel \theta \parallel }_2^2\hfill \\ \hfill & {\mathrm{s}.\mathrm{t}.\parallel \theta \parallel }_1\le {\gamma }_s,{\parallel \theta \parallel }_2^2\le {\gamma }_e\hfill \end{array}$$

At each round, $x_t$ is drawn from a normal distribution with fixed covariance, $y_t$ is drawn from a Bernoulli distribution, and the true parameter vector $w_{\mathrm{true}}\left(t\right)$ varies smoothly over time in a sinusoidal manner.

We explored the influence of initial parameter values on our algorithm. Different ${\alpha }_0$ values in our algorithm were compared. There are two criteria for measuring the quality of an online algorithm, so a multi-objective planning approach was adopted to make a mixed measurement. We used AHP(Analytical Hierarchy Process) to determine the weights of the two quantities. The two measurements correspond to scale value 3 in the 1-9 scale method. So the weight of regret is 0.25 and the weight of constraint violation is 0.75. Our mixed measurement $mix$ is defined as follows:

$$mix=0.25*regret+0.75*violation$$

In Figure 1, in adaptive filtering, we found that the algorithm performs best when ${\alpha }_0=0.2$. And in online logistic regression, the algorithm performs best when ${\alpha }_0=0.5$.

Then we compared our algorithm with PSGD algorithm. The comparison results are as follows:

Figure 2. We compared our algorithm with PSGD algorithm. (a) The regret of adaptive filtering. (b)The constraint violation of adaptive filtering. (c) The regret of online logistic regression. (d) The constraint violation of online logistic regression.

Figure 2. We compared our algorithm with PSGD algorithm. (a) The regret of adaptive filtering. (b)The constraint violation of adaptive filtering. (c) The regret of online logistic regression. (d) The constraint violation of online logistic regression.

we compared the performance of two algorithms in adaptive filtering and online logistic regression. The figures show that MSALM attains lower regret and constraint violation bounds than PSGD. This result indicates that our algorithm has the ability to converge more rapidly toward the theoretical optimum while adhering better to the constraints. According to the experimental results, the conclusion can be obtained. When choosing a suitable ${\alpha }_0$, MSALM is superior to the existing algorithm.

4.2. Experiments Under Existing Models

In order to test the practicality of our algorithm, we applied our algorithm in energy dispatch problem and network resource allocation.

Time-varying smart grid energy dispatch problem is one of the core tasks of modern smart grids. This problem is to economically and reliably coordinate multiple heterogeneous energy sources while meeting the changing electricity demand. The objective functions of this problem include power balance constraints and resource capacity constraints. The mathematical model of time-varying smart grid energy dispatch is as follows:

$$\underset{x\in {\mathbb{R}}^n}{min}{c}^{\top }x+{\displaystyle \frac{1}{2}}{x}^{\top }Qx$$

$$\mathrm{s}.\mathrm{t}.\sum _{i=1}^{n_g}{x}_i-\sum _{j=1}^{n_s}{x}_{n_g+j}-\sum _{k=1}^{n_d}{x}_{n_g+n_s+k}=d$$

$$0\le x_i\le u_i,i=1,\dots ,n$$

At each round, c, Q are drawn from independent normal distributions with fixed standard deviations. Their means evolve over time according to a random walk with a decaying step size.

Online network resource allocation is a key challenge in modern computing systems. The key task is to efficiently allocate limited resources to continuously arriving real-time tasks or user requests. The objective function consists of two parts: Quality of Service cost and resource usage cost. The mathematical model of online network resource allocation is as follows:

$$\underset{x\in {\mathbb{R}}^n}{min}\sum _{i=1}^n{(d_i-x_i)}^2+\alpha \sum _{i=1}^n{x}_i$$

$$\mathrm{s}.\mathrm{t}.x_i\le c_i,i=1,\dots ,n$$

At each round, the demand vector is drawn from a multivariate normal distribution with fixed standard deviation, and is then truncated to be nonnegative. Its mean vector varies smoothly over time following a sinusoidal pattern with a slow linear trend.

We conducted simulation experiments on two mathematical models, and the results are as follows,

Figure 3. Our algorithm is applied to energy dispatch problem and online network resource allocation. (a) Regret of time-varying smart grid energy dispatch. (b)Constraint violation of time-varying smart grid energy dispatch. (c) Regret of online network resource allocation. (d) Constraint violation of online network resource allocation.

Figure 3. Our algorithm is applied to energy dispatch problem and online network resource allocation. (a) Regret of time-varying smart grid energy dispatch. (b)Constraint violation of time-varying smart grid energy dispatch. (c) Regret of online network resource allocation. (d) Constraint violation of online network resource allocation.

We took ${\alpha }_0=0.7$ and ${\alpha }_0=0.37$ for testing, respectively. The simulation experiments show that regret and constraint violation have a sublinear convergent bound of T, demonstrating that our algorithmic solution can gradually approach the theoretically optimal solution while adhering to the constraints. The results show that our algorithm can be applied to time-varying smart grid energy dispatch problem and the online network resource allocation problem.

4.3. Experiment Combining Our Algorithm with Supervised Learning

We combined MSALM with supervised learning to solve path planning problem. For obtaining an explicit function that is used in our algorithm, we applied parameter regression methods in supervised learning. First of all, 10 flight trajectories’ data for the same flight at the same time slot on different dates were randomly selected. Afterwards, we utilized these data to train a mathematical model for flight trajectories through parameter regression methods.B-spline interpolation was used to obtain the explicit function of the flight trajectory. In order to find the appropriate number of B-spline interpolation control points, we compared the errors between the fitted trajectory and 10 known trajectories under different numbers of control points. The results are as follows:

Table 1. Fitting error with different numbers of control points.

Number of Control Points	3D RMSE (m)	Mean Error (m)	Max Error (m)
6	16212.93	14630.05	42953.94
7	15278.20	13697.46	28088.96
8	10301.73	9577.57	20571.04
9	7767.29	7094.96	15264.24
10	6056.13	5466.36	12094.13
11	6474.88	5956.19	12599.98
12	6416.03	5793.27	12159.70

Table 1. Fitting error with different numbers of control points.

Number of Control Points	3D RMSE (m)	Mean Error (m)	Max Error (m)
6	16212.93	14630.05	42953.94
7	15278.20	13697.46	28088.96
8	10301.73	9577.57	20571.04
9	7767.29	7094.96	15264.24
10	6056.13	5466.36	12094.13
11	6474.88	5956.19	12599.98
12	6416.03	5793.27	12159.70

According to the above table, the accuracy reaches its highest when the trajectory has 10 control points. So we obtained the fitting function in this case. The coordinates of the feature points are as follows:

Table 2. Coordinates of the control points.

Control Point	X (m)	Y (m)	Z (m)
1	478288.923	4423806.307	1199.714
2	483007.653	4412762.620	1939.867
3	491176.780	4411013.545	1022.683
4	540450.109	4386975.935	6298.608
5	613313.197	4283290.653	6170.731
6	753322.541	4268797.429	6188.807
7	792318.111	4171779.364	6157.780
8	773451.251	4061749.319	3778.580
9	782481.198	4033694.042	2106.506
10	782988.940	4014086.869	1142.003

Table 2. Coordinates of the control points.

Control Point	X (m)	Y (m)	Z (m)
1	478288.923	4423806.307	1199.714
2	483007.653	4412762.620	1939.867
3	491176.780	4411013.545	1022.683
4	540450.109	4386975.935	6298.608
5	613313.197	4283290.653	6170.731
6	753322.541	4268797.429	6188.807
7	792318.111	4171779.364	6157.780
8	773451.251	4061749.319	3778.580
9	782481.198	4033694.042	2106.506
10	782988.940	4014086.869	1142.003

We set some random situations as constraints to simulate the real flight conditions. The constraints include 9 dynamic obstacle avoidance constraints and 4 airspace boundary constraints. The mathematical model of dynamic obstacle avoidance constraints has the following form:

$$g_{k,i}=d_{safe,k}^2\left(t\right)-\left|\right|p_i-c_k\left(t\right){\left|\right|}^2\le 0$$

where $c_k\left(t\right)$ is the obstacle center coordinates, $d_{safe,k}\left(t\right)$ is the safe distance, $p_i$ is the control point with obstacle avoidance constraint. We set 3 obstacles and the 1st, 5th, and 10th control points with obstacle avoidance constraints, so $k=1,2,3$ and $i\in \{1,5,10\}$.

The mathematical model of airspace boundary constraints has the following form:

$$x_{min}\le x\le x_{max}$$

$$y_{min}\le y\le y_{max}$$

Then we use our algorithm to solve this online optimization problem. The result of the experiment is as follows:

Figure 4. This is a figure in which our algorithm is applied to flight path planning. (a) The regret of flight path planning (b) The constraint violation of flight path planning.

Figure 4. This is a figure in which our algorithm is applied to flight path planning. (a) The regret of flight path planning (b) The constraint violation of flight path planning.

In this figure, the results of the simulation experiment show that regret and constraint violation of our algorithm have a sublinear convergent bound of T, demonstrating progressive convergence to the theoretical optimum under compliance with the constraint. This indicates that the aircraft can make decisions that gradually approach the optimal choice during the flight and stay as safe as possible in the face of emergencies. This proves that our algorithm can combine with machine learning methods to be applied to similar path planning problems.

5. Conclusions

In this paper, we present a new online stochastic augmented Lagrangian method, the MSALM, for solving online stochastic optimization problems with time-varying distributions. At each round, we construct the model functions for the objective and constraint functions based on their properties, which reduces the computational complexity. The step size is designed in a dynamic form and decreases as t increases to accelerate convergence. Under standard assumptions, we proved that our algorithm achieves sublinear regret and constraint violation bounds of T. Simulation experiments demonstrate the performance and practical utility of the MSALM algorithm. Additionally, in the context of path planning, we combined our algorithm with supervised learning to further demonstrate its extensibility.