A Novel Symmetrical Inertial Alternating Direction Method of Multipliers with Proximal Term for Nonconvex Optimization with Applications

1. Introduction

In recent years, nonconvex optimization problems have found widespread applications in science and engineering. For instance, Mohammadreza et al. [1] investigated the optimization of local nonconvex objective functions in time-varying networks based on the gradient tracking algorithm. And then, the researchers had explored optimizing nonconvex objective functions in multi-node networks under imperfect data exchange links. Zhang et al. [2] pointed out that traditional optimization methods often lead to target feature compression and information loss in motor imagery decoding, thereby reducing classification performance. Further, to address the high dimensionality and small sample size characteristics of motor imagery signals, Zhang et al. [2] proposed a nonconvex sparse regularization model constructed using the Cauchy function. This approach enables more accurate extraction of target features across multiple datasets while effectively suppressing noise interference. In addition, Tiddeman and Ghahremani [3] combined wavelet transforms with principal component analysis to propose a class of principal component waveform networks for solving linear inverse problems, they fully utilized the symmetry in wavelet transforms during the wavelet decomposition, ensuring the effectiveness of image reconstruction. For more related works, one can see [4,5,6] and the references therein.

It is well known that recovering sparse signals from incomplete observations is an important research direction in practical applications. The core objective is to find the optimal sparse solution to a system of linear equations, which can be formulated as the following model [7]:

$${minc\parallel x\parallel }_0+{\displaystyle \frac{1}{2}}\parallel Ax-b\parallel ,$$

where A is the measurement matrix, b is the observed data, x is a sparse signal, $c>0$ is a regularization parameter, and ${\parallel ·\parallel }_0$ denotes the ${\ell }_0$-norm. However, Chartrand and Staneva [8] pointed out that the above problem represents a class of problems that are fundamentally difficult to solve. To overcome this challenge, Zeng et al. [9] proposed a relaxed objective function by replacing the $l_0$ regularization with the $l_{{\displaystyle \frac{1}{2}}}$ regularization, the problem is transformed into a more tractable nonconvex optimization problem. Therefore, adopting this modification becomes more reasonable in signal recovery problems, which leads to the following two-block nonconvex optimization problem:

$$\begin{array}{cc}\hfill min& {c\parallel x\parallel }_{{\displaystyle \frac{1}{2}}}^{{\displaystyle \frac{1}{2}}}+{\displaystyle \frac{1}{2}}{\parallel Ax-b\parallel }^2,\hfill \end{array}$$

(1)

where ${\parallel x\parallel }_{{\displaystyle \frac{1}{2}}}^{{\displaystyle \frac{1}{2}}}={\sum }_{i=1}^n{\left|x_i\right|}^{{\displaystyle \frac{1}{2}}}$. In general, we would consider introducing an auxiliary variable y to reformulate the problem (1) as follows

$$\begin{array}{cc}\hfill min& {c\parallel x\parallel }_{{\displaystyle \frac{1}{2}}}^{{\displaystyle \frac{1}{2}}}+{\displaystyle \frac{1}{2}}{\parallel y\parallel }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ax-y=b.\hfill \end{array}$$

(2)

Zeng et al. [9] pointed out that the iterative soft-thresholding algorithm can be used to solve the regularization problem, which was validated in context of problem (2). Meanwhile, Chen and Selesnick [10] performed a performance validation of model (2) using an improved overlapping shrinkage algorithm. In addition, further related works can be found in [11,12].

In statistical optimization, certain penalty methods exhibit limitations, such as vulnerability to data circumvention and biased estimation of significant variables [14]. To address these issues, Fan and Li [14] proposed the smoothly clipped absolute deviation (SCAD) penalty function. They developed optimization algorithms to solve non-concave penalized likelihood problems and demonstrated that this method possesses asymptotic oracle properties. Remarkably, with appropriate regularization parameter selection, the results can achieve nearly identical performance to the known true model. SCAD penalty problem proposed by scholars can be conceptually understood as

$$\begin{array}{cc}\hfill min& {\displaystyle \sum _{i=1}^n}{h}_k\left(\right|x_i\left|\right)+{\displaystyle \frac{1}{2}}{\parallel y\parallel }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ax-y=b,\hfill \end{array}$$

(3)

with $A\in {\mathbb{R}}^{m\times n}$, $y,b\in {\mathbb{R}}^m$, $x={(x_1,x_2,\dots ,x_n)}^T\in {\mathbb{R}}^n$ and the penalty function $h_{\kappa }$ in the objective, we refer readers to (26) later. As shown above, problems of the forms (2) and (3) can be generalized to the following nonconvex optimization problem:

$$\begin{array}{cc}\hfill min& f\left(x\right)+g\left(y\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ax+y=b,\hfill \end{array}$$

(4)

where f is a lower semicontinuous function from ${\mathbb{R}}^n$ to $\mathbb{R}$, and $g:{\mathbb{R}}^m\to \mathbb{R}$ is a differentiable function whose gradient is L-Lipschitz continuous with $L>0$. Here, A denotes a matrix in ${\mathbb{R}}^{m\times n}$, $x\in {\mathbb{R}}^n$ and $y\in {\mathbb{R}}^m$. Variants of model (4) have found applications in various fields, such as statistical learning [15,16,17], penalized zero-variance discriminant analysis [18] and image reconstruction [19,20].

It is well known that the alternating direction method of multipliers (ADMM) has gained widespread attention due to its balance between performance and efficiency. When the subproblems are independent, ADMM exhibits a unique symmetry. In fact, with appropriately designed update steps, this symmetry ensures that the convergence of ADMM is independent of the order in which the subproblems are updated [21]. In recent years, as nonconvex optimization problems have gained increasing attention, the convergence analysis of ADMM in nonconvex settings has become a research hotspot. Hong et al. [22], recognizing the strong empirical performance of ADMM on nonconvex problems but the lack of theoretical guarantees, not only established the convergence theory for nonconvex ADMM but also overcame the limitation on the number of variable blocks. Wang et al. [23] demonstrated that incorporating the Bregman distance into ADMM can effectively simplify the computation of subproblems, emphasizing the feasibility of ADMM in nonconvex settings. Ding et al. [24] proposed a class of Semi-Proximal ADMM for solving low-rank matrix recovery problems. In the presence of noisy matrix data, by minimizing the nuclear norm, they effectively addressed the issues of Gaussian noise and related mixed noise. Guo et al. [25] provided insights into solving large-scale nonconvex optimization problems using ADMM. For more related work, readers may refer to [26,27,28,29].

The inertial acceleration technique, derived from the heavy-ball method, utilizes information from previous iterations to construct affine combinations [30]. Additionally, we observe that inertial techniques can employ different extrapolation strategies during the optimization process to enhance convergence speed. In their study of the general inertial proximal gradient method, Wu and Li [31] proposed two distinct extrapolation strategies to flexibly adjust the algorithm’s convergence rate. Chen et al. [32] investigated an inertial proximal ADMM and established the global convergence of iterates under appropriate assumptions. In fact, Chao et al. [33] also discovered that embedding the inertial term into the y-subproblem can significantly improve the algorithm’s convergence speed. Moreover, compared to the standard inertial update step ${\overline{x}}^k=x^k+\eta (x^k-x^{k-1})$, Wang et al. [34] considered a different update scheme, ${\overline{x}}^k=x^k+\eta (x^k-{\overline{x}}^k)$. This update step not only preserves the acceleration effect of inertia but also reduces computational errors introduced by the inertial term updates.

Unfortunately, the work of Wang et al. only considered the inertial update step for x. Inspired by the work in [34], we propose a novel symmetrical inertial alternating direction method of multipliers with proximal term (NIP-ADMM). Building upon Wang et al.’s inertial update step, we introduce an additional inertial update for y and incorporate ${\overline{y}}^k$ into the x-subproblem update, this form of inertial update ensures that the primal variables are treated equally, thereby achieving faster acceleration. To simplify the computation of the subproblems, we introduce an approximation term in the x-subproblem, which under certain conditions, allows the nonconvex subproblem to be transformed into an approximate projection-type problem. Furthermore, since $g\left(y\right)$ is convex, this property ensures that $g\left(y\right)$ is well-defined, this enables us to abandon the traditional ADMM update scheme and instead adopt a gradient descent approach. This method requires only the computation of gradients at each iteration, significantly reducing computational complexity. Consequently, it offers substantial advantages when handling high-dimensional or large-scale datasets.

The structure of this paper is as follows. In Section 2, we review essential results required for further analysis. We present NIP-ADMM and analyze its convergence in Section 3. Numerical experiment and application to signal recovery in Section 4 highlight the benefits of the majorization and inertial techniques. Lastly, in Section 5, we sent a conclusion.

2. Preliminaries

In this section, we introduce key notations and definitions that are essential for the results to be developed and are utilized in the subsequent sections.

Assume $\langle x,y\rangle =x^{T}y$, $\parallel x\parallel =\sqrt{\langle x,x\rangle }$. If matrix Q is a positive definite (semi-definite positive) matrix, then we have $Q\succ 0$$(Q⪰0)$. Given any $n\times n$ matrix $Q⪰0$ and a vector $x\in {\mathbb{R}}^n$, let ${\parallel x\parallel }_Q:=\sqrt{x^{T}Qx}$ be Q-norm of x. For the matrix G, we define ${\lambda }_{{min}_{\left(G\right)}}$ and ${\lambda }_{{max}_{\left(G\right)}}$ as the smallest and largest eigenvalues of $G^{T}G$, respectively. If we denote $f:{\mathbb{R}}^n\to (-\infty ,+\infty ]$, then the domain of the function is defined as $domf=\{x\in {\mathbb{R}}^n|f\left(x\right)<+\infty \}$.

Definition 1. 

Let $S\subseteq {\mathbb{R}}^n$. Then the distance from point $x\in {\mathbb{R}}^n$ to S is defined as $d(x,S)={inf}_{y\in S}\parallel y-x\parallel$. In particular, if $S=\varnothing$, then $d(x,S)=+\infty$.

Definition 2. 

For a differentiable convex function $F:{\mathbb{R}}^n\to \mathbb{R}$, the Bregman distance is defined by

$${▵}_F(p,q)=F\left(p\right)-F\left(q\right)-\langle \nabla F\left(q\right),p-q\rangle ,$$

where $p,q\in {\mathbb{R}}^n$.

Definition 3. 

Assume $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ is a proper lower semicontinuous function.

(i)

The Frechet sub-differential of f at $x\in \mathrm{dom}f$ is denoted by $\widehat{\partial }f\left(x\right)$ and defined as:

$$\widehat{\partial }f\left(x\right)=\left\{\overline{x}\in {\mathbb{R}}^n:\underset{y\ne x}{lim}\underset{y\to x}{inf}{\displaystyle \frac{f\left(y\right)-f\left(x\right)-〈\overline{x},y-x〉}{\parallel y-x\parallel }}\ge 0\right\}.$$

Among others, we set $\widehat{\partial }f\left(x\right)=\varnothing$ when $x\notin \mathrm{dom}f$.

(ii)

The limiting sub-differential of f at $x\in \mathrm{dom}f$ is written as $\partial f\left(x\right)$ and defined by

$$\partial f\left(x\right)=\left\{\overline{x}\in {\mathbb{R}}^n:\exists x_k\to x,f\left(x_k\right)\to f\left(x\right),{\widehat{x}}_k\in \widehat{\partial }f\left(x_k\right),{\widehat{x}}_k\to \overline{x}\right\}.$$

Proposition 1. 

The sub-differential of a lower semicontinuous function f possesses several fundamental and significant properties as follows:

(i)

From Definition 3, which implies that $\widehat{\partial }f\left(x\right)\subseteq \partial f\left(x\right)$ holds for all $x\in {\mathbb{R}}^n$, and given that $\partial f\left(x\right)$ is a closed set, $\widehat{\partial }f\left(x\right)$ is also a closed set.

(ii)

Suppose that $(x^k,y^k)$ is a sequence that converges to $(x^{★},y^{★})$, and $f\left(x^k\right)$ converges to $f\left(x^{★}\right)$ with $y^k\in \partial f\left(x^k\right)$. Then, by the definition of the sub-differential, we have $y^{★}\in \partial f\left(x^{★}\right)$.

(iii)

If $x^{★}$ is a local minimum of f, then it follows that $0\in \partial f\left(x^{★}\right)$.

(vi)

Assuming that $g:{\mathbb{R}}^n\to \mathbb{R}$ is a continuously differentiable function, we can derive:

$$\partial (f+g)\left(x\right)=\partial f\left(x\right)+\nabla g\left(x\right).$$

Definition 4. 

We consider the point $(x^{★},y^{★},{\lambda }^{★})$ to be a critical point of the augmented Lagrangian function ${\widehat{L}}_{\beta }(x,y,\lambda )$ if it satisfies the following conditions:

$$\left\{\begin{array}{c}{A}^T{\lambda }^{★}\in \partial f\left(x^{★}\right),\hfill \\ B^T{\lambda }^{★}=\nabla g\left(y^{★}\right),\hfill \\ A{x}^{★}+B{y}^{★}=b.\hfill \end{array}\right.$$

Definition 5 

([36]). (Kurdyka-Łojasiewicz property (KLP)) Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper lower semicontinuous function. For $\widehat{p}\in \partial f$ ($\partial f\ne \varnothing$), if there exists $\varsigma \in (0,+\infty )$, a neighborhood U of $\widehat{p}$, and a function $\phi \in Q_{\phi }$, here $Q_{\phi }$ is the set of the concave function $\phi :[0,\varsigma )\to [0,+\infty )$, then for any $p\in \{p\in U\mid f\left(\widehat{p}\right)<f\left(p\right)<f\left(\widehat{p}\right)+\varsigma \}$, the following inequality holds

$${\phi }^{\prime }(f\left(p\right)-f\left(\widehat{p}\right))d(0,\partial f\left(p\right))\ge 1,$$

and we call that f satisfies KLP.

Lemma 1 

([35]). Suppose that the matrix $B\in {\mathbb{R}}^{r\times p}$ is a non-zero matrix, and let ${\mu }_B$ denote the smallest positive eigenvalue of the matrix $B{B}^T$. Then for each $u\in {\mathbb{R}}^P$, the following holds:

$$\parallel P_{Q^T}u\parallel \le {\displaystyle \frac{1}{{\mu }_B}}\parallel Bu\parallel .$$

Lemma 2 

([36]). Assume $B(x,y)=f\left(x\right)+g\left(y\right)$, where $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ and $g:{\mathbb{R}}^m\to \mathbb{R}\cup \{+\infty \}$ are both proper lower semicontinuous functions. Then, for any $(x,y)\in \mathrm{dom}B=\mathrm{dom}f\times \mathrm{dom}g$, we can obtain

$$\partial B(x,y)={\partial }_{x}f(x,y)\times {\partial }_{y}g(x,y).$$

Lemma 3 

([37]). (Uniformized KLP) Let Ω be a compact set and $Q_{\phi }$ be the same as in Definition 5. If a proper lower semicontinuous function $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ is fixed at a point in Ω and satisfies the KLP at every point on Ω, and there exist $\varrho >0$, $\varsigma >0$, and $\phi \in Q_{\phi }$ such that for any $\widehat{x}\in \mathsf{\Omega }$ and $x\in \{x\in {\mathbb{R}}^n:d(x,\Omega )<\varrho \}\cap [f\left(\widehat{x}\right)<f\left(x\right)<f\left(\widehat{x}\right)+\varsigma ]$, then the following inequality is satisfied:

$${\phi }^{\prime }(f\left(x\right)-f\left(\widehat{x}\right))d(0,\partial f\left(x\right))\ge 1.$$

Lemma 4 

([38]). If the function $c:{\mathbb{R}}^n\to \mathbb{R}$ is continuously differentiable, and $\nabla c$ is Lipschitz continuous with constant $L\ge 0$, then for any $x,y\in {\mathbb{R}}^n$, the following result holds:

$$|c\left(y\right)-c\left(x\right)-\langle \nabla c\left(x\right),y-x\rangle |\le {\displaystyle \frac{L}{2}}{\parallel y-x\parallel }^2.$$

3. Algorithm and Convergence Analysis

In this section, we first present the definition of the augmented Lagrangian function associated with problem (4) as follows:

$$\begin{array}{c}{L}_{\beta }(x,y,\lambda )=f\left(x\right)+g\left(y\right)+\langle \lambda ,Ax+y-b\rangle +{\displaystyle \frac{\beta }{2}}{\parallel Ax+y-b\parallel }^2,\end{array}$$

(5)

where $\lambda$ denotes the augmented Lagrange multiplier, and $\beta >0$ is a penalty parameter. Following this, we propose the NIP-ADMM for solving the problem (4), the proposed algorithm is outlined below:

Remark 1.

(i)

In NIP-ADMM, the inertial parameters η and θ are both in $(0,1]$, and S is a positive semi-definite matrix.

(ii)

The update scheme for the y-subproblem adopts the gradient descent method, where ${\nabla }_y{L}_{\beta }$ is the gradient of the function L with respect to y, and γ is called the learning rate.

(iii)

The inertial structure we adopted employs a structurally balanced acceleration strategy. This update strategy is mathematically symmetric, with the only distinction being the values of the parameters η and θ.

According to Algorithm 1, the optimality conditions for NIP-ADMM are obtained as

$$\left\{\begin{array}{c}0\in \partial f\left(x^{k+1}\right)-A^T{\lambda }^k+\beta A^T(A{x}^{k+1}+{\overline{y}}^k-b)+S(x^{k+1}-{\overline{x}}^k),\hfill \\ 0=\nabla g\left(y^k\right)+{\lambda }^k+\beta (A{x}^{k+1}+y^k-b)-{\displaystyle \frac{1}{\gamma }}(y^{k+1}-y^k).\hfill \end{array}\right.$$

(6)

Before concluding this section, we present the following fundamental assumptions, which are essential for the convergence analysis.

Algorithm 1: NIP-ADMM

Initialization: Input $x^1,y^1$, and ${\lambda }^1$, let ${\overline{x}}^0=x^1$, and ${\overline{y}}^0=y^1$. Given constants $\eta ,\theta ,\gamma ,\beta$. Set $k=1$. While "not converge" Do $1^{\circ }$ Compute $({\overline{x}}^k,{\overline{y}}^k)=(x^k,y^k)+\theta (x^k-{\overline{x}}^{k-1},0)+\eta (0,y^k-{\overline{y}}^{k-1})$. $2^{\circ }$ Execute $argmin\{L_{\beta }(x,{\overline{y}}^k,{\lambda }^k)+{\displaystyle \frac{1}{2}}\parallel x-{\overline{x}}^k{\parallel }_S^2$ to determine $x^{k+1}$. $3^{\circ }$ Calculate $y^{k+1}=y^k-\gamma {\nabla }_y{L}_{\beta }(x^{k+1},y^k,{\lambda }^k)$. $4^{\circ }$ Update dual variable ${\lambda }^{k+1}={\lambda }^k+\beta (A{x}^{k+1}+y^{k+1}-b)$. $5^{\circ }$ Let $k=k+1$. End While Output: output $(x^{k+1},y^{k+1},{\lambda }^{k+1})$ of the problem (4).

Assumption 1. (i) $f:{\mathbb{R}}^{n_1}\to \mathbb{R}\cup \{+\infty \}$ is a proper lower semicontinuous function. $g:{\mathbb{R}}^{n_2}\to \mathbb{R}$ is continuously differentiable, and $\nabla g$ is Lipschitz continuous with a Lipschitz constant $l_g>0$.

(ii)

S is a positive semidefinite matrix.

(iii)

For convenience, we introduce the following symbols:

$$\begin{array}{cc}\hfill & \zeta =(x,y,\lambda ),{\zeta }^k=(x^k,y^k,{\lambda }^k),{\zeta }^{*}=(x^{*},y^{*},{\lambda }^{*}),\xi ={\displaystyle \frac{1}{\gamma }}-\beta ,\hfill \\ \hfill & {\widehat{\zeta }}^k=(x^k,y^k,{\lambda }^k,{\overline{x}}^k,x^{k-1},y^{k-1}),{\sigma }_0={\displaystyle \frac{1}{\gamma }}-{\displaystyle \frac{L+\beta }{2}}-{\displaystyle \frac{2{\xi }^2}{\beta }}-{\displaystyle \frac{2{(\xi +L)}^2}{\beta }},\hfill \\ \hfill & {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)=L_{\beta }\left({\zeta }^k\right)+{\displaystyle \frac{{(\xi +L)}^2}{\beta }}\parallel y^{k+1}-y^k{\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel {\overline{x}}^{k+1}-x^k\parallel }_S^2.\hfill \end{array}$$

(iv)

To analyze the monotonicity of $\left\{{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)\right\}$, we set ${\sigma }_0>0$.

Lemma 5. 

If Assumption 1 holds, for any $k\ge 1$, then

$$\begin{array}{cc}\hfill & {\widehat{L}}_{\beta }(x^{k+1},y^{k+1},{\lambda }^{k+1},{\overline{x}}^{k+1},x^k,y^k)-{\widehat{L}}_{\beta }(x^k,y^k,{\lambda }^k,{\overline{x}}^{k-1},x^{k-1},y^{k-1})\hfill \\ \hfill & \le -{\displaystyle \frac{1-{\eta }^2}{2}}\parallel x^k-{\overline{x}}^{k-1}{\parallel }_S^2-{\sigma }_0{\parallel y^{k+1}-y^k\parallel }^2,\hfill \end{array}$$

(7)

where $\eta \in (0,1]$ is the inertial parameter in Algorithm 1.

Proof. 

According to the definition of the Lagrangian function, one gets

$$\begin{array}{cc}\hfill & L_{\beta }(x^{k+1},y^{k+1},{\lambda }^{k+1})-L_{\beta }(x^{k+1},y^{k+1},{\lambda }^k)\hfill \\ \hfill & =\langle {\lambda }^{k+1}-{\lambda }^k,A{p}^{k+1}+q^{k+1}-b\rangle ={\displaystyle \frac{1}{\beta }}{\parallel {\lambda }^{k+1}-{\lambda }^k\parallel }^2,\hfill \end{array}$$

(8)

and we also can know that

$$\begin{array}{cc}\hfill & L_{\beta }(x^{k+1},y^{k+1},{\lambda }^k)-L_{\beta }(x^{k+1},y^k,{\lambda }^k)\hfill \\ \hfill & =g\left(y^{k+1}\right)-g\left(y^k\right)-\langle {\lambda }^k,y^{k+1}-y^k\rangle +{\displaystyle \frac{\beta }{2}}{\parallel A{x}^{k+1}+y^{k+1}-b\parallel }^2\hfill \\ \hfill & -{\displaystyle \frac{\beta }{2}}{\parallel A{x}^{k+1}+y^k-b\parallel }^2.\hfill \end{array}$$

(9)

It follows from (6), (9) and Lemma 4 that

$$\begin{array}{cc}\hfill & g\left(y^{k+1}\right)-g\left(y^k\right)-\langle {\lambda }^k,y^{k+1}-y^k\rangle \hfill \\ \hfill & \le \langle \nabla g\left(y^k\right),y^{k+1}-y^k\rangle +{\displaystyle \frac{L}{2}}{\parallel y^{k+1}-y^k\parallel }^2-\langle {\lambda }^k,y^{k+1}-y^k\rangle \hfill \\ \hfill & =\langle {\lambda }^{k+1}+({\displaystyle \frac{1}{\gamma }}-\beta )(y^k-y^{k+1}),y^{k+1}-y^k\rangle +{\displaystyle \frac{L}{2}}{\parallel y^{k+1}-y^k\parallel }^2-\langle {\lambda }^k,y^{k+1}-y^k\rangle \hfill \\ \hfill & =({\displaystyle \frac{L}{2}}+\beta -{\displaystyle \frac{1}{\gamma }}){\parallel y^{k+1}-y^k\parallel }^2+\langle {\lambda }^{k+1}-{\lambda }^k,y^{k+1}-y^k\rangle ,\hfill \end{array}$$

and we get

$$\begin{array}{cc}\hfill & {\displaystyle \frac{\beta }{2}}\parallel A{x}^{k+1}+y^{k+1}{-b\parallel }^2-{\displaystyle \frac{\beta }{2}}{\parallel A{x}^{k+1}+y^k-b\parallel }^2\hfill \\ \hfill & ={\displaystyle \frac{\beta }{2}}\parallel A{x}^{k+1}+y^{k+1}{-b\parallel }^2-{\displaystyle \frac{\beta }{2}}{\parallel (A{x}^{k+1}+y^{k+1}-b)+(y^k-y^{k+1})\parallel }^2\hfill \\ \hfill & ={\displaystyle \frac{\beta }{2}}\parallel A{x}^{k+1}+y^{k+1}{-b\parallel }^2-{\displaystyle \frac{\beta }{2}}(\parallel A{x}^{k+1}+y^{k+1}{-b\parallel }^2+\parallel y^k-y^{k+1}{\parallel }^2)\hfill \\ \hfill & +\langle \beta (A{x}^{k+1}+y^{k+1}-b),y^k-y^{k+1}\rangle \hfill \\ \hfill & =-{\displaystyle \frac{\beta }{2}}{\parallel y^k-y^{k+1}\parallel }^2+\langle {\lambda }^k-{\lambda }^{k+1},y^k-y^{k+1}\rangle .\hfill \end{array}$$

Combining the above two formulas, one can declare

$$\begin{array}{c}\hfill L_{\beta }(x^{k+1},y^{k+1},{\lambda }^k)-L_{\beta }(x^{k+1},y^k,{\lambda }^k)\le ({\displaystyle \frac{L+\beta }{2}}-{\displaystyle \frac{1}{\gamma }}){\parallel y^{k+1}-y^k\parallel }^2.\end{array}$$

(10)

Since $x^{k+1}$ is the optimal solution to the subproblem with respect to x, one knows that

$$\begin{array}{cc}\hfill & L_{\beta }(x^{k+1},y^k,{\lambda }^k)-L_{\beta }(x^k,y^k,{\lambda }^k)\le {\displaystyle \frac{1}{2}}\parallel x^k-{\overline{x}}^k{\parallel }_S^2-{\displaystyle \frac{1}{2}}{\parallel x^k-{\overline{x}}^{k+1}\parallel }_S^2\hfill \\ \hfill & \le {\displaystyle \frac{1}{2}}\parallel x^k-{\overline{x}}^{k-1}{\parallel }_S^2-{\displaystyle \frac{1}{2}}\parallel x^k-{\overline{x}}^{k+1}{\parallel }_S^2-{\displaystyle \frac{1-{\eta }^2}{2}}{\parallel x^k-{\overline{x}}^{k-1}\parallel }_S^2.\hfill \end{array}$$

(11)

Noticing Algorithm 1 and (6), one can see

$$({\displaystyle \frac{1}{\gamma }}-\beta )(y^k-y^{k+1})=\nabla g\left(y^k\right)-{\lambda }^{k+1}.$$

Thus, it is natural to derive the following process:

$$\begin{array}{cc}\hfill & \parallel {\lambda }^{k+1}-{\lambda }^k{\parallel }^2={\parallel \nabla g\left(y^k\right)-\nabla g\left(y^{k-1}\right)+({\displaystyle \frac{1}{\gamma }}-\beta )(y^{k+1}-y^k)-({\displaystyle \frac{1}{\gamma }}-\beta )(y^k-y^{k-1})\parallel }^2\hfill \\ \hfill & \le 2{(L+{\displaystyle \frac{1}{\gamma }}-\beta )}^2\parallel y^k-y^{k-1}{\parallel }^2+2{({\displaystyle \frac{1}{\gamma }}-\beta )}^2{\parallel y^{k+1}-y^k\parallel }^2.\hfill \end{array}$$

(12)

Combining (8) and Equations (10)–(12), one can draw the following conclusions:

$$\begin{array}{cc}\hfill & L_{\beta }(x^{k+1},y^{k+1},{\lambda }^{k+1})+{\displaystyle \frac{{(\xi +L)}^2}{\beta }}\parallel y^{k+1}-y^k{\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel {\overline{x}}^{k+1}-x^k\parallel }_S^2\hfill \\ \hfill & \le L_{\beta }(x^k,y^k,{\lambda }^k)+{\displaystyle \frac{{(\xi +L)}^2}{\beta }}\parallel y^k-y^{k-1}{\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel {\overline{x}}^k-x^{k-1}\parallel }_S^2\hfill \\ \hfill & -{\displaystyle \frac{1-{\eta }^2}{2}}\parallel x^k-{\overline{x}}^{k-1}{\parallel }_S^2-{\sigma }_0{\parallel y^{k+1}-y^k\parallel }^2,\hfill \end{array}$$

(13)

where $\xi ={\displaystyle \frac{1}{\gamma }}-\beta$ and ${\sigma }_0={\displaystyle \frac{1}{\gamma }}-{\displaystyle \frac{L+\beta }{2}}-{\displaystyle \frac{2{\xi }^2}{\beta }}-{\displaystyle \frac{2{(\xi +L)}^2}{\beta }}$, and we obtain the desired conclusion. □

According to Assumption 1 with ${\sigma }_0>0$ and $\eta \in (0,1]$, the monotonic non-increasing property of the sequence $\left\{{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)\right\}$ is guaranteed.

Lemma 6. 

If the sequence ${\zeta }^k:=\{x^k,y^k,{\lambda }^k\}$ generated by the algorithm is bounded, then we have

$$\parallel {\zeta }^{k+1}-{\zeta }^k{\parallel }^2<+\infty .$$

Proof. 

Since $\left\{{\zeta }^k\right\}$ is bounded, it is evident that ${\widehat{\zeta }}^k$ is also bounded. Moreover, there exists an accumulation point, let us assume it to be ${\zeta }^{★}$, and there exists a subsequence $\left\{{\zeta }_j^k\right\}$ of $\left\{{\zeta }^k\right\}$ such that

$$\begin{array}{c}\hfill \underset{j\to \infty }{lim\; inf}{\widehat{L}}_{\beta }\left({\zeta }^{k_j}\right)\ge {\widehat{L}}_{\beta }\left({\zeta }^{★}\right),\end{array}$$

(14)

which implies that $\left\{{\widehat{L}}_{\beta }\left({\zeta }^{k_j}\right)\right\}$ is bounded from below. From Lemma 5 and the condition $k\ge 2$, it follows that

$$\begin{array}{cc}\hfill & \sum _{k=2}^n{\sigma }_0\parallel y^{k+1}-y^k{\parallel }^2+\sum _{k=2}^n{\displaystyle \frac{1-{\theta }^2}{2}}{\parallel x^k-{\overline{x}}^{k-1}\parallel }^2\hfill \\ \hfill & \le {\widehat{L}}_{\beta }\left({\zeta }^2\right)-{\widehat{L}}_{\beta }\left({\zeta }^{★}\right).\hfill \end{array}$$

(15)

Given ${\sigma }_0>0$, $\theta \in [0,1)$, and S is a positive semi-definite matrix, it is evident that one can derive that

$$\begin{array}{c}\hfill \sum _{k=0}^n{\sigma }_0\parallel y^{k+1}-y^k{\parallel }^2<\infty ,\sum _{k=0}^n{\displaystyle \frac{1-{\theta }^2}{2}}{\parallel x^k-{\overline{x}}^{k-1}\parallel }^2<\infty .\end{array}$$

(16)

By the inertial relationship, the following conclusion can be obtained:

$$\begin{array}{cc}\hfill \parallel x^{k+1}-x^k{\parallel }^2& =\parallel x^{k+1}-{\overline{x}}^k+{\overline{x}}^k-x^k{\parallel }^2\hfill \\ \hfill & =\parallel x^{k+1}-{\overline{x}}^k+\theta (x^k-{\overline{x}}^{k-1}){\parallel }^2\hfill \\ \hfill & \le 2\parallel x^{k+1}-{\overline{x}}^k{\parallel }^2+2\theta {\parallel x^k-{\overline{x}}^{k-1}\parallel }^2,\hfill \\ \hfill \parallel y^{k+1}-y^k{\parallel }^2& =\parallel y^{k+1}-{\overline{y}}^k+{\overline{y}}^k-y^k{\parallel }^2\hfill \\ \hfill & =\parallel y^{k+1}-{\overline{y}}^k+\eta (y^k-{\overline{y}}^{k-1}){\parallel }^2\hfill \\ \hfill & \le 2\parallel y^{k+1}-{\overline{y}}^k{\parallel }^2+2\eta {\parallel y^k-{\overline{y}}^{k-1}\parallel }^2.\hfill \end{array}$$

(17)

Combining (12), (16), and (17), we have

$$\sum _{k=0}^n\parallel x^{k+1}-x^k{\parallel }^2<\infty ,\sum _{k=0}^n\parallel y^{k+1}-y^k{\parallel }^2<\infty ,\sum _{k=0}^n{\parallel {\lambda }^{k+1}-{\lambda }^k\parallel }^2,$$

thus, we have $\parallel {\zeta }^{k+1}-{\zeta }^k{\parallel }^2<+\infty$. □

Now we give subsequential convergence analysis of NIP-ADMM.

Theorem 1. (Subsequential Convergence) The sequence $\left\{{\zeta }^k\right\}$ generated by NIP-ADMM is bounded, and assume S and $\widehat{S}$ are the sets of cluster points of $\left\{{\zeta }^k\right\}$ and $\left\{{\widehat{\zeta }}^k\right\}$, respectively. Under the assumptions and the conditions of Lemma 5, we have the following conclusion:

(i)

M and $\widehat{M}$ are two non-empty compact sets. As $k\to 0$, it follows that $d({\zeta }^k,M)\to 0$ and $d({\widehat{\zeta }}^k,\widehat{M})\to 0$.

(ii)

${\zeta }^{★}\in M\iff {\widehat{\zeta }}^{★}\in \widehat{M}$.

(iii)

$M\subseteq crit{L}_{\beta }$.

(iv)

The sequence $\left\{{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)\right\}$ converges, and ${\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right)=\underset{k\in \mathbb{N}}{inf}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)=\underset{k\to \infty }{lim}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)$.

Proof.

(i)

Based on the definitions of M and $\widehat{M}$, the conclusion can be satisfied.

(ii)

Combining Lemma 5 with the definitions of ${\zeta }^{★}$ and ${\widehat{\zeta }}^{★}$, we obtain the desired conclusion.

(iii)

Let $\zeta \in M$, then we obtain that a subsequence $\left\{{\zeta }^{k_j}\right\}$ of $\left\{{\zeta }^k\right\}$ can converge to ${\zeta }^{★}$. By combining Lemma 5, as $k\to +\infty$, one has $\parallel {\zeta }^{k+1}-{\zeta }^k\parallel =0$, which implies $\underset{j\to +\infty }{lim}{\zeta }^{k_j}={\zeta }^{★}$. On one hand, noting that $x^{k+1}$ is the optimal solution to the x-subproblem, we have

$$\begin{array}{cc}\hfill & f\left(x^{k+1}\right)-\langle {\lambda }^k,A{x}^{k+1}\rangle +{\displaystyle \frac{\beta }{2}}\parallel A{x}^{k+1}+{\overline{y}}^k{-b\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel x^{k+1}-{\overline{x}}^k\parallel }_S^2\hfill \\ \hfill \le & f\left(x^{★}\right)-\langle {\lambda }^k,A{x}^{★}\rangle +{\displaystyle \frac{\beta }{2}}\parallel A{x}^{★}+{\overline{y}}^k{-b\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel x^{★}-{\overline{x}}^k\parallel }_S^2.\hfill \end{array}$$

(18)

From Lemma 6, we know that $\underset{k\to +\infty }{lim}\parallel x^{k+1}-{\overline{x}}^k\parallel =0$, and combining this with $\underset{k_j\to +\infty }{lim}{\zeta }^{k_j}=\underset{k_j\to +\infty }{lim}{\zeta }^{k_j+1}={\zeta }^{★}$, we conclude that the equality $\underset{k_j\to +\infty }{lim}supf\left(x^{k_j+1}\right)\le f\left(x^{★}\right)$ holds. On the other hand, since $f(·)$ is a lower semi-continuous function, we deduce that $\underset{k_j\to +\infty }{lim}f\left(x^{k_j+1}\right)\ge f\left(x^{★}\right)$, so one gets

$$\begin{array}{cc}\hfill \underset{k_j\to +\infty }{lim}f\left(x^{k_j+1}\right)& =f\left(x^{★}\right).\hfill \end{array}$$

(19)

Moreover, given the closedness of $\partial f$ and the continuity of $\nabla g$, along with $k=k_j\to +\infty$ and the optimality condition of NIP-ADMM (6), we assert that

$$\left\{\begin{array}{c}{A}^T{\lambda }^{★}\in \partial f\left(x^{★}\right),\hfill \\ B^T{\lambda }^{★}=\nabla g\left(y^{★}\right),\hfill \\ A{x}^{★}+B{y}^{★}=b,\hfill \end{array}\right.$$

and ${\zeta }^{★}\in \mathrm{crit}{L}_{\beta }$ as established in Definition 4.

(v)

Let ${\widehat{\zeta }}^{★}\in \widehat{M}$, and assume that there exists a subsequence $\left\{{\zeta }^{k_j}\right\}$ of $\left\{{\zeta }^k\right\}$ that converges to ${\zeta }^{★}$. Combining the relations (14), (19), and the continuity of g, we have

$$\underset{k_j\to +\infty }{lim}{\widehat{L}}_{\beta }\left({\zeta }^{k_j}\right)={\widehat{L}}_{\beta }\left({\zeta }^{★}\right).$$

Considering that $\widehat{L}\left({\widehat{\zeta }}^k\right)$ is monotonically non-increasing, it follows that $\widehat{L}\left({\widehat{\zeta }}^k\right)$ is convergent. Consequently, for any ${\widehat{\zeta }}^{★}\in \widehat{S}$, the relationship can be established as

$${\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right)=\underset{k\in \mathbb{N}}{inf}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)=\underset{k\to \infty }{lim}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right).$$

□

Based on the definition of the augmented Lagrangian function and the semidefiniteness of the matrix S, the following can be defined with ${\zeta }^k=(x^k,y^k,{\lambda }^k)$:

$$\begin{array}{cc}& \left\{\begin{array}{cc}\hfill {ϵ}_1^{k+1}& =A^T({\lambda }^k-{\lambda }^{k+1})+\beta (y^{k+1}-y^k)+S(x^k-x^{k+1}),\hfill \\ \hfill {ϵ}_2^{k+1}& ={\nabla }_S\left(y^{k+1}\right)-{\nabla }_S\left(y^k\right)+({\lambda }^k-{\lambda }^{k+1})+\beta (y^{k+1}-y^k)\hfill \\ & +{\displaystyle \frac{1}{\gamma }}(y^k-y^{k+1})+{\displaystyle \frac{2{(ϵ+L)}^2}{\beta }}(y^{k+1}-y^k),\hfill \end{array}\right.\hfill \end{array}$$

(20)

$$\begin{array}{cc}& \left\{\begin{array}{cc}\hfill {ϵ}_3^{k+1}& =-(A{x}^{k+1}+y^{k+1}-b),\hfill \\ \hfill {ϵ}_4^{k+1}& =S(x^{k+1}-x^k),\hfill \\ \hfill {ϵ}_5^{k+1}& =-S(x^{k+1}-x^k),\hfill \\ \hfill {ϵ}_6^{k+1}& =-{\displaystyle \frac{2{(ϵ+L)}^3}{\beta }}(y^{k+1}-y^k).\hfill \end{array}\right.\hfill \end{array}$$

(21)

Then, the following result can be obtained.

Lemma 7. 

Let $({ϵ}_1^{k+1},{ϵ}_2^{k+1},{ϵ}_3^{k+1},{ϵ}_4^{k+1},{ϵ}_5^{k+1},{ϵ}_6^{k+1})$ be contained in $\partial {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{k+1}\right)$. Then, there exists $\psi >0$ and $k\ge 1$ such that

$$d(0,\partial {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{k+1}\right))\le \psi \left(\parallel y^{k+1}-y^k\parallel +\parallel x^{k+1}-{\overline{x}}^k\parallel +\parallel y^k-y^{k-1}\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel \right).$$

Proof. 

By the definition of ${\widehat{L}}_{\beta }(·)$ and ${\widehat{\zeta }}^k=(x^k,y^k,{\lambda }^k,{\widehat{x}}^k,x^{k-1},y^{k-1})$, we can derive that

$$\left\{\begin{array}{cc}\hfill {\partial }_x{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=& \partial f\left(x^{k+1}\right)-A^{\top }{\lambda }^{k+1}+\beta (A{x}^{k+1}-y^{k+1}-b),\hfill \\ \hfill {\partial }_y{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=& \nabla g\left(y^{k+1}\right)-{\lambda }^{k+1}+\beta (A{x}^{k+1}-y^{k+1}-b)+{\displaystyle \frac{2{(\epsilon +L)}^2}{\beta }}(y^{k+1}-y^k),\hfill \\ \hfill {\partial }_{\lambda }{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=& -(A{x}^{k+1}+y^{k+1}-b),{\partial }_{\overline{x}}{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=S({\overline{x}}^{k+1}-x^k),\hfill \\ \hfill {\partial }_{\widehat{x}}{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=& -S({\overline{x}}^{k+1}-x^k),{\partial }_{\overline{y}}{\widehat{L}}_{\beta }\left({\zeta }^{k+1}\right)=-{\displaystyle \frac{2{(\epsilon +L)}^3}{\beta }}(y^{k+1}-y^k).\hfill \end{array}\right.$$

Combining the above expression with the optimality conditions of NIP-ADMM (6), which means

$$\left\{\begin{array}{cc}\hfill {ϵ}_1^{k+1}=& A^{\top }({\lambda }^k-{\lambda }^{k+1})+\beta (y^{k+1}-{\overline{y}}^k)+S({\overline{x}}^k-x^{k+1}),\hfill \\ \hfill {ϵ}_2^{k+1}=& \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)+({\lambda }^k-{\lambda }^{k+1})+\beta (y^{k+1}-y^k)+{\displaystyle \frac{1}{\gamma }}(y^k-y^{k+1})\hfill \\ \hfill & +{\displaystyle \frac{2{(\epsilon +L)}^2}{\beta }}(y^{k+1}-y^k),\hfill \\ \hfill {ϵ}_3^{k+1}=& -(A{x}^{k+1}+y^{k+1}-b),{ϵ}_4^{k+1}=S({\overline{x}}^{k+1}-x^k),\hfill \\ \hfill {ϵ}_5^{k+1}=& -S({\overline{x}}^{k+1}-x^k),{ϵ}_6^{k+1}=-{\displaystyle \frac{2{(\epsilon +L)}^3}{\beta }}(y^{k+1}-y^k).\hfill \end{array}\right.$$

(22)

It is easy to see from Lemma 2 that $({ϵ}_1^{k+1},{ϵ}_2^{k+1},{ϵ}_3^{k+1},{ϵ}_4^{k+1},{ϵ}_5^{k+1},{ϵ}_6^{k+1})\in \partial {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{k+1}\right)$. Moreover, since $g(·)$ has a Lipschitz continuous gradient with respect to L, we get

$$\parallel \nabla g\left(y^{k+1}\right)-\nabla g\left(y^k\right)\parallel \le L\parallel y^{k+1}-y^k\parallel ,$$

therefore, according to (20), there exists a positive real number ${\psi }_1$ such that

$$\begin{array}{cc}\hfill \parallel ({ϵ}_1^{k+1},{ϵ}_2^{k+1},{ϵ}_3^{k+1},{ϵ}_4^{k+1},{ϵ}_5^{k+1},{ϵ}_6^{k+1})\parallel & \le {\psi }_1(\parallel {\lambda }^{k+1}-{\lambda }^k\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel \hfill \\ \hfill & +\parallel x^{k+1}-{\overline{x}}^k\parallel +\parallel y^{k+1}-y^k\parallel ).\hfill \end{array}$$

Furthermore, combining this with (12), we know that there exists ${\psi }_2>0$ and $k>1$ such that:

$$\parallel {\lambda }^{k+1}-{\lambda }^k\parallel \le {\psi }_2(\parallel y^k-y^{k-1}\parallel +\parallel y^{k+1}-y^k\parallel ),$$

thus, by selecting $\psi =max\{{\psi }_1,{\psi }_2\}$ and $k>1$, we can further conclude that

$$\begin{array}{cc}\hfill d(0,\partial {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{k+1}\right))& \le \parallel ({ϵ}_1^{k+1},{ϵ}_2^{k+1},{ϵ}_3^{k+1},{ϵ}_4^{k+1},{ϵ}_5^{k+1},{ϵ}_6^{k+1})\parallel \hfill \\ \hfill & \le \psi \left(\parallel y^{k+1}-y^k\parallel +\parallel x^{k+1}-{\overline{x}}^k\parallel +\parallel y^k-y^{k-1}\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel \right),\hfill \end{array}$$

this concludes the proof. □

Theorem 2. (Global convergence) Suppose the sequence $\left\{{\zeta }^k\right\}$ generated by NIP-ADMM is bounded, and the assumptions hold. If ${\widehat{L}}_{\beta }$ is a KL function, then

$$\parallel {\zeta }^{k+1}-{\zeta }^k\parallel <+\infty .$$

Moreover, the sequence $\left\{{\zeta }^k\right\}$ converges to a critical point of $L_{\beta }$.

Proof. 

From Theorem 1, we know that $\underset{k\to \infty }{lim}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)={\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right)$. For any ${\widehat{\zeta }}^{★}\in \widehat{M}$, the proof process needs to consider the following two cases:

(i)

For any $k_0>1$ and given that ${\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{\left(k_0\right)}\right)={\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right)$, it follows from Lemma 1 and Lemma 5 that there exists a constant $K>0$ such that

$$K(\parallel x^k-{\overline{x}}^{k-1}{\parallel }^2+\parallel y^{k+1}-y^k{\parallel }^2)\le {\overline{L}}_{\beta }\left({\widehat{\zeta }}^k\right)-{\overline{L}}_{\beta }\left({\widehat{\zeta }}^{k+1}\right)\le {\overline{L}}_{\beta }\left({\widehat{\zeta }}_0^k\right)-{\overline{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right).$$

It is clear that $K=min\left\{{\sigma }_0,{\displaystyle \frac{1-{\eta }^2}{2}}{\lambda }_{{min}_{\left(S\right)}}\right\}$. As a result, we have $\parallel y^{k+1}-y^k\parallel <+\infty$ and $\parallel x^k-{\overline{x}}^{k-1}\parallel <+\infty$. Combining (12) and (17), it follows that $\parallel x^{k+1}-x^k\parallel <+\infty$ and $\parallel {\lambda }^{k+1}-{\lambda }^k\parallel <+\infty$. Finally, for any $k>k_0$, we conclude that $\parallel {\zeta }^{k+1}-{\zeta }^k\parallel <+\infty$, and the result holds.

(ii)

Assume that for any $k>0$, the inequality $L_{\beta }\left({\widehat{\zeta }}^k\right)>L_{\beta }\left({\widehat{\zeta }}^{★}\right)$ holds. Since

$$\underset{k\to \infty }{lim}d({\widehat{\zeta }}^k,\widehat{M})=0,$$

it follows that for any ${ϵ}_1>0$, there exists $k_1>0$ such that for all $k\ge k_1$, we have:

$$d({\widehat{\zeta }}^k,\widehat{M})<{ϵ}_1.$$

Moreover, noting that

$$\underset{k\to \infty }{lim}{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)={\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right),$$

it implies that for any ${ϵ}_2>0$, there exists $k_2>0$ such that for all $k>k_2$, the inequality holds:

$$L_{\beta }\left({\widehat{\zeta }}^k\right)<L_{\beta }\left({\widehat{\zeta }}^{★}\right)+{ϵ}_2.$$

Hence, given ${ϵ}_1$ and ${ϵ}_2$, when $k=max\{k_1,k_2\}$, we have

$$d({\widehat{\zeta }}^k,\widehat{M})<{ϵ}_1,L_{\beta }\left({\widehat{\zeta }}^{★}\right)<L_{\beta }\left({\widehat{\zeta }}^k\right)<L_{\beta }\left({\widehat{\zeta }}^{★}\right)+{ϵ}_2.$$

And, based on Lemma 3, the following holds for all $k>\overline{k}$, it can be deduced that

$${\phi }^{\prime }\left({\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right)-{\widehat{L}}_{\beta }\left({\widehat{\zeta }}^{★}\right)\right)d(0,\partial {\widehat{L}}_{\beta }\left({\widehat{\zeta }}^k\right))\ge 1.$$

Furthermore, using the concavity of $\phi$, we derive the following:

$$\begin{array}{cc}\hfill \phi (L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))-\phi (L_{\beta }\left({\widehat{\zeta }}^{k+1}\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))& \ge {\phi }^{\prime }(L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))\hfill \\ \hfill & \times \left(L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{k+1}\right)\right).\hfill \end{array}$$

Noting the fact that ${\phi }^{\prime }\left({\widehat{L}}_{\beta }\left(w^k\right)-{\widehat{L}}_{\beta }\left(w^{★}\right)\right)>0$, together with the conclusion obtained in Lemma 7, we can infer that

$$\begin{array}{cc}\hfill L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{k+1}\right)& \le {\displaystyle \frac{\phi (L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))-\phi (L_{\beta }\left({\widehat{\zeta }}^{k+1}\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))}{{\phi }^{\prime }(L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))}}\hfill \\ \hfill & \le {\Pi }_{\left[\phi \right],[k+1,k]}\psi T_{[k,k+1]},\hfill \end{array}$$

(23)

where ${\Pi }_{\left[\phi \right],[k,k+1]}$ represents $\phi (L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))-\phi (L_{\beta }\left({\widehat{\zeta }}^{k+1}\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right)),$ and $T_{[k,k+1]}$ represents $\parallel y^{k+1}-y^k\parallel +\parallel x^{k+1}-{\overline{x}}^k\parallel +\parallel y^k-y^{k-1}\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel .$ Combining Lemma 5, we can rewrite (22) as follows

$$\begin{array}{c}\hfill \varphi (\parallel x^k-{\overline{x}}^{k-1}{\parallel }^2+\parallel y^{k+1}-{\overline{y}}^k{\parallel }^2)\le {\Pi }_{\left[\phi \right],[k+1,k]}\psi T_{[k,k+1]},\end{array}$$

(24)

where (22) can be equivalently expressed as

$$\sqrt{2\parallel x^k-{\overline{x}}^{k-1}{\parallel }^2+2{\parallel y^{k+1}-{\overline{y}}^k\parallel }^2}\le 2\sqrt{{\displaystyle \frac{2\psi }{\varphi }}}{T}_{[k,k+1]}^{{\displaystyle \frac{1}{2}}}{\Pi }_{\left[\phi \right],[k+1,k]}^{{\displaystyle \frac{1}{2}}}.$$

By applying the Cauchy-Schwarz inequality and multiplying both sides by 6, we obtain

$$6{\left(\parallel x^k-{\overline{x}}^{k-1}{\parallel }^2+{\parallel y^{k+1}-{\overline{y}}^k\parallel }^2\right)}^{{\displaystyle \frac{1}{2}}}\le \sqrt{{\displaystyle \frac{2\psi }{\varphi }}}{T}_{[k,k+1]}^{{\displaystyle \frac{1}{2}}}{\Pi }_{\left[\phi \right],[k+1,k]}^{{\displaystyle \frac{1}{2}}}.$$

Then, by further applying the fundamental inequality, we can deduce that

$$\begin{array}{cc}\hfill 6\left(\parallel x^k-{\overline{x}}^{k-1}\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel \right)\le & 2T_{[k,k+1]}^{{\displaystyle \frac{1}{2}}}\sqrt{{\displaystyle \frac{18\psi }{\varphi }}}{\Pi }_{\left[\phi \right],[k+1,k]}^{{\displaystyle \frac{1}{2}}}\hfill \\ \hfill \le & T_{[k,k+1]}+{\displaystyle \frac{18\psi }{\varphi }}{\Pi }_{\left[\phi \right],[k+1,k]}\hfill \\ \hfill =& \parallel y^{k+1}-y^k\parallel +\parallel x^{k+1}-{\overline{x}}^k\parallel +\parallel y^k-y^{k-1}\parallel \hfill \\ \hfill & +\parallel y^{k+1}-{\overline{y}}^k\parallel +{\displaystyle \frac{18\psi }{\varphi }}{\Pi }_{\left[\phi \right],[k+1,k]}\hfill \\ \hfill \le & 2\parallel y^{k+1}-{\overline{y}}^k\parallel +2\parallel y^k-{\overline{y}}^{k-1}\parallel +\parallel x^{k+1}-{\overline{x}}^k\parallel \hfill \\ \hfill & +\parallel y^{k-1}-{\overline{y}}^{k-2}\parallel +{\displaystyle \frac{18\psi }{\varphi }}{\Pi }_{\left[\phi \right],[k+1,k]}.\hfill \end{array}$$

(25)

Next, summing up (23) from $k=p+1$ to $k=z$ and rearranging the terms, one gets

$$\begin{array}{cc}\hfill 5\sum _{k=p+3}^z\parallel x^k-{\overline{x}}^{k-1}\parallel +\sum _{k=p+3}^z\parallel y^{k+1}-{\overline{y}}^k\parallel \le & 3\parallel y^{p+1}-{\overline{y}}^p\parallel +\parallel y^{p+2}-{\overline{y}}^{p+1}\parallel \hfill \\ \hfill & +\parallel x^{p+1}-{\overline{x}}^p\parallel -3\parallel y^{z+1}-{\overline{y}}^z\parallel \hfill \\ \hfill & -\parallel y^{z+2}-{\overline{y}}^{z+1}\parallel -\parallel x^{z+1}-{\overline{x}}^{z+1}\parallel \hfill \\ \hfill & +{\displaystyle \frac{18\psi }{\varphi }}{\Pi }_{\left[\phi \right],[p+1,z+1]}.\hfill \end{array}$$

Furthermore, as $0\le \phi (L_{\beta }\left({\widehat{\zeta }}^k\right)-L_{\beta }\left({\widehat{\zeta }}^{★}\right))$ and m approaches positive infinity, we can conclude that

$$\sum _{k=p+1}^{+\infty }(5\parallel x^k-{\overline{x}}^{k-1}\parallel +\parallel y^{k+1}-{\overline{y}}^k\parallel )<+\infty ,$$

which implies

$$\sum _{k=0}^{+\infty }\parallel x^k-{\overline{x}}^{k-1}\parallel <+\infty ,\sum _{k=0}^{+\infty }\parallel y^k-{\overline{y}}^{k-1}\parallel <+\infty .$$

Based on the relationship between (12) and (17), we can assert that

$$\sum _{k=0}^{+\infty }\parallel x^k-x^{k-1}\parallel <+\infty ,\sum _{k=0}^{+\infty }\parallel y^k-y^{k-1}\parallel <+\infty ,\sum _{k=0}^{+\infty }\parallel {\lambda }^k-{\lambda }^{k-1}\parallel <+\infty .$$

This demonstrates that $\left\{{\zeta }^k\right\}$ forms a Cauchy sequence, which ensures its convergence. By applying Theorem 1, it follows that $\left\{{\zeta }^k\right\}$ converges to a critical point of $L_{\beta }$.

□

4. Numerical Simulations

In this section, we demonstrate the application of NIP-ADMM in signal recovery and SCAD penalty problems. To verify the effectiveness of the algorithm, we compare it with Bregman modification of ADMM (BADMM) proposed by Wang et al. [23] and inertial proximal ADMM (IPADMM) proposed by Pi et al. [32]. All codes were implemented in MATLAB 2024b and executed on a Windows 11 system equipped with an AMD Ryzen R9-9900X CPU.

4.1. Signal Recovery

In this subsection on signal recovery, we consider the previously mentioned model (2).

$$\begin{array}{cc}\hfill min& {c\parallel x\parallel }_{{\displaystyle \frac{1}{2}}}^{{\displaystyle \frac{1}{2}}}+{\displaystyle \frac{1}{2}}{\parallel y\parallel }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ax-y=b,\hfill \end{array}$$

(26)

where ${\parallel x\parallel }_{{\displaystyle \frac{1}{2}}}^{{\displaystyle \frac{1}{2}}}={\sum }_{i=1}^n{\left|x_i\right|}^{{\displaystyle \frac{1}{2}}}$, and $A\in {\mathbb{R}}^{m\times n},b\in {\mathbb{R}}^m$,$y\in {\mathbb{R}}^m$, $x\in {\mathbb{R}}^n$. To evaluate the effectiveness of NIP-ADMM, we compare it with BADMM [23] and IPADMM [32]. We construct the following framework to solve problem (24). We set $S=e_{0}I-\beta A^{\top }A$, where I denotes the identity matrix.

$$\left\{\begin{array}{cc}\hfill & ({\overline{x}}^k,{\overline{y}}^k)=(x^k,y^k)+\theta (x^k-{\overline{x}}^{k-1},0)+\eta (0,y^k-{\overline{y}}^{k-1}),\hfill \\ \hfill & x^{k+1}=H\left({\displaystyle \frac{1}{e_0}}\left(-A^{\top }{\lambda }^k+\beta A^{\top }({\overline{y}}^k+b)+(e_{0}I-\beta A^{\top }A){\overline{x}}^k\right),{\displaystyle \frac{2c}{e_0}}\right),\hfill \\ \hfill & y^{k+1}=y^k-\gamma (y^k+{\lambda }^k-\beta (A{x}^{k+1}-y^k-b)),\hfill \\ \hfill & {\lambda }^{k+1}={\lambda }^k+\beta (A{x}^{k+1}-y^{k+1}-b).\hfill \end{array}\right.$$

Here, $H(·,·)$ represents the half-shrinkage operator proposed by Xu et al. [39], which is defined as

$$H(x,e)={\{h_e\left(x_1\right),h_e\left(x_2\right),\dots ,h_e\left(x_n\right)\}}^T,$$

where the function $h_e\left(x_i\right)$ for $i=1,2,\dots ,n$ is defined by

$$\left\{\begin{array}{cc}{h}_e\left(x_i\right)={\displaystyle \frac{2x_i}{3}}\left(1+cos{\displaystyle \frac{2}{3}}\left(\pi -arccos\left({\displaystyle \frac{e}{8}}{\left({\displaystyle \frac{x_i}{3}}\right)}^{-{\displaystyle \frac{3}{2}}}\right)\right)\right),\hfill & \left|x_i\right|>{\displaystyle \frac{\sqrt[3]{54}}{4}}{e}^{{\displaystyle \frac{2}{3}}},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

In this setup, the entries of matrix A are drawn from a standard normal distribution, with each column adjusted for normalization. The starting vector $x_0$ is initialized as a sparse vector, containing at least 100 non-zero components. The values ${\overline{x}}^0,{\overline{y}}^0,x^0,{\lambda }^0$ are set initially to zero. To simulate the observation vector with added noise, we generate b using $b=A{x}^0+v$, where $v\sim N(0,{10}^{-3}I)$. For the regularization parameter c, we compute

$$c=0.1\parallel A^T{b\parallel }_{\infty }.$$

Based on Assumption 1, the parameters are chosen as $\beta =3$ and $e_0=10$. The error trend is depicted as ${log}_{10}(\parallel x^k-x^{*}\parallel )\mathrm{and}{log}_{10}(\parallel y^k-y^{*}\parallel ).$ At the $(k+1)$-th iteration, the primal residual is expressed as $r^{k+1}=A{x}^{k+1}+y^{k+1}-b,$ while the Dual residual is represented as $s^{k+1}=\beta A^T(x^{k+1}-x^k).$ Termination occurs when both conditions are met: $\parallel r^{k+1}{\parallel }_2\le \sqrt{n}\times {10}^{-4}+{10}^{-3}·max\left(\parallel A{x}^k{\parallel }_2,{\parallel y^k\parallel }_2\right)$ and $\parallel s^{k+1}{\parallel }_2\le \sqrt{n}\times {10}^{-4}+{10}^{-3}·{\parallel A^T{\lambda }^k\parallel }_2.$ During the experiments, to satisfy Assumption 1, we set $\gamma =0.3$. Table 1 shows that when $m=n=1000$, selecting the inertial parameter values $\theta =0.8$ and $\eta =0.75$ produces satisfactory results. Therefore, in subsequent experiments, we also adopt $\theta =0.8$ and $\eta =0.75$. The metrics include the number of iterations (Iter), CPU running time (CPUT) and the objective function value (Obj). To better present the experimental results, we retain two decimal places for Obj and four decimal places for CPUT.

The numerical results consistently demonstrate the superior performance of NIP-ADMM compared to BADMM and IPADMM (see Table 2): for $m=n=1000$, NIP-ADMM shows faster convergence in terms of both objective value and error reduction (see Figure 1); for $m=3000$ and $n=4000$, the inclusion of the inertial term further highlights its effectiveness (see Figure 2); and for larger-scale models with $m=n=6000$, it is proven that NIP-ADMM is more suitable than BADMM and IPADMM for handling large-scale problems (see Figure 3).

4.2. SCAD Penalty Problem

We note that the smoothly clipped absolute deviation (SCAD) penalty problem in statistics can be formulated as the following model [14,31]:

$$\begin{array}{cc}\hfill min& \sum _{i=1}^n{h}_k\left(\right|x_i\left|\right)+{\displaystyle \frac{1}{2}}{\parallel y\parallel }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ax-y=b,\hfill \end{array}$$

(27)

with $A\in {\mathbb{R}}^{m\times n},b\in {\mathbb{R}}^m$,$y\in {\mathbb{R}}^m$, $x\in {\mathbb{R}}^n$ and the penalty function $h_{\kappa }(·)$ in the objective is defined as:

$$h_{\kappa }\left(\theta \right)=\left\{\begin{array}{cc}\kappa \theta ,\hfill & \theta \le \kappa ,\hfill \\ {\displaystyle \frac{-{\theta }^2+2c\kappa \theta -{\kappa }^2}{2(c+1)}},\hfill & \kappa <\theta \le c\kappa ,\hfill \\ {\displaystyle \frac{(c+1){\kappa }^2}{2}},\hfill & \theta >c\kappa ,\hfill \end{array}\right.$$

(28)

where $c>2$ and $\kappa >0$, being the knots of the quadratic spline function. In the reference signal recovery subsection, we similarly set $\psi \left(x\right)={\displaystyle \frac{1}{2}}{\parallel x\parallel }_S^2$ and $\varphi \left(y\right)=0$, where $S=\mu I-\beta A^{\top }A$. For the problem (25), the x-subproblem can be expressed as

$$\begin{array}{cc}\hfill x^{k+1}& =argmin\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)-\langle p^k,Ax\rangle +{\displaystyle \frac{\beta }{2}}\parallel Ax-{\overline{y}}^k{-b\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel x-{\overline{x}}^k\parallel }_{\mu I-\beta A^{T}A}^2\right\}\\ \hfill & =argmin\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)+{\displaystyle \frac{\mu }{2}}{∥x-{\displaystyle \frac{\beta A^T({\overline{y}}^k+b+\frac{p^k}{\beta }-A{\overline{x}}^k)}{\mu }}∥}^2\right\}.\end{array}$$

On the one hand, the x-subproblem can be equivalently formulated as:

$$\underset{x\in {\mathbb{R}}^n}{min}\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)+{\displaystyle \frac{1}{2\nu }}{\parallel x-q\parallel }^2\right\}.$$

On the other hand, under the condition that $1+\nu \le c$, we can update x using the following rule [31]:

$$x_i:=\left\{\begin{array}{cc}\mathrm{sign}\left(q_i\right)\left(\right|q_i{|-\kappa \nu )}_{+},\hfill & |q_i|\le (1+\nu )\kappa ,\hfill \\ {\displaystyle \frac{(c-1)q_i-\mathrm{sign}\left(q_i\right)\kappa \nu }{c-1-\nu }},\hfill & (1+\nu )\kappa <|q_i|\le c\kappa ,\hfill \\ q_i,\hfill & |q_i|>c\kappa ,\hfill \end{array}\right.$$

where ${(·)}_{+}$ represents the positive part operator, which is defined as ${\left(x\right)}_{+}=max(0,x)$, applying NIP-ADMM to solve the problem (25), one yields that

$$\left\{\begin{array}{c}{\overline{x}}^k=x^k+\theta (x^k-{\overline{x}}^{k-1}),{\overline{y}}^k=y^k+\eta (y^k-{\overline{y}}^{k-1}),\hfill \\ x^{k+1}=argmin\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)+{\displaystyle \frac{\mu }{2}}{\parallel x-{\displaystyle \frac{-A^{\top }{\lambda }^k+\beta A^{\top }({\overline{y}}^k+b)+{\mu }_0{\overline{x}}^k-A^{\top }A{\overline{x}}^k}{\mu }}\parallel }^2\right\},\hfill \\ y^{k+1}=y^k-\gamma (y^k-{\lambda }^k-\beta (A{x}^{k+1}-y^k-b)),\hfill \\ {\lambda }^{k+1}={\lambda }^k-\beta (A{x}^{k+1}-y^{k+1}-b).\hfill \end{array}\right.$$

Similarly, the update scheme of BADMM can be represented by the following procedure:

$$\left\{\begin{array}{c}{x}^{k+1}=argmin\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)+{\displaystyle \frac{\mu }{2}}{\parallel x-{\displaystyle \frac{\beta A^T(y^k+b+\frac{{\lambda }^k}{\beta }-A{x}^k)+\mu x^k}{\mu }}\parallel }^2\right\},\hfill \\ y^{k+1}={\displaystyle \frac{1}{1+\beta }}\left(\beta (A{x}^{k+1}-b)-{\lambda }^k\right),\hfill \\ {\lambda }^{k+1}={\lambda }^k-\beta (A{x}^{k+1}-y^{k+1}-b).\hfill \end{array}\right.$$

Utilize the IPADMM to address model (25) and derive the following iterative scheme:

$$\left\{\begin{array}{c}({\overline{x}}^k,{\overline{y}}^k,{\overline{\lambda }}^k)=(x^k,y^k,{\lambda }^k)+\theta (x^k-x^{k-1},y^k-y^{k-1},{\lambda }^k-{\lambda }^{k-1}),,\hfill \\ x^{k+1}=argmin\left\{\sum _{i=1}^n{g}_{\kappa }\left(\right|x_i\left|\right)+{\displaystyle \frac{\mu }{2}}{\parallel x-{\displaystyle \frac{-A^{\top }{\overline{\lambda }}^k+\beta A^{\top }({\overline{y}}^k+b)+{\mu }_0{\overline{x}}^k-A^{\top }A{\overline{x}}^k}{\mu }}\parallel }^2\right\},\hfill \\ {\lambda }^{k+1}={\overline{\lambda }}^k-\beta (A{x}^{k+1}-{\overline{y}}^k-b),\hfill \\ y^{k+1}=y^k-\gamma (y^k-{\lambda }^{k+1}-\beta (A{x}^{k+1}-y^k-b)).\hfill \end{array}\right.$$

In this experiment, we generate a random $m\times n$ matrix A, and perform row and column normalization. Here, we choose to generate a vector z of dimension n with a sparsity ratio of ${\displaystyle \frac{100}{m}}$. The vector b is represented as the sum of $Az$ and a Gaussian noise vector with zero mean and variance $0.001$. The initial variables $x_0$ and $y_0$ are set as zero vectors, serving as the starting point for optimization. To improve numerical efficiency, in this experiment, we set $c=5$ and $\kappa =0.1$. Under the condition that Assumption 1 is satisfied, we configure $\gamma =0.1$, $\beta =12$, and $\mu =100$ for NIP-ADMM and other algorithms. The stopping criterion for the updates is set as

$$max\left\{\parallel x^{k+1}-x^k\parallel ,\parallel y^{k+1}-y^k\parallel \right\}\le {10}^{-2}.$$

In Table 3, we set $m=n=1000$. The results in the table support our choice of the inertial parameters $\eta =0.9$ and $\theta =0.9$. Under these conditions, the NIP-ADMM requires the fewest iterations and the least running time. Therefore, we selected the inertial parameters $\eta =0.9$ and $\theta =0.9$ for the experiments.

To further investigate the convergence behavior of the algorithms, we plot the update curves of the objective function (left figure) and the iteration error (right figure) against the number of iterations for each algorithm under three different dimensions (see Figure 4, Figure 5, and Figure 6). The numerical results demonstrate that NIP-ADMM achieves nearly the same performance as IPADMM and BADMM but with significantly fewer iterations.

Figure 4. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=1000$ and $n=1000$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=1000$.

Figure 4. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=1000$ and $n=1000$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=1000$.

Figure 5. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=1500$ and $n=1500$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=1500$.

Figure 5. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=1500$ and $n=1500$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=1500$.

Figure 6. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=3000$ and $n=3000$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=3000$.

Figure 6. Comparison of convergence when $m=1000$ and $n=1000$: (a) The objective value when $m=3000$ and $n=3000$. (b) $log\parallel x^k-x^{★}\parallel$ and $log\parallel y^k-y^{★}\parallel$ under $m=n=3000$.

In Table 4, we present the test results of NIP-ADMM, BADMM and IPADMM under different dimensions. Although there are slight discrepancies in the Obj values between the two methods, by focusing on the metrics of Iter and CPUT, it is evident that NIP-ADMM demonstrates a significant advantage over BADMM and IPADMM. This advantage becomes even more pronounced in higher-dimensional scenarios.

5. Conclusion

The Purpose of this is to propose a novel symmetrical inertial ADMM with proximal term for solving nonconvex two-block optimization problems. Under certain conditions, if the objective function satisfies Kurdyka-Łojasiewicz property, the sequence generated by the algorithm globally converges to a stationary point. In numerical experiments, we apply the algorithm to signal recovery and SCAD penalty problems, and its superiority is validated. Notably, by continuously adjusting the inertial parameters, we identify a set of parameters that enhances the convergence speed of the algorithm.

Furthermore, we believe that future work could explore whether the convergence of the algorithm can be guaranteed when the objective function is non-separable. Additionally, it would be worthwhile to investigate whether introducing inertial terms into the y-subproblem and the multiplier $\lambda$ could further accelerate the convergence speed of the algorithm.