Hybrid Gauss-Newton Method for Convex Inclusion and Convex-Composite Optimization Problems

1. Introduction

Let X and Y be Banach spaces, and let K represent a closed convex subset of Y. Two inclusion problems have been extensively studied [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. The first problem is defined as

$$F\left(x\right)\in K,$$

(1)

where $F:X\to Y$ is a differentiable operator in the Fréchet sense. The second problem is a convex composite optimization problem:

$$\underset{x\in X}{min}(h\circ F)\left(x\right),$$

(2)

where h is a real-valued convex function on Y and F is as defined in (1). If $h(·)=d(·,K)$, the distance function related to K, and (2) is solvable, then (2) reduces to (1). Many optimization and mathematical programming problems can be reduced to solving (1) or (2) [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. Regarding problem (1), Robinson [25,26] developed the extended Newton method under the conditions that K is a closed convex cone and X is a reflexive space. Let $x_0\in X$.

Algorithm $A\left(x_0\right)$ : 

Given $x_n$, $n=0,1,2,\dots$, compute $x_{n+1}$ as follows: First, define the set

$$D_{\infty }:=\{d\in X:F\left(x\right)+F^{\prime }\left(x\right)d\in K\mathrm{for}\mathrm{all}x\in X\}.$$

(3)

If $D_{\infty }\left(x\right)\ne \varnothing$, pick $d_n\in D_{\infty }\left(x_n\right)$ to satisfy $\parallel d_n\parallel ={min}_{d\in D_{\infty }\left(x_n\right)}\parallel d\parallel$, and set $x_{n+1}=x_n+d_n.$ In the case when $D_{\infty }\left(x\right)=\varnothing$ for some $x\in X$, the algorithm is not well-defined. It is said to be well-defined if at least one sequence is generated by this algorithm. Let $T_{x_0}$ be a convex process given by

$$T_{x_0}d=F^{\prime }\left(x_0\right)d\mathrm{for}\mathrm{all}d\in X.$$

(4)

Robinson in [25,26] provided two conditions:

$$\mathrm{Range}\left(T_{x_0}\right)=Y,$$

(5)

and $F^{\prime }$ is Lipschitz continuous with modulus H, $T_{x_0}$ is normed with $\parallel T_{x_0}^{-1}\parallel <\infty$, and

$$\parallel x_1-x_0\parallel \le {\displaystyle \frac{1}{2H\parallel T_{x_0}^{-1}\parallel }}.$$

(6)

Under these conditions, Algorithm $A\left(x_0\right)$ generates a sequence $\left\{x_n\right\}$ converging to some $x^{*}$ solving (1). Numerous authors have presented relevant convergence results lacking the affine invariant property for the operator F [14]. Later, in the elegant work by Li and Ng in [20], convergence conditions addressing this problem under weaker conditions were introduced. Their study also relied on the new concept of the weak-Robinson condition for convex processes introduced in Section 2, Subsection 2.1. The semi-local convergence of the two algorithms is presented in Section 2, Subsection 2.2. The hybrid methods are studied in Section 2, Subsection 2.3. The concluding remarks appear in Section 3 and the paper ends with discussion in Section 4.

2. Materials and Methods

2.1. Weak-Robinson Condition and Properties of Convex Processes

Let $U(x,\rho )$ denote the open ball in X with center $x\in X$ and radius $\rho >0$. Also, let $d(x,S)$ denote the distance from x to S. We assume familiarity with the concept of convex process $T:X\to 2^Y$ as defined by Rockafellar [27,28], as well as its properties. We list only some in order to make the study as self-contained as possible. The following sets are needed:

$$\begin{array}{cc}\hfill D\left(T\right)& =\{x\in X:T_x\ne \varnothing \},\hfill \\ \hfill R\left(T\right)& =\bigcup \{T_x:x\in D\left(T\right)\},\hfill \end{array}$$

and

$$T^{-1}y=\{x\in X:y\in T_x\}\mathrm{for}\mathrm{all}y\in Y.$$

Proposition 1 

([28]). Let $B_1,B_2$ be convex processes with $D\left(B_1\right)=D\left(B_2\right)=X$ and $R\left(B_1\right)=Y$. Suppose that $\parallel B_1^{-1}\parallel \parallel B_2\parallel <1$ and that $(B_1+B_2)\left(x\right)$ is closed for all $x\in X$. Then, the following assertions hold:

$$R(B_1+B_2)=Y$$

and

$$\parallel {(B_1+B_2)}^{-1}\parallel \le {\displaystyle \frac{\parallel B_1^{-1}\parallel }{1-\parallel B_1^{-1}\parallel \parallel B_2\parallel }}.$$

Next, we present Lipschitz-type conditions connected to convex processes for comparison.

Definition 1. 

Let $E:X\to \mathcal{L}(X,Y)$, where $\mathcal{L}(X,Y)$ denotes the space of linear operators from X to Y. Let $T:Y\to 2^Z$ be a convex process, where Z is also a Banach space. The pair $(T,E)$ is said to be center-Lipschitz continuous on $U(x_0,{\rho }_0)({\rho }_0>0)$ if

$$\parallel T\left(E\left(x\right)-E\left(x_0\right)\right)\parallel \le L_0\parallel x-x_0\parallel ,$$

for all $x\in U(x_0,{\rho }_0)$ and some $L_0>0$. Set $\rho =min\{{\rho }_0,{\displaystyle \frac{1}{L_0}}\}$.

Definition 2. 

Let E and T be as in Definition 1. Then, the pair $(T,E)$ is said to be restricted-Lipschitz continuous on $U(x_0,\rho )$ if

$$\parallel T\left(E\left(x\right)-E\left(y\right)\right)\parallel \le L\parallel x-y\parallel$$

for all $x,y\in U(x_0,\rho )$ and some $L>0$.

Definition 3. 

Let E and T be as in Definition 1. Then, the pair $(T,E)$ is said to be Lipschitz continuous on $U(x_0,{\rho }_0)$ if

$$\parallel T\left(E\left(x\right)-E\left(y\right)\right)\parallel \le L_1\parallel x-y\parallel ,$$

for all $x,y\in U(x_0,{\rho }_0)$ and some $L_1>0$.

Remark 1. 

It follows from these Definitions, since $\rho \le {\rho }_0$, that

$$L_0\le L_1$$

(7)

and

$$L\le L_1.$$

(8)

The convergence analysis in [4] used $L_1$. However, the proofs can be repeated with L replacing $L_1$. Note that $L_0=L_0\left(U(x_0,{\rho }_0)\right)$ and $L_1=L_1\left(U(x_0,{\rho }_0)\right)$, while $L=L\left(L_0,U(x_0,{\rho }_0)\right)$. We assume that

$$L_0\le L.$$

(9)

Otherwise, i.e., if $L\le L_0$, the results hold with $L_0$ replacing L.

Definition 4. 

The problem (1) is said to satisfy the Weak-Robinson condition at $x_0$ on $U(x_0,{\rho }_0)$ if

$$-F\left(x_0\right)\in R\left(T_{x_0}\right)\mathrm{and}R\left(F^{\prime }\left(x\right)\right)\subseteq R\left(T_{x_0}\right)\mathrm{for}\mathrm{all}x\in U(x_0,{\rho }_0).$$

(10)

It follows that for the problem (1), the Robinson condition at $x_0$ implies the Weak-Robinson condition at $x_0$ on X.

Proposition 2. 

Let $x_0\in X$ and $r\in \left(0,{\displaystyle \frac{1}{L_0}}\right]$. Suppose that the problem (1) satisfies the Weak-Robinson condition at $x_0\in U(x_0,r)$ and $(T_{x_0}^{-1},F^{\prime })$ is $L_0$-Lipschitz continuous on $U(x_0,r)$. Then, for all $x\in U(x_0,r)$, the following assertions hold:

$$R\left(T_{x_0}\right)=R\left(T_x\right),D\left(T_x^{-1}{F}^{\prime }\left(x_0\right)\right)=X$$

(11)

and

$$\parallel T_x^{-1}{F}^{\prime }\left(x_0\right)\parallel \le {\displaystyle \frac{1}{1-L_0\parallel x-x_0\parallel }}.$$

(12)

Additionally, if X is reflexive, then

$$D_{\infty }\left(x\right)\ne \varnothing .$$

(13)

Proof. 

Simply replace $L_1$ by the smallest $L_0$ that is actually needed in the proof of Proposition 2.3 in [4]. □

2.2. The Gauss-Newton Method for the Problem

We assume in this section that X is reflexive, h is a continuous convex function, and the set K is the set of minimum points defined as

$$K:=argminh.$$

(14)

For each $\Delta \in (0,+\infty ]$ and $x\in X$, let $D_{\Delta }\left(x\right)$ be the set of all $d\in X$ such that $\parallel d\parallel \le \Delta$ and

$$h\left(F\left(x\right)+F^{\prime }\left(x\right)d\right)=min\left\{h\left(F\left(x\right)+F^{\prime }\left(x\right)d_0\right):d_0\in X,\parallel d_0\parallel \le \Delta \right\}.$$

(15)

Then, $d\in D_{\Delta }\left(x\right)$ if and only if d solves the convex minimization problem:

$$min\left\{h\left(F\left(x\right)+F^{\prime }\left(x\right)d_0\right):d_0\in X,\parallel d_0\parallel \le \Delta \right\}.$$

(16)

Define

$$D_{\Delta }\left(x\right)=\left\{d\in X:\parallel d\parallel \le \Delta ,F\left(x\right)+F^{\prime }\left(x\right)d\in K\right\}.$$

(17)

Let $r\in [1,+\infty )$, $x_0\in X$, and $\Delta \in (0,+\infty ]$. Then, the Gauss-Newton method for solving the problem (2) is given by the Algorithm:

Algorithm $A(r,\Delta ,x_0)$: 

Given $x_n,n=0,1,\dots$, compute $x_{n+1}$ as follows: If

$$h\left(F\left(x_n\right)\right)=min\left\{h\left(F\left(x_n\right)+F^{\prime }\left(x_n\right)d\right):d\in X,\parallel d\parallel \le \Delta \right\},$$

(18)

then stop. Otherwise, select $d_n\in D_{\Delta }\left(x_n\right)$ so that $\parallel d_n\parallel \le rd\left(0,D_{\Delta }\left(x_n\right)\right)$, and let $x_{n+1}=x_n+d_n$.

Define the parameters $\beta$ and $\alpha$ by

$$\beta =r\parallel T_{x_0}^{-1}\left(-F\left(x_0\right)\right)\parallel ,\alpha ={\displaystyle \frac{r}{1+(r-1)L\beta }},\mathrm{and}{\alpha }_1={\displaystyle \frac{r}{1+(r-1)L_1\beta }}.$$

(19)

The majorizing functions for all $t\ge 0$ are given by

$$\begin{array}{ccc}\hfill g_0\left(t\right)& =& \beta -t+{\displaystyle \frac{\alpha L_0}{2}}{t}^2,\hfill \\ \hfill g\left(t\right)& =& \beta -t+{\displaystyle \frac{\alpha L}{2}}{t}^2,\hfill \\ \hfill \mathrm{and}& & \hfill \\ \hfill g_1\left(t\right)& =& \beta -t+{\displaystyle \frac{{\alpha }_1{L}_1}{2}}{t}^2.\hfill \end{array}$$

It follows from (19), (7), and these definitions that

$$g_0\left(t\right)\le g_1\left(t\right)$$

(20)

and

$$g\left(t\right)\le g_1\left(t\right).$$

(21)

Define the majorizing sequences $\left\{s_n\right\},\left\{t_n\right\}$ by

$$s_0=0,s_1=r,s_{n+1}=s_n-{\displaystyle \frac{g\left(s_n\right)}{g_0^{\prime }\left(s_n\right)}},$$

$$t_0=0,t_1=r,t_{n+1}=t_n-{\displaystyle \frac{g\left(t_n\right)}{g^{\prime }\left(t_n\right)}},$$

and

$$u_0=0,u_1=r,u_{n+1}=u_n-{\displaystyle \frac{g_1\left(u_n\right)}{g_1^{\prime }\left(u_n\right)}}.$$

The sequence $\left\{u_n\right\}$ is used in [4]. Here, we use $\left\{t_n\right\}$ although $\left\{s_n\right\}$ can also be used according to the proof of Theorem 3.1. Next, a Kantorovich condition is provided for the convergence of the sequence $\left\{t_n\right\}$ also for the sequence $\left\{s_n\right\}$.

Lemma 1. 

Suppose

$$\beta (r+1)L\le 1.$$

(22)

Then, the sequence $\left\{t_n\right\}$ is increasingly convergent to $t^{*}$ which is the smallest of the two roots of the function g guaranteed to exist by (22). Moreover for

$$\begin{array}{ccc}\hfill \theta & =& {\displaystyle \frac{1-L\beta -\sqrt{1-2L\beta -(r^2-1){\left(L\beta \right)}^2}}{\beta rL}}<1,\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill t_n& =& {\displaystyle \frac{{\sum }_{j=0}^{2^n-2}{\theta }^j}{{\sum }_{j=0}^{2^n-1}{\theta }^j}}{t}^{*},n=1,2,\dots .\hfill \end{array}$$

(24)

Proof. 

Simply replace $L_1,g_1$ by $L,g$ in the proof of Lemma 3.1 in [4] . □

Remark 2. 

The Kantorovich condition in [4] is

$$\beta (r+1)L_1\le 1.$$

(25)

It follows that

$$\left(\right)\Rightarrow \left(\right)$$

(26)

but not necessarily vice versa unless $L_1=L$. Moreover,

$$\theta \le {\theta }_1,$$

(27)

where ${\theta }_1$ is given by (23) with L replaced by $L_1$. Thus, the new results are at least as good as those in [4].

It is also worth noting that the improved results are obtained under the same computational effort, since in practice the computation of $L_1$ requires that of $L_0$ and L as special cases. Next, we present the semi-local convergence result for the algorithm $A(r,\Delta ,x_0)$.

Theorem 1. 

Suppose that the problem (1) satisfies the Weak-Robinson condition at $x_0$ on $U(x_0,t^{*})$ and that $(T_{x_0}^{-1},F^{\prime })$ is $L_0$-center-Lipschitz continuous on $U(x_0,{\displaystyle \frac{1}{L_0}})$ and L-restricted-Lipschitz on $U(x_0,t^{*})$. Suppose that

$$\beta \le min\left\{\Delta ,{\displaystyle \frac{1}{L(r+1)}}\right\}.$$

(28)

Then, any sequence $\left\{x_n\right\}$ generated by the algorithm $A(r,\Delta ,x_0)$ is convergent to some $x^{*}$ solving (1). Moreover, the following assertions hold:

$$\begin{array}{cc}\hfill \parallel x_n-x_{n-1}\parallel \le t_n-t_{n-1},& n=1,2,\dots ,\\ \hfill F\left(x_{n-1}\right)+F^{\prime }\left(x_{n-1}\right)(x_n-x_{n-1})\in K,& n=1,2,\dots \end{array}$$

(29)

and

$$\parallel x^{*}-x_n\parallel \le {\displaystyle \frac{{\theta }^{2^n-1}}{{\sum }_{j=0}^{2^n-1}{\theta }^j}}{t}^{*}.$$

(30)

Proof. 

Simply replace ${\theta }_1,L_1,g_1,u_n$ by $\theta ,L,g,t_n$, respectively, in the proof of Theorem 3.1 in [4]. □

Theorem 2. 

Suppose that the problem (1) satisfies the Weak-Robinson condition at $x_0$ on $U(x_0,t^{*})$, $L_0,L\in (0,+\infty )$, $\beta \ge \parallel T_{x_0}^{-1}\left(-F\left(x_0\right)\right)\parallel$, $r=1$, and $\left(T_{x_0}^{-1},F^{\prime }\right)$ is $L_0$-center-Lipschitz on $U(x_0,{\displaystyle \frac{1}{L_0}})$, L-Lipschitz continuous on $U(x_0,\rho )$, and $2\beta L\le 1$. Then, any sequence $\left\{x_n\right\}$ generated by the Algorithm $A\left(x_0\right)$ converges to a solution $t^{*}$ of the problem (1). Moreover, the following error assertion holds:

$$\parallel x^{*}-x_n\parallel \le {\displaystyle \frac{p^{2^n-1}}{{\sum }_{j=0}^{2^n-1}{p}^j}}{t}^{*},$$

where

$$p={\displaystyle \frac{1-L\beta -\sqrt{1-2\beta L}}{L\beta }}.$$

Proof. 

Replace $q_1,L_1,g_1,u_n$ by $p,L,g,t_n$ in the proof of Theorem 3.2 in [4]. □

Remark 3. 

(i)

Theorem 2 can be reduced to a weaker version of Robinson’s [25,26] for $\beta =\parallel T_{x_0}^{-1}\left(-F\left(x_0\right)\right)\parallel$, ${\rho }_1={\displaystyle \frac{1-\sqrt{1-2H\parallel T_{x_0}^{-1}\parallel \beta }}{H\parallel T_{x_0}^{-1}\parallel }}$, and $\parallel x_1-x_0\parallel \le {\displaystyle \frac{1}{2H\parallel T_{x_0}^{-1}\parallel }}$.

(ii)

The Kantorovich conditions for the convergence of the sequence $\left\{t_n\right\}$ can be weakened further.

Indeed, let us demonstrate this in the case of Algorithm $A(r,x_0,\Delta )$ and Theorem 3.4. According to the proof of Theorem 3.5:

$$r_0=0,r_1=\beta ,r_2=\beta +{\displaystyle \frac{L_0{\beta }^2}{2(1-L_0\beta )}},$$

(31)

$$r_{n+2}=r_{n+1}+{\displaystyle \frac{L{(r_{n+1}-r_n)}^2}{2(1-L_0{r}_{n+1})}}.$$

The sequence majorizing $\left\{x_n\right\}$ in [4], i.e., $\left\{u_n\right\}$, can be written as:

$$u_0=0,u_1=\beta ,u_{n+1}=u_n+{\displaystyle \frac{L_1{(u_n-u_{n-1})}^2}{2(1-L_1{u}_n)}}=u_n-{\displaystyle \frac{g_1\left(u_n\right)}{g_1^{\prime }\left(u_n\right)}}.$$

(32)

Moreover, the convergence conditions are given in [2] and [4], respectively, as:

$$\lambda =\overline{L}\beta \le 1$$

(33)

and

$$\mu =2L_1\beta r\le 1,$$

(34)

where $\overline{L}={\displaystyle \frac{1}{4}}\left(4L_0+\sqrt{L_{0}L+8L_0^2}+\sqrt{L_{0}L}\right)$. Notice that:

$$\mu \le 1\Rightarrow \lambda \le 1,$$

(35)

holds and ${\displaystyle \frac{\lambda }{\mu }}\to 0$ as ${\displaystyle \frac{L_0}{L_1}},{\displaystyle \frac{L}{L_1}}$ approaches 0. Thus, the new convergence condition is infinitely weaker than the original Kantorovich condition (33). Moreover, the following hold:

$$0\le r_{n+1}-r_n\le u_{n+1}-u_n$$

and

$$r^{*}=\underset{n\to +\infty }{lim}{r}_n\le t^{*}.$$

Hence, the error bounds on the distances $\parallel x_{n+1}-x_n\parallel$ are at least as tight and the information on the location of the solution is at least as precise. The results can also be rewritten using majorant functions replacing the Lipschitz constants $L_0,L$, and $L_1$ (see [2,3,4,5]).

2.3. The Implementation of the Algorithms

The iterates in both algorithms are computed by finding the expensive $F^{\prime }{\left(x_n\right)}^{-1}$. In the papers [3,4], we developed a hybrid method to address this difficulty. Specifically, we define the method

$$F\left(x_n\right)+F_1\left(x_n\right)(x_{n+1}-x_n)=0,$$

(36)

where $F_1=M{B}_k^{-1}\left(x_n\right)$, k is a natural number, $M\in \mathcal{L}(X,Y)$ is an invertible operator, $\Delta =M^{-1}(M-F^{\prime }\left(x\right))$, and $B_k=I+\Delta +\dots +{\Delta }^k$. Here, ${\displaystyle B_{\infty }{M}^{-1}=\underset{k\to +\infty }{lim}{B}_k{M}^{-1}=F^{\prime }}$.

Similarly, we introduce hybrid methods to replace both Algorithms. We demonstrate this with Algorithm $A\left(x_0\right)$. The method for $A(r,\Delta ,x_0)$ follows along the same lines. We take $M=F^{\prime }\left(x_0\right)$ for simplicity. As in [3], define the parameters:

$$a=a(\parallel x-x_0\parallel )\ge \parallel A\parallel ,$$

$$\overline{b_k}={\displaystyle \frac{a(1-a^k)}{1-a}},b_k={\displaystyle \frac{1}{1-\overline{b_k}}},$$

$$b_{\infty }={\displaystyle \frac{1-a}{1-2a}},\lambda (\parallel x-x_0\parallel )={\lambda }_k(\parallel x-x_0\parallel )=b_k{b}_{\infty }{\displaystyle \frac{a^{k+1}}{1-a}}.$$

Define the sequence $\left\{v_n\right\}$ with $v_0=0$, $v_1=\beta$,

$$\begin{array}{ccc}\hfill v_2& =& \beta +{\displaystyle \frac{\frac{L_0}{2}{\beta }^2+\lambda \left(\beta \right)}{1-L_0\beta }},\hfill \\ \hfill v_{n+2}& =& v_{n+1}+{\displaystyle \frac{\frac{L}{2}{(v_{n+1}-v_n)}^2+{\lambda }_k\left(v_{n+1}\right)}{1-L_0{v}_{n+1}}}.\hfill \end{array}$$

(37)

The sequence $\left\{v_n\right\}$ shall be shown to majorizing for the sequence $\left\{x_n\right\}$ generated by the Algorithm defined next.

Algorithm $A{\left(x_0\right)}^{\prime }$: 

Given $x_n,n=0,1,\dots ,$ compute $x_{n+1}$ as follows: First define the set

$$D_{\infty }^{\prime }\left(x\right):=\left\{d\in X:F\left(x\right)+F_1\left(x\right)d\in K\right\}\mathrm{for}\mathrm{all}x\in X.$$

(38)

If $D_{\infty }^{\prime }\left(x\right)\ne \varnothing$, pick $d_n\in D_{\infty }\left(x_n\right)$ to satisfy $\parallel d_n\parallel ={min}_{d\in D_{\infty }\left(x_n\right)}\parallel d\parallel$ and set $x_{n+1}=x_n+d_n$.

Let us provide a convergence criterion for the sequence $\left\{v_n\right\}$. Suppose that there exists $0\le \rho <{\displaystyle \frac{1}{L_0}}$ such that for each $n=0,1,2,\dots$,

$$v_n\le \rho .$$

(39)

It follows by (39) and (38) that $0\le v_n\le v_{n+1}\le \rho$, and there exists $v^{*}\in [0,\rho ]$ such that ${\displaystyle v^{*}=\underset{n\to \infty }{lim}{v}_n}$. The limit point $v^{*}$ is the unique least upper bound of the sequence $\left\{v_n\right\}$. We can show the analog of Theorem 2 for the sequence $\left\{x_n\right\}$ generated by Algorithm $A{\left(x_0\right)}^{\prime }$.

Theorem 3. 

Suppose that the problem (1) satisfies the weak-Robinson condition at $x_0$ on $U(x_0,v^{*}),L_0,L\in (0,+\infty ),\beta \ge \parallel T_{x_0}^{-1}\left(-F\left(x_0\right)\right)\parallel ,r=1$, and $\left(T_{x_0}^{-1},F^{\prime }\right)$ is $L_0$-center Lipschitz on $U(x_0,{\displaystyle \frac{1}{L_0}})$ and L-Lipschitz on $U(x_0,\rho )$,

$$a<{\displaystyle \frac{1}{2}},$$

(40)

and (39) holds. Then, any sequence $\left\{x_n\right\}$ generated by Algorithm $A{\left(x_0\right)}^{\prime }$ converges to a solution $x^{*}$ of the problem (1). Moreover, the following error estimate holds:

$$\parallel x^{*}-x_n\parallel \le v^{*}-v_n.$$

(41)

Proof. 

Notice that for each $x\in U(x_0,v^{*})$,

$$\begin{array}{ccc}\hfill \parallel B_k-I\parallel & =& \parallel \Delta +{\Delta }^2+\dots +{\Delta }^k\parallel \hfill \\ & \le & \parallel \Delta \parallel +{\parallel \Delta \parallel }^2+\dots +{\parallel \Delta \parallel }^k\hfill \\ & \le & a+a^2+\dots +a^k={\displaystyle \frac{a(1-a^k)}{1-a}}=\overline{b_k}<1\left(\mathrm{by}\left(\right)\right).\hfill \end{array}$$

So, $B_k^{-1}\in \mathcal{L}(Y,X)$ and $\parallel B_k^{-1}\parallel \le {\displaystyle \frac{1}{1-\overline{b_k}}}=b_k$. Then, we can write

$$\begin{array}{ccc}\hfill F_1\left(x\right)& =& F^{\prime }\left(x_0\right)B_k^{-1}\left(x\right)\hfill \\ \hfill & =& F^{\prime }\left(x_0\right)\left(B_k^{-1}\left(x\right)-B_{\infty }^{-1}\left(x\right)+B_{\infty }^{-1}\left(x\right)\right)\hfill \\ \hfill & =& F^{\prime }\left(x_0\right)B_{\infty }^{-1}\left(x\right)+F^{\prime }\left(x_0\right)\left(B_k^{-1}\left(x\right)-B_{\infty }^{-1}\left(x\right)\right)\hfill \\ \hfill & =& F^{\prime }\left(x_0\right)B_{\infty }^{-1}\left(x\right)+F^{\prime }\left(x_0\right)B_k^{-1}\left(x\right)\left(B_{\infty }\left(x\right)-B_k\left(x\right)\right)B_{\infty }^{-1}\left(x\right).\hfill \end{array}$$

(42)

We also need the estimate

$$\begin{array}{ccc}\hfill \parallel B_k^{-1}\left(x\right)\left(B_{\infty }\left(x\right)-B_k\left(x\right)\right)B_{\infty }^{-1}\left(x\right)\parallel & \le & b_k\left(a^{k+1}+·\right)b_{\infty }\hfill \\ \hfill & =& b_k{b}_{\infty }{\displaystyle \frac{a^{k+1}}{1-a}}\hfill \\ \hfill & =& {\lambda }_k(\parallel x-x_0\parallel ).\hfill \end{array}$$

(43)

By replacing $F^{\prime }$ by $F_1$ in the proof of Theorem 2 and using (43), we get

$$\parallel x_{n+1}-x_n\parallel \le v_{n+1}-v_n,$$

(44)

leading to the existence of $x^{*}\in U(x_0,v^{*})$ solving $\left(\right)$. Then, from (44), we have for each $i=0,1,2,\dots$

$$\parallel x_{n+i}-x_n\parallel \le v_{n+i}-v_n.$$

(45)

Finally, by letting $i\to +\infty$ in (45), we obtain (41). □

Remark 4. 

(i)

It is worth noticing that the condition (39) is weaker than the Kantorovich condition $2\beta L\le 1$ used in the Theorem 2 or (33) since both of these conditions imply (39) but not necessarily vice versa. Thus, (39) can replace these stronger conditions in the Theorem 2 provided that the sequence $\left\{v_n\right\}$ (for $k=\infty$) replaces the sequence $\left\{t_n\right\}$.

(ii)

The results of the Theorem 3 can be presented in a more general setting if M is not necessarily chosen to be $F^{\prime }\left(x_0\right)$ as follows: Simply replace $E\left(x_0\right)$ by M in the Definition 1 and $F^{\prime }\left(x_0\right)$ by M in (12) where $M\in \mathcal{L}(X,Y)$ is an invertible operator. Then, the conclusions of the Theorem 3 hold this more general setting.

(iii)

The results can be further extended if the Lipschitz continuity is replaced by the generalized continuity of the operator $F^{\prime }$ along the lines of our work in [2,3,4,5].

3. Concluding Remarks

Remark 5. 

The results in this work can also be expressed using majorant functions instead of the Lipschitz constants $L_0,L,$ and $L_1$. This approach allows for even tighter error estimates (see [1,2,3,4,5]).

For instance, the constant $L_0$ in Definition 1 can be replaced with a function ${\phi }_0:[0,+\infty )\to [0,+\infty )$ that is continuous and non-decreasing. Consequently, for all $x\in U(x_0,{\rho }_0)$, we obtain:

$$\parallel T(E\left(x\right)-E\left(x_0\right))\parallel \le {\phi }_0(\parallel x-x_0\parallel ).$$

(46)

If ${\phi }_0\left(t\right)=L_0\left(t\right)$, (46) reduces to the one stated in Definition 1. Similarly, $L_1$ and L in Definition 2 and Definition 3 can be replaced accordingly.

Let us introduce some parameters and real functions that play a role in the semi-local convergence of the Gauss-Newton algorithm under generalized continuity conditions used to control the derivative $f^{\prime }$ of the operator f. Set $A=[0,+\infty )$.

Suppose: $\left(C_1\right)$ There exists a function ${\psi }_0$ : $A\to A$, which is continuous and nondecreasing and a parameter $\alpha >0$ such that the equation $\alpha {\psi }_0\left(t\right)-1=0$ has at least one positive solution.Denote the smallest such solution by $\xi$. Set $A_0=[0,\xi ).$

$\left(C_2\right)$ There exists a function $\psi :A_0\to A$ which is continuous and nondecreasing. Let $\beta \ge 0$ be a parameter. Define the scalar sequence $\left\{s_n\right\}$ for $c>0$, $s_0=0$, $s_1=\beta$,

$$s_2=s_1+{\displaystyle \frac{c{\int }_0^1{\psi }_0\left((1-\theta \right)\left(s_1-s_0\right))d\theta \left(s_1-s_0\right)}{1-{\psi }_0\left(s_1\right)}}$$

(47)

and each $n=1,2,\dots$ by

$$s_{n+2}=s_{n+1}+{\displaystyle \frac{c{\int }_0^1\psi \left((1-\theta )\left(s_{n+1}-s_n\right)\right)d\theta \left(s_{n+1}-s_n\right)}{1-{\psi }_0\left(s_{n+1}\right)}}.$$

The sequence $\left\{s_n\right\}$ shall be shown to be majorizing in Theorem 6.1. But, first a convergence condition is provided for the sequence $\left\{s_n\right\}$.

$\left(C_3\right)$ There exists ${\xi }_0\in [0,\xi )$ such that for all $n=0,1,2,\dots$

$${\psi }_0\left(s_n\right)<1\mathrm{and}{s}_n\le {\xi }_0.$$

It follows by this condition and the definition of the sequence $\left\{s_n\right\}$ that for each $n=0,1,2,\dots ,$$0\le s_n\le s_{n+1}\le {\xi }_0$ and there exists $s^{*}$ such that ${\displaystyle s^{*}=\underset{n\to \infty }{lim}{s}_n.}$ It is known from calculus that $s^{*}$ is the least upper bound of the sequence $\left\{s_n\right\}$ which is unique.

The scalar functions ${\psi }_0$ and $\psi$ connect to the operators on the Gauss-Newton Algorithm. Let $c\in [1,+\infty ),\Delta \in (0,+\infty )$ and $x_0\in X$ be such that $-f\left(x_0\right)\in Range\left(T\left(x_0\right)\right)$. Define $\beta$ by

$$\beta =c\parallel T{\left(x_0\right)}^{-1}\left(-f\left(x_0\right)\right)\parallel$$

(48)

$\left(C_4\right)$ Let $T:Y\to 2^Z$ be a convex process, where Z is also a Banach space. The pair ($T^{-1}{f}^{\prime }$) is center-generalized continuous on $U\left[x_0,{\rho }_0\right)$ for $\rho >0$, i.e., $∥T^{-1}\left(f^{\prime }\left(x\right)-f^{\prime }\left(x_0\right)\right)∥\le {\psi }_0\left(∥x-x_0∥\right)$ for all $x\in U\left(x_0,{\rho }_0\right).$

Set

$${\overline{\xi }}_0=min\{{\rho }_0,\xi \}.$$

(49)

$\left(C_5\right)$ Let $f^{\prime },T$ be as in the condition $\left(C_4\right)$. The pair $\left(T^{-1},f^{\prime }\right)$ is restricted generalized continuous on $U\left(x_0,\overline{\xi }\right)$, i.e., $∥T^{-1}\left(f^{\prime }\left(x\right)-f^{\prime }\left(y\right)\right)∥\le \psi \left(∥x-y∥\right)$ for all $x,y\in U(x_0,\overline{\xi })$.

$\left(C_6\right)\beta \le \Delta$.

$\left(C_7\right)U\left[x_0,s^{*}\right]\subset U\left[x_0,{\rho }_0\right]$.

Next, the main semi-local convergence of the Gauss-Newton algorithm is shown under the conditions $\left(C_1\right)-\left(C_7\right)$.

Theorem 4. 

Suppose that the inclusion (1) satisfies the weak-Robinson condition at $x_0$ on $U\left(x_0,s^{*}\right)$ and ($C_1$)-($C_7$) hold. Then, the Gauss-Newton Algorithm $A\left(c,\Delta ,x_0\right)$ is well defined and any sequence $\left\{x_n\right\}$ converges to some $x^{*}\in U\left[x_0,s^{*}\right]$ such that $F\left(x^{*}\right)\in K.$ Moreover, the following items hold for all $n=1,2,\dots$

$$∥x_n-x_{n-1}∥\le s_n-s_{n-1},$$

(50)

$$f\left(x_{n-1}\right)+f^{\prime }\left(x_{n-1}\right)\left(x_n-x_{n-1}\right)\in K$$

(51)

and

$$\parallel x^{*}-x_n\parallel \le s^{*}-s_n.$$

(52)

Proof. 

It follows by the weak-Robinson condition that

$$D_{\infty }\left(x_0\right)=T^{-1}\left(x_0\right)\left(-f\left(x_0\right)\right)\ne 0.$$

(53)

So, by the definition of $\beta$, (47), (48) and the condition ($\left.C_6\right)$, we get

$$cd\left(0,D_{\infty }\left(x_0\right)\right)=c∥T{\left(x_0\right)}^{-1}\left(-F\left(x_0\right)\right)∥=\beta \le s_1-s_0\le \Delta$$

(54)

By the reflexivity of X, (54) and since $c\ge 1$,there exists $d\in D_{\infty }\left(x_0\right)$ such that $\parallel d\parallel \le \beta \le \Delta$. Hence, $d\left(0,D_{\Delta }\left(x_0\right)\right)=d\left(0,D_{\infty }\left(x_0\right)\right)$ and $D_{\Delta }\left(x_0\right)\ne \varnothing$. Thus, the iterate $x_1$ is well defined and $f\left(x_0\right)+f^{\prime }\left(x_0\right)d_0\in K;$ so, (51) holds for $n=1$.

Moreover, by (54) and the Gauss-Newton Algorithm $A\left(c,\Delta ,x_0\right):∥d_0∥\le cd\left(0,D_{\infty }\left(x_0\right)\right)\le s_1-s_0$, i.e., $∥x_1-x_0∥\le s_1,-s_0$. So,(50) holds sot $n=1$. Suppose that (50) and (51) hold for $n=1,2,\dots ,m$. Then, we get

$$\begin{array}{cc}\hfill ∥x_m-x_0∥& \le ∥x_m-x_{m-1}∥+∥x_{m-1}-x_{m-2}∥+\dots +∥x_1-x_0∥\hfill \\ \hfill & \le \left(s_m-s_{m-1}\right)+\left(s_{m-1}-s_{m-2}\right)+\dots +\left(s_1-s_0\right)=s_m.\hfill \end{array}$$

Similarly, we have $∥x_{m-1}-x_0∥\le s_{m-1}\le s_m$.

Then, by the condition $\left(C_7\right)$$\theta x_m+(1-\theta )x_{m-1}\in U\left(x_0,s^{*}\right)\subset B(x_0,\xi )$ for all $\theta \in [0,1],$ so $D_{\infty }\left(x_m\right)\ne 0$. Hence, the iterate $x_{m+1}$ exists and (51) holds for $n=m+1$. Furthermore, for $\theta =1:\parallel x_m-x_0\parallel \le s^{*}\le \xi$. By applying the condition ( $C_4$ ) $\parallel T{\left(x_0\right)}^{-1}\left(f^{\prime }\left(x_m\right)-f^{\prime }\left(x_0\right)\right)\parallel \le {\psi }_0\left(∥x_m-x_0∥\right)<{\psi }_0\left(s^{*}\right)\le 1.$ Then, by the weak-Robinson condition, the Proposition (2) is applicable to $x_m$ replacing x leading to

$$D\left(T{\left(x_m\right)}^{-1}{f}^{\prime }\left(x_0\right)\right)=X$$

(55)

and

$$∥T^{-1}\left(x_0\right)f^{\prime }\left(x_0\right)∥\le {\displaystyle \frac{1}{1-{\psi }_0\left(∥x_m-x_0∥\right)}}\le {\displaystyle \frac{1}{1-{\psi }_0\left(s_m\right)}}$$

(56)

Thus, we get

$$\left.T{\left(x_0\right)}^{-1}{\int }_0^1\left(\left(\theta x_m+(1-\theta )x_{m-1}\right)-f^{\prime }\left(x_{m-1}\right)\right)-x_m\right)d\theta \left(x_{m-1}-x_m\right)\ne 0,$$

(57)

and by the induction hypothesis, (47) and the condition $\left(C_s\right)$

$$\begin{array}{cc}\hfill & \parallel T{\left(x_0\right)}^{-1}\left[-f\left(x_m\right)+f\left(x_{m-1}\right)-f^{\prime }\left(x_{m-1}\right)\left(x_{m-1}-x_m\right)\right]\parallel \hfill \\ \hfill & \le {\int }_0^1\psi ((1-\theta )\left(s_m-s_{m-1}\right))d\theta \left(s_{m-1}-s_{m-1}\right).\hfill \end{array}$$

Notice that this estimate holds for $\psi$ replaced by ${\psi }_0$ if $m=1$. Next, we must show

$$\varnothing \ne \left(T^{-1}{\left(x_m\right)}^{-1}{f}^{\prime }\left(x_0\right)T{\left(x_0\right)}^{-1}\left[-f\left(x_m\right)+f\left(x_{m-1}\right)\left(x_{m-1}\right)\left(x_m\right)\right]\right.\subseteq D_{\infty }\left(x_m\right)$$

(58)

It follows from (55) and (56) that the set in the middle of (58) is non-empty. To show the rest in (58), set $w=-f\left(x_m\right)+f\left(x_{m-1}\right)+f^{\prime }\left(x_{m-1}\right)\left(x_m-x_{m-1}\right)$ and $d\in \left(T^{-1}\left(x_m\right)f^{\prime }\left(x_0\right)\right)T{\left(x_0\right)}^{-1}\left(w\right)$, so $d\in \left(T{\left(x_m\right)}^{-1}{g}^{\prime }\left(x_0\right)\right)v$ for some $v\in T{\left(x_0\right)}^{-1}\left(w\right)$. We must show $d\in D_{\infty }\left(x_m\right)$. But we have $f^{\prime }\left(x_m\right)d\in f^{\prime }\left(x_0\right)v+K$ and $f^{\prime }\left(x_0\right)v\in w+K.$

So, $f^{\prime }\left(x_m\right)d\in w+K+K=w+K$, since K is a cone. By the definition of w and the induction hypothesis, it follows that

$$f\left(x_m\right)+f^{\prime }\left(x_m\right)d\in f\left(x_{m-1}\right)+f^{\prime }\left(x_{m-1}\right)\left(x_m-x_{m-1}\right)+K\subset K+K=K,$$

Thus, $d\in D_{\infty }\left(x_m\right)$, showing the right-hand side of the assertion (58). Hence,we can get from (56) and (58) in turn,

$$\begin{array}{cc}\hfill cd\left(0,D_{\infty }\left(x_m\right)\right)& \le \parallel T{\left(x_m\right)}^{-1}{F}^{\prime }\left(x_0\right)\parallel \parallel T{\left(x_0\right)}^{-1}\left(-F\left(x_m\right)+F\left(x_{m-1}\right)+F^{\prime }\left(x_{m-1}\right)\left(x_m-x_{m-1}\right)\parallel \right.\hfill \\ \hfill & \le c{\displaystyle \frac{{\int }_0^1\psi \left((1-\theta )\left(s_m-s_{m+1}\right)\right)d\theta \left(s_m-s_{m-1}\right)}{1-{\psi }_0\left(s_m\right)}}\hfill \\ \hfill & =s_{m+1}-s_m.\hfill \end{array}$$

(59)

Notice that the sequence $s_{m+1}-s_m$ is decreasing, so $s_{m+1}-s_m\le s_1-s_0=\beta \le \Delta$. Thus, by (59)$d\left(0,D_{\infty }\left(x_m\right)\right)\le cd\left(0,D_{\infty }\left(x_m\right)\right)\le \Delta$, which together with $D_{\infty }\left(x_m\right)\ne 0$ imply that there exists $d_0\in X$ with $\parallel d_0\parallel \le \Delta$ such that $F\left(x_m\right)+F^{\prime }\left(x_m\right)d_0\in K$. Hence, we have $D_{\Delta }\left(x_m\right)\ne \varnothing$ and $d\left(0,D_{\Delta }\left(x_m\right)\right)=d\left(0,D_{\infty }\left(x_m\right)\right)$. Thus, it follows by (59) that $∥x_{m+1}-x_m∥=\parallel d_m\parallel \le cd\left(0,D_{\Delta }\left(x_m\right)\right)\le s_{m+1}-s_m$. Thus, the induction for (50) and (51) is completed. It follows that the sequence $\left\{x_m\right\}$ is fundamental in X (since $s_m$ is also fundamental as convergent). Therefore,there exists $x^{*}\in X$ such that ${\displaystyle \underset{m\to +\infty }{lim}{x}_m=x^{*}}$ and $F\left(x^{*}\right)\in K$. Finally,from (50) for $i=0,1,2\dots$ and the triangle inequality, we obtain

$$∥x_{m+i}-x_m∥\le s_{m+i}-s_m$$

leading to (52) if $i\to +\infty$ □

Remark 6. 

label=()

Let us specialize the functions ${\psi }_0$ and ψ to be ${\psi }_0\left(t\right)=L_{0}t$ and $\psi \left(t\right)=Lt$. Then, the condition $\left(C_3\right)$ becomes ${\left(C_3\right)}^{\prime }:L_0{s}_n<1$ and $s_n\le {\displaystyle \frac{1}{L_0}}$ , since ${\xi }_0={\displaystyle \frac{1}{L_0}}$ by the condition $\left(C_1\right)$. Notice that the sequence $\left\{s_n\right\}$ becomes by the definition of the functions ${\psi }_0$ and ψ

$$\begin{array}{cc}\hfill & s_0=0,s_1=\beta ,s_2=s_1+{\displaystyle \frac{c{L}_0{\left(s_1-s_0\right)}^2}{2\left(1-s_1{L}_0\right)}}\hfill \\ \hfill & s_{m+1}=s_m+{\displaystyle \frac{2L{\left(s_m-s_{m-1}\right)}^2}{2\left(1-L_0{s}_m\right)}}\hfill \end{array}$$

(60)

According to the proof of Theorem 6.1, the sequence $\left\{s_n\right\}$ given by (60) majorizes $\left\{x_n\right\}$ provided that the conditions $\left(C_3\right)$ and $\left(C_6\right)$ hold. These conditions are weaker than (1). Indeed, this is the case, since

$$\begin{array}{c}\hfill 1\le c\le \alpha (1+(c-1)L\beta )\le \alpha (1+(c-1)L{s}_n),\end{array}$$

so

$$\begin{array}{c}\hfill {\displaystyle \frac{c}{\alpha }}{(1-L{s}_n)}^{-1}\le {(1-\alpha L{s}_n)}^{-1}\le -{\psi }^{\prime }{\left(s_n\right)}^{-1}.\end{array}$$

Hence, the sequence $\left\{s_n\right\}$ can be replaced by the less tight $\left\{t_n\right\}$ given by the formula (18). Notice that the sequence $\left\{t_n\right\}$ converges provided that the $\beta \le {\displaystyle \frac{1}{(c+1)L}}$ (see (27)). Consequently, the sequence $\left\{s_n\}\right.$ given by (60) is tighter than $\left\{t_n\}\right.$ used in [18] and the sufficient convergence conditions $\left(C_3\right)$ wood $\left(C_6\right)$ are weaker than $\xi \le {\displaystyle \frac{1}{\left(C+1\right)L_1}}$ used in [18]. We conclude that under these choices of the functions ${\psi }_0$ and ψ, the results of Theorem 6.1 specialize to the ones in Theorem 1. Clearly, the same is true for Theorem 2 if we take $c=1$ in Theorem 6.1.

lbbel=()

It is well known that generalized continuity provides even sharper bounds on the derivative than Lipschitz or Hölder or other conditions.

4. Discussion

The applicability of two Gauss-Newton methods for solving inclusion and convex-composition optimization problems for Banach space-valued operators is extended using more precise majorizing sequences and under the same computational cost, providing weaker yet sufficient semi-local convergence criteria and more precise error estimates on the distances $\parallel x_{n+1}-x_n\parallel$ and $\parallel x^{*}-x_n\parallel$. Moreover, the implementation of these algorithms is addressed by developing the corresponding hybrid Gauss-Newton methods. The Lipschitz constants can be replaced by the generalized continuity of $F^{\prime }$.

In future research, the ideas presented in this study shall be used to extend the applicability of other iterative methods used to solve these problems.