7. The iterative algorithms Methods
Numerical methods for solving general equations have been extensively studied. However, the development of numerical algorithms for systems of equations presents greater challenges, as it requires the simultaneous convergence of solutions for multiple equations. Consequently, research in this area remains relatively limited and technically demanding.
For the matrix system (
3), existing studies on numerical algorithms have predominantly focused on the real-number domain. Over the years, ongoing efforts have resulted in the development of more efficient iterative methods, thereby expanding both the types of solutions and the practical applicability of these algorithms.
This section provides a review of a series of studies by Peng et al. from 2006 to 2021, which investigated various solution types, including symmetric solutions, minimum-norm solutions, least-squares solutions, and the optimization problem
for given
with
representing the set of solutions of (
3) [
75,
76,
77,
78,
79,
80,
82,
83,
84,
85,
86,
87].
Sheng and Chen were among the first to propose a finite iterative method for solving the general solution of the matrix equation system (
3) [
75]. The algorithm presented in [
75] is outlined as follows.
|
Algorithm 1:General solution for (3) over [75] |
-
Require:
Matrices , , , , , , and the initial matrix .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set k = 1.
-
Ensure:
Step 2: If , , then stop; else, .
-
Ensure:
-
Ensure:
Step 4: Go to step 2.
|
Theorem 39 (Convergence of Algorithm 1 in [
75]).
For any given initial matrix , a solution to the system of matrix equations (3) can be obtained in at most iterations. When the initial iteration matrix is taken as , where H and are arbitrary, the matrix obtained by the Algorithm 1 iteration is the minimum-norm solution of the system of matrix equations (3).
Remark 23.
The problem (44) is equivalent to the minimum-norm solution of the following system:
Let , and . Then, the system (45) equivalent to
By using Algorithm 1, we can obtain the unique minimum-norm solution of the system of linear matrix equations (46).
Ding et al. considered the numerical solutions to the system of matrix equations (
3) [
76]. Among their contributions, they provided the constraint conditions for solving the unique solution of the matrix equation system (
3). Stochastic gradient algorithms and least-squares algorithms were developed to iteratively generate approximate solutions, and the algorithm was later extended to a more general form of matrix equation system.
Prior to that, some notations are introduced. Let
where
for
. Then, the block-matrix star product
★ is defined as
Below is the expressions for the iterative solution of the matrix equations system (
3). Let
,
,
,
,
and
. Define
The matrix equations system (
3) has a unique solution if and only if
. In this case, the unique solution is given by
and the corresponding homogeneous matrix equations
,
have a unique solution
.
If
and
are non-square matrices with full column rank, and
and
are non-square matrices with full row rank, then [
76] have the gradient based iterative algorithm described as follows:
where
Next, we present the convergence of the iterative formula (
47).
Theorem 40 (Convergence of (
47) in [
76]).
If the matrix equations in (3) have a unique solution X, then the iterative solution given by the sequence (47) converges to X.
On the other hand, when
and
are non-square matrices with full column rank, and
and
are non-square matrices with full row rank, the least-squares iterative sequence based iterative algorithm is described as
where
Theorem 41 (Convergence of (
48) in [
76]).
If the matrix system (3) has a unique solution X, then the iterative solution given by the algorithm in (48) converges to X.
Remark 24.
The iterative sequences (47) and (48) can be also applied to the generalized matrix equations:
The gradient iterative based solution can be expressed as
Similarly, one can easily give the the least-squares based iterative algorithm solution to the matrix equations in (49):
We next present the symmetric solutions of the system (
3) over
[
77,
78,
79,
80]. In addressing this problem, various algorithms have been continuously refined to solve symmetric solutions of the system (
3), resulting in further reductions in the computational cost per iteration.
First applied by Peng et al., the iterative method is used to obtain symmetric solutions of the system (
3) [
77]. The following outlines the iterative algorithm designed to solve the symmetric solutions of the system (
3) over
in [
77].
|
Algorithm 2:Symmetric solutions for (3) over [77] |
-
Require:
Matrices , , , , , and the initial matrix .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set .
-
Ensure:
-
Ensure:
if or , then
-
Ensure:
Stop.
-
Ensure:
else
-
Ensure:
, go to Step 2.
-
Ensure:
end if
|
Theorem 42 (Convergence of Algorithm 2 in [
77]).
If the system of equations (3) is consistent, then for any initial matrix , a solution can be obtained within a finite number of iterations. The minimum-norm solution can be obtained by choosing the initial iteration matrix . Additionally, the problem (44) can be equivalently transformed into solving (46).
Later, Chen et al. proposed a LSQR iterative method for symmetric solutions to the system of matrix equations (
3) [
78]. The system of matrix equations (
3) can be processed as
Hence, the vector form
and
in the LSQR algorithm can be rewritten as the following matrix form:
Then, we obtain the matrix form iteration LSQR method for solving the system of matrix equations (
3) and the least-squares problem (
57).
|
Algorithm 3:Symmetric solution for (3) over [78] |
-
Require:
Matrices , , , , , and the initial matrix .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Step 2: Repeat the process until the stopping criteria are met.
-
Ensure:
-
Ensure:
Step 4: Go to step 2.
|
Theorem 43 (Convergence of Algorithm 3 in [
78]).
Algorithm 3 possesses the finite termination property, and the specific stopping criteria can be found in [78]. Let the initial iteration matrix be . Where and are arbitrary matrices. In particular, if , the solution obtained by Algorithm 3 is the unique minimum-norm symmetric solution of the system of matrix equations (3).
Remark 25.
For a given arbitrary matrix and , the optimize problem (44) can be considered as
The solution can be obtained by applying Algorithm 3 to the modified equations with right-hand sides as
respectively. The solution can be expressed as
Li et al. proposed an efficient algorithm for computing symmetric solutions of the system of matrix equations (
3) [
79]. This algorithm outperforms previous methods in terms of speed and computational cost per iteration, as it involves only matrix-matrix multiplications at each step, making it well-suited for parallel implementation. The solution is formulated as the intersection of closed convex sets and is computed via the alternating projection method.
Let
M be a closed convex subset of a real Hilbert space
H, and
. The projection of
u onto
M, denoted by
, is the point in
M closest to
u, which satisfying the following equation
The system (
3) is solved in [
79] using the alternating projection method. When the sets intersect, this method finds a point in their intersection. For solving (
3), two sets are defined as
and
If the system (
3) is consistent, then
, and the intersection point
is the solution of (
3). Thus, solving system (
3) is equivalent to finding the intersection of
,
, and
. For given matrix
, we can obtain the projections
,
, and
of matrix
Z on
,
, and
as
respectively. Based on these preparatory results, the proposed algorithm is presented below.
|
Algorithm 4:Symmetric solution for (3) over [79] |
-
Require:
Matrices , , , , , , and the initial matrix .
-
Ensure:
The solution X of the ( 3).
-
Ensure:
Step 1: Set , , , .
-
Ensure:
fordo
-
Ensure:
-
Ensure:
end for
-
Ensure:
Step 3:.
|
Remark 26. Compared to Algorithm 2 in [77] and Algorithm 3 in [78], Algorithm 4 requires fewer computational resources per step when solving the symmetric solution of the system of matrix equations (3).
Theorem 44 (Convergence of Algorithm 4 in [
79]).
If the system of matrix equations (3) is consistent, the matrix sequence generated by Algorithm 4 converges to the solution of (3).
Wu and Zeng proposed an alternating direction method of multipliers (ADMM) to solve the symmetric solution of the optimize problem (
44) [
80]. They introduced two equivalent constrained optimization problems for the matrix least-squares problem (
44), which are formulated as
with
, and
with
. Both problems are formulated such that (
44) holds.
Theorem 45 (Constrained optimization problem (
50) in [
80]).
The problem (50) admits matrices as solutions if and only if there exist matrices , , and that satisfy the equations below.
Theorem 46 (Constrained optimization problem (
51) in [
80]).
Matrices are solutions of the constrained optimization problem (51) if and only if there exists matrices , , and such that the following equations hold.
The augmented Lagrangians corresponding to the constrained optimization problems (
50) and (
51) are given by
with
,
,
,
,
,
,
, and
with
,
,
,
,
,
,
, where
are penalty parameters.
Based on the ADMM approach, the variables
and
Z are minimized at each iteration step, after which the Lagrange multipliers
are updated according to the steepest ascent principle [
81]. The following two iterative algorithms correspond to the Lagrange functions (
52) and (
53).
|
Algorithm 5:Least-squares symmetric solution for (3) over [80] |
-
Require:
Matrices , , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Step 1: Choose the initial matrices and the parameters .
Set .
-
Ensure:
Step 2: Exit if a stopping criterion has been met.
-
Ensure:
-
Ensure:
Step 4: Set and go to Step 1.
|
|
Algorithm 6:Least-squares symmetric solution for (3) over [80] |
-
Require:
Matrices , , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Step 1: Choose the initial matrices and the parameters .
Set .
-
Ensure:
Step 2: Exit if a stopping criterion has been met.
-
Ensure:
-
Ensure:
Step 4: Set and go to Step 1.
|
Remark 27. The sequences generated by Algorithms 5 and 6 start with initial matrices and , along with parameters . The sequence then converges to the unique solution of the matrix least-squares problem (44).
Further work on symmetric solutions of the system (
3) can be found in [
82] and [
83], where bisymmetric solutions are discussed.
Cai and Chen proposed an iterative algorithm to compute the least-squares bisymmetric solutions of the system of matrix equations (
3) [
82]. They define a matrix function on
as
Before introducing the algorithm in [
82], a theorem that serves as its foundation is presented.
Theorem 47 (Bi-symmetry solutions for (
3) over
[
82]).
A matrix is a solution of the system (3) if and only if it satisfies the following matrix equation:
For convenience, the components of equation (55) are denoted as follows:
The iterative algorithm for computing the least-squares bisymmetric solutions of the system of matrix equations (
3) is given.
|
Algorithm 7:Least-squares bisymmetric solution for (3) over [82] |
-
Require:
Matrices , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set k = 1.
-
Ensure:
Step 2: If then stop; else, .
-
Ensure:
-
Ensure:
Step 4: Go to step 2.
|
Remark 28.
The sequence in Algorithm 7 is orthogonal in the finite dimensional matrix space . Therefore, there exists a positive integer t such that , and the solution of (3) can be obtained in a finite number of iterations. In Algorithm 7, the initial iteration matrix is given by
where and are arbitrary matrices. In particular, if , Algorithm 7 will compute the bisymmetric minimum-norm solution of (3).
Remark 29.
The optimal approximation solution to (44) can be derived. Let and . We have
Hence, the problem (44) is equivalent to
Denote and Substituting these into equation (55) obtains
This transforms the problem into solving the least-squares problem
The least-squares bisymmetric solution to (56) can be computed using Algorithm 7, and the optimal approximation solution is given by
Liu et al. [
83] introduced a novel iterative method for computing the bisymmetric minimum-norm solution of the system of matrix equations (
3). This method demonstrates improved speed and stability compared to Algorithm 7 by Cai et al. [
82].
For a given matrix
, the bisymmetry condition holds if and only if
The algorithm is described next.
|
Algorithm 8:Bisymmetric minimum-norm solution for (3) over [83] |
-
Require:
Matrices , , , , , , and the initial matrix .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Step 2: Repeat until the stopping criteria have been met.
-
Ensure:
-
Step 3: For , compute
Step 4: Go to step 2.
|
Remark 30.
The stopping criteria for Algorithm 8 can be defined as
where ϵ is a small tolerance.
Theorem 48 (Convergence of Algorithm 8 in [
83]).
The solution generated by Algorithm 8 is the bisymmetric minimum-norm solution of (3). In the absence of round-off errors, the algorithm is guaranteed to terminate in at most iterations.
Several algorithms for computing reflexive solutions of the system (
3) have been presented in [
84,
85,
86,
87]. The earlier algorithms offer specific forms of reflexive solutions, while the later ones yield more general solutions.
Peng et al. [
84] proposed an efficient algorithm for computing the least-squares reflexive solution of the system of matrix equations (
3). In their work, the problem is transformed into the optimized problem
Before presenting the algorithm, the concept of gradient matrix is introduced. Let be a continuous and differentiable function. The gradient of on is denoted as .
Theorem 49 (Reflexive solution for (
3) over
[
84]).
A matrix is a solution of the system of matrix equations (3) if and only if
For clarity, we define the following notations:
Then the corresponding iterative algorithm in [
84] is presented.
|
Algorithm 9:Reflexive solution for (3) over [84] |
-
Require:
Matrices , , , , , , and the initial matrix .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set
-
Ensure:
whiledo
-
Ensure:
-
Ensure:
-
Ensure:
end while
|
Theorem 50 (Convergence of Algorithm 9 in [
84]).
For an arbitrary initial matrix , Algorithm 9 generates a solution to (57) in a finite number of iterations. In particular, if the initial matrix is chosen as , the unique minimum-norm solution of (57) can be obtained within a finite number of iterations using Algorithm 9.
Remark 31.
When (3) is consistent, then
Suppose that , , , then the optimal approximation solution X of the (57) is equivalent to the minimum-norm reflexive solution of the following minimum residual problem
Dehghan and Hajarian proposed an iterative algorithm for computing the reflexive solution of the system of matrix equations (
3) [
85]. Their method refines and extends the algorithm originally introduced in [
84]. Given a matrix
, the following is the iterative algorithm for solving the reflexive solution of the matrix equation system (
3).
|
Algorithm 10:Reflexive solution for (3) over [85] |
-
Require:
Matrices , , , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set k = 1.
-
Ensure:
Step 2: If , then stop; else, .
-
Ensure:
-
Ensure:
Step 4: Go to step 2.
|
Theorem 51 (Convergence of Algorithm 10 in [
85]).
For any given initial matrix ,a solution to the system of matrix equations (3) can be obtained in a finite number of iterations in the absence of round-off errors. When (3) is consistent, and if we choose the initial iteration matrix as
where and are arbitrary. In particularly, if , then the solution obtained by Algorithm 10 is the minimum-norm reflexive solution of the system (3).
Remark 32. The optimal approximation solution to (44) for a given matrix can be derived from the minimum-norm reflexive solution of the following system (46). Let , , and , where . Then, using Algorithm 10 and the initial matrix from (58), the minimum-norm solution can be obtained. In this case, the unique solution to (44) can be computed and is given by .
Chen et al. proposed an iterative algorithm for computing the generalized reflexive solution of the system of matrix equations (
3). The iterative algorithm for obtaining the generalized reflexive solution is presented first, followed by an explanation of its convergence [
86].
|
Algorithm 11:Generalized reflexive solution for (3) over [86] |
-
Require:
Matrices , , , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set k = 1.
-
Ensure:
Step 2: If , then stop. Else go to Step 3.
-
Ensure:
-
Ensure:
Step 4: If , then stop. Else, let . Go to Step 3.
|
Theorem 52 (Convergence of Algorithm 11 in [
86]).
When the system (3) is consistent, a solution can be obtained within a finite number of iterations for any initial matrix , assuming there are no round-off errors. If the system (3) is consistent and the initial matrix is chosen as
where and are arbitrary matrices, or in particular , the unique minimum-norm generalized reflexive solution to (3) can be obtained within a finite number of iterations using Algorithm 11.
Remark 33.
When is a solution to (3), the solvability and solution of (3) are equivalent to
Yin and Huang proposed an iterative algorithm for computing the least-squares generalized reflexive solution of the system of matrix equations (
3). In their work, Algorithm 12 is employed to find a solution
that satisfies (
57) [
87].
Let
be defined as in (
54). Since the set
is unbounded, open, and convex,
is a continuous, differentiable, and convex function on
.
Theorem 53 (Generalized reflexive solutions for (
3) over
[
87]).
is a solution of the system of matrix equations (3), if and only if
For simplicity, we introduce the following notations:
Then, the result in [
87] is given.
|
Algorithm 12:Least-squares generalized reflexive solution for (3) over [87] |
-
Require:
Matrices , , , , , , , , and .
-
Ensure:
The solution matrix X.
-
Ensure:
-
Set k = 1.
-
Ensure:
Step 2: If , then stop. Else go to Step 4.
-
Ensure:
-
Ensure:
Step 5: If , then stop. Else, let . Go to Step 4.
|
Theorem 54 (Convergence of Algorithm 12 in [
87]).
For any initial matrix , Algorithm 12 generates a solution to (3) within a finite number of iterations, assuming no round-off errors occur. In particular, if the initial matrix is chosen as , Algorithm 12 produces the unique minimum-norm generalized reflexive solution to (57) within a finite number of iterations.
Remark 34.
Note that for (57), we have
Let , , . The problem of (44) is equivalent to finding the minimum-norm generalized reflexive solution of a new corresponding minimum residual problem
This section introduces iterative methods for solving system (
3). The development of existing algorithms has become more refined, with increasingly comprehensive methods for obtaining various types of solutions. However, there is still significant room for further advancement in numerical techniques for solving matrix equations. Most current algorithms are designed primarily for real-number domains, and future research could focus on extending these methods to more general numerical settings. Additionally, due to the high computational cost of numerical algorithms, their practical application to large-scale matrix equations remains limited.