Application of Multivariant Optimization Algorithm in Multiobjective Optimization

1. Introduction

The multiobjective optimization problem is a key research direction in the field of optimization. In scientific research and engineering application, most practical optimization problems involve the need of simultaneously optimizing multiple objective functions in the design process. To make the problem even complex, these multiple objectives may be mutually constraining. Therefore, it is imperative to study multiobjective optimization algorithms. Over the recent years, a variety of swarm intelligence optimization algorithms have sprung up in an endless stream, and quite a few of them have demonstrated excellent performance in solving multiobjective optimization problems. Against large-scale multiobjective optimization problems with complicated backgrounds, literature [1] proposes a competition-based swarm optimization (CSO) algorithm, which adopts a two-stage update strategy to boost the search efficiency to an enormous extent. Literature [2] proposes a new-type multiobjective sine cosine algorithm (MOSCA) for the prediction of a wind speed hybrid prediction system. Domestic and foreign scholars have achieved good results by introducing some new mechanisms and evolution strategies into the multiobjective optimization field. Literature [3] introduces the Levy disturbance and the niched optimization technology to improve the effectiveness of the multiobjective sparrow algorithm and applies this algorithm to optimize the capacity configuration of wind-sunlight diesel batteries. Literature [4] proposes a novel congestion strategy to improve the multiobjective sparrow algorithm and adopts multiple test functions and a disc brake design to test the algorithm performance. Literature [5] proposes an evolution algorithm (DMOES) based on a new-type evolution strategy, which involves a simulated niche of codirectional magnetic particles to approximate the Pareto front and adopts two strategies, the non-dominated solution and the dominated solution, to accelerate convergence, hence can obtain a good Pareto front at low computational cost.

Multivariant optimization algorithm (MOA) [6] is a swarm intelligence optimization algorithm proposed in 2013. Taking full advantage of the characteristic of sufficiently large modern compute memory, the algorithm establishes a computer data structure based structural body, which manages searching atoms in the form of multiple linked lists and employs search individuals to alternate between global and local searches to implement the procedure of optimization. This algorithm is compact-structured, with low computational complexity, process memory, high convergence and computational accuracy [7]. As proven by literatures [8,9,10], this algorithm has good asymptotic property, convergence, and reachability. Later, some scholars applied it in dynamic route optimization [11], data clustering [12], three-dimensional container loading problem [13] and other practical applications and achieved good results.

Based on MOA, this paper proposes a multiobjective multivariant optimization algorithm (MMOA). The algorithm has improved the global and local searching mechanisms by adopting a good point set [14] for initialization, increasing the population diversity and avoiding its dependence on the initial solutions. A nonlinear adaptive search radius is introduced to supersede, and avoid the blindness due to, the random radius in the primitive MOA, and the search range is narrowed by enabling the search radius to decrease gradually with the increase of the number of iterations. The way the search radius is processed balances the global and local searches of the algorithm and increases the convergence precision of the algorithm. Levy Flight [15] and Sine chaotic mapping [16] are introduced to disturb the local search and increase the population diversity. What is boosted is not just the optimization efficiency and convergence rate but also the optimization accuracy of the algorithm.

2. MOA

MOA is a multivariant search swarm intelligence optimization algorithm. Its basic thought is to construct a structural body composed of a horizontal linked list that saves global optimal solutions in order and a vertical linked list that saves local optimal solutions in order. Individuals with different functions are arranged in the structural body in a certain pattern, each performing its own functions and sharing information. The searching individuals are updated continually during iterations, and the search range is gradually narrowed until the global optimal solution and multiple local optimal solutions are found ultimately. The structural body is shown in Figure 1. By function of the searching individuals in the algorithm, the individuals responsible for global search are called global atoms (Ga’s), whereas the individuals responsible for local search are called local atoms (La’s). Ga’s are arranged in a queue pattern, whereas La’s are arranged in a stack pattern, composing an upper triangular structural body. The global optimal solution and multiple local optimal solutions are to be found in the end through alternate iterations between global search and local search.

Ga’s are generated by Equation (1):

$$\{\begin{array}{l}Ga=[h_1,\cdots ,h_{Dim}]\\ h_i=unifrnd({\min }_i,{\max }_i)\end{array}$$

(1)

where $Dim$ represents the dimension of the problem; ${\max }_i$ and ${\min }_i$ are the upper and lower limits of the $i$^th dimension of the solution space; $unifrnd({\min }_i,{\max }_i)$ is a function that generates a random number uniformly distributed between ${\min }_i$ and ${\max }_i$. La’s are generated by equation (2):

$$\{\begin{array}{l}La=Ga+R\times [l_1,\cdots ,l_{Dim}]\\ l_i=unifrnd(-1,1)\end{array}$$

(2)

where $R$ is the search radius, and $unifrnd(-1,1)$ is a function that generates a random number uniformly distributed between -1 and 1.

3. MMOA

To implement the MMOA, fast non-dominated sorting [14] and novel crowding distance are adopted to adjust the external archiving strategy, with searching atoms randomly selected as Ga’s from external archives. The good point set is adopted for population initialization and to ensure population diversity and avoid the dependence of the algorithm on the initial solutions. The nonlinear adaptive search radius is introduced to decrease gradually with the increase of the number of iterations and narrow the search range, thus balancing the global and local searches and enhances the convergence precision of the algorithm. To improve the update formula for local atoms, Levy Flight and chaotic mapping are introduced to increase population diversity and optimization efficiency.

3.1. Good Point Set Initialization

Given that there are multiple mutually constraining objective functions in a multiobjective optimization problem, the problem-solving process is even complex. To avoid the dependence of the algorithm on the initial solutions, the good point set [15] theory proposed by Loo-keng Hua et al. is adopted to construct the solution space of initialized searching atoms. The basic theory of good point set is introduced below: Assuming $G_s$ is a unit cube in $S$-dimensional Euclidean space, and $r\in G_s$, if

$$P_n(k)=\left\{\left(\left\{r_1^{(n)}\cdot k\right\},\left\{r_2^{(n)}\cdot k\right\},\cdots ,\left\{r_s^{(n)}\cdot k\right\}\right),1\le k\le n\right\}$$

(3)

with its deviation $\varphi (n)=C(r,\epsilon )n^{-1+\epsilon }$, where $C(r,\epsilon )n^{-1+\epsilon }$ is a constant depending only on $r$ and arbitrary positive number $\epsilon$, then $P_n(k)$ is called a good point set, with $r$ being the good point(s). $\left\{r_s^{(n)}\cdot k\right\}$ denotes taking the decimal part, and $n$ denotes the number of points. Take $r=\left\{2\cos (2\pi k/p),1\le k\le s\right\}$, where $p$ is the smallest prime number satisfying $\left(p-3\right)/2\ge s$. Map the good points into the search space to get:

$$h_i(j)=({\max }_j-{\min }_j)\cdot \left\{r_j^{(i)}\cdot k\right\}+{\min }_j$$

(4)

From Figure 2 , one can find that the points in the good point set are more uniformly distributed in the 2D space. Adopting the good point set for initialization favors the searching atoms finding the global optimal solution while avoiding premature convergence.

3.2. Nonlinear Search Radius

In single-objective MOA, the search radius decreases linearly with the gradual increase in the number of iterations. The search radius determines the size of the search space of the algorithm. However, most problems with the need for optimization are nonlinear and noncontinuous. Therefore, a linear search space is to the disadvantage of searching for optimal solutions. Here a nonlinear adaptive search radius is adopted as shown in equation (5):

$$r=0.7+\alpha \cdot e^{\cos (\frac{2\pi t}{t+T})},$$

(5)

where α = 0.1825, t denotes the current number of iterations, and T denotes the maximum number of iterations. The search radius decreases nonlinearly with the increase of the number of iterations. At the initial stage of iterations, the radius is relatively large, which favors global search; with the increase of the number of iterations, the search radius decreases nonlinearly, and the local search predominates progressively.

3.3. Global Search

In multiobjective optimization, for two arbitrary solutions $x_1$ and $x_2$ of objective functions $f_i(x)$, if $f_i(x_1)<f_i(x_2)$, then $x_1$ dominates $x_2$, denoted as $x_1\prec x_2$. If there exist $i$ and $j$ such that $f_i(x_1)<f_i(x_2)$ and $f_j(x_1)>f_j(x_2)$, then neither $x_1$ nor $x_2$ is dominated. A set of solutions are non-dominated, i.e., Pareto optimal solutions, if none of the solutions is dominated by any solution. The essence of multiobjective optimization is to find a set of Pareto optimal solutions such that the values of the objective functions are minimized.

MMOA adopts fast non-dominated sorting, an external archiving strategy , and a novel crowding distance to obtain Ga’s. The Ga’s are generated by randomly drawing individuals from external archives. Assume $P$ is the set of solutions, $n_i$ is the number of solutions dominated by solution $i$ in the population, and $S_i$ is the set of dominated solutions. Fast non-dominated sorting is in essence a cyclic classification process. It begins by finding all solutions sets for $n=0$ that cannot be dominated by any other solution, denoted as class $F_1$-1, and removes these solutions from $P$. Next, this process is looped by continuing to search for non-dominated solution sets in $P$, denoted as class $F_2$-2. The classification continues until all solutions have been classified. To ensure diversity, a certain distance should be kept between one solution and another in a same class, which requires calculating the crowding distance for each solution by the concrete formula of equation (6).

$$D_i=\{\begin{array}{l}\frac{\left|f_i(j+1)-f_i(j)\right|}{\left|f_i(j-1)-f_i(j)\right|},f_i(j+1)-f_i(j)<f_i(j-1)-f_i(j)，\\ \frac{\left|f_i(j-1)-f_i(j)\right|}{\left|f_i(j+1)-f_i(j)\right|},f_i(j-1)-f_i(j)<f_i(j+1)-f_i(j)，\end{array}$$

(6)

where $D_i$ is the distance ratio of searching atom $j$ at the $i$ th objective function. The total crowding distance of searching atom $j$ at multiobjective functions is $D_s={\displaystyle \sum _{i=1}^N{D}_i}$, where $N$ is the number of objective functions. As $D_S$ approaches $N$, the uniformity of solution distribution is better.

3.4. Local Search

To enhance the local search capacity of MMOA, this paper improves the update formula for La’s by introducing Levy Flight to control La’s. Levy Flight [16] was proposed by French mathematician Paul Levy. It provides a random walk mechanism to control local search, increase the diversity of searching atoms in the algorithm, and avoid the algorithm from running into the local optima in the later phase. Levy Flight is formulated by equations (7) and (8):

(7)

(8)

The update formula for the La’s in the local linked list is given by equation (9):

$$La=\{\begin{array}{l}Ga+R\cdot Levy(\cdot ),\begin{array}{cc}if& rand>0.5，\end{array}\\ Ga+C_1\cdot Sine(Dim)\cdot (Gpbest-Ga)+C_2\cdot rand(Dim)\cdot (Ggbest-La),else，\end{array}$$

(9)

where $Levy(\cdot )$ denotes Levy Flight, $Dim$ denotes the dimension in the problem, $C_1=2$, and $Sine(\cdot )$ denotes Sine chaotic mapping. Sine chaotic mapping [17] is a classical one-dimensional unimodal mapping, mathematically expressed by equation (10):

$$X_{t+1}=\mu \sin (\pi t_t)，$$

(10)

where $X_t\in (0,1),\mu =0.99$ . A chaotic phenomenon occurs for $\mu \in (0.87,0.93)\cup (0.95,1)$.

$Gpbest$ denotes the historical optimal solution, and $Ggbest$ denotes the global optimal solution. ${\mathrm{C}}_2=2.5-\frac{{\mathrm{e}}^{\frac{\mathrm{t}}{\mathrm{T}}}-1}{\mathrm{e}-1}$, decreasing nonlinearly with the increase of the number of iterations.

3.5. Algorithm Flow

According to the foregoing presented strategy, the algorithm flow of the MMOA proposed in this paper is detailed as below:

• Initialize the population Np using the good point set; set the maximum size Nr of external archives and maximum number of iterations T; perform non-dominated sorting among the initial solutions, update the external archives, and record the global optimum and local optima;
• While (t < T)
• Select Np individuals randomly as the current global atoms from external archives;
• for i = 1: Np
• for j = 1: Np - i+1
• Using formula (9), locally develop each local atom based on the upper triangular structural body, update the positions of current local atoms, and calculate their fitness;
• Perform non-dominated sorting among all current atoms and add non-dominated individuals into the external archives;
• end for
• end for
• Update the local optimal solutions and the global optimal solution;
• Update the parameters R and C2;
• t = t + 1;
• end while;
• Output all solutions in the set of external archives

4. Simulation experiment

4.1. Experiment Setup

To verify the performance of MMOA, this paper compares MMOA with NSGA-II [14], multiobjective evolutionary algorithm based on decomposition (MOEA/D) [18], non-dominated sorting whale optimization algorithm (NSWOA) [19], multiobjective Grey Wolf Optimizer (MOGWO) [20], and non-dominated sorting Moth-Flame Optimization (NDSMFO) [21]. To ensure objectivity, the main parameters in all algorithms are set as: individual number in or size of the largest set of external archives = 150, population size = 30, maximum number of iterations = 200. The parameters in all comparative algorithms are set in accord with the corresponding literatures.

• Performance indexes

There are many performance evaluation indexes for MOA, which are divided into three major classes: convergence, diversity, and composite indexes evaluating convergence and diversity. This paper selects one diversity index and two composite indexes to evaluate the algorithm performance [22].

• Inverted Generational Distance (IGD)

A composite index evaluating the algorithm performance, it calculates the mean value of the minimum distances from real Pareto individuals to the approximate solution set work out by the algorithm. It is mathematically expressed by equation (11).

$$IGD(S,P^{\ast })=\frac{{\displaystyle {\sum }_{x\in P\ast }d(x,S)}}{\left|P\ast \right|}，$$

(11)

where $d(x,S)$is the Euclidean distance from individual $x$ to the closest solution in $S$, and $\left|P^{\ast }\right|$ is the $P^{\ast }$-base. The smaller the value of IGD, the better the convergence and diversity of $S$, which is more capable to approximate to the ensemble of real Pareto individuals. When $IGD(S,P^{\ast })=0，S\subseteq P^{*}$.

2.

Spatial Index (SP)

This evaluates the distribution of the individuals in the approximate solution set of Pareto in the objective space. It is mathematically expressed by equation (12):

$$SP(S)=\sqrt{{\displaystyle \sum _{i=1}^{\left|S\right|}\frac{{(d_i-\overline{d})}^2}{\left|S\right|-1}}}，$$

(12)

where $PF$ represents the known real Pareto, $d_i$ is the Manhattan distance between two closest solution vectors on the non-dominated boundary, and $\overline{d}$ is the average distance. The smaller the value of SP, the better the spatial distribution of solution set $S$.

3.

Hypervolume (HV)

A composite index evaluating the algorithm, it measures the volume of the dominated objective space bounded by $r^{*}$. It is mathematically expressed by equation (13).

$$HV(S)=VOL({\displaystyle \underset{x\in S}{\cup }[f_1(x),r_1^{\ast }]\times \cdots [f_n(x),r_n^{\ast }]})，$$

(13)

where $r^{\ast }=(r_1^{\ast },r_2^{\ast },\cdots ,r_n^{\ast })$ is a set of reference points preset to be distributed in the objective space, and $VOL(\cdot )$ is Lebesgue measure. The higher the value of $HV$, the more approximate $S$ is to the ensemble of real Pareto individuals, and the better the diversity in the objective space.

To verify the validity of this algorithm, this paper selects four ZDT series dual-objective test functions and two DTLZ series triple-objective test functions to examine the solving performance of the algorithm. The selected test functions are shown in Table 1.

4.2. Experimental Results and Analysis

To avoid randomness, each of the test functions runs independently 10 times, and the listed test data are averages of the 10 times. The average runtimes are shown in Figure 3. MMOA achieves superior results on most of the test functions to the other comparing functions. During the algorithm iteration process, the environment is time-varying, and yet the algorithm adopts the nonlinear adaptive search radius $r$, avoiding the blind search and shortening its runtime.

Tabels 2–4 correspond respectively to the means and mean square errors of IGD, SP and HV on the six test functions achieved by MMOA versus the other five comparing algorithms. The boldface terms highlight the optimal IGD and SP values for each test function.

From Table 2, the MMOA in this paper achieves five optimal IGD means over the six test functions, while NSWOA achieves one optimal value. The convergence effect on DTLZ6 is next only to NSWOA. Moreover, the results of both algorithms have the same order of magnitude. MMOA introduces Levy Flight and Sine chaotic mapping for disturbance purposes and to improve the local atoms, conduct full local search, and improve the convergence precision.

A comparative analysis of the SP performance indexes of the six algorithms via Table 3 reveals that MMOA achieves four optimal values and MOGWO achieves two on the six test functions, while none of the other comparing algorithms achieves an optimal value on the distributive performance index. MMOA adopts fast non-dominated sorting, the external archiving strategy and the novel crowding distance to acquire global atoms, giving full consideration to the spatiality of solutions, which endows it with a good distributive performance in Pareto solution sets of multiobjective optimization problems, as indicated by the small SP means.

The data presented in Table 4 show that, despite the three optimal values achieved by MMOA and NSWOA each, the gap between them is quite narrow. In MMOA, the good point set is adopted for the initialization of searching atoms, increasing the population diversity, further guaranteeing the spatial diversity of solutions, and making the HV values of MMOA relatively high.

To further compare the performances of MMOA and the other comparing algorithms, this paper conducts a t-test with the results of the two performance indexes, IGD and SP. In terms of 5% level of significance, a comparative analysis is made between MMOA and the other five algorithms. The analysis results are shown in TABLEs III and IV, where “+”, “-” and “=” signify the performance of MMOA is superior, inferior, and approximate to the corresponding comparing algorithm, respectively. From the two tables, one can find the performance indexes of MMOA are significantly superior to NSGAII, MOEA/D and NSMFO amid most of the test functions, except that it is approximate or inferior to MOGWO and NSWOA on only a few of the functions. From the above experiments, MMOA demonstrates superior convergence precision and distributive performance.

To intuitively display the convergency and distributivity of the solution set derived by each algorithm, Figure 4 present the Pareto fronts of the six algorithms on some of the test problems.

4.3. Engineering Application

To test the validity of MMOA in solving practical engineering problems, this paper applies it to the problem of a disc brake design [23]. The disc brake design is a typical multiobjective engineering design problem. The problem involves two objective functions: the braking time $f_1(x)$ and the braking mass $f_2(x)$ of the disc brake, subject to five constraints $g_1(x),\cdots g_5(x)$, which denote the minimum distance between radii, the maximum length of the brake, pressure, temperature, and torque, respectively. The problem is formulated as equation (14):

$$\begin{array}{l}Minimize：f_1(x)=4.9\times {10}^{-5}(x_2^2-x_1^2)(x_4-1)，\\ \begin{array}{ccccc}& & & & f_2\end{array}(x)=\frac{(9.82\times {10}^6)\times (x_2^2-x_1^2)}{\left(x_2^3-x_1^3\right)x_4{x}_3}，\\ {\begin{array}{ccccc}& & & s.t.:& g_1(x)=20+x_1-x\end{array}}_2，\\ \begin{array}{cccccc}& & & & & \begin{array}{l}{g}_2(x)=2.5+(x_4+1)-30，\\ g_3(x)=\frac{x_3}{3.14\times {(x_2^2-x_1^2)}^2-0.4}，\end{array}\end{array}\\ \begin{array}{cccccc}& & & & & g_4(x)=\end{array}\frac{2.22\times {10}^{-3}{x}_3(x_2^3-x_1^3)}{{(x_2^2-x_1^2)}^2-1}，\\ \begin{array}{cccccc}& & & & & g_5(x)=\end{array}\frac{900-2.66\times {10}^{-2}{x}_3{x}_4(x_2^3-x_1^3)}{{(x_2^2-x_1^2)}^2}，\\ \begin{array}{cccccc}& & & & & \begin{array}{l}55\le x_1\le 80,75\le x_2\le 110，\\ 1000\le x_3\le 3000,2\le x_4\le 20，\end{array}\end{array}\end{array}$$

(14)

where the four design variables $x_1,x_2,x_3,x_4$ represent the disc’s outer radius, inner radius, joint force, and number of friction surfaces, respectively.

In the algorithm the population size is set as 50, the individual number in or size of the largest set of external archives as 200, and the maximum number of iterations as 500, with the same settings for the rest of the parameters. Figure 5 shows the results of optimization by MMOA and by the five algorithms selected in this paper. From Figure 7, MMOA renders a smoother result.

Table 5 presents the spatial evaluator SP and runtime. The SP value of MMOA is the smallest, showing that MMOA has the optimal distributivity of solutions. The runtime, although not the shortest, and slightly inferior to NSWOA, is at the same order of magnitude as NSWOA. Therefore, MMOA can be applied to solve multiobjective engineering problems.

5. Conclusions

At present, various new-type swarm intelligence algorithms have emerged in an endless stream. Their use in combination with novel mechanisms in solving multiobjective optimization problems has become a new research hot spot. Based on MOA, this paper has proposed the multiobjective MOA (MMOA), adopted the test functions and the disc braker design model to test the algorithm, and compared it with five optimization algorithms. The experimental results have shown that the solution set of MMOA demonstrates excellent convergence, spatial distributivity, and scalability. The innovation points achieved by the algorithm in this paper include:

• adopting the good point set for initialization of searching atoms, which is conducive to increasing population diversity and avoiding the dependence of the algorithm on the initial solutions;
• implementing the nonlinear adaptive search radius, which has the capacity of balancing global search avoiding blind search and accelerating optimization with effectiveness;
• improving the global atom setting method in the single-objective MOA, configuring external archives through fast non-dominated sorting and novel crowding distance, and randomly selecting from external archives searching atoms as global atoms for global search;
• improving the formula for local atoms in the single-objective MOA, and introducing Levy Flight and Sine chaotic mapping, by which to impose perturbation as the search radius decreases to increase population diversity and boost the algorithm’s optimization efficiency and convergence rate of the algorithm.

Future research will focus on the practical application of MMOA. In recent years, despite the wide attention to the data privacy issue paid by both the industrial and academic communities, data sharing remains hard to implement due to isolated data islands and privacy protection in practical application. Federated learning is an effective approach to break isolated data islands and protect data privacy by building a swarm intelligence model. However, how to address the communication overhead while ensuring accuracy is a problem expected to be resolved by federated learning. With the focus on such practical application problems in the future, MMOA will be employed to optimize the model structure of federated learning, reduce the communication cost, and enhance accuracy.