Improving the Giant Armadillo Optimization Method

1. Introduction

Global optimization targets to discover the global minimum of an optimization problem problem by exploring the entire search space. Typically, a global optimization method aims to discover the global minimum of a continuous function $f:S\to R,S\subset R^n$ and hence the global optimization problem is formulated as:

$$x^{*}=\arg \underset{x\in S}{min}f\left(x\right).$$

(1)

The set S is defined as:

$$S=\left[a_1,b_1\right]\otimes \left[a_2,b_2\right]\otimes \dots \left[a_n,b_n\right]$$

The vectors a and b stand for the left and right bounds respectively for the point x. A systematic review of the optimization procedure can be found in the work of Rothlauf [1]. Global optimization refers to techniques that seek the optimal solution to a problem, mainly using traditional mathematical methods, for example methods that try to locate either maxima or minima [2,3,4]. Each optimization problem consists of the decision variables, the problem constraints and the objective function [5]. The main objective in optimization is to assign appropriate values to the decision variables, so that the objective function is optimized. Problem solving techniques in optimization are divided into deterministic and stochastic approaches [6]. The most common techniques in the first category are interval methods [7,8]. In interval techniques the set S is divided through a number of iterations into subareas that may contain the global minimum using some criteria. Nevertheless, stochastic optimization methods are used in the majority of cases, because they can be programmed more easily and they do not require any priory information about the objective function. Such techniques may include Controlled Random Search methods [9,10,11], Simulated Annealing methods [12,13], Clustering methods [14,15,16] etc. Systematic reviews of stochastic methods can be found in the work of Pardalos et al [17] or in the work of Fouskakis et al [18]. Furthermore, due to the widespread use of parallel computing techniques in recent years, a number of techniques have been developed that exploit such architectures [19,20].

A group of stochastic programming techniques that have been developed to handle optimization problems are evolutionary techniques. These techniques are biological inspired, heuristic and population-based [21,22]. Some techniques that belong to evolutionary techniques are for example Ant Colony Optimization methods [23,24], Genetic algorithms [25,26,27], Particle Swarm Optimization (PSO) methods [28,29], Differential Evolution techniques [30,31], evolutionary strategies [32,33], evolutionary programming [34], genetic programming [35] etc. These methods have been with success in a series of practical problems from many fields, for example biology [36,37], physics [38,39], chemistry [40,41], agriculture [42,43], economics [44,45].

Recently, Alsayyed et al [46] introduced a new bio-inspired metaheuristic algorithm called Giant Armadillo Optimization (GAO). This algorithm aims to replicate the behavior of giant armadillos in the real-world [47]. The new algorithm is based on the giant armadillo’s hunting strategy of heading towards prey and digging termite mounds.

Owaid et al present a method [48] concerning the decision-making process in organizational and technical systems management problems, which also uses giant armadillo agents. The article presents a method for maximizing decision-making capacity in organizational and technical systems using artificial intelligence. The research is based on giant armadillo agents that are trained with the help of artificial neural networks [49,50] and in addition a genetic algorithm is used to select the best one.

This article focuses on enhancing the effectiveness and the speed of the GAO algorithm by proposing some modifications and more specifically:

• Application of termination rules, that are based on asymptotic considerations and they are defined in the recent bibliography. This addition will achieve early termination of the method and will not waste computational time on iterations that do not yield a better estimate of the global minimum of the objective function.
• A periodic application of a local search procedure. By using local optimization, the local minima of the objective function will be found more efficiently, which will also lead to a faster discovery of the global minimum.

The new method was tested on a series of objective problems found in the relevant literature and it is compared against an implemented Genetic Algorithm and a variant of the PSO technique. This paper has the following structure: in Section 2 the steps of the proposed method are described in detail, in Section 3 the benchmark functions are listed as well as the experimental results and finally in Section 4 some conclusions and guidelines for future work are provided.

2. The proposed method

The GAO algorithm mimics the process of natural evolution and initially generates a population of candidate solutions, that are possible solutions of the objective problem. The GAO algorithm aims to evolve the population of solutions through iterative steps. The algorithm is divided into two stochastic phases: the exploration phase, where the candidate solutions are updated with a process that mimics the attack of armadillos on termite mounds and the exploitation phase, where the solutions are updated similar to digging in termite mounds. The basic steps of the GAO algorithm are presented below:

1.

Initialization step

• Set $N_c$ as the number of armadillos in the population.
• Set $N_g$ the maximum number of allowed generations.
• Initialize randomly the $N_c$$g_i,i=1,\dots ,N_c$ armadillos in S.
• Set iter=0.
• Set $p_l$ the local search rate.

2.

Evaluation step

• For $i=1,\dots ,N_c$do Set $f_i=f\left(g_i\right)$.
• endfor

3.

Computation step

• For  $i=1,\dots ,N_c$do

  (a)

  Phase 1: Attack on termite mounds

  -

  Construct the termite mounds set ${\mathrm{TM}}_i=\left\{g_{k_i}:f_{k_i}<f_{i}and{k}_i\ne i\right\}$

  -

  Select the termite mound ${\mathrm{STM}}_i$ for armadillo i.

  -

  Create a new position $g_i^{\mathrm{P}1}$ for the armadillo according to the formula: $g_{i,j}^{P1}=g_{i,j}+r_{i,j}\left({\mathrm{STM}}_{i,j}-I_{i,j}{g}_{i,j}\right)$ where $r_{i,j}$ are random numbers in $[0,1]$ and $I_{i,j}$ are random numbers in $[1,2]$ and $j=1,\dots ,n$

  -

  Update the position of the armadillo i according to:

  $$g_i=\left\{\begin{array}{c}\begin{array}{cc}{g}_i^{\mathrm{P}1},& f\left(g_i^{\mathrm{P}1}\right)\le f_i\\ g_i& \mathrm{otherwise}\end{array}\hfill \end{array}\right.$$

  (b)

  Phase 2: Digging in termite mounds

  -

  Calculate a new trial position

  $$g_{i,j}^{\mathrm{P}2}=g_{i,j}+(1-2r_{i,j})\frac{b_j-a_j}{\mathrm{iter}}$$

  where $r_{i,j}$ are random numbers in $[0,1]$.

  -

  Update the position of the armadillo i according to:

  $$g_i=\left\{\begin{array}{c}\begin{array}{cc}{g}_i^{\mathrm{P}2},& f\left(g_i^{\mathrm{P}2}\right)\le f_i\\ g_i& \mathrm{otherwise}\end{array}\hfill \end{array}\right.$$

  (c)

  Local search. Draw a random number $r\in [0,1]$. If $r\le p_l$ then a local optimization algorithm is applied to $g_i$. Some local search procedures found in the optimization literature are the BFGS method [51], the Steepest Descent method [52], the L-Bfgs method [53] for large scaled optimization etc. A BFGS variant of Powell [54] was used in the current work as the local search optimizer.

• endfor

4.

Termination Check Step

• Set iter=iter+1.
• For the valid termination of the method, two termination rules that have recently appeared in the literature are proposed here and they are based on asymptotic considerations. The first stopping rule will be called DoubleBox in the conducted experiments and it was introduced in the work of Tsoulos in 2008 [55]. This termination rule is based on calculating the variance of the best function value discovered by the optimization method in each iteration. The second termination rule was introduced in the work of Charilogis et al [56] and will be called Similarity in the experiments. In this termination termination technique, at every iteration the difference between the current best value and the previous best value is calculated and the algorithm terminates when this difference is zero for a number of predefined iterations.
• If the termination criteria are not hold then goto step 3.

3. Experiments

This section will begin by detailing the functions that will be used in the experiments. These functions are widespread in the modern global optimization literature and have been used in many research works. Next, the experiments performed using the current method will be presented and a comparison will be made with two commonly used techniques in the field of global optimization, such as genetic algorithms and particle swarm optimization.

3.1. Experimental Functions

The proposed method was tested on a series of benchmark functions available from the related literature [57,58]. The definitions for the functions are listed subsequently.

• Bf1 function. The function Bohachevsky 1 is defined as:

  $$f\left(x\right)=x_1^2+2x_2^2-\frac{3}{10}cos\left(3\pi x_1\right)-\frac{4}{10}cos\left(4\pi x_2\right)+\frac{7}{10}$$

  with $x\in {[-100,100]}^2$.
• Bf2 function. The Bohachevsky 2 function is defined as:

  $$f\left(x\right)=x_1^2+2x_2^2-\frac{3}{10}cos\left(3\pi x_1\right)cos\left(4\pi x_2\right)+\frac{3}{10}$$

  with $x\in {[-50,50]}^2$.
• Branin function with the following definition: $f\left(x\right)={\left(x_2-\frac{5.1}{4{\pi }^2}{x}_1^2+\frac{5}{\pi }{x}_1-6\right)}^2+10\left(1-\frac{1}{8\pi }\right)cos\left(x_1\right)+10$ with $-5\le x_1\le 10,0\le x_2\le 15$.
• Camel function defined as:

  $$f\left(x\right)=4x_1^2-2.1x_1^4+\frac{1}{3}{x}_1^6+x_1{x}_2-4x_2^2+4x_2^4,x\in {[-5,5]}^2$$
• Easom defined as:

  $$f\left(x\right)=-cos\left(x_1\right)cos\left(x_2\right)exp\left({\left(x_2-\pi \right)}^2-{\left(x_1-\pi \right)}^2\right)$$

  with $x\in {[-100,100]}^2$.
• Exponential function defined as:

  $$f\left(x\right)=-exp\left(-0.5\sum _{i=1}^n{x}_i^2\right),-1\le x_i\le 1$$

  The global minimum is located at $x^{*}=(0,0,...,0)$ with value $-1$. The cases of $n=4,8,16,32$ were used in the conducted experiments.
• Gkls function. $f\left(x\right)=Gkls(x,n,w)$ a function with w local minima and dimension n. This function is provided in [59] with $x\in {[-1,1]}^n$. The values $n=2,3$ and $w=50$ were used in the conducted experiments.
• Goldstein and Price function

  $$\begin{array}{ccc}\hfill f\left(x\right)& =& \left[1+{\left(x_1+x_2+1\right)}^2\right.\hfill \\ & & \left(19-14x_1+3x_1^2-14x_2+6x_1{x}_2+3x_2^2\right)]\times \hfill \\ & & [30+{\left(2x_1-3x_2\right)}^2\hfill \\ & & \left(18-32x_1+12x_1^2+48x_2-36x_1{x}_2+27x_2^2\right)]\hfill \end{array}$$
• Griewank2 function. The function is given by

  $$f\left(x\right)=1+\frac{1}{200}\sum _{i=1}^2{x}_i^2-\prod _{i=1}^2\frac{cos\left(x_i\right)}{\sqrt{\left(i\right)}},x\in {[-100,100]}^2$$

  The global minimum is located at the $x^{*}=(0,0,...,0)$ with value 0.
• Griewank10 function defined as:

  $$f\left(x\right)=\sum _{i=1}^n\frac{x_i^2}{4000}-\prod _{i=1}^{n}cos\left(\frac{x_i}{\sqrt{i}}\right)+1$$

  with $n=10$.
• Hansen function. $f\left(x\right)={\sum }_{i=1}^{5}icos\left[(i-1)x_1+i\right]{\sum }_{j=1}^{5}jcos\left[(j+1)x_2+j\right]$, $x\in {[-10,10]}^2$.
• Hartman 3 function defined as:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^3{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^3$ and $a=\left(\begin{array}{ccc}3& 10& 30\\ 0.1& 10& 35\\ 3& 10& 30\\ 0.1& 10& 35\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{ccc}0.3689& 0.117& 0.2673\\ 0.4699& 0.4387& 0.747\\ 0.1091& 0.8732& 0.5547\\ 0.03815& 0.5743& 0.8828\end{array}\right)$$
• Hartman 6 function given by:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^6{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^6$ and $a=\left(\begin{array}{cccccc}10& 3& 17& 3.5& 1.7& 8\\ 0.05& 10& 17& 0.1& 8& 14\\ 3& 3.5& 1.7& 10& 17& 8\\ 17& 8& 0.05& 10& 0.1& 14\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{cccccc}0.1312& 0.1696& 0.5569& 0.0124& 0.8283& 0.5886\\ 0.2329& 0.4135& 0.8307& 0.3736& 0.1004& 0.9991\\ 0.2348& 0.1451& 0.3522& 0.2883& 0.3047& 0.6650\\ 0.4047& 0.8828& 0.8732& 0.5743& 0.1091& 0.0381\end{array}\right)$$
• Potential function, the well - known Lennard-Jones potential[60] is used as a test function here and it is defined as:

  $$V_{LJ}\left(r\right)=4ϵ\left[{\left(\frac{\sigma }{r}\right)}^{12}-{\left(\frac{\sigma }{r}\right)}^6\right]$$

  (2)

  The values $N=3,5$ were adopted in the conducted experiments.
• Rastrigin function defined as:

  $$f\left(x\right)=x_1^2+x_2^2-cos\left(18x_1\right)-cos\left(18x_2\right),x\in {[-1,1]}^2$$
• Rosenbrock function.

  $$f\left(x\right)=\sum _{i=1}^{n-1}\left(100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2\right),-30\le x_i\le 30.$$

  The values $n=4,8,16$ were used in the provided experiments.
• Shekel 7 function.

  $$f\left(x\right)=-\sum _{i=1}^7\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

  with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 3& 5& 3\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\end{array}\right)$.
• Shekel 5 function.

  $$f\left(x\right)=-\sum _{i=1}^5\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

  with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\end{array}\right)$.
• Shekel 10 function.

  $$f\left(x\right)=-\sum _{i=1}^{10}\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

  with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 5& 3& 3\\ 8& 1& 8& 1\\ 6& 2& 6& 2\\ 7& 3.6& 7& 3.6\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\\ 0.7\\ 0.5\\ 0.6\end{array}\right)$.
• Sinusoidal function defined as:

  $$f\left(x\right)=-\left(2.5\prod _{i=1}^{n}sin\left(x_i-z\right)+\prod _{i=1}^{n}sin\left(5\left(x_i-z\right)\right)\right),0\le x_i\le \pi .$$

  . The values $n=4,8,16$ and $z=\frac{\pi }{6}$ were examined in the conducted experiments.
• Test2N function defined as:

  $$f\left(x\right)=\frac{1}{2}\sum _{i=1}^n{x}_i^4-16x_i^2+5x_i,x_i\in [-5,5].$$

  The function has $2^n$ local minima and the values $n=4,5,6,7$ were used in the conducted experiments.
• Test30N function defined as:

  $$f\left(x\right)=\frac{1}{10}{sin}^2\left(3\pi x_1\right)\sum _{i=2}^{n-1}\left({\left(x_i-1\right)}^2\left(1+{sin}^2\left(3\pi x_{i+1}\right)\right)\right)+{\left(x_n-1\right)}^2\left(1+{sin}^2\left(2\pi x_n\right)\right)$$

  with $x\in [-10,10]$. The function has ${30}^n$ local minima and the values $n=3,4$ were used in the conducted experiments.

3.2. Experimental Results

The used software was coded in ANSI-C++ with the assistance of the freely available Optimus optimization environment, that can be downloaded from https://github.com/itsoulos/GlobalOptimus/ (accessed on 14 April 2024). The experiments were conducted on an AMD Ryzen 5950X with 128GB of RAM, running Debian Linux. In all experimental tables, the numbers in cells denotes average function calls for 30 independent runs. In each run different seed for the random number generator was used. The decimal numbers in parentheses symbolize the success rate of the method in finding the global minimum of the objective function. If this number does not exist, then the method succeeded in finding the global minimum in all 30 runs. The simulation parameters for the used optimization techniques are listed in Table 1.

The experimental results for the comparison of the proposed method against other methods found in the literature are outlined in Table 2. The following applies to this table:

1.

The column PROBLEM denotes the objective problem.

2.

The column GENETIC denotes the average function calls for the Genetic algorithm. The same number of armadillos and chromosomes and particles was used in the conducted experiments in order to be a fair comparison between the algorithms. Also, the same number of maximum generations and the same stopping criteria were utilized among the different optimization methods.

3.

The column PSO stands for the application of a Particle Swarm Optimization method in the objective problem. The number of particles and the stopping rule in the PSO method are the same as in proposed method.

4.

The column PROPOSED represents the experimental results for the Gao method with the suggested modifications.

5.

The final row denoted as AVERAGE stands for the average results for all the used objective functions.

The statistical comparison for the previous experimental results is depicted in Figure 1. The previous experiments and their subsequent statistical processing demonstrate that the proposed method significantly outperforms Particle Swarm Optimization in terms of the number of function calls, since it requires 20% fewer function calls on average to efficiently find the global minimum. In addition, the proposed method appears to have similar efficiency in terms of required function calls to that of the Genetic Algorithm.

The reliability of the termination techniques was tested with one more experiment, in which both proposed termination rules were used, and the experimental results for the test benchmark are presented in Table 3. Also, the statistical comparison for the experiment is shown graphically in Figure 2.

From the statistical processing of the experimental results, one can find that the termination method using the Similarity criterion requires a lower number of function calls than DoubleBox stopping rule to achieve the goal, which is to effectively find the global minimum. Furthermore, there is no significant difference in the success rate of the two termination techniques as reflected in the success rate in finding the global minimum, which rate remains high for both techniques (around 97%).

Moreover, the effect of the periodical application of the local search technique is explored in the experiments shown in Table 4, where the local search rate increases from 0.5% to 5%.

As expected, the success rate in finding the global minimum increases as the rate of application of the local minimization technique increases. For the case of the current method this rate increases from 92% to 97% in the experimental results. This finding demonstrates that if this method is combined with effective local minimization techniques, it can lead to more efficient finding of the global minimum for the objective function.

4. Conclusions

Two modifications for the Giant Armadillo Optimization method was suggested in this article. These modifications aimed to improve the efficiency and the speed of the underlying global optimization algorithm. The first modification suggested the periodically application of a local optimization procedure to randomly selected armadillos from the current population. The second modification utilized some stopping rules from the recent bibliography in order to prevent the method from unnecessary iterations, when the global minimum was already discovered. The modified global optimization method was tested against two other global optimization methods from the relevant literature and more specific an implementation of the Genetic Algorithm and a Particle Swarm Optimization variant on a series of well - known test functions. In order to have a fair comparison between these methods, the same number of test solutions (armadillo or chromosomes) as well as the same termination rule were used. The present technique after comparing the experimental results shows that it clearly outperforms the particle optimization and has similar behavior to that of the genetic algorithm. Also, after a series of experiments it was shown that the Similarity termination rule outperforms the DoubleBox termination rule in terms of function calls, without reducing the effectiveness of the proposed method in the task of locating the global minimum.

Since the experimental results show to be extremely promising further efforts can be made for the development of the technique in various fields. For example, an extension could be to develop a termination rule that exploits the particularities of the particular global optimization technique. Among the future extensions of the application may be the use of parallel computing techniques to speed up the optimization process, such as the incorporation of the MPI [61] or the OpenMP library [62]. For example, in this direction it could be investigated to parallelize the technique in a similar way as genetic algorithms using islands [63,64].