A Class of Local Search Based Anytime Algorithms for Continuous Distributed Constraint Optimization Problems

1. Introduction

The Distributed Constraint Optimization Problems (DCOPs) [1,2,3,4] are an efficient framework to solve the cooperative problems of multi-agent systems. In DCOPs, agents communicate directly or indirectly to coordinate their value assignments of variables to optimize the aggregate constraint utilities, where the variables are discrete and the constraint utilities are presented in tabular form. DCOPs have been successfully applied to model the problems in the real world like distributed meeting scheduling [5,6], sensor networks [7], multi-robot coordination [8,9], smart grid [10,11], and smart home [12,13]. Some improved methods such as [14,15,16,17] and some improvements of key metrics like privacy [18,19,20] have been proposed. In addition, there are some extensions of DCOPs like Asymmetric DCOPs [21,22,23,24] and Dynamic DCOPs [25,26,27].

Over the years, researchers have proposed several different types of algorithms to solve DCOPs, including exact search-based algorithms [28,29], exact inference-based algorithms [2,30], approximate search-based algorithms [31], approximate inference-based algorithms [7], and approximate sampling-based algorithms [32,33]. Although the DCOP algorithms have excellent performance for discrete problems, it is not satisfactory for solving continuous problems, such as target tracking sensor orientation. Therefore, Continuous DCOPs (C-DCOPs) [34] were proposed to model problems with continuous variables, and the constraint utilities are functional forms. Correspondingly, researchers proposed new C-DCOP algorithms to deal with the modification of the C-DCOP formulation, including Continuous Max-Sum (CMS) [34], Hybrid CMS (HCMS) [35], Bayesian Distributed Pseudo-tree Optimization Procedure (B-DPOP) [36], Particle Swarm Based Continuous DCOP (PCD) [37], Exact Continuous DPOP (EC-DPOP) [38] and extensions (Approximate Continuous DPOP (AC-DPOP), Clustered AC-DPOP (CAC-DPOP), and Continuous DSA (C-DSA)), Continuous Cooperative Constraint Approximation (C-CoCoA) [39], and Particle Swarm with Local Decision Based Continuous DCOP (PCD-LD) [40].

The CMS algorithm is a continuous extension of discrete Max-Sum (MS) algorithm, which approximates the utility functions as piece-wise linear functions to solve C-DCOPs. Similarly, HCMS improves the solution quality according to the non-linear optimization method based on the MS algorithm. Later, B-DPOP combines the Bayesian Optimization framework with DPOP to solve C-DCOPs. After that, Hoang et al. propose three extensions of the inference-based DPOP algorithm and one extension of search-based Distributed Stochastic Algorithm (DSA): EC-DPOP, AC-DPOP, CAC-DPOP, and C-DSA, where the first algorithm provides an exact approach to solve general C-DCOPs, and the second and third provide approximate methods. In addition, C-DSA extends DSA to the continuous domain. C-CoCoA is a semi-greedy non-iterative algorithm that can get a high-quality solution with less execution time and overhead.

Unfortunately, none of the above C-DCOP algorithms is an anytime algorithm. An anytime algorithm should have two characteristics: $\left(i\right)$ The algorithm can be stopped at any time to output a solution, which means it is iterative; $\left(ii\right)$ The algorithm guarantees the monotonicity of the solution quality over time. PCD introduces the Particle Swarm Optimization (PSO) algorithm to solve C-DCOPs, and PCD-LD combines local decisions with PCD to improve the solution quality. They are two iterative algorithms and collect all utilities by constructing a Breadth First Search (BFS) pseudo-tree to guarantee the monotonicity of the solution quality. Although PCD and PCD-LD are anytime algorithms, the utility passing in the pseudo-tree causes privacy violations [41].

In the wake of the above analysis, we extend Maximum Gain Message (MGM) [31] algorithm so that it’s able to solve C-DCOPs. Three main reasons prompt us to customize MGM for C-DCOPs as follows:

• We consider that anytime property is an important problem concerned for the C-DCOP algorithm. Since communication may be halted arbitrarily, an algorithm without the anytime property risk being terminated at an unsatisfactory assignment combination. Therefore, an anytime algorithm guarantees lower bounds on performance in anytime environments when given acceptable starting conditions.
• The MGM algorithm is an iterative, search-based algorithm that performs a distributed local search, and it can guarantee the monotonicity of the solution quality through local interactions. In other words, the MGM algorithm is an anytime algorithm.
• Compared with the anytime algorithms using the BFS pseudo-tree, the MGM algorithm solves problems without any restriction on the graph structure. Specifically, MGM uses a basic constraint graph and local interactions to maintain the privacy of agents.

Therefore, we provide three MGM-based C-DCOP algorithms, namely Continuous MGM (C-MGM), Parallel C-MGM (C-PMGM), and Parallel Differential Search C-MGM (C-PDSM). Firstly, C-MGM is an approximation C-DCOP algorithm that is similar to MGM. In C-MGM, each agent compares the gain of its local utility with neighbors to strictly increase the aggregate utility. Then, C-PMGM extends C-MGM and generates a competing solution for each solution to solve C-DCOPs in parallel. Finally, C-PDSM is an improvement based on C-PMGM that uses differential search to update competing solutions. We empirically evaluate the performance of proposed algorithms on benchmark problems, and the results show that our algorithms outperform the state-of-the-art C-DCOP algorithms.

The remainder of this paper is organized as follows. The next section introduces the necessary background, including the DCOP and C-DCOP frameworks, and the standard MGM algorithm. The details of our algorithms are described in Section 3. Then, we present theoretical analysis in Section 4. Afterward, Section 5 reports the empirical evaluation of our algorithms against the state-of-the-art algorithms. Finally, we provide conclusions and future work in Section 6.

2. Background

In this section, we provide the definition of the DCOP framework. Then, we introduce the C-DCOP framework and provide an example of a C-DCOP. Finally, we describe the MGM algorithm, which is the basis of our proposed algorithms.

2.1. Distributed Constraint Optimization Problems

A Distributed Constraint Optimization Problem (DCOP) is a tuple $\langle \mathbf{A},\mathbf{X},\mathbf{D},\mathbf{F},\alpha \rangle$, where

• $\mathbf{A}={\left\{a_i\right\}}_{i=1}^n$ is a set of agents, an agent can control one or more variables.
• $\mathbf{X}={\left\{x_i\right\}}_{i=1}^m$ is a set of discrete variables, each variable is controlled by one of the agents.
• $\mathbf{D}={\left\{D_x\right\}}_{x\in \mathbf{X}}$ is a set of discrete domains and each variable $x\in \mathbf{X}$ takes value from the set $D_x$.
• $\mathbf{F}={\left\{f_i\right\}}_{i=1}^l$ is a set of utility functions and each utility function is defined over a set of variables: $f_i\in \mathbf{F}:{\prod }_{x\in {\mathbf{x}}^{f_i}}{D}_x\to \mathbb{R}$, where the ${\mathbf{x}}^{f_i}\subseteq \mathbf{X}$ is the scope of $f_i$.
• $\alpha :$ is a mapping function that associates each variable to one agent. In this paper, we assume one agent controls only one variable.

In this paper, we assume that one agent only controls one variable, thus $n=m$ and the terms "agent" and "variable" could be used interchangeably. In addition, we assume that all utility functions are binary functions between two variables, thus $f_{ij}\in \mathbf{F}:D_{x_i}\times D_{x_j}\to \mathbb{R}$ and the ${\mathbf{x}}^{f_{ij}}=\{x_i,x_j\}$.

A solution of a DCOP is an assignment combination $X^{*}$ to all variables that are consistent with their respective domains. The goal of the solution is to minimize1 the aggregate utility as shown in Equation (1), where the aggregate utility is the sum of utilities across all the applicable utility functions.

$$\begin{array}{c}\hfill X^{*}=\underset{d_i\in D_{x_i},d_j\in D_{x_j}}{\arg \min }{\sum }_{f_{ij}\in \mathbf{F}}{f}_{ij}(x_i=d_i,x_j=d_j)\end{array}$$

(1)

2.2. Continuous Distributed Constraint Optimization Problems

The Continuous DCOP (C-DCOP) framework extends the DCOP framework by modeling the variables as continuous variables. It is defined by a tuple $\langle \mathbf{A},\mathbf{X},\mathbf{D},\mathbf{F},\alpha \rangle$, where $\mathbf{A}$, $\mathbf{F}$, and $\alpha$ are the same as DCOP. The set of variables $\mathbf{X}$ and the set of domains $\mathbf{D}$ are defined as follows:

• $\mathbf{X}={\left\{x_i\right\}}_{i=1}^m$ is the set of continuous variables.
• $\mathbf{D}={\left\{D_x\right\}}_{x\in \mathbf{X}}$ is the set of continuous domains and each continuous variable takes any value from the domain $D_x=[L{B}_x,U{B}_x]$, where $LB$ and $UB$ represent the lower and upper bounds of the domain, respectively.

The goal of a C-DCOP is the same as Equation (1), where the variables x and domain $D_x$ are continuous. Figure 1 provides an example of a C-DCOP, where Figure 1(a) is a constraint graph with three variables controlled by the agents and three constraints. Figure 1(b) defines the utility functions corresponding to the constraints and gives the domain of variables.

2.3. Maximum Gain Message

The MGM algorithm is an anytime, local search algorithm for solving DCOPs. Its main idea is to guarantee the agent with maximum gain changes its variable through competition between neighboring agents. In this decision method, only one agent can change its value in each local, which can completely avoid the uncertainty of the result caused by the simultaneous assignment of neighboring agents. Therefore, the solution quality improves steadily with iterations.

The framework of MGM is presented in Algorithm 1. Firstly, each agent randomly selects a value and sends it to neighbors (Algorithm 1: Lines 2-3). After receiving these values from neighbors, the agent assumes that the neighbors’ values remain unchanged and chooses a new value that maximizes the local gain. Then, the agent sends the local gain to neighbors and simultaneously receives gains from neighbors. If the agent with the maximum gain in the current local, it’s assigned the new value, otherwise it remains the chosen random value. Finally, the agent sends the updated value to neighbors and executes the loop (Algorithm 1: Lines 5-12) until the condition is met.

Algorithm 1: Maximum Gain Message Algorithm (MGM)

3. Our Algorithms

In this section, we introduce the three proposed algorithms in detail. Let us define a solution set $\mathit{S}={\left\{S_k\right\}}_{k=1}^K$ containing K solutions, where $S_k={\{S_k.x_i\}}_{i=1}^n$ represents an assignment combination (i.e. a solution) of n variables controlled by agents. Therefore, an agent $x_i$ holds the value assignments of K solutions on the $i^{th}$ dimension, denoted as $\mathit{S}.x_i={\{S_k.x_i\}}_{k=1}^K$. In addition, we use $S_k.x_i$ to represent the variable of the $k^{th}$ solution held by agent $x_i$, and its local utility is ${\mathcal{L}}_i(S_k,S_k)={\sum }_{j\in N_i}{f}_{ij}(S_k.x_i,S_k.x_j)$, where $(k,k)$ represents two variables on the $k^{th}$ solution and $N_i$ represents the neighbors of agent $x_i$. For example, we construct a solution set with two solutions for Figure 1 as $\mathit{S}=\{S_1,S_2\}$, where $S_1=\{S_1.x_1,S_1.x_2,S_1.x_3\}$ and $S_2=\{S_2.x_1,S_2.x_2,S_2.x_3\}$. For agent $x_1$, its value assignments are $\mathit{S}.x_1=\{S_1.x_1,S_2.x_1\}$ and the local utility of $S_1.x_1$ is ${\mathcal{L}}_i(S_1,S_1)$. It is worth noting that since we consider minimizing the aggregate utility, we use absolute gain to represent the local utility reduced, which is a negative value.

3.1. Continuous Maximum Gain Message

Algorithm 2 shows the process of the Continuous Maximum Gain Message (C-MGM) algorithm, including three phases: Initialization, Calculation, and Comparison.

Algorithm 2: Continuous MGM (C-MGM)

Initialization: Firstly, C-MGM constructs a solution set for K solutions. Then, each agent assigns a random value from its domain to each solution on the current dimension and generates a variable $G_i$ to record the gain value of agent $x_i$. Finally, the agent $x_i$ sends ValueMessage($\{i,S_r.x_i\}$) to all neighbors (Algorithm 2: Lines 1-7), where $S_r.x_i$ is a random value in $\mathit{S}.x_i$.

Calculation: After receiving ValueMessage($\{j,S_r.x_j\}$) from neighbors, the agent $x_i$ calculates the local utility for each solution according to its value assignments $\mathit{S}.x_i$, respectively (Algorithm 2: Lines 12-14). We define ${\mathcal{L}}_i(S_r,S_r)$ to represent the basic local utility of agent $x_i$, which is the local utility assuming that the values of agent $x_i$ and its neighboring agents are not changed, calculated by ${\sum }_{j\in N_i}{f}_{ij}(S_r.x_i,S_r.x_j)$. Then, the $G_i$ is assigned a difference value between the minimum and basic local utility (Algorithm 2: Lines 15-16). Finally, the agent $x_i$ sends GainMessage($\{i,G_i\}$) to neighbors.

Comparison: Similarly, the agent $x_i$ enters the comparison phase after receiving GainMessage($\{j,G_j\}$) from each neighbor $N_{i_j}\in N_i$. If the current agent with the minimum gain value among all gain values of neighbors and the gain is less than 0, the agent $x_i$ assigns the value that minimizes local utility to $S_r.x_i$ (Algorithm 2: Lines 18-20). In general, after the size of the domain is determined, the variable of the standard MGM algorithm can take a finite number of values from its domain, but there is an infinite number of values to be selected in the variable of C-MGM. We consider updating the value assignments for each agent to search its domain and avoid falling in the local optimum by adding randomness. Therefore, the agent $x_i$ updates each value $S_k.x_i$ to a random value from its domain $D_{x_i}$ (Algorithm 2: Line 22).

3.2. Parallel C-MGM

We consider solving K solutions in parallel based on C-MGM. Therefore, we propose the Parallel C-MGM (C-PMGM) algorithm as shown in Algorithm 3, which also consists of three phases: Initialization, Calculation, and Comparison.

Algorithm 3: Parallel C-MGM (C-PMGM)

Initialization: Except for the similar initialization as C-MGM, C-PMGM additionally generates a competing solution set $\mathit{C}$, which is the same as the solution set $\mathit{S}$. In addition, each agent uses a gain set $\mathit{G}$ to record the gain values of K solutions (Algorithm 3: Lines 7-9). It is worth noting that the ValueMessage($\{i,\mathit{S}.x_i\}$) sent by agent $x_i$ in C-PMGM to neighbors is the value assignments instead of a random value in $\mathit{S}.x_i$ (Algorithm 3: Line 10).

Calculation: Since the computational overhead of solving K solutions according to the way of C-MGM is expensive, we consider using the values from the corresponding competing solution set to calculate the gains for K solutions (Algorithm 3: Lines 11-16), which means that the indexes of the two solution sets ($\mathit{S}$ and $\mathit{C}$) are consistent. Later, the agent $x_i$ sends GainMessage($\{i,{\mathit{G}}_i\}$) to neighbors.

Comparison: Similar to C-MGM, the agent $x_i$ in C-PMGM compares K gains with all neighbors. For each gain $G_{k,i}$ in the gain set of agent $x_i$, if the gain value is the smallest in the current local and less than 0, the value $S_k.x_i$ is assigned to its competing value $C_k.x_i$ (Algorithm 3: Lines 18-21). Since an agent only calculates one local utility for a solution instead of K local utilities (such as Algorithm 2: Lines 13-14), the idea is to search as many competing values as possible. Therefore, the agent $x_i$ resets each competing value $C_k.x_i$ to a value from its domain, whether or not the value $S_k.x_i$ is changed.

3.3. Parallel Differential Search C-MGM

Based on C-PMGM, we construct a heuristic method to guide the exploitation of the competing solution set. Specifically, we extend the update of value assignments in C-PMGM by combining a differential search, named Parallel Differential Search C-MGM (C-PDSM). The details of C-PDSM can be found in Algorithm 4.

Algorithm 4: Parallel Differential Search C-MGM (C-PDSM)

The difference between C-PDSM and C-PMGM is the update of value assignments (Algorithm 4: Line 22). For a gain set in one agent, the agent filters each absolute gain and adds it to an absolute gain set $\mathsf{\Omega }$ (Algorithm 4: Lines 26-29). Since the differential search method we used requires at least two absolute gains, we provide two ways to update value assignments (Algorithm 4: Lines 30-34): $\left(i\right)$ If the number of absolute gains is less than two, the agent update the value assignments of the competing solution set through random search. $\left(ii\right)$ Otherwise, the agent updates value assignments by differential search, as shown in Equation (2). Our motivation is that the current value learns the exploitation ability from other values with absolute gain, which can improve the convergence speed of the algorithm.

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill C_k.x_i=C_k.x_i& +\omega *(C_{{\arg \min }_k\left\{\mathsf{\Omega }\right\}}.x_i-C_k.x_i)\hfill \\ & +\omega *(C_{{\arg \max }_k\left\{\mathsf{\Omega }\right\}}.x_i-C_k.x_i)\hfill \end{array}\end{array}$$

(2)

where $\omega$ is a scaling factor $\in [0,2]$, which controls the amplification of the differential variation [42]. The "${\arg \min }_k\left\{\mathsf{\Omega }\right\}$" and "${\arg \max }_k\left\{\mathsf{\Omega }\right\}$" represent the indexes of competing solution set with the minimum and maximum absolute gain values, respectively.

3.4. An Example of Algorithms

We take Figure 1 as an example to realize the three phases of C-MGN, C-PMGM, and C-PDSM.

In the Initialization phase, we assume that the size of the solution set is $K=2$ and the solution sets of the three algorithms are all $\mathit{S}=\{S_1=\{3,1,5\},S_2=\{-2,2,4\}\}$. Therefore, the value assignments held by agent $x_1$ are $\mathit{S}.x_1=\{S_1.x_1=3,S_2.x_1=-2\}$, the value assignments of agent $x_2$ are $\mathit{S}.x_2=\{1,2\}$, and the value assignments of agent $x_3$ are $\mathit{S}.x_3=\{5,4\}$.

• For C-MGM, the values randomly selected by each agent are $S_r.x_1=3$, $S_r.x_2=2$, and $S_r.x_3=4$.
• For C-PMGM and C-PDSM, we assume the competing solution sets both are $\mathit{C}=\{C_1=\{-4,-1,0\},C_2=\{-3,3,-5\}\}$ and the scaling factor $\omega$ in C-PDSM is 1.4.

In the Calculation phase, the agent $x_i$ calculates the local utility and gain after receiving the random value $S_r.x_j$ or value assignments $\mathit{S}.x_j$ from neighbors $N_{i_j}\in N_i$. Let’s take the agent $x_1$ as an example for calculation.

• For C-MGM:
  ${\mathcal{L}}_1(S_1,S_r)=f(3,2)+f(3,4)=46$
  ${\mathcal{L}}_1(S_2,S_r)=f(-2,2)+f(-2,4)=-19$
  ${\mathcal{L}}_1(S_r,S_r)=f(3,2)+f(3,4)=46$
  $G_1=-19-46=-65$;
• For C-PMGM and C-PDSM:
  ${\mathcal{L}}_1(S_1,S_1)=f(3,1)+f(3,5)=43$
  ${\mathcal{L}}_1(C_1,S_1)=f(-4,1)+f(-4,5)=-146$
  $G_{1,1}=-146-43=-189$;
  ${\mathcal{L}}_1(S_2,S_2)=f(-2,2)+f(-2,4)=-19$
  ${\mathcal{L}}_1(C_2,S_2)=f(-3,2)+f(-3,4)=-62$
  $G_{2,1}=-62-(-19)=-43$;

In the Comparison phase, each agent compares the gains to action. Simultaneously, there are different way to update value assignments in C-MGM, C-PMGM, and C-PDSM. We assume that the agent $x_1$ with maximum gain among its neighbors, and its actions and updates are as follows:

• For C-MGM:
  $S_r.x_1={\arg \min }_{S_k}\{46,-19\}=S_2.x_1=-2$;
  $\mathit{S}.x_1=\{1,-1\}$ (assumed random values);
• For C-PMGM:
  $S_1.x_1=C_1.x_1=-4$, $S_2.x_1=C_2.x_1=-3$;
  $\mathit{C}.x_1=\{2,-2\}$ (assumed random values);
• For C-PDSM:
  $S_1.x_1=C_1.x_1=-4$, $S_2.x_1=C_2.x_1=-3$;
  Since both gain values (-189 and -43) are less than 0, $\left|\mathsf{\Omega }\right|=2$.
  $C_1.x1=-4+1.4*[-4-(-4)]+1.4*[-3-(-4)]=-2.6$
  $C_2.x1=-3+1.4*[-4-(-3)]+1.4*[-3-(-3)]=-4.4$;

Finally, C-MGM outputs a solution as $X^{*}=\{-2,2,4\}$. The two solutions output by C-PMGM and C-PDSM are $X_1^{*}=\{-4,1,5\}$ and $X_2^{*}=\{-3,2,4\}$, the difference between C-PMGM and C-PDSM is that the value assignments of agent $x_1$ in the competing solution set are $\mathit{C}.x_1=\{2,-2\}$ and $\mathit{C}.x_1=\{-2.6,-4.4\}$, respectively.

4. Theoretical Analysis

In this section, we prove that C-MGM, C-PMGM, and C-PDSM are three anytime algorithms, and we provide some theoretical properties of our algorithms in terms of communication, computation, and memory. We assume that the graph structure is a binary constraint graph $G=(N,E)$, the number of nodes is $\left|A\right|$, and the number of edges in the constraint graph is $\left|F\right|$. In addition, we define one iteration as the complete process of the algorithm completing Calculation and Comparison phase.

Theorem 1.

C-MGM, C-PMGM, and C-PSDM are three anytime algorithms.

Proof. 

On the one hand, the three algorithms are iterative, which means they can be terminated at any time and output solutions. On the other hand, we assume that the aggregate utility (U) of C-MGM at t iteration is ${\mathbf{U}}_t={\sum }_{f_{ij}\in \mathbf{F}}{f}_{ij}$, which represents the sum of utility on all constraints. When an agent calculates local utility, the same constraint is calculated twice by two neighboring agents, the aggregate utility can be modified as ${\mathbf{U}}_t=\frac{1}{2}{\sum }_{x_i\in \mathbf{X}}{\mathcal{L}}_i$. Therefore, the aggregate utility of the $t+1$ iteration in C-MGM can represent ${\mathbf{U}}_{t+1}=\frac{1}{2}{\sum }_{x_i\in \mathbf{X}}{\mathcal{L}}_i+{\sum }_{x_i\in \mathcal{X}}{G}_i$, where $\mathcal{X}\subseteq \mathbf{X}$ represents a set of variables that change their values. In each local, since only one variable with local utility reduced changes its value, we can get that ${\sum }_{x_i\in \mathcal{X}}{G}_i<0$. Therefore, ${\mathbf{U}}_{t+1}<{\mathbf{U}}_t$, which means the aggregate utility strictly decreases with increasing iterations. Similarly, C-PMGM and C-PDSM solve K solutions in parallel, the aggregate utilities’ change of one solution in them is the same as that of C-MGM. Hence, C-MGM, C-PMGM, and C-PSDM are three anytime algorithms. □

Theorem 2.

The total number of messages of C-MGM, C-PMGM, and C-PDSM with t iterations all are $4t\left|F\right|$.

Proof. 

For each constraint $f_{ij}(x_i,x_j)$ in one iteration, the agents of the three algorithms need to send value assignments to neighbors in the Calculation phase and vice versa, there are two messages per edge. Therefore, the number of messages is $2\left|F\right|$ in this phase, where $\left|F\right|$ is the number of constraints (edges). In the Comparison phase, the agents of the three algorithms share gain or gain sets with neighbors to compare their values, thus the number of messages is also $2\left|F\right|$. Finally, the total number of messages of C-MGM, C-PMGM, and C-PDSM with t iterations all are $t*\left(2\right|F|+2|F\left|\right)=4t\left|F\right|$. □

Theorem 3.

In one iteration, the message size of C-MGM, C-PMGM, and C-PDSM are respectively 1, K, and K, where K is the number of solutions.

Proof. 

In the Calculation phase, the agent in C-MGM sends a random value to its neighbors to calculate the local utility, the message size of the random value is 1. However, the agents in C-PMGM and C-PDSM send value assignments to neighbors. The size of value assignments is K, where K is the number of solutions. In the Comparison phase, the size of the gain message in C-MGM is 1, but the size of gain sets in the other two algorithms is K. Hence, since $max\{1,1\}=1$, $max\{K,K\}=K$, and $max\{K,K\}=K$, the message size of C-MGM, C-PMGM, and C-PDSM are respectively 1, K, and K in one iteration. □

Theorem 4.

In one iteration, the computational overhead of one agent in C-MGM, C-PMGM, and C-PSAM are respectively $K\left|N\right|+1$, $K\left(2\right|N|+1)$, and $2K\left(\right|N|+1)$, where K is the number of solutions and $\left|N\right|$ is the number of neighbors. Further, the overall computational overhead of C-MGM, C-PMGM, and C-PSAM in one iteration are $2K\left|F\right|+\left|A\right|$, $K\left(4\right|F|+|A\left|\right)$, and $K\left(4\right|F|+2|A\left|\right)$, respectively.

Proof. 

We define the number of neighbors of one agent as $\left|N\right|$. For C-MGM, the agent $x_i$ needs to calculate the local utility for K solutions in one iteration, including the basic local utility. In addition, the agent calculates the gain once. Therefore, in one iteration, the computational overhead of one agent in C-MGM is $K\left|N\right|+1$. Since ${\sum }_{x_i\in X}|N_i|=2\left|F\right|$, and the overall computational overhead of C-MGM is ${\sum }_{x_i\in X}K|N_i|+1=2K\left|F\right|+\left|A\right|$.

For C-PMGM, the agent calculates the basic local utility and competing local utility (Algorithm 3: Line 15) for K solutions with neighbors, and the agent calculates the gain set, including K gains. Hence, the computational overhead of one agent in C-PMGM is $2K\left|N\right|+K=K\left(2\right|N|+1)$, the overall computational overhead is $K\left(4\right|F|+|A\left|\right)$.

Compared to C-PMGM, an agent of C-PDSM additionally updates the value of K solutions based on C-PMGM. Therefore, the computational overhead of one agent in C-PDSM in one iteration is $2K\left|N\right|+K+K=2K\left(\right|N|+1)$, and the overall computational overhead is $K\left(4\right|F|+2|A\left|\right)$. □

5. Experimental Results

In this section, we empirically evaluate the solution quality of our three algorithms and the state-of-the-art algorithms, PCD, C-CoCoA, and PCD-LD. Firstly, we set initialization parameters for each algorithm. Specially, we evaluate the performance of C-PDSM with different scaling factors to find the most suitable scaling factor for comparison experiments. Then, we compare the solution quality of the six algorithms when the algorithm runs completely. Finally, we use the runtime to evaluate the performance of all algorithms.

Although C-MGM, C-PMGM, and C-PDSM can use any form of the utility function, we use the utility function in the form of $a{x}^2+bx+cxy+dy+e{y}^2+f$ to fairly show the results for consistency with researchers’ work [38,40], where $a,b,c,d,e$, and f are random numbers in the range $[-5,5]$. Furthermore, we set the domain of each variable to $[-50,50]$ and the number of iterations to 500. It is worth mentioning that we choose the best solutions in C-PMGM and C-PDSM to present comparisons. For each benchmark problem, we independently run each algorithm on 30 different sets of function coefficients ($a,b,c,d,e$, and f) and take the average solution quality as the experimental result. The experiments are carried out on a computer equipped with Intel(R) Core(TM) i5-10500 CPU, 3.10GHz processor, and 8GB RAM.

We choose three benchmark problems and set their parameters: $\left(i\right)$Random Graphs: we use the Erdös-Rényi model [43] to generate random graphs with two different settings: D1: Sparse random graphs ($p=0.1$) and D2: dense random graphs ($p=0.6$), where p represents the probability of connection between two nodes in a graph; $\left(ii\right)$D3: Scale-free Networks: we refer to the Barabási-Albert (BA) [44] model to generate scale-free networks with an initial set of $m_1=10$ connected agents, and at each iteration, a new agent that connected to $m_2=3$ other agents is added to the current network; and $\left(iii\right)$ D4: Small-world Networks: we use the Newman-Watts model [45] to provide small-world networks. The number of nearest nodes is set to 8 and the probability of connection adding is set to 0.1.

5.1. Parameter Configuration

In C-MGM, C-PMGM, and C-PDSM, K represents the size of the solution set, which is also the number of solutions that need to be optimized. Considering the previous work of some researchers using the solution set [37,40], we uniformly set K to 1000. For the scaling factor in C-PDSM, we use different scaling factors to compare the solution quality of C-PDSM on sparse random graphs with 100 agents. Figure 2 shows the convergence curves of C-PDSM with different scaling factors, where an obvious parameter setting is $\omega =1.0$. When $\omega <1.0$, the convergence speed of C-PDSM becomes slower as $\omega$ decreases. Conversely, the solution quality of C-PDSM with different parameters is similar when $\omega >1.0$. However, we can observe that there is a slight superiority in terms of convergence when $\omega =1.6$. Therefore, we set $\omega$ to 1.6 for C-PDSM. In addition, we set the parameters for competing algorithms according to the references [37,39,40].

5.2. Comparison of Solution Quality

We show convergence curves of our algorithms and competing algorithms on three benchmark problems with 100 agents. Since C-CoCoA is a non-iterative algorithm, we use a straight line to represent its solution quality.

Figure 3a and Figure 3b show the solution quality of the six algorithms on sparse and dense random graphs, respectively. It can be seen that the solution quality of our three algorithms is significantly better than that of competing algorithms. Compared to performance on sparse random graphs, our three algorithms require more iterations to improve the solution quality. The most critical reason is that each agent has more neighbors in the dense configuration. Since only one of the two neighboring agents is allowed to change its value in our algorithms, there are fewer agents to improve their local utilities in one iteration. Therefore, our algorithms converge slower. Nevertheless, the solution quality of the three algorithms is superior to competing algorithms at about 100 iterations.

The convergence curves of the six algorithms on scale-free and small-world networks are presented in Figure 3c and Figure 3d, respectively. The performance of our algorithms is similar to that on sparse random graphs. C-MGM can obtain a high-quality solution and converge after a few iterations, and C-PMGM can improve the solution quality as the number of iterations increases. Compared to C-MGM, C-PDSM can converge faster since it uses heuristics to enhance the exploitation of solutions.

Further, we evaluate the solution quality of the six algorithms on each benchmark problem with 80 and 90 agents, respectively. The detailed results are shown in Table 1. We can observe clearly that the solution quality of C-MGM, C-PMGM, and C-PDSM significantly outperforms that of competing algorithms on different benchmark problems with varying numbers of agents, and C-PDSM has the best solution quality among all algorithms.

5.3. Comparison of Algorithms Using Runtime

We compare the best solution quality reached by C-MGM, C-PMGM, C-PDSM, and competing algorithms on each benchmark problem after runtime in 1000 milliseconds (ms).

Table 2 lists the solution quality of algorithms on benchmark problems with different numbers of agents. Since C-MGM uses multiple local utilities to find the maximum gain compared to C-PMGM and C-PDSM, it can quickly find a high-quality solution. In addition, the solution quality of C-MGM outperforms competing algorithms on each benchmark problem except for the comparison with C-CoCoA on dense random graphs, the reason is that C-MGM converges slower due to the increase of neighbors.

6. Conclusion and Future Work

Researchers propose C-DCOPs to model distributed problems with continuous variables as an extension of DCOPs. In this paper, we design three algorithms C-MGM, C-PMGM, and C-PDSM for solving C-DCOPs and prove that they are three anytime algorithms. Since the C-MGM algorithm resets value assignments to expend the search space and chooses the maximum gain, it can obtain high-quality solutions with few iterations. The C-PMGM algorithm extends C-MGM by searching multiple solutions in parallel, which can significantly improve the solution quality. Based on C-PMGM, C-PDSM combines differential search to strengthen the exploitation of solutions, which can effectively improve the convergence speed and the solution quality. Extensive experiments on three benchmark problems show that C-MGM, C-PMGM, and C-PDSM outperform the state-of-the-art C-DCOP algorithms. In future work, we plan to improve the convergence speed of the three algorithms on some dense graphs.