A Reinforcement Learning–Driven Multi-Agent Cooperative Grey Wolf Algorithm for Influence Maximization

1. Introduction

The fast development of the Internet and mobile devices has turned social media platforms like Facebook, Twitter, and Weibo into major routes for the spread of information. The interaction data in large-scale created by these platforms has inspired a considerable amount of research work on social network analysis, such as rumor control [1], impact maximization [2,3,4], community detection [5,6] and viral marketing [7,8]. Among these, influence maximization (IM) is a fundamental problem: given a diffusion model, we want to find a small seed set to maximize the total impact spread in the network. The subject has drawn constant interest due to its theoretical depth and practical importance [9,10].

The IM problem is known to be NP-hard [11]. Early work mostly used greedy algorithms like CELF [12], which provide approximation guarantees but are computationally expensive for large scale networks. Since then, metaheuristic algorithms have been proposed as a realistic alternative, with a good trade-off between solution quality and the computational cost [13].

Different metaheuristic methodologies have been applied to IM. Jiang et al. [14] were the first to propose an influence evaluation function based on EDV, which was optimized via simulated annealing. Other works investigated different algorithms. Tang et al. [15,16] used discretized bat algorithm and shuffled frog-leaping algorithm to maximize the LIE function. Zareie et al. [17] defined an objective based on the entropy of the network and solved the problem using GWO. Fan et al. [18] proposed a discrete Harris Hawks Optimization method. Qiu et al. [19] proposed a local-influence-diminishing search strategy, which leads to LIDDE. Zhu et al. [20] proposed PHEE, a phased hybrid method combining an evolutionary search with random range partition and adaptive simulated annealing to reduce the possibility of falling into local optima. Overall, these research have greatly extended the applicability of metaheuristic approaches to IM.

Among these algorithms, the Grey Wolf Optimizer (GWO) [21] has received much attention due to its few control parameters, conceptually simple mechanism, and powerful global search capacity in continuous optimization. Several improvements have been proposed: Zhang et al. [22] enhanced the exploration mechanism against premature convergence, Yuan et al. [23] integrated elite opposition-based learning with a chaotic k-best gravitational search method to adaptively tune step sizes, Zhang et al. [24] incorporated a scale-free network topology to strengthen interactions between individuals, and Gupta et al. [25] designed a memory-based GWO with crossover and greedy selection to preserve the exploration-exploitation balance.

However, two unique shortcomings of GWO’s continuous-domain techniques are apparent and they are not suitable for discrete combinatorial problems such as IM. First, the shift from exploration to exploitation is inflexible. The control parameter a declines linearly and cannot adapt to the real dynamics of the search process. Achieving the proper balance between global and local search is challenging in the extremely non-convex solution space of IM. Second, the static three-level leadership structure (α, β, δ) constrains the ω wolves into passive following behavior, which also rapidly decreases population diversity and increases the risk of premature convergence.

To address these limitations, hybridization of reinforcement learning (RL) with swarm intelligence algorithms is of considerable interest. Reinforcement Learning (RL) allows an agent to learn a decision policy by interacting with the environment, and its capacity to generate adaptive decisions makes it a perfect companion to metaheuristic search. Goulart et al. [26] integrated RL and PSO for autonomous pH management in electroplating wastewater treatment. Hybrid PSO-Q-learning techniques have also been proved to be useful for mobile robot path planning [27] and underwater localization [28]. Zamfirache et al. [29] used RL with GWO to train policy neural networks, where GWO was employed to tune the weights of the network. Q-learning is used by Ahmad et al. [30] for node selection for IM. EGWO was proposed by [31] by Emary et al. in which each individual is assigned with dynamic exploration parameters learned by RL and neural networks for feature selection and weight optimization. These results in several fields indicate that the coupling of metaheuristic algorithms with learning-based methods is a promising avenue for hard combinatorial optimization.

However, a recurrent issue in applying RL to GWO is the lack of proper state representation. Mostly, the present approaches depend on the single state variable such as the iteration number or the best fitness value, which can not characterize the complex dynamics of population evolution including the change of diversity and convergence tendency. This poor state representation lowers the quality of the learned policies and limits the practical utility of RL-guided search. Inspiration from above observations leads to the proposal of this work, which is a reinforcement learning based multi-role cooperative Grey Wolf Optimizer called Multiple-roles GWO. The primary contributions are: First, we propose a multi-dimensional state vector that jointly represent population diversity, fitness improvement rate and iteration progress, which drives a Q-learning based adaptive search mechanism that dynamically modifies exploration and exploitation decisions in real-time. Second, based on the idea of social division of labor , the exploitation phase is reformulated as a four-role cooperative system consisting of leaders , explorers , followers and losers , each performing a different search strategy to overcome the behavioral homogeneity of traditional GWO .

2. Related Work

2.1. Grey Wolf Optimizer

In 2014, Mirjalili et al. [21] introduced the grey wolf optimizer (GWO), which is inspired by the hunting behavior and social structure of grey wolf packs. The procedure separates the population into four groups that reflect the best, second-best, third-best, and other possible solutions, denoted by α, β, δ, and ω, respectively. The three leader wolves simulate the surrounding, hunting and attacking stages of wolf predation to drive the $\omega$ wolves towards the promising parts of the solution space during the search. (1) randomly initialize a population and the α, β, The optimization process is six-step: (2) the ω wolves adjust their positions relative to the leaders; (3) boundary conditions are imposed to keep all solutions inside the feasible search area; (4) the fitness values are recalculated for the new positions; (5) the leadership roles are redistributed according to the new fitness values; and (6) steps (2)–(5) are repeated until a stopping condition is satisfied.

The position update involves two fundamental formulas. The distance vector D in Eq. (2.1) represents the separation between wolf and leader, while the position update algorithm in Eq. (2.2) advances the wolf towards its prey. The coefficient vectors A and C are given in Eqs. (2.3) and (2.4), where r1 and r2 are random values within the range of [0, 1], and an is the convergence factor that linearly decreases from 2 to 0.

$$\overrightarrow{D}=|\overrightarrow{C}\times \overrightarrow{X_p}(t)-\overrightarrow{X}(t)|$$

(2.1)

$$\overrightarrow{X}(t+1)=\overrightarrow{X_p}(t)-\overrightarrow{A}*\overrightarrow{D}$$

(2.2)

$$\overrightarrow{A}=2\overrightarrow{a}\times r_1-\overrightarrow{a}$$

(2.3)

$$\overrightarrow{C}=2r_2$$

(2.4)

2.2. Q-learning Reinforcement Learning Algorithm

Q-learning is a model-free reinforcement learning technique in which the agent learns an optimal policy by direct contact with the environment. The three main components are states, actions and Q-value function Q(s, a) that predicts the expected cumulative reward from taking action an in state s.

Here’s how the method works. Build a Q-table to store the Q-values for each state-action pair. At each step, the agent chooses an action based on the $\epsilon$-greedy policy, with probability $(1-\epsilon)$ of choosing the action with the highest Q-value and a probability of $\epsilon$ of choosing a random action. The desired action is performed and the corresponding next state and reward are recorded. The Q-value is updated according to Eq. (2.5). This technique is repeated until it converges.

$$Q(s,a)\leftarrow Q(s,a)+\eta [r+\gamma \max Q(s\text{'},a\text{'})-Q(s,a)]$$

(2.5)

3. Reinforcement Learning–Based Multi-Role Grey Wolf Algorithm

In this part, the Multiple-roles GWO is introduced to overcome the two main constraints of the conventional GWO for IM: the rigid search transition mechanism and the static leadership hierarchy. The algorithm is improved in two ways. First, an RL-based multi-dimensional state vector is defined to allow the dynamic evaluation of the population state so that the algorithm can adapt its search behavior according to the real-time feedback. Second, inspired by the principle of social division of labor, the exploitation phase is transformed into a four-role cooperative structure with leaders, explorers, followers and losers, each of which adopts a different search technique. This greatly improves local search coverage.

3.1. Overall Framework of the Algorithm

The overall framework of the proposed algorithm is illustrated in Figure 3.1.

3.2. Population Initialization

A degree-centrality-based initialization technique is utilized to improve the diversity and quality of the initial population. The k nodes with highest degree are chosen to build a base set as indicated in Algorithm 1. Each individual is initialized in a randomized way: a random number chosen evenly from [0,1] decides if each element is taken from this base set or replaced by a non-duplicate node through the Replace() method. Then the location vectors of all individuals are recorded.

3.3. Adaptive Search Mechanism Based on Reinforcement Learning

In the traditional GWO, the search behavior is controlled by a linearly decreased coefficient which has no flexibility to adapt to dynamic needs of discrete solution spaces. For this purpose, we propose a high-level RL-based adaptive phase transition mechanism, which is the decision center of the algorithm to adjust the balance between exploration and exploitation based on the current global search state.

3.3.1. State and Action Space Construction

In reinforcement learning, a state is a representation of information about the environment and is used to make decisions. Most of the existing algorithms use one state variable, e.g. iteration count or best fitness, which is not enough to reflect the complicated dynamic features of the population during the optimization. Such a constraint limits the precision of learned decisions and the possible effectiveness of an algorithm.

To this end we develop a multi-dimensional state vector that gives the agent an overall evaluation of the present search state. The state vector s₁(t) has three indications (Eq. (3.1)):

$$s{1}_i(t)=[D_{pop}(x_i^t),R(t),\varphi (t)]$$

(3.1)

Here, d denotes the population diversity, which serves as a key metric for measuring the degree of variation among individuals in the population. Its computation is given by Eq. (3.2):

$$D_{pop}(x_i^t)=1-{\displaystyle \frac{1}{NK}}{\displaystyle \sum _{i=1}^N\frac{|x_i(t)\cap x_{\alpha }(t)|}{k}}$$

(3.2)

Here, N denotes the population size, $k$ is the seed set size, $x_i(t)$ represents the position vector of individual i, and $x_{\alpha }(t)$ denotes the position vector of the current best solution. This formula measures diversity by computing the average overlap of all individuals with the best solution. A higher $D_{pop}$ value indicates a more dispersed population and greater exploration potential, whereas a lower value suggests population convergence and an increased risk of premature trapping in local optima. Incorporating this parameter enables the agent to actively maintain population diversity and mitigate early convergence.

$$R(t)={\displaystyle \frac{f({x_i}^t)-f({x_i}^{t-1})}{f({x_i}^{t-1})}}$$

(3.3)

A positive and relatively high value of $R(t)$ indicates that the current search strategy is effective and should continue to exploit the solution space. Conversely, when $R(t)$ is close to or less than zero, it suggests that the current strategy is of limited effectiveness, and a change in strategy or a shift toward exploration may be required.

The iteration progress factor is defined as t/tmax, where tmax is the maximum number of iterations. This factor indicates the current stage of the optimization process and enables the agent to formulate strategies consistent with whether the search is in the early, middle, or late phase. The action space is a binary discrete set, allowing the agent to select action a from this set. The action space is formally defined in Eq. (3.4):

$$A_1=[a_1,a_2]$$

(3.4)

Here, a1 represents a global exploration action and a2 a local exploitation action.

3.3.2. Decision and Reward Mechanism

The Q-learning algorithm is used in this framework. Given the current global state s1, the agent selects an action from the action space A1 via the ε-greedy policy. The reward function R1 is tied directly to changes in overall population fitness, rewarding phase transitions that improve solution quality. Through continuous interaction, the algorithm learns under which global conditions a phase transition most effectively advances the search. The action selection rule is defined in Eq. (3.5):

$$A{1}_t={\{}_{A_{random},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}}^{\arg \max Q_1(S{1}_t,a), \mathrm{if} \mathrm{r}\text{>}\epsilon }$$

(3.5)

Here, argmax Q(s1, a) is the action with the highest Q-value for the current state, and arand is a randomly selected action. The variable r is a uniform random number in [0,1], and the exploration rate ε is set to 0.1. When r ≥ ε, the agent exploits accumulated knowledge by selecting the historically best action for state s1; otherwise, with probability ε, a random action is chosen to explore alternative strategies. The Q-values are initialized randomly to ensure sufficient exploration in the early stages.

After executing action A1, the agent receives reward r1 and transitions to the new state s1′. The Q-value is updated according to Eq. (3.6):

$$Q1(S{1}_t,A{1}_t)\leftarrow Q1(S{1}_t,A{1}_t)+\eta [R{1}_{t+1}+\gamma \max Q(S{1}_{t+1},A{1}_{t+1})-Q1(S{1}_t,A{1}_t)]$$

(3.6)

In Eq. (3.6), Q(s₁, A₁) is the Q-value of taking action A₁ in the current state s₁. s₁′ and a′ represent the next state and the next action after executing the current action, respectively. r₁ is the reward obtained from taking the current action, computed according to Eq. (3.7). γ is the discount factor, set to 0.9, which balances the relative importance of immediate and future rewards. η is the learning rate, set to 0.1, which controls the extent to which new information updates the Q-value.

The immediate reward r1 is defined in Eq. (3.7).

$$R{1}_{t+1}={\{}_{-1, \mathrm{otherwise}}^{1{, \mathrm{if} \mathrm{LIE}(\mathrm{x}}_i^{t+1}{\left)\text{>}\mathrm{LIE}\right(\mathrm{x}}_i^t)}$$

(3.7)

3.4. Reinforcement Learning–Based Exploration Phase

In the typical GWO, the leader wolves (α, β, δ) completely drive the search process. The design mirrors the cooperative hunting of wolf packs in stable surroundings. In reality, sudden shifts are often hit by wolf packs, like the habitat changing, prey vanishing to unknown places or natural barriers. In this scenario, wolves tend to cluster around the alpha for protection, this behavior algorithmically translates into fast convergence to the current best solution, limiting the variety of the population and risking stagnation of the search.

To solve this, we take use of the adaptive reactions of wolves when under survival pressure and employ RL to replicate dispersed decision making across the population as the search state degrades. Instead of mindlessly following the leader for all individuals, the environmental signal is the current search state (e.g., diversity level, fitness feedback). To implement this, we train a Q-learning agent to decide the best search actions for every member based on real-time assessments of the population. This helps the population to retain its cooperative behavior, while distributing positions more effectively, conserving diversity, and lowering the risk of stagnation.

3.4.1. Network Node Stratification and Candidate Pool Building

Nodes are stratified according to their structural role in the network, so as to explore the network in different structural areas of the network and to prevent over-concentration on specific node types. Here we apply k-core decomposition. A K-core is the largest subgraph where every node has degree K or above. The coreness of a node is the maximum K such that the node belongs to a K-core. The higher the coreness, the more important the node in the graph structure, and it is often related to stronger connectedness and influence potential.

Based on these topological attributes of the network, all network nodes V are divided into three candidate pools by coreness value to support differentiated guidance in the exploration phase.

The Core Candidate Pool (Pc) consists of nodes with coreness equal to or above the upper quartile (Q3) of all coreness values. These nodes are located in the innermost levels of the network and have a crucial role in the influence propagation. The Middle Candidate Pool (Pm) is composed of nodes with coreness values between the lower quartile (Q1) and the higher quartile (Q3) and represents the main structural backbone of the network. The Periphery Candidate Pool (Pp) consists of nodes that have a coreness value below the lower quartile (Q1) at the periphery of the network. The stratification rule is given directly in Eq. (3.8).

$$\begin{array}{l}{C}_{core}=\{v\in V|Coreness(v)\ge {\mathrm{Q}}_{0.75}(Coreness(v))\}\\ C_{middle}=\{v\in V|Q_{0.25}Coreness(v)\le Coreness(v)\le {\mathrm{Q}}_{0.75}(Coreness(v))\}\\ C_{periphery}=\{v\in V|Coreness(v)\le {\mathrm{Q}}_{0.25}(Coreness(v))\}\end{array}$$

(3.8)

Here, Q1 and Q3 denote the lower and upper quartiles of coreness values, respectively. By constructing these three candidate pools, the algorithm can draw replacement targets from structurally distinct node sets during exploration, achieving more balanced search intensity across both the core and peripheral regions of the network.

3.4.2. Design of State and Action Spaces

$$\mathrm{S}{2}_i(t)=[D_{\alpha }(X_i^t),f(X_i^t)]$$

(3.9)

d is the population diversity defined in Eq. (3.2), which measures the average spatial overlap of all individuals with the current best solution. f is the fitness value of the current individual, indicating the quality of its current position vector.

The action space defines three node-replacement strategies corresponding to the three candidate pools, allowing the algorithm to modify solutions by targeting nodes in different structural layers. The action space is formally defined in Eq. (3.10):

$$A2=[a_{core},a_{\mathrm{mi}d},a_{pre}]$$

(3.10)

Here, a₁ replaces a randomly selected node vᵢ in the current solution with a node from the core candidate pool Pᴄ; a₂ replaces a node with one drawn from the middle candidate pool Pₘ; and a₃ replaces a node with one drawn from the periphery candidate pool Pₚ.

Q-learning is used here to allow each agent to learn which exploration action to take in a given state. We define a bi-objective reward function R2 that credits both direct improvements in individual fitness and contributions to population diversity, achieving a principled balance between exploitation and exploration.

The agent selects action A₂ based on its current state s₂ and the exploration rate ε using the ε-greedy policy. The action selection rule is as follows:

$$A{2}_t={\{}_{A_{random},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{otherwise}}^{\arg \max Q2(S2t,A2), \mathrm{if} \mathrm{r}\text{>}\epsilon }$$

(3.11)

Here, r is a uniform random number in [0,1], argmax Q is the greedy action, and arand is a random action. The exploration rate ε is set to 0.1. When r > ε, the agent selects the action with the highest expected reward according to the Q-table; otherwise, a random action is chosen with probability ε to explore potentially beneficial search directions. The Q-table is initialized with random values to encourage sufficient early exploration.

After executing action A2, the agent receives reward r2 and transitions to state s2′. The corresponding Q-value is updated according to Eq. (3.12).

$$Q2(S{2}_t,A{2}_t)=Q2(S{2}_t,A{2}_t)+\eta [R{2}_{t+1}+\gamma \max Q2(S{2}_{t+1},A{2}_{t+1})-Q2(S{2}_t,A{2}_t)]$$

(3.12)

In Eq. (3.12), Q(s₂, A₂) is the Q-value of taking action A₂ in the current state s₂. s₂′ and a′ denote the next state and action after the current action is executed. r₂ is the reward for taking the current action, computed according to Eq. (3.13). The discount factor γ, set to 0.9, balances the relative weight of current and future rewards. The learning rate η, set to 0.1, determines the degree to which new information influences the Q-value update.

$$R{2}_{t+1}=sign(\mathsf{\Delta }f)+sign(\mathsf{\Delta }D)$$

(3.13)

Here, Δf is the fitness gain and Δd is the change in population diversity.

$$sign(\mathsf{\Delta }x)={\{}_{-1, \mathrm{otherwise}}^{1, \mathrm{if} \mathsf{\Delta }\mathrm{x}\text{>}0}$$

(3.14)

3.5. Multi-role Cooperative Exploitation Phase

In typical GWO, during the exploitation phase, all individuals move along a common trajectory towards the positions of the α, β and δ leaders. The homogeneity of behavior restricts the population power for effective search and often leads to convergence to local optima. We suggest a multi-role cooperative exploitation mechanism based on the theory of social division of labor, which claims that the specialization of activities can improve collective efficiency by dividing subtasks to various persons. The main novelty consists in replacing the rigid hierarchical structure with a dynamic role-assignment mechanism where each participant individually selects the most suitable role with respect to the current search environment.

3.5.1. Building the Role Library and Defining Functions

There are four different jobs outlined. Each role has its own tasks and search technique.

Leader role is leader for heavy exploitation. Leaders leverage the collected information via regular small-step neighborhood searches in the known high-quality solution regions.

The Explorer job is responsible for leading the hunt for new areas. Explorers do broad jump-based searches in the neighborhood of current best solutions to prevent stagnation and discover new candidate optima.

The Follower’s job is to gather information. Followers learn from and imitate successful individuals . Leaders influence followers , but followers also communicate with other followers to produce fresh candidate solutions .

The Role of the Loser adds Diversity. Losers purposefully stray from the present search paradigm and adopt a “survival of the fittest” method to encourage diversity in the population and avoid premature convergence.

Together, these four roles comprise a complete cooperative system ensuring behavioral variation and concerted effort during the exploitation phase.

3.5.2. Role Adaptive Selection Mechanism Based on Q-learning

The RL architecture utilized here is the same as shown in Section 3.4, i.e. state perception, action selection, reward calculation and Q-value update. The main difference is the action space which is now defined over the four roles instead of candidate pools, as stated in Eq. (3.15):

$$A3=[a_{leader},a_{\exp lore},a_{follow},a_{loser}]$$

(3.15)

Through continuous learning, the agent determines which role yields the greatest long-term reward given an individual's current state. Consequently, a high-fitness individual is not always assigned as a Leader; when exploration of new directions is warranted, it may be dynamically assigned as an Explorer. Equally, a low-fitness individual is not inherently a Loser; in contexts requiring information integration, it may be assigned as a Follower.

3.5.3. Multi-role Search Strategies

(a) Leader Search Strategy. Leaders typically occupy high-quality regions of the search space and continue intensive exploitation by refining their current solutions. The position update for a leader is defined in Eq. (3.16):

$$X_i^{t+1}=X_i^t\otimes p$$

(3.16)

Here, Op(·) is the discrete position probability replacement operator, which replaces nodes based on a threshold as described in Eq. (3.17). The time-decaying replacement probability p is given in Eq. (3.18): with probability p, a randomly selected node in the current solution is replaced by one of its randomly chosen neighboring nodes.

$$x_{ij}^{\mathrm{t}+1}={\{}_{replace(x_{ij}^t,{\mathsf{\Gamma }}_1(X_i^t)), \mathrm{otherwise}}^{x_{ij}^t,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{if} \mathrm{p}\le 0.3}$$

(3.17)

Here, N1(x) denotes the first-order neighbors of the current solution x.

$$p=d\text{·}\exp (-{\displaystyle \frac{t}{r_1{\text{·}\mathrm{T}}_{\max }}})$$

(3.18)

Here, λ = 0.8 is the decay coefficient, t is the current iteration, tmax is the maximum number of iterations, and r is a uniform random number in [0,1].

(b) Explorer Search Strategy. Explorers search in the direction spanning the best and worst current solutions, seeking new candidates within this range. This approach builds on prior knowledge of good solutions while actively preventing the algorithm from over-exploiting narrow regions. The position update for an explorer is defined in Eq. (3.19):

$$X_i^{t+1}=(X_i^t\backslash X_{\alpha }^t)\cup random({\mathsf{\Gamma }}_2(X_i^t))$$

(3.19)

Here, x* is the current best solution and x*\ is the best solution with shared nodes removed, allowing the agent to search the remaining local area. The term N2(x) draws randomly from the second-order neighborhood of the current solution for broader exploration.

(c) Follower Search Strategy. Followers draw guidance from leaders and adapt accordingly. However, relying solely on leader guidance can restrict followers' ability to explore independently. To address this, we establish a communication channel between explorers and followers, allowing individuals in different roles to share useful information and improve solution quality. When explorers identify new regions, followers can access these findings through cooperative learning to improve their search coverage.

At each step, each follower samples r2 in [0,1]. If r2 < p2, the follower continues to learn from the leader; otherwise, it learns from another follower to investigate alternative regions. The probability p2 is set to 0.7. The learning target selection is defined in Eq. (3.20):

$$X_{learn}={\{}_{X_{best\_\exp lore}, \mathrm{otherwise}}^{X_{\alpha },{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \mathrm{if} \mathrm{r}}_2{\text{<}\mathrm{p}}_2}$$

(3.20)

The follower then updates its position according to the chosen target to stabilize its search, as defined in Eq. (3.21):

$$X_i^{t+1}=X_i^t\oplus H(r_3(X_i^t)\cap X_{learn})$$

(3.21)

σ(·) is a decision control function whose output is restricted to 0 or 1. For a given parameter θ, σ(θ) is defined as σ(θ) = 1 if θ ≥ τ, and σ(θ) = 0 otherwise, where τ is a threshold factor defined in Eq. (3.22).

$$h_j(x_{ij})={\{}_{1, \mathrm{if} x_{ij}\ge 0.6}^{0, \mathrm{if} x_{ij}<0.6 }$$

(3.22)

The decision operator σ determines whether each element of the current position vector x is retained as a candidate seed node, as defined in Eq. (3.23):

$$x_{ij}^{t+1}={\{}_{\mathrm{Re}place(x_{ij}^t,V), otherwise}^{x_{ij}^t,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\mathrm{if} h_j(x_{ij})=0 }$$

(3.23)

(d) Loser Search Strategy. Losers are individuals with low fitness, analogous to wolves separated from the pack or injured during a hunt. To improve overall search quality in subsequent iterations, these individuals are replaced under a "survival of the fittest" principle by reinitializing them in regions more conducive to effective search. The replenishment process is defined in Eq. (3.24):

$$X_i^{t+1}=(X_{\alpha }^t-X_i^t)\cup random(\mathrm{deg}ree(X_i^t))$$

(3.24)

Here, x∩x* retains the nodes shared between the current solution and the best solution x*. vₘₐˣ₋ᵈᵉᴳ is the node with the highest degree centrality in the social network, and random() denotes a uniformly random draw from the set of high-degree nodes used for replenishment.

4. Experiments

The proposed algorithm was implemented in Python and executed on a machine equipped with an Intel Core i7-8750H CPU running Windows 10, with 16 GB of RAM.

4.1. Datasets

Experiments were conducted on six real-world social networks. The PGP (Pretty Good Privacy) network models a social sharing environment for the exchange of confidential information. NetHEPT and GrQc are co-authorship networks capturing collaborations among researchers across multiple academic disciplines. CondMat and CaAstroph are scientific collaboration networks drawn from papers submitted to the Condensed Matter and Astrophysics categories of ArXiv, respectively. p2p-Gnutella31 is a peer-to-peer file-sharing network. Table 4.1 summarizes the structural statistics of the six networks.

4.2. Comparative Algorithms

The proposed algorithm was evaluated with six baseline approaches. The proposed algorithm is Multiple-roles GWO and it is performed for 150 iterations. LIDDE [19] is a differential evolution algorithm with a local-influence-diminishing objective function (EDIV), where the three weighting factors $\alpha$, $\beta$, $\gamma$ are all set to 1 and all other parameters are the same as in the original study. CELF [12] is a greedy algorithm based on the submodularity and lazy forward evaluation to rank the nodes based on the marginal benefit; the effect is calculated based on 10000 Monte Carlo simulations . PHEE [20] is a hybrid method combining random-range-based differential evolution with adaptive simulated annealing, with λ = 0.7, p ∈ [0.1, 0.5], mp = 0.1, and cp = 0.6. CoreQ [30] combines K-core decomposition and Q-learning for seed selection, with RL parameters η = 0.1, γ = 0.9, ε = 0.5. DPSO [32] is a discrete version of PSO decomposing and reconstructing search behavior based on network structure, with learning factors c₁ = c₂ = 2, inertia weight 0.8, and a maximum of 100 iterations. GWIM [17] uses an entropy based function to measure the influence of nodes. All values are set according to the original paper.

4.3. Parameter Analysis

In this section, we investigate the effect of important parameters on the performance of Multiple-roles GWO. All the experiments were conducted on the genuine social networks with the seed set size as 50. The findings were averaged over 30 independent runs for statistical reliability.

4.3.1. Population Number

With RL parameters fixed at η = 0.1, γ = 0.9, ε = 0.1 and tmax = 150, we examined the effect of different population sizes. This is demonstrated in Figure 4.1 (a) that the population improvement from 5 to 10 leads to a significant improvement of LIE function value for the six networks . Increasing further to 20, 50 or 100 did not produce any significant incremental improvement in LIE but a large increase in running time, as illustrated in Figure 4.1(b). Therefore, a population size of 10 was chosen to provide a balance between efficiency and effectiveness.

4.3.2. Number of Iterations

With population size fixed at 10 and RL parameters set to η = 0.1, γ = 0.9, ε = 0.1, we examined the effect of the maximum iteration count tmax. Figure 4.2(a) shows that the LIE value improved steadily as tmax increased, stabilizing at tmax = 150 with no notable improvement thereafter. Running time increased with iteration count but remained within a reasonable range, as shown in Figure 4.2(b). Considering both convergence speed and computational cost, tmax was set to 150.

4.3.3. Q-learning Parameter Optimization

Since Q-learning performance depends on hyperparameters including the learning rate η, discount factor γ, and exploration rate ε, parameter tuning experiments were conducted on three real-world networks of varying sizes. The seed set size was fixed at 50 and results were averaged over 30 independent runs. Values of η, γ, and ε from {0.1, 0.5, 0.9} were tested over 150 iterations. As shown in Figure 4.3, Figure 4.4 and Figure 4.5, the best configuration is η = 0.1, γ = 0.9, ε = 0.1.

4.4. Ablation Study

An ablation study was conducted by varying the seed set size k to evaluate the contribution of the proposed components. The fitness values of Multiple-roles GWO, DGWO (the standard discrete Grey Wolf Optimizer), and DPSO were recorded and compared.

Figure 4.6 compares fitness values across methods under the IC model with propagation probability p = 0.01 and varying seed set sizes k. All algorithms show increasing fitness with k, with Multiple-roles GWO consistently outperforming the others.

4.5. Influence Propagation

To thoroughly evaluate the Multiple-roles GWO on the IM problem, systematic comparisons were conducted on six real-world social networks under the Independent Cascade (IC) model, benchmarked against both classical and state-of-the-art algorithms. Propagation probabilities were set to p = 0.01 and p = 0.05, and seed set sizes were varied from 10 to 50. Influence was estimated using 10,000 Monte Carlo simulations, and all results are averaged over 30 independent runs to control for randomness.

Figure 4.7 and Figure 4.8 show the influence propagation performance of all comparison methods on six social networks under propagation probability 0.01 and 0.05 correspondingly. The suggested Multiple-roles GWO has a better overall performance, achieving the influence spread far better (or comparable to) existing heuristic algorithms in most of the testing circumstances. The performance of numerous-roles GWO approaches to that of CELF, the greedy algorithm that guarantees the maximum accuracy, in numerous scenarios, indicating the strong optimization potential of the suggested technique.

In small-scale networks (Figure 4.7(a)-(c)), the number of nodes and edges are limited, the search space is relatively confined, and most algorithms can successfully find near-optimal seed sets and the differences in influence spread are not significant. For large scale networks (Figure 4.7 (d)–(f)) the structural complexity increases rapidly and the search space increases exponentially. In such a situation, Multiple-roles GWO has a clear advantage: it greatly beats other heuristic algorithms and reduces the gap with CELF considerably. CELF has great accuracy by thorough examination, but its computing overhead makes it unfeasible for large-scale applications. Multiple-roles GWO provides a better trade-off between solution quality and computational cost as compared to LIDDE, CoreQ, and PHEE. GWIM suffers from the estimate bias inherent in entropy-based calculations, while DPSO is prone to local optima despite its semi-greedy technique.

4.6. Execution Time

In practical IM applications, the efficiency of the calculation is as crucial as the quality of the optimization, especially for large-scale networks, where scalability is a key issue. The running times were recorded for k = 50 seed set size on six real-world networks with p = 0.01 and p = 0.05, averaged over 30 separate trials. Times are presented in seconds on a logarithmic scale to illustrate variations of orders of magnitude (Figure 4.9).

Figure 4.9 demonstrates a distinct stratification in the running times. CoreQ is the fastest since its heuristic initialization through RL-based search reduces the amount of evaluations needed. PHEE, GWIM and LIDDE are in the middle tier. LIDDE has a relative greater cost in this tier since it needs to analyze the influence of individuals to all nodes of the network. Multiple-roles GWO has a similar computational level as LIDDE. DPSO is slower since it tends to spend several iterations in unpromising regions before finding good candidates. CELF has the highest cost due to the repeated Monte Carlo simulations to calculate marginal gain. Multiple-roles GWO yields good efficiency-effectiveness trade-off with competitive computing cost and same or better influence distribution over most networks.

4.7. Analyse statistique

To validate the statistical significance of the performance improvements, pairwise statistical comparisons between Multiple-roles GWO and each of the other six baseline algorithms among the six networks were carried out by the paired Wilcoxon signed-rank test at the significance threshold α = 0.05. Each comparative sample was obtained by averaging the results of 30 independent runs and evaluated using SPSS. The findings are shown in Table 4.2, where N⁺ and N⁻ represent the number of networks in which the proposed method performed better and worse than the baseline, respectively. The Z and p values show the size of the difference and the statistical significance of the difference, respectively, with a p value less than .05 being statistically significant.

The multiple-roles GWO outperforms PHEE, CoreQ, DPSO and GWIM significantly in statistical terms for all the six networks, somewhat better than LIDDE, and on par with CELF. These results validate the efficiency and robustness of the suggested optimization approach.

5. Conclusions

In this paper, we suggest a Multi-roles GWO, which is a Grey Wolf Optimizer based on reinforcement learning and role-based collaboration to solve the influence maximization problem in social networks. The program provides a multi-dimensional state perception mechanism for adaptive search, a Q-learning based dynamic phase transition technique and a four-role cooperative exploitation framework. These components together enhance search efficiency, maintain population variety and reduce early convergence. The simulation results on six real-world social networks reveal the superiority of Multiple-roles GWO over regular discrete GWO, DPSO, PHEE, CoreQ and GWIM, substantial performance over LIDDE, and equivalent performance with CELF, and computational tractability.

Future work will be dedicated to the application of the Multiple-roles GWO to more demanding scenarios, e.g., dynamic networks and multi-objective optimization. Further refinements to the adaptive decision-making process may be made through the integration of principles from cognitive research such as attention mechanisms and metacognitive monitoring. We will also talk about other sectors of application such as social network marketing and public opinion management.