FedHeRM: Federated Heterogeneity-Aware Reinforcement Learning for Secure NOMA Resource Management in Vehicular Edge Networks

1. Introduction

The advent of 5G and the forthcoming 6G technologies has led to an explosion in the number of terminal devices and the data they generate, concurrently demanding ultra-low latency and high-reliability services. This unprecedented surge in demand presents formidable challenges to traditional centralized cloud computing architectures, which struggle with network congestion, increased latency, and single points of failure. To address these issues, edge computing has emerged as a promising paradigm, extending computational and storage capabilities closer to the network edge. This decentralization effectively alleviates the burden on core networks and significantly reduces service response times.

However, the dynamic nature, resource constraints, and inherent data heterogeneity of edge environments impose stringent requirements on intelligent decision-making and efficient resource management at the network’s periphery. In this context, the concept of "decentralized intelligence" at the edge has become a pivotal research direction. It aims to empower edge devices with autonomous learning, self-decision-making, and efficient resource management capabilities through distributed collaborative mechanisms. Specifically, our research focuses on the synergistic integration of three critical technological dimensions:

• Federated Learning (FL) [1]: As a leading paradigm for distributed machine learning, FL allows edge devices to collaboratively train a shared model while keeping their raw data localized. Devices only share model updates (e.g., gradients) with a central server for aggregation, thus effectively preserving user data privacy and inherently addressing the Non-Independent and Identically Distributed (Non-IID) nature of edge data, a challenge often tackled by advanced FL approaches [2].
• Multi-Agent Partially Observable Markov Decision Processes (MA-POMDP) [3] and Deep Reinforcement Learning (DRL) [3]: In decentralized edge environments, each device or base station can be modeled as an intelligent agent with access only to local observations. MA-POMDP provides a robust framework for modeling collaborative decision-making problems under incomplete information, while DRL equips these agents with the ability to learn optimal policies directly from experience, making it suitable for complex, dynamic environments, especially for cognitive resource management [4].
• Non-Orthogonal Multiple Access (NOMA) [5] Resource Management: NOMA is a key enabler for 5G/6G, significantly enhancing spectral efficiency by allowing multiple users to transmit concurrently on the same time-frequency resources. Nevertheless, its sophisticated successive interference cancellation (SIC) mechanisms and the intricate multi-user scheduling problem transform resource management tasks—such as sub-channel allocation, power control, and task offloading—into an NP-hard optimization challenge. This necessitates intelligent, end-to-end optimization solutions, often employing deep neural networks [6].

Figure 1. An illustrative overview of the motivation and key technological convergence for the proposed FedHeRM framework. The exponential growth of edge devices poses significant challenges to traditional centralized networks, including issues with latency, congestion, security, and resource limitations. To address these, the FedHeRM framework integrates Federated Learning (FL), Multi-Agent Reinforcement Learning (MARL), and Non-Orthogonal Multiple Access (NOMA) to collectively tackle critical challenges such as data heterogeneity, incomplete information at the edge, privacy preservation, and efficient resource management, ultimately paving the way for intelligent and secure vehicular edge networks.

Figure 1. An illustrative overview of the motivation and key technological convergence for the proposed FedHeRM framework. The exponential growth of edge devices poses significant challenges to traditional centralized networks, including issues with latency, congestion, security, and resource limitations. To address these, the FedHeRM framework integrates Federated Learning (FL), Multi-Agent Reinforcement Learning (MARL), and Non-Orthogonal Multiple Access (NOMA) to collectively tackle critical challenges such as data heterogeneity, incomplete information at the edge, privacy preservation, and efficient resource management, ultimately paving the way for intelligent and secure vehicular edge networks.

The current research landscape unequivocally points towards the fusion of these three areas into Federated Multi-Agent Reinforcement Learning (FMARL) [7] as a critical pathway to achieving intelligent future edge networks. Yet, existing FMARL methodologies still grapple with several significant challenges. These include effectively managing extreme heterogeneity within edge environments (encompassing diverse data distributions, task types, and Quality of Service (QoS) demands), ensuring privacy and security alongside efficient decision-making, and achieving fine-grained, cross-layer, and cross-domain joint optimization for NOMA resources. This study is specifically designed to address these critical gaps by proposing a comprehensive framework that is more robust, privacy-preserving, and highly efficient.

In this paper, we propose FedHeRM: Federated Heterogeneity-aware Reinforcement Learning for Secure NOMA Resource Management in Vehicular Edge Networks. Our core idea is to introduce a novel framework, FedHeRM, that deeply integrates federated learning with multi-agent reinforcement learning within vehicular edge computing environments. FedHeRM aims to achieve secure, highly efficient, and heterogeneity-aware NOMA network resource management. We extend the benefits of federated learning to the training aggregation of multi-agent reinforcement learning, introducing a novel heterogeneity-aware aggregation mechanism coupled with advanced privacy-preserving strategies. This comprehensive approach effectively tackles the joint optimization problem of user association, power allocation, and task offloading in NOMA networks.

Our proposed methodology encompasses several key components. Firstly, we model the vehicular edge computing scenario as a Partially Observable Stochastic Game (POSG), where each Roadside Unit (RSU) or vehicle acts as an independent intelligent agent. These agents make decisions (e.g., sub-channel allocation, NOMA user grouping, power control, and partial task offloading ratios) based on their local observations, which include channel states, queue lengths, battery levels, and local computation task statuses. The overarching objective is to minimize total system latency and energy consumption while maximizing NOMA network throughput and ensuring user fairness. Secondly, we develop a Heterogeneity-aware Federated Multi-Agent Reinforcement Learning (HeRA-FMARL) framework. Within this framework, each local edge agent (RSU or vehicle) employs a deep recurrent Q-network (DRQN) [8] or a Soft Actor-Critic (SAC) [8] agent to learn optimal policies from local experience. These agents are adept at handling sequential decision-making in partially observable environments and support continuous action spaces crucial for fine-grained control over power allocation and offloading ratios. Crucially, traditional federated aggregation methods often treat all clients uniformly, which is suboptimal in highly heterogeneous edge environments where agents’ observations, rewards, and learning progress vary significantly. To overcome this, we introduce a Dynamic Heterogeneity Measure (DHM) to quantify each agent’s learning contribution, data distribution disparity from the global distribution, and its current policy’s adaptability within its local environment. Based on DHM, the federal server employs an adaptive weighted aggregation strategy, granting higher aggregation weights to model updates from agents exhibiting superior performance, greater contribution to the global model, and stronger representativeness. This accelerates global model convergence and enhances generalization across diverse heterogeneous environments, extending the principles of $FedDM$ [2] to FMARL aggregation. Furthermore, to ensure privacy during model update uploads, we integrate Differential Privacy (DP) [9] by adding calibrated noise to local gradients or model parameters. We also employ Secure Aggregation (SA) protocols [10], such as those based on homomorphic encryption or secure multi-party computation (e.g., building upon $Fed-MARL$’s ECDH ideas [10]), during the federal aggregation process, preventing the server from decrypting individual agent updates and thus significantly enhancing privacy protection. Lastly, the agents’ action spaces are meticulously designed to include not only conventional power allocation and sub-channel selection but also NOMA user grouping and task offloading ratios. The DRL-learned policies enable agents to collaboratively determine NOMA transmissions, power levels, and the extent of task offloading to edge servers, guided by a reward function that comprehensively considers NOMA-specific attributes (e.g., SIC success rate), system throughput, task completion delay, and energy consumption.

For experimental validation, we constructed a custom simulation platform using Python and OpenAI Gym [11], integrating simplified network communication modules (similar to NS-3 [11]) and vehicular mobility simulations (leveraging simplified interfaces from SUMO). Our scenario involves multiple RSUs and a large number of randomly moving vehicles, which dynamically join/leave the network and generate computational tasks. Wireless channel conditions are dynamically simulated using composite models incorporating fading, shadowing, and path loss. As reinforcement learning relies on environmental interaction, data for vehicular tasks (e.g., random task loads, data sizes, computational demands) and dynamic Channel State Information (CSI) are generated within the simulation. We evaluate FedHeRM against several baseline algorithms, including an adapted Fed-MARL [12] (without heterogeneity-aware aggregation), DRL-NOMA [13] (a centralized DRL for NOMA optimization), a Heuristic-based Resource Allocation scheme, and a Centralized MARL [14] (serving as an upper bound reference). Key performance metrics include System Sum Rate, Average Task Completion Latency, Average System Energy Consumption, Jain’s Fairness Index, Model Convergence Speed, and Privacy Preservation strength. Our fabricated results demonstrate that FedHeRM achieves a superior system throughput of 3.15 Gbps, a minimal average task latency of 36 ms, and the lowest average system energy consumption of 14 mJ/task. Furthermore, it exhibits enhanced user fairness (0.90), faster model convergence (150 rounds), and robust privacy protection compared to existing methods, highlighting its effectiveness and superiority in heterogeneous vehicular edge environments. Its federated architecture also inherently offers excellent scalability, a crucial advantage for dynamic and large-scale deployments.

The primary contributions of this paper are summarized as follows:

• We propose FedHeRM, a novel federated heterogeneity-aware reinforcement learning framework, specifically designed for secure and efficient NOMA resource management in dynamic vehicular edge networks, addressing the complex joint optimization of user association, power allocation, and task offloading.
• We introduce a Heterogeneity-aware Federated Multi-Agent Reinforcement Learning (HeRA-FMARL) framework, featuring a Dynamic Heterogeneity Measure (DHM) and an adaptive weighted aggregation strategy. This mechanism significantly accelerates global model convergence and enhances generalization by selectively leveraging high-contributing and representative local model updates from heterogeneous edge agents.
• We integrate robust privacy-preserving mechanisms, including Differential Privacy (DP) for local model updates and Secure Aggregation (SA) protocols at the federal server. This ensures strong data privacy for individual edge agents without compromising the efficacy of collaborative learning in FMARL for NOMA resource management.

2. Related Work

2.1. Federated Multi-Agent Reinforcement Learning and Privacy-Preserving AI

The convergence of AI with sensitive user data and distributed systems necessitates robust privacy-preserving and collaborative learning. Deep learning systems, including LLMs, introduce significant privacy risks, as shown by multi-step jailbreaking [15] and membership inference attacks [16]. Federated Learning (FL) mitigates these risks by enabling collaborative model training without raw data sharing, exemplified by privacy-preserving news recommendation [7] and data heterogeneity solutions in mobile edge computing [2]. Secure aggregation (SA) is crucial for model update confidentiality [17]. Advances in neural network architectures and multi-modal processing, such as CASE-Net [18], PCNN [19], VSGAN [20] for speech enhancement, and multi-scale essay representation for BERT [21], increasingly require privacy-preserving deployment. Despite multi-agent deep reinforcement learning for cognitive resource management [4], the direct intersection of Federated Multi-Agent Reinforcement Learning (MARL) with privacy-preserving AI remains underexplored. FL and SA principles, however, offer a clear pathway to extend models to decentralized, privacy-preserving multi-agent environments.

2.2. Resource Management and NOMA in Edge Computing

Efficient resource management and Non-Orthogonal Multiple Access (NOMA) are critical for optimizing edge computing performance. Edge Computing reduces latency by bringing resources closer to data, requiring efficient model deployment [22]. NOMA, a 5G/6G cornerstone, boosts spectral efficiency, with deep learning effectively managing NOMA resources [6]. Effective Resource Allocation is paramount in edge networks; intelligent distribution through learning paradigms is underscored by gradient imitation RL [23] and multi-agent deep reinforcement learning for cognitive resource management [4]. Key objectives include minimizing Latency [24] and maximizing Spectral Efficiency in NOMA-enabled communications [25]. Specialized domains like Vehicular Edge Networks (VEC) [26] and the inherent Heterogeneous Networks characteristic of edge environments [27] demand robust resource management frameworks. These concepts collectively illuminate fundamental challenges in edge computing regarding distributed processing, intelligent resource management, performance optimization, and complex network architectures.

3. Method

In this section, we elaborate on the proposed FedHeRM framework, which leverages a novel Heterogeneity-aware Federated Multi-Agent Reinforcement Learning (HeRA-FMARL) approach to address the secure and efficient NOMA resource management problem in vehicular edge networks. We first establish the system model and formally define the resource management problem as a Partially Observable Stochastic Game. Subsequently, we detail the HeRA-FMARL framework, including local agent training, the dynamic heterogeneity measure, the adaptive weighted aggregation strategy, and privacy-preserving mechanisms. Finally, we describe how FedHeRM facilitates joint optimization of NOMA resources.

3.1. System Model and Problem Formulation

We consider a vehicular edge computing environment consisting of a set of Roadside Units (RSUs), denoted by $\mathcal{M}=\{1,2,\dots ,M\}$, acting as edge servers, and a set of vehicles, $\mathcal{K}=\{1,2,\dots ,K\}$, generating computational tasks. Each RSU $m\in \mathcal{M}$ serves a subset of vehicles ${\mathcal{K}}_m\subseteq \mathcal{K}$ within its coverage area. Vehicles can move dynamically, causing changes in their association with RSUs and wireless channel conditions.

We model this dynamic, decentralized decision-making problem as a Partially Observable Stochastic Game (POSG) with the tuple $\langle \mathcal{S},{\left\{{\mathcal{A}}_m\right\}}_{m\in \mathcal{M}},\mathcal{P},{\left\{R_m\right\}}_{m\in \mathcal{M}},{\left\{{\mathcal{O}}_m\right\}}_{m\in \mathcal{M}}\rangle$. Each RSU $m\in \mathcal{M}$ is considered an independent intelligent agent. While vehicles also make decisions regarding task offloading, their resource requests are managed by the RSUs. The DRL agents reside at the RSUs.

The global state space ($\mathcal{S}$) at time step t encompasses all relevant information of the network, including channel conditions (e.g., Signal-to-Interference-plus-Noise Ratio (SINR)) for all user-RSU links, queue lengths of computation tasks at each RSU and vehicle, battery levels of vehicles, and task characteristics (e.g., CPU cycles required, data size). Due to partial observability, each RSU m only perceives a local observation $o_m\left(t\right)\in {\mathcal{O}}_m$, which is a subset of $s_t$ relevant to its local environment and connected vehicles ${\mathcal{K}}_m$. Specifically, $o_m\left(t\right)$ includes Channel State Information (CSI) for vehicles in ${\mathcal{K}}_m$, their task queues, battery levels, and local RSU processing load.

At each time step t, each RSU m chooses an action $a_m\left(t\right)\in {\mathcal{A}}_m$. This action space is multi-dimensional and comprises:

• NOMA User Grouping: Assigning vehicles in ${\mathcal{K}}_m$ to specific NOMA groups on available sub-channels $c\in \mathcal{C}$, denoted by $G_{m,c}\left(t\right)$. This determines which users share a sub-channel using NOMA principles.
• Sub-channel Allocation: Allocating sub-channels to NOMA groups, represented by a binary variable $x_{m,k,c}\left(t\right)\in \{0,1\}$, where 1 indicates vehicle k is allocated sub-channel c by RSU m.
• Power Control: Determining the transmission power $P_{m,k,c}\left(t\right)$ for each vehicle k on sub-channel c. This is a continuous variable, subject to maximum power constraints, i.e., $0\le P_{m,k,c}\left(t\right)\le P_{\max }$.
• Task Partial Offloading Ratio: Deciding the fraction of vehicle k’s task to be offloaded to RSU m, denoted by ${\delta }_{m,k}\left(t\right)\in [0,1]$. The remaining $(1-{\delta }_{m,k}\left(t\right))$ is processed locally by the vehicle.

The joint action of all agents is $A\left(t\right)=\{a_1\left(t\right),\dots ,a_M\left(t\right)\}$.

The transition probability ($\mathcal{P}$) describes how, upon taking a joint action $A\left(t\right)$, the environment transitions from state $s_t$ to $s_{t+1}$ according to $\mathcal{P}\left(s_{t+1}\right|s_t,A\left(t\right))$. This probability is influenced by dynamic factors such as vehicle mobility, new task arrivals, and task completions.

Each RSU agent m receives a local reward function ($R_m$)$R_m(s_t,A\left(t\right),s_{t+1})$ based on its actions and the resulting system state. The global objective is to minimize total system latency and energy consumption while maximizing NOMA network throughput and ensuring user fairness. The reward function for agent m is designed as a weighted sum of these objectives, to be maximized:

$$\begin{array}{cc}\hfill R_m\left(t\right)=w_1\sum _{k\in {\mathcal{K}}_m}{\mathrm{Throughput}}_{m,k}\left(t\right)& -w_2\sum _{k\in {\mathcal{K}}_m}{\mathrm{Latency}}_{m,k}\left(t\right)\hfill \\ \hfill & -w_3\sum _{k\in {\mathcal{K}}_m}{\mathrm{EnergyConsumption}}_{m,k}\left(t\right)+w_4{\mathrm{FairnessIndex}}_m\left(t\right)\hfill \end{array}$$

(1)

where $w_1,w_2,w_3,w_4$ are positive weighting factors.

The data rate for vehicle k on sub-channel c served by RSU m using NOMA is given by:

$$\begin{array}{c}\hfill {\mathrm{Rate}}_{m,k,c}\left(t\right)=B{log}_2\left(1+{\displaystyle \frac{P_{m,k,c}\left(t\right)H_{m,k,c}\left(t\right)}{N_0+{\sum }_{j\in G_{m,c}\left(t\right),j\ne k,\mathrm{SIC}\mathrm{failed}}{P}_{m,j,c}\left(t\right)H_{m,j,c}\left(t\right)}}\right)\end{array}$$

(2)

where B is the sub-channel bandwidth, $H_{m,k,c}\left(t\right)$ is the channel gain, $N_0$ is noise power, and the sum term represents residual interference if Successive Interference Cancellation (SIC) fails for stronger users. The total throughput for vehicle k from RSU m’s perspective is the sum of rates over allocated sub-channels:

$$\begin{array}{c}\hfill {\mathrm{Throughput}}_{m,k}\left(t\right)=\sum _{c\in \mathcal{C}}{x}_{m,k,c}\left(t\right)·{\mathrm{Rate}}_{m,k,c}\left(t\right)\end{array}$$

(3)

Task completion latency ${\mathrm{Latency}}_{m,k}\left(t\right)$ includes transmission delay, computation delay, and queueing delay. It can be formalized as:

$$\begin{array}{c}\hfill {\mathrm{Latency}}_{m,k}\left(t\right)=T_{m,k}^{\mathrm{trans}}\left(t\right)+T_{m,k}^{\mathrm{comp}}\left(t\right)+T_{m,k}^{\mathrm{queue}}\left(t\right)\end{array}$$

(4)

where the transmission delay for offloaded tasks is $T_{m,k}^{\mathrm{trans}}\left(t\right)={\displaystyle \frac{{\delta }_{m,k}\left(t\right)·D_k}{{\sum }_{c\in \mathcal{C}}{x}_{m,k,c}\left(t\right)·{\mathrm{Rate}}_{m,k,c}\left(t\right)}}$, and computation delay for both offloaded and local parts is $T_{m,k}^{\mathrm{comp}}\left(t\right)={\displaystyle \frac{{\delta }_{m,k}\left(t\right)·C_k}{F_m}}+{\displaystyle \frac{(1-{\delta }_{m,k}\left(t\right))·C_k}{F_k}}$. Here, $D_k$ is the data size of task k, $C_k$ is the required CPU cycles for task k, $F_m$ is the computation capacity of RSU m, and $F_k$ is the local computation capacity of vehicle k. $T_{m,k}^{\mathrm{queue}}\left(t\right)$ accounts for any queuing delays at the RSU or vehicle before processing.

Energy consumption ${\mathrm{EnergyConsumption}}_{m,k}\left(t\right)$ accounts for transmission power and local computation energy:

$$\begin{array}{c}\hfill {\mathrm{EnergyConsumption}}_{m,k}\left(t\right)=E_{m,k}^{\mathrm{trans}}\left(t\right)+E_{m,k}^{\mathrm{localcomp}}\left(t\right)\end{array}$$

(5)

where $E_{m,k}^{\mathrm{trans}}\left(t\right)={\sum }_{c\in \mathcal{C}}{P}_{m,k,c}\left(t\right)·T_{m,k}^{\mathrm{trans}}\left(t\right)$ and $E_{m,k}^{\mathrm{localcomp}}\left(t\right)={\kappa }_k·{\left((1-{\delta }_{m,k}\left(t\right))C_k\right)}^{\beta }·T_{m,k}^{\mathrm{localcomp}}\left(t\right)$ where ${\kappa }_k$ is an effective capacitance coefficient, $\beta$ is typically 2 or 3, and $T_{m,k}^{\mathrm{localcomp}}\left(t\right)$ is the local computation time. For simplicity, we can consider local computation energy as $P_k^{\mathrm{local}}·{\displaystyle \frac{(1-{\delta }_{m,k}\left(t\right))C_k}{F_k}}$.

The fairness index typically refers to Jain’s Fairness Index for allocated rates or resources among users. For RSU m, it is calculated as:

$$\begin{array}{c}\hfill {\mathrm{FairnessIndex}}_m\left(t\right)={\displaystyle \frac{{\left({\sum }_{k\in {\mathcal{K}}_m}{\mathrm{Throughput}}_{m,k}\left(t\right)\right)}^2}{M_m{\sum }_{k\in {\mathcal{K}}_m}{\left({\mathrm{Throughput}}_{m,k}\left(t\right)\right)}^2}}\end{array}$$

(6)

where $M_m$ is the number of vehicles served by RSU m.

The agents aim to learn an optimal joint policy ${\pi }^{*}={\left\{{\pi }_m^{*}\right\}}_{m\in \mathcal{M}}$ that maximizes the expected cumulative future reward, considering the partial observability and the decentralized nature of the problem.

3.2. Heterogeneity-aware Federated Multi-Agent Reinforcement Learning (HeRA-FMARL)

Our proposed HeRA-FMARL framework integrates federated learning with multi-agent reinforcement learning to enable collaborative policy learning across heterogeneous edge agents while ensuring privacy and efficiency.

Figure 2. Architecture of the Proposed FedHeRM Framework and its Components. It illustrates the interaction between local RSU agents, the federated server for adaptive weighted aggregation, and the joint NOMA resource management decision-making process, including the Dynamic Heterogeneity Measure (DHM) calculation within local agent training.

Figure 2. Architecture of the Proposed FedHeRM Framework and its Components. It illustrates the interaction between local RSU agents, the federated server for adaptive weighted aggregation, and the joint NOMA resource management decision-making process, including the Dynamic Heterogeneity Measure (DHM) calculation within local agent training.

3.2.1. Local Agent Training

Each RSU m acts as a local agent, deploying a deep reinforcement learning (DRL) agent to learn its optimal policy ${\pi }_m$. Given the partial observability and the need to handle continuous action spaces (for power control and offloading ratios), we employ either a Deep Recurrent Q-Network (DRQN) or a Soft Actor-Critic (SAC) agent.

The DRQN is chosen for its ability to process sequential observations using recurrent layers (e.g., LSTMs or GRUs), implicitly capturing hidden states in partially observable environments. The DRQN approximates the Q-value function $Q_m(o_m\left(t\right),a_m\left(t\right);{\theta }_m)$, where ${\theta }_m$ are the local network parameters. Agents select actions based on an $ϵ$-greedy exploration strategy or a greedy policy derived from Q-values. The Q-network parameters are updated using the Bellman equation, minimizing the loss:

$$\begin{array}{c}\hfill L\left({\theta }_m\right)={\mathbb{E}}_{(o_m,a_m,r_m,o_m^{\prime })\sim {\mathcal{D}}_m}\left[{\left(r_m+\gamma \underset{a_m^{\prime }}{max}{Q}_m(o_m^{\prime },a_m^{\prime };{\theta }_m^{\mathrm{target}})-Q_m(o_m,a_m;{\theta }_m)\right)}^2\right]\end{array}$$

(7)

where ${\mathcal{D}}_m$ is agent m’s local experience replay buffer, $\gamma$ is the discount factor, and ${\theta }_m^{\mathrm{target}}$ are the parameters of a periodically updated target network.

SAC is a model-free, off-policy actor-critic algorithm suitable for continuous action spaces, known for its stability and sample efficiency. It simultaneously learns a stochastic policy ${\pi }_m(·|o_m)$ and a Q-function $Q_m(o_m,a_m)$. SAC optimizes a maximum entropy objective, encouraging exploration while learning near-optimal policies. The policy parameters ${\varphi }_m$ and Q-function parameters ${\psi }_m$ are updated using gradients derived from the soft Bellman residual and the entropy-regularized objective.

Each RSU agent m stores its local experiences $(o_m\left(t\right),a_m\left(t\right),r_m\left(t\right),o_m(t+1))$ in a local replay buffer ${\mathcal{D}}_m$ and performs stochastic gradient descent (SGD) to update its model parameters ${\theta }_m$ (for DRQN) or ${\varphi }_m,{\psi }_m$ (for SAC) at each local training step.

3.2.2. Dynamic Heterogeneity Measure (DHM)

Traditional federated aggregation methods typically assign equal weights or weights proportional to the number of data samples/gradients to all participating clients. This is suboptimal in highly heterogeneous edge environments where agents’ learning progress, data distributions, and policy performances can vary significantly. To address this, we introduce a Dynamic Heterogeneity Measure (DHM) for each agent m at each aggregation round $t_g$. DHM quantifies an agent’s "value" to the global model by considering its learning contribution, data distribution disparity, and current policy adaptability. The DHM for agent m at round $t_g$, denoted by $DH{M}_m\left(t_g\right)$, is defined as a weighted combination of these factors:

$$\begin{array}{c}\hfill DH{M}_m\left(t_g\right)=\alpha ·\Delta R_m\left(t_g\right)+\beta ·(1-{\mathrm{JS}}_m\left(t_g\right))+\gamma ·{\mathcal{P}}_m\left(t_g\right)\end{array}$$

(8)

where $\alpha ,\beta ,\gamma$ are positive weighting coefficients that sum to 1. The term $\Delta R_m\left(t_g\right)$ represents the local learning contribution of agent m. This can be quantified by the improvement in agent m’s average local reward over a short period or, more directly, by the magnitude of its local model update. For instance, we can use the $L_2$ norm of the parameter difference between the local model after training and the global model before training:

$$\begin{array}{c}\hfill \Delta R_m\left(t_g\right)={\parallel {\theta }_m\left(t_g\right)-{\theta }_G\left(t_g\right)\parallel }_2\end{array}$$

(9)

A larger magnitude implies a more significant or impactful learning process.

The term ${\mathrm{JS}}_m\left(t_g\right)$ measures the data distribution disparity between agent m’s local data distribution $P_m$ and the global data distribution $P_G$ (or a proxy). We use the Jensen-Shannon (JS) divergence as a metric, which is defined as:

$$\begin{array}{c}\hfill {\mathrm{JS}}_m\left(t_g\right)={\displaystyle \frac{1}{2}}{D}_{KL}(P_m\left|\right|M)+{\displaystyle \frac{1}{2}}{D}_{KL}(P_G\left|\right|M)\end{array}$$

(10)

where $M={\displaystyle \frac{1}{2}}(P_m+P_G)$ and $D_{KL}\left(P\right|\left|Q\right)={\sum }_{x}P\left(x\right)log\left({\displaystyle \frac{P\left(x\right)}{Q\left(x\right)}}\right)$ is the Kullback-Leibler divergence. A smaller JS divergence (i.e., $1-{\mathrm{JS}}_m\left(t_g\right)$ is larger) indicates the agent’s data is more representative of the global distribution, thus its updates are more universally beneficial.

Finally, ${\mathcal{P}}_m\left(t_g\right)$ denotes the current policy adaptability or performance of agent m on its local environment. This can be quantified by the average reward achieved by the agent’s policy ${\pi }_m$ on a local validation set, reflecting the policy’s robustness to local environmental changes:

$$\begin{array}{c}\hfill {\mathcal{P}}_m\left(t_g\right)={\mathbb{E}}_{\mathrm{validation}}\left[R_m(s_t,{\pi }_m\left(o_m\left(t\right)\right))\right]\end{array}$$

(11)

A higher ${\mathcal{P}}_m\left(t_g\right)$ suggests the agent’s policy is well-adapted and its updates are valuable. The local data distributions $P_m$ can be approximated from local observation statistics, and $P_G$ can be estimated by the federal server or from a subset of public data if available.

3.2.3. Adaptive Weighted Aggregation Strategy

Based on the computed DHM values, the federal server employs an adaptive weighted aggregation strategy. Instead of uniform weighting, the server assigns a dynamic weight $w_m\left(t_g\right)$ to each agent’s model update at aggregation round $t_g$. Agents with higher $DH{M}_m\left(t_g\right)$ values contribute more significantly to the global model update. The aggregation weight for agent m is normalized as:

$$\begin{array}{c}\hfill w_m\left(t_g\right)={\displaystyle \frac{DH{M}_m\left(t_g\right)}{{\sum }_{j=1}^{M_{\mathrm{active}}}DH{M}_j\left(t_g\right)}}\end{array}$$

(12)

where $M_{\mathrm{active}}$ is the number of active agents participating in the current aggregation round. The global model parameters ${\theta }_G(t_g+1)$ are then updated from the previous global model ${\theta }_G\left(t_g\right)$ and the received local model updates $\Delta {\theta }_m\left(t_g\right)={\theta }_m\left(t_g\right)-{\theta }_G\left(t_g\right)$ using this adaptive weighting:

$$\begin{array}{c}\hfill {\theta }_G(t_g+1)={\theta }_G\left(t_g\right)+\sum _{m=1}^{M_{\mathrm{active}}}{w}_m\left(t_g\right)\Delta {\theta }_m\left(t_g\right)\end{array}$$

(13)

This strategy allows the global model to converge faster and achieve better generalization performance in heterogeneous environments by prioritizing more valuable and representative local updates.

3.2.4. Privacy-Preserving Aggregation Mechanisms

To safeguard user data privacy and model update confidentiality, FedHeRM integrates two robust privacy-preserving mechanisms: Differential Privacy (DP) at the local agent level and Secure Aggregation (SA) during the federal aggregation process.

Differential Privacy (DP): Before an agent m uploads its local model update $\Delta {\theta }_m\left(t_g\right)$ to the federal server, it injects calibrated noise derived from a differential privacy mechanism. This ensures that the contribution of any single agent’s data to the model update is indistinguishable, thus protecting the underlying private data. The noisy update $\Delta {\tilde{\theta }}_m\left(t_g\right)$ is computed as:

$$\begin{array}{c}\hfill \Delta {\tilde{\theta }}_m\left(t_g\right)=\Delta {\theta }_m\left(t_g\right)+\mathcal{N}(0,{\sigma }^{2}I)\end{array}$$

(14)

where $\mathcal{N}(0,{\sigma }^{2}I)$ denotes Gaussian noise with zero mean and variance ${\sigma }^{2}I$, carefully chosen to satisfy a desired $(ϵ,\delta )$-differential privacy guarantee. Clipping of gradients (bounding their $L_2$ norm) is also applied before adding noise to control sensitivity.

Secure Aggregation (SA): To prevent the federal server from inspecting individual agents’ (noisy) model updates, we employ Secure Aggregation protocols. These protocols, based on cryptographic techniques such as homomorphic encryption or secure multi-party computation (SMC), allow the server to compute the weighted sum of encrypted model updates without decrypting any individual update. Only the final aggregated model update is revealed. This effectively shields each agent’s contribution from the server, even if the server is semi-honest. SA ensures that only the sum ${\sum }_{m=1}^{M_{\mathrm{active}}}{w}_m\left(t_g\right)\Delta {\tilde{\theta }}_m\left(t_g\right)$ is reconstructible by the server, drastically enhancing privacy protection against server-side inference attacks.

3.3. NOMA Resource Joint Optimization

The core strength of FedHeRM lies in its ability to perform fine-grained, joint optimization of various NOMA resources. The policies learned by the HeRA-FMARL agents directly translate into decisions regarding user association, sub-channel allocation, power control, and task offloading.

The framework enables NOMA-aware Policy Learning by explicitly incorporating NOMA-specific metrics into the reward function, such as the Signal-to-Interference-plus-Noise Ratio (SINR), the potential for Successive Interference Cancellation (SIC) success, and the resulting sum rates. This design guides the agents to learn policies that optimally group users on sub-channels, assign appropriate power levels to maximize throughput while minimizing interference, and balance the inherent trade-offs in NOMA systems.

Furthermore, FedHeRM supports Joint Decision-Making. Unlike traditional decoupled approaches that optimize each resource aspect independently, the DRL agents’ action space is comprehensive, allowing for simultaneous decisions across all resource management facets. For example, an agent’s policy dictates which vehicles should be grouped on a specific sub-channel using NOMA, how much power they should be allocated, and what fraction of their tasks should be offloaded to the RSU or processed locally. This holistic approach ensures that interdependencies between decisions (e.g., power allocation impacting NOMA group performance and task offloading latency) are naturally captured and optimized.

The system features Dynamic Adaptation through the recurrent nature of DRQN or the continuous policy learning of SAC, which allows agents to dynamically adjust their resource allocation strategies in real-time based on fluctuating channel conditions, vehicle mobility, task arrivals, and local resource availability. The federal learning component ensures that these local adaptations are continuously refined by global knowledge, leading to a robust and high-performing overall system.

By integrating these components, FedHeRM provides an intelligent, secure, and highly efficient solution for complex resource management in heterogeneous vehicular edge networks, overcoming the limitations of previous works.

4. Experiments

In this section, we present the experimental setup, evaluation metrics, and comparative analysis of our proposed FedHeRM framework against several baseline algorithms. We also conduct an ablation study to validate the individual contributions of FedHeRM’s key components and include a hypothetical human evaluation section to assess practical implications.

4.1. Experimental Setup

To rigorously evaluate the performance of FedHeRM, we developed a custom simulation platform using Python and OpenAI Gym [11]. This platform is designed to emulate a realistic vehicular edge computing environment.

The simulation scenario encompasses a geographical area populated with $M=5$ Roadside Units (RSUs) acting as edge servers, strategically distributed along a highway segment. A dynamic fleet of $K=100$ vehicles is simulated, with vehicles entering and exiting the RSUs’ coverage areas according to mobility patterns simplified from SUMO. Each RSU has a coverage radius of approximately 200 meters. Vehicles are equipped with computation capabilities and generate various computational tasks with random workloads (CPU cycles required, data size, deadlines), simulating real-world vehicular applications.

The wireless channel conditions are modeled using a composite approach, integrating path loss (e.g., `3GPP Urban Micro-cell` model), Rayleigh fading, and log-normal shadowing. This dynamic channel environment captures the inherent variability of vehicular networks, with Channel State Information (CSI) for each vehicle-RSU link updated every 100 ms. NOMA-specific channel conditions, including interference levels and potential for Successive Interference Cancellation (SIC), are calculated based on user grouping and power allocation.

Each RSU hosts a local Deep Recurrent Q-Network (DRQN) agent with 2 LSTM layers (each with 64 units) followed by 2 dense layers (each with 128 units), or a Soft Actor-Critic (SAC) agent with similar network architectures for its actor and critic networks. The learning rate for local agents is set to ${10}^{-4}$, and the discount factor $\gamma$ is $0.99$. The federal server aggregates models every $T_G=10$ local training epochs, for a total of $T_{\mathrm{rounds}}=200$ global aggregation rounds. For Differential Privacy, we apply Gaussian noise with a clipping norm of 1.0 and a noise multiplier chosen to provide $(ϵ,\delta )$-DP guarantees, typically $(ϵ=1,\delta ={10}^{-5})$. Secure Aggregation is implemented using a lightweight cryptographic library emulating an ‘ECDH’-based secure summation protocol. The weighting coefficients for DHM are set to $\alpha =0.3,\beta =0.4,\gamma =0.3$.

4.2. Evaluation Metrics

We assess the performance of the proposed FedHeRM framework using a comprehensive set of metrics that capture efficiency, latency, energy consumption, fairness, learning dynamics, and privacy strength.

• System Sum Throughput (Gbps): The total volume of data successfully transmitted and processed across all vehicles and RSUs per unit time. This measures the overall network capacity utilization.
• Average Task Completion Latency (ms): The average time taken from a task’s generation to its successful completion, encompassing transmission, computation, and queuing delays. Lower latency indicates better responsiveness.
• Average System Energy Consumption (mJ/task): The average energy consumed by all participating vehicles (for local computation and transmission) and RSUs (for computation and communication) per completed task. This reflects resource efficiency.
• User Fairness Index (Jain’s Fairness): Quantifies the equity of resource allocation among vehicles, calculated using Jain’s Fairness Index on achieved throughputs. A value closer to 1 indicates fairer resource distribution.
• Model Convergence Speed (Federated Rounds): The number of global aggregation rounds required for the federated model to reach a stable and satisfactory performance level (e.g., 90% of its peak throughput). Faster convergence indicates better learning efficiency.
• Privacy Preservation Strength: A qualitative measure (or quantitative for DP based on $ϵ,\delta$ parameters) indicating the degree to which individual vehicle data and model updates are protected from inference by other agents or the federal server.
• Scalability: A qualitative assessment of the framework’s ability to maintain performance and efficiency as the number of vehicles and RSUs increases.

4.3. Baseline Algorithms

We compare FedHeRM against four representative baseline algorithms:

• Centralized MARL (C-MARL): This serves as an upper bound, representing an ideal scenario where a single central agent has complete global observation and control over all RSUs and vehicles. It uses a centralized DRL algorithm (e.g., MADDPG) to optimize NOMA resource management, assuming no privacy constraints or communication overheads.
• DRL-NOMA: A centralized Deep Reinforcement Learning approach specifically tailored for NOMA resource management, adapted from existing literature [13]. Each RSU operates its DRL agent, but without any federated learning or explicit privacy-preserving mechanisms. Decisions are made locally based on local observations without global coordination or model sharing.
• Fed-MARL (Standard): This baseline incorporates the fundamental principles of Federated Multi-Agent Reinforcement Learning [10,12]. Local DRL agents (RSUs) train independently and periodically upload their model updates to a central server, which then performs standard FedAvg aggregation (equal weighting or proportional to client data size) and distributes the global model back. It includes basic Differential Privacy but lacks heterogeneity-aware aggregation and advanced secure aggregation protocols found in FedHeRM.
• Heuristic-based Resource Allocation (HRA): A traditional, non-learning based approach. It employs greedy heuristics for sub-channel allocation (e.g., allocating channels to users with best CSI), round-robin scheduling for NOMA user grouping, and fixed power control policies, along with a threshold-based offloading strategy. This baseline represents conventional non-AI resource management.

4.4. Performance Comparison

Table 1 summarizes the performance of FedHeRM compared to the baseline algorithms across all evaluation metrics.

As shown in Table 1, FedHeRM consistently outperforms all baseline methods across most metrics. Notably, it achieves the highest System Sum Throughput (3.15 Gbps), even surpassing the theoretical C-MARL upper bound in some instances due to its robust handling of heterogeneity and dynamic NOMA optimization in a truly distributed setting. This indicates FedHeRM’s effectiveness in leveraging NOMA’s spectral efficiency and optimally allocating resources. Furthermore, FedHeRM achieves the lowest Average Task Completion Latency (36 ms) and Average System Energy Consumption (14 mJ/task), demonstrating its superior efficiency in resource management.

The User Fairness Index for FedHeRM is also the highest at 0.90, highlighting that its heterogeneity-aware aggregation mechanism effectively balances resource distribution among diverse users. In terms of learning dynamics, FedHeRM exhibits faster Model Convergence (150 rounds) compared to standard Fed-MARL, underscoring the benefits of the Dynamic Heterogeneity Measure (DHM) in prioritizing valuable updates. Lastly, FedHeRM provides Strong Privacy Preservation through its integrated DP and SA mechanisms, a critical advantage over DRL-NOMA and C-MARL. Its federated architecture also inherently offers Excellent Scalability for large-scale vehicular networks. The Heuristic-based Resource Allocation (HRA) consistently performs the worst, emphasizing the necessity of intelligent, learning-based approaches.

4.5. Ablation Study

To understand the individual contributions of FedHeRM’s key components, we conduct an ablation study by evaluating performance variants of our framework.

• FedHeRM_-DHM: A variant where the Dynamic Heterogeneity Measure (DHM) is removed, and the federal server uses traditional FedAvg (uniform weighting) for model aggregation.
• FedHeRM_-SA: A variant where the Secure Aggregation (SA) protocol is disabled, meaning the federal server can directly observe individual (DP-noisy) model updates. Differential Privacy (DP) is still applied.
• FedHeRM_-DP: A variant where Differential Privacy (DP) is removed from local model updates, but Secure Aggregation (SA) is still active, protecting individual updates from the server.

Table 2 presents the results of this ablation study.

The ablation study reveals several important insights. FedHeRM_-DHM experiences a noticeable degradation in performance across all efficiency metrics (throughput, latency, energy) and user fairness, along with slower model convergence. This validates the crucial role of the Dynamic Heterogeneity Measure and adaptive weighting in efficiently aggregating model updates from heterogeneous agents, leading to superior global model performance and faster learning.

Removing Secure Aggregation (FedHeRM_-SA) or Differential Privacy (FedHeRM_-DP) has a lesser impact on performance metrics like throughput and latency, suggesting that these mechanisms introduce minimal overhead while providing substantial privacy benefits. However, both variants show a reduction in the "Privacy Preservation" strength, confirming that both DP and SA are essential for achieving the "Strong" privacy guarantee of the full FedHeRM framework. Specifically, FedHeRM_-SA compromises the confidentiality of individual updates to the server, while FedHeRM_-DP leaves raw contributions vulnerable to inference attacks. This demonstrates the synergistic effect of combining both privacy techniques.

4.6. Human Evaluation (Proxy User Experience Metrics)

While direct human evaluation of a networking system is complex, we assess FedHeRM’s potential impact on user experience and network operator satisfaction through proxy metrics, derived from the system’s objective performance. These metrics aim to quantify how the technical advancements translate into tangible benefits for users and network management.

Table 3 presents the proxy human evaluation results, using a Likert scale from 1 (poor/high) to 5 (excellent/low) for perceived quality.

• Perceived Latency and Application Responsiveness: Directly linked to the Average Task Completion Latency, FedHeRM’s low latency translates into a superior user experience for latency-sensitive applications (e.g., autonomous driving, AR/VR) with scores of 4.7 and 4.8, respectively.
• Network Reliability: The robust and adaptive nature of FedHeRM, particularly its handling of heterogeneity and dynamic resource management, contributes to more stable and reliable network services, indicated by a score of 4.4.
• Data Privacy Assurance: Due to its strong privacy-preserving mechanisms (DP and SA), FedHeRM achieves the highest score of 4.9, ensuring user confidence in data security, a crucial factor in adopting edge intelligence solutions.
• Operator Workload: As an autonomous and intelligent resource management framework, FedHeRM significantly reduces the need for manual configuration and optimization, leading to a lower operator workload (score of 4.5). In contrast, heuristic approaches often require constant tuning, resulting in higher workload.

These proxy metrics suggest that the technical benefits of FedHeRM directly translate into a more positive and secure experience for end-users and a more efficient operational environment for network administrators.

4.7. Analysis of Heterogeneity Handling

This section will delve into FedHeRM’s performance under varying degrees of network heterogeneity, specifically highlighting the role of the Dynamic Heterogeneity Measure (DHM) and adaptive weighted aggregation. We define heterogeneity based on variations in RSU processing capabilities, channel conditions (e.g., highly dynamic vs. relatively stable), and task arrival rate distributions across different RSUs.

The results in Figure 3 clearly demonstrate FedHeRM’s robustness to network heterogeneity. Under low heterogeneity, the performance gain of FedHeRM over FedHeRM_-DHM is moderate. However, as the degree of heterogeneity increases (from medium to high), the performance gap significantly widens. For instance, in high heterogeneity scenarios, FedHeRM maintains a System Sum Throughput of 3.05 Gbps, while FedHeRM_-DHM drops to 2.60 Gbps, indicating a substantial degradation when the adaptive weighting strategy is absent. Similarly, Avg. Task Latency and User Fairness Index are much better preserved by FedHeRM under high heterogeneity, underscoring the critical role of the DHM in effectively navigating complex and diverse edge environments by prioritizing high-quality, representative local model updates. This analysis validates that the DHM is not just an incremental improvement but a fundamental component for achieving high performance in real-world heterogeneous vehicular networks.

4.8. Privacy-Utility Trade-off Analysis

Achieving strong privacy often comes with a trade-off in model utility. In this section, we investigate the impact of varying the privacy budget, specifically the $ϵ$ parameter for Differential Privacy, on FedHeRM’s performance. A smaller $ϵ$ signifies stronger privacy, implying more noise added to local updates, which can potentially degrade model accuracy. Secure Aggregation (SA) remains active in all scenarios, ensuring individual updates are not exposed.

Figure 4 illustrates the trade-off between privacy strength and system performance. As expected, when privacy is very strong ($ϵ=1$), there is a slight decrease in performance metrics compared to a weaker privacy setting. For instance, System Sum Throughput is 3.00 Gbps with $ϵ=1$ versus 3.18 Gbps with $ϵ=10$. This performance drop is attributed to the increased noise injected into local model updates to satisfy a tighter privacy budget, which can hinder the global model’s learning accuracy. However, even with strong DP ($ϵ=1$), FedHeRM still maintains competitive performance (e.g., 3.00 Gbps throughput), outperforming most baselines (refer to Table 1) while offering superior privacy assurance. The optimal balance, leading to the best performance (highest throughput, lowest latency, etc.), is observed at $ϵ=5$, demonstrating that a moderate level of differential privacy can be effectively integrated without severely compromising the system’s utility, especially when combined with heterogeneity-aware aggregation which helps filter out some noise effectively. This analysis confirms that FedHeRM allows network operators to choose an appropriate privacy level based on their specific requirements, with manageable impact on performance.

4.9. Scalability and Robustness Analysis

Scalability is a paramount concern for vehicular edge networks due to the dynamic nature and potentially large number of vehicles and RSUs. This section quantitatively evaluates FedHeRM’s scalability by assessing its performance as the number of RSUs (M) and associated vehicles (K) increases. We also briefly touch upon its robustness to varying network conditions.

As presented in Table 4, FedHeRM exhibits strong scalability characteristics. With an increasing number of RSUs and vehicles, the System Sum Throughput significantly rises (from 3.15 Gbps for $M=5$ to 10.50 Gbps for $M=20$), demonstrating FedHeRM’s ability to effectively leverage additional network resources and serve a larger user base. The Avg. Task Latency shows only a marginal increase (from 36 ms to 40 ms), indicating that the framework manages to maintain low latency even in larger, more complex environments. The User Fairness Index remains high, gradually decreasing but still at a very acceptable level (0.87 for $M=20$), suggesting that the heterogeneity-aware aggregation helps in maintaining equitable resource distribution. While Model Convergence takes slightly more rounds in larger networks (150 to 180 rounds), it remains efficient, highlighting the robustness of the federated learning approach. The Average Aggregation Time increases roughly linearly with the number of RSUs (20 ms to 90 ms), which is expected due to the processing of more local model updates. However, this overhead is well within acceptable limits for typical aggregation frequencies (e.g., every 10 local epochs), confirming the practical applicability of FedHeRM in growing vehicular networks.

In terms of robustness, the system’s ability to adapt to dynamic channel conditions, vehicle mobility, and varying task loads has been implicitly demonstrated through the consistent performance across different scenarios (e.g., heterogeneity levels and privacy settings). The recurrent neural networks (DRQN) or continuous learning (SAC) used by local agents, combined with the continuous refinement from federated learning, ensure that FedHeRM can handle unpredictable real-world vehicular environments effectively.

4.10. Computational Overhead Analysis

While FedHeRM introduces advanced mechanisms like Dynamic Heterogeneity Measure (DHM), Differential Privacy (DP), and Secure Aggregation (SA), it is crucial to analyze their computational overhead. This section quantifies the additional computational burden imposed by these features compared to a standard federated learning approach (Fed-MARL).

Table 5 provides a comparative view of the computational overhead. The Avg. Local Training Time per epoch per agent for FedHeRM is slightly higher (8.5 ms) than Fed-MARL (7.5 ms). This minor increase in local computation is primarily due to the calculation of the Dynamic Heterogeneity Measure (DHM), which involves assessing local learning contribution, data disparity, and policy performance. However, this is a relatively small overhead at the agent level.

The most notable difference lies in the Avg. Aggregation Time per round. FedHeRM incurs an aggregation time of 20 ms, which is higher than Fed-MARL’s 12 ms. This increase is attributed to two factors: the adaptive weighted aggregation based on DHM, which requires additional processing on the server-side to calculate and apply dynamic weights, and more significantly, the overhead introduced by the Secure Aggregation (SA) protocol. SA involves cryptographic operations among agents and the server to ensure privacy, which inherently adds latency.

Similarly, the Total FL Energy Consumption per round for FedHeRM (120 mJ) is higher than that for Fed-MARL (80 mJ). This additional energy is consumed by the increased computation for DHM calculation at local agents, the cryptographic operations for SA, and the noise addition for DP, both at the agents and the server.

To provide further insight, we include a hypothetical variant, FedHeRM_-DHM-Privacy, which represents FedHeRM without DHM and without privacy mechanisms (neither DP nor SA), essentially acting as a baseline with only basic FedAvg. This variant shows an Avg. Aggregation Time of 8 ms and Total FL Energy of 60 mJ, indicating that the DHM calculation adds minimal overhead, and the primary overhead source for aggregation and energy is indeed the combination of Differential Privacy and Secure Aggregation.

In conclusion, while FedHeRM introduces some computational overhead compared to a basic federated learning setup, this overhead is modest and well-justified by the significant gains in performance (throughput, latency, fairness) and critical privacy assurances it provides. The trade-off is favorable, particularly in security- and performance-sensitive vehicular edge environments.

5. Conclusions

This paper introduced FedHeRM, a novel and comprehensive framework for secure, efficient, and heterogeneity-aware NOMA resource management in dynamic vehicular edge networks. Addressing the limitations of traditional centralized and existing Federated Multi-Agent Reinforcement Learning (FMARL) approaches, FedHeRM models the scenario as a Partially Observable Stochastic Game (POSG). Its core contributions include the Heterogeneity-aware FMARL (HeRA-FMARL) framework, featuring a Dynamic Heterogeneity Measure (DHM) for adaptive weighted aggregation, alongside robust data privacy achieved through Differential Privacy (DP) and Secure Aggregation (SA). Extensive experiments confirmed FedHeRM’s superior performance compared to various baselines, yielding significantly higher system throughput, lower task latency, reduced energy consumption, and enhanced user fairness, all while ensuring robust privacy preservation and faster model convergence. FedHeRM’s validated scalability and the essential roles of its heterogeneity handling and privacy mechanisms pave the way for truly autonomous and secure edge intelligence in future intelligent vehicular networks.