Hierarchical Expert Multi-Agent Framework for Causal Root Cause Localization in Cloud-Native Microservices

1. Introduction

Cloud-native architectures split applications into microservices that interact through complex and dynamic dependencies. This design improves scalability and resilience, but it makes root cause localization harder because anomalies can spread across services and heterogeneous telemetry data. The task is to separate true faults from propagated effects with strict limits on latency and accuracy. Traditional methods that use static graphs or single-modality inputs often fail when workloads change or when data are multimodal.

Recent work with graph neural networks and causal inference improves dependency modeling, but these approaches often misidentify hub services as root causes because of structural bias. Large language model based methods can analyze logs and metrics, but they face hallucination, context window limits, and overhead in coordination. Flat frameworks also increase communication bottlenecks as system scale grows.

HEMA-RCL is a hierarchical expert multi-agent framework that combines different large models to achieve accurate and efficient diagnosis. The layered design reduces communication cost. Dynamic agent generation adapts to new failures with small parameter overhead. Belief propagation with causal metrics helps avoid hub misidentification. Multimodal alignment combines metrics, logs, and traces in a consistent way. Context-aware prompting reduces hallucination and improves scalability for real-world use.

2. Related Work

Microservice diagnosis uses traces and metrics: Zhang et al. [1] exploit traces for fine granularity, while Wang et al.[2] integrate metrics for multimodality; both struggle with scalability and class imbalance.

Causal approaches refine localization: Xin et al.[3] apply causal reasoning, and Zhu et al.[4] extend to instance-level services; both rely on brittle, stable structures.

Graph methods improve accuracy at higher cost: Sun et al.[5] use graph autoencoders for interpretability, and Wang et al.[6] leverage knowledge graphs, increasing computation and maintenance.

Trace-based anomaly detection is sensitive yet fragile: Panahandeh et al.[7] detect deviations but degrade with noise or missing data. Broader modeling advances aid adaptability: Tian et al.[8] introduce cross-attention for heterogeneous tasks, Guan[9] applies ML to healthcare prediction, and Zhu and Liu[10] enhance NER via LoRA fine-tuning. Persistent gaps remain in scalability, robustness, and adaptation to novel failures.

3. Methodology

HEMA-RCL tackles microservice root-cause localization via a hierarchical expert multi-agent system coordinating heterogeneous LLMs: a DeepSeek-V2 (236B) Orchestrator routes to Qwen-72B and Qwen-14B specialists and dynamically spawns Qwen-7B sub-agents for compute–accuracy efficiency. For reproducibility, the appendix provides pseudocode for the Orchestrator main loop and belief-propagation message passing, detailing input formats, update rules, and termination criteria. Spawning is KL-gated on detected fault-pattern distributions, and agent disagreement is resolved by factor-graph belief propagation that fuses historical reliability with transfer entropy for causal inference. On 10,000 fault-injection scenarios, HEMA-RCL attains 87.6% top-1 root-cause accuracy and reduces mean time to detection to 38.7 s.

4. Algorithm and Model

4.1. HEMA-RCL Architecture Overview

HEMA-RCL coordinates heterogeneous LLMs in a unified diagnostic stack. DeepSeek-V2 (236B) orchestrates long-context reasoning; Qwen-72B performs high-throughput metric analysis; Qwen-14B mines log patterns. The system uses seven agent types:

$$\begin{array}{cc}\hfill {\mathcal{S}}_{HEMA}=\{& {\mathcal{A}}_{orch},{\mathcal{A}}_{mad},{\mathcal{A}}_{lpm},{\mathcal{A}}_{tr},\hfill \\ \hfill & {\mathcal{A}}_{ci},{\left\{{\mathcal{A}}_{ds}^i\right\}}_{i=1}^{N_{dynamic}},{\mathcal{A}}_{cb}\}\hfill \end{array}$$

(1)

where $N_{dynamic}$ is resource-bounded.

4.2. Hierarchical Expert System with LLM Specialization

Flat multi-agent designs incur quadratic coordination; with $>5$ agents, message passing reaches $67\%$ runtime. We adopt a three-layer hierarchy to reduce coordination while preserving information fidelity. The orchestrator uses grounded context with a compact history. History is compressed via ${\mathrm{PCA}}_{k=32}$ on recent features and centroids of similar retrieved contexts. Resource allocation uses a composite complexity metric with online weights $\alpha =0.3$, $\beta =0.2$, $\gamma =0.25$, $\delta =0.25$. The Metric Anomaly Detection Agent (MAD) fuses FFT-based spectral features, seasonal LSTM states over a sliding window, and wavelet coefficients. A learnable threshold, updated toward a conservative baseline, mitigates false positives after distribution shifts.

$${\pi }_{orch}(a_t\mid s_t)=\mathrm{DeepSeek}-\mathrm{V}2\left({\mathrm{Prompt}}_{grounded}(·)\right)$$

(2)

$$\begin{array}{cc}\hfill \mathcal{C}\left(s_t\right)& =\alpha H\left({\mathbf{M}}_t\right)+\beta {\parallel {\mathbf{L}}_t\parallel }_0\hfill \\ \hfill & +\gamma \mathrm{depth}\left({\mathcal{T}}_t\right)+\delta {\mathrm{VAR}}_{temporal}\left({\mathbf{M}}_t\right)\hfill \end{array}$$

(3)

4.3. Dynamic Agent Spawning Mechanism

Pre-configured agents degrade on novel faults (up to $40\%$ accuracy loss). We trigger spawning via a two-stage KL-gated policy (Figure 1):

$$\begin{array}{c}\hfill p(\mathrm{spawn}\mid {\mathcal{F}}_t)=\sigma \left({\mathbf{w}}^{\top }\left[{\mathcal{D}}_{KL}({\mathcal{F}}_t\parallel {\mathcal{F}}_{known}),{\mathcal{R}}_{available},{\mathcal{U}}_{current}\right]\right)\end{array}$$

(4)

Rapid adaptation uses rank-16 LoRA on Qwen-7B, cutting trainable parameters by $98.7\%$ while retaining $94\%$ performance:

$${\mathcal{A}}_{ds}^{new}=\mathrm{Qwen}-7\mathrm{B}\left({\theta }_{base}\right)+{\mathrm{LoRA}}_{r=16}\left(\Delta \theta \left({\mathcal{F}}_t\right)\right)$$

(5)

4.4. Belief Propagation-based Consensus Mechanism

Majority voting fails under correlated errors. We use factor-graph belief propagation with time-decayed reliability (Figure 2):

$$r_i\left(t\right)={\displaystyle \frac{{\sum }_{\tau =1}^t\mathbb{I}\left[{\mathrm{correct}}_i\left(\tau \right)\right]e^{-\alpha (t-\tau )}}{{\sum }_{\tau =1}^t{e}^{-\alpha (t-\tau )}}}$$

(6)

Messages encode semantic compatibility and historical agreement:

$$\begin{array}{c}\hfill {\psi }_{ij}(x_i,x_j)=exp\left(-\lambda d_{semantic}(x_i,x_j)·{\displaystyle \frac{1}{\sqrt{r_i{r}_j}}}·(1+{ϵ}_{ij})\right)\end{array}$$

(7)

Damping stabilizes loopy graphs:

$${\mu }_{i\to j}^{(t+1)}=(1-\rho ){\mu }_{i\to j}^{\left(t\right)}+\rho {\widehat{\mu }}_{i\to j}^{(t+1)}$$

(8)

This reduces consensus time from $8.3$ to $2.1$ s.

4.5. Causal Inference Enhancement

Correlation-based selection overweights hubs (43% accuracy). We incorporate transfer entropy with adaptive delay:

$$\begin{array}{c}\hfill \mathrm{TE}(u\to v)=\sum _{t}p(v_{t+\Delta t},v_t,u_t)log{\displaystyle \frac{p(v_{t+\Delta t}\mid v_t,u_t)}{p(v_{t+\Delta t}\mid v_t)}}\end{array}$$

(9)

$$\begin{array}{c}\hfill \Delta t_{optimal}=arg\underset{\Delta t\in [1,10]}{max}\left[\mathrm{TE}(u\to v;\Delta t)-{\lambda }_{penalty}\Delta t\right]\end{array}$$

(10)

We integrate with GNNs via position-agnostic attention:

$$\begin{array}{c}\hfill {\mathbf{h}}_v^{(l+1)}=\sigma \left({\mathbf{W}}_{self}^{\left(l\right)}{\mathbf{h}}_v^{\left(l\right)}+\sum _{u\in \mathcal{N}\left(v\right)}{\alpha }_{vu}^{\left(l\right)}{\mathbf{W}}_{msg}^{\left(l\right)}{\mathbf{h}}_u^{\left(l\right)}+{\mathbf{b}}_{struct}\left(\mathcal{G}\right)\right)\end{array}$$

(11)

where ${\mathbf{b}}_{struct}\left(\mathcal{G}\right)$ is structure-encoded and position-agnostic.

4.6. Data Preprocessing

Figure 3 summarizes the pipeline.

4.6.1. Multi-modal Data Alignment and Temporal Synchronization

Metrics arrive every 15 s; logs are event-driven; traces are asynchronous. Naive fusion degraded accuracy by 31%. We align to a unified grid, interpolate irregular logs, progressively encode traces, and apply robust normalization.

$${\mathcal{T}}_{unified}=\{{\tau }_k:{\tau }_k=t_0+k·\Delta t_{base},k\in [0,K]\}$$

(12)

Traces use progressive completion with confidence $\alpha =min\left(1,n_{complete}/n_{expected}\right)$:

$${\mathbf{t}}_{prog}\left({\tau }_k\right)=\alpha {\mathbf{t}}_{complete}\left({\tau }_k\right)+(1-\alpha ){\widehat{\mathbf{t}}}_{partial}\left({\tau }_k\right)$$

(13)

Robust scaling uses MAD:

$${\tilde{m}}_{ij}={\displaystyle \frac{m_{ij}-median\left({\mathbf{m}}_j\right)}{MAD\left({\mathbf{m}}_j\right)·1.4826}}$$

(14)

This alignment and scaling reduced false positives by 43%.

4.6.2. Adaptive Feature Engineering and Dimensionality Reduction

Standard PCA preserved only 67% fault-relevant variance despite 95% total variance. We compute feature importance, aggregate multi-scale statistics (windows $s\in \{1,5,15,60\}$; mean, max, gradient, FFT), and learn a supervised low-dimensional embedding with fault-aware reconstruction.

$$\begin{array}{cc}\hfill {\mathcal{I}}_{importance}\left(f_i\right)& ={\omega }_{1}I(f_i;Y_{fault})+{\omega }_2\sum _j|TE(f_i\to f_j)|\hfill \\ \hfill & +{\omega }_3{VAR}_{conditional}\left(f_i\mid Y_{fault}\right)\hfill \end{array}$$

(15)

$${\mathcal{L}}_{reduce}=\parallel \mathbf{X}-\widehat{\mathbf{X}}{\parallel }_2^2+\beta \sum _{i\in {\mathcal{F}}_{fault}}\parallel {\mathbf{x}}_i-{\widehat{\mathbf{x}}}_i{\parallel }_2^2+\gamma {\parallel {\mathbf{W}}_{encoder}\parallel }_1$$

(16)

This reduced dimensionality by 94% and improved fault-detection F1 by 12%.