Multi-Modal Hierarchical Spatio-Temporal Network with Gradient-Boosting Integration for Cloud Resource Prediction

I. Introduction

Accurate prediction of multiple resource metrics is essential for scheduling, autoscaling and fault mitigation in heterogeneous cloud systems. Predictors must reconcile dense time-series signals, sparse categorical configuration data, and intermittent log events. Existing solutions split into two classes: sequence models (e.g., TCN, RNN) that capture temporal dependencies, and gradient boosting machines that handle tabular heterogeneity and categorical splits but lack native long-range temporal modeling.

Recent applied studies report that hybrid designs combining engineered statistical summaries with learned temporal embeddings reduce prediction variance under distribution shift and improve operator-level interpretability; these findings motivate explicit LightGBM feedback and correlation-guided fusion in our design.

This paper proposes MHST-GB, a multi-modal hierarchical spatio-temporal ensemble that jointly optimizes deep temporal representations and LightGBM predictions. Contributions are: (1) a dual-path architecture combining modality-specific neural encoders and LightGBM; (2) a correlation-guided attention mechanism with LightGBM-driven feedback; (3) a targeted training regime using focal MSE and curriculum scheduling; (4) empirical evaluation demonstrating consistent gains on standard metrics.

II. Related Work

Luo et al. [1] presented Gemini-GraphQA that combines language models and graph encoders. Their work shows how graph-aware design can improve reasoning and execution. Duc et al. [2] studied workload prediction in cloud-edge environments. They focused on proactive allocation of resources and showed the importance of prediction for large-scale systems.

Rondón-Cordero et al. [3] reviewed ensemble learning methods for energy consumption forecasting. They explained how ensemble models give better performance than single models but also discussed the cost of complexity. Bawa et al. [4] used ensemble methods for workload prediction in cloud systems. Their results showed higher accuracy and confirmed the advantage of model integration in cloud resource management.

Krishnan [5] examined AI methods for dynamic allocation and auto-scaling in cloud computing. The work showed that AI can improve efficiency but also needs careful design to avoid overhead. Sanjalawe et al. [6] gave a review of AI-based job scheduling. They covered different techniques and pointed out that scalability and fairness remain open problems.

Huang [7] proposed a framework for demand prediction using a mix of statistical analysis and neural models. This work showed that hybrid approaches can handle both continuous and discrete data in cloud systems. Sunder et al. [8] designed a hybrid deep learning model for smart building energy load forecasting. Their study proved that combining multiple neural architectures can capture complex spatio-temporal behavior.

Singh et al. [9] developed an ensemble-based system for water quality classification. They showed that ensemble methods can work well for heterogeneous data beyond cloud environments. Khoramnejad and Hossain [10] studied generative AI for optimization in wireless networks. Their survey described the use of generative models for resource allocation and pointed to challenges in integrating such systems with current network infrastructure.

III. Methodology

We propose MHST-GB, a hybrid model combining spatio-temporal deep networks with LightGBM for multi-resource prediction in heterogeneous cloud environments. Four modalities—configuration, monitoring, logs, and faults—are encoded via TCN, BiGRU, and CNN, then fused through attention and fed into LightGBM. This design captures temporal patterns and heterogeneous feature interactions efficiently. Curriculum learning, adaptive rates, and focal loss enhance performance. See Figure 1.

IV. Model Overview

MHST-GB jointly models heterogeneous, multi-modal, and irregular cloud workload data. Given M metrics and T time steps, we learn:

$${\widehat{y}}_t^{\left(m\right)}={\mathcal{F}}_{\Theta }\left({\mathbf{X}}_t^{\mathrm{conf}},{\mathbf{X}}_{1:t}^{\mathrm{mon}},{\mathbf{X}}_{1:t}^{\mathrm{task}},{\mathbf{X}}_{1:t}^{\mathrm{fault}}\right)$$

(1)

Here ${\widehat{y}}_t^{\left(m\right)}$ denotes the prediction for metric m at time t; ${\mathbf{X}}^{\mathrm{conf}}$ denotes static node configuration features; ${\mathbf{X}}_{1:t}^{\mathrm{mon}}$, ${\mathbf{X}}_{1:t}^{\mathrm{task}}$ and ${\mathbf{X}}_{1:t}^{\mathrm{fault}}$ denote monitoring, parsed task logs and fault streams over the input window respectively.

We adopt a dual-path design: deep encoders extract spatio-temporal features, while LightGBM models tabular sparsity and non-linear interactions.

A. Configuration Encoder

Static configuration ${\mathbf{X}}^{\mathrm{conf}}$ is encoded via residual MLP:

$${\mathbf{h}}^{\mathrm{conf}}=\sigma \left({\mathbf{W}}_2\sigma \left({\mathbf{W}}_1{\mathbf{X}}^{\mathrm{conf}}+{\mathbf{b}}_1\right)+{\mathbf{b}}_2\right)$$

(2)

In this equation ${\mathbf{X}}^{\mathrm{conf}}\in {\mathbb{R}}^{d_{\mathrm{conf}}}$; ${\mathbf{W}}_1\in {\mathbb{R}}^{d\times d_{\mathrm{conf}}}$ and ${\mathbf{W}}_2\in {\mathbb{R}}^{d\times d}$ produce ${\mathbf{h}}^{\mathrm{conf}}\in {\mathbb{R}}^d$. Activation $\sigma (·)$ denotes ReLU in our implementation.

B. Monitoring Metric Encoder

Monitoring input is processed by TCN with dilated convolution:

$${\mathbf{o}}_t=\sum _{k=0}^{K-1}{w}_k·{\mathbf{x}}_{t-d·k}$$

(3)

Here K is kernel size, d the dilation factor, $w_k$ the convolutional kernel weights and ${\mathbf{x}}_{·}$ the multivariate monitoring vector per time step. TCN outputs are projected to the shared embedding dimension d before fusion.

C. Task Event Sequence Encoder

Parsed task logs ${\varphi }_{\log }\left({\mathbf{X}}_{1:t}^{\mathrm{task}}\right)$ are passed into BiGRU:

$${\mathbf{h}}_t^{\mathrm{task}}=\mathrm{BiGRU}\left({\varphi }_{\log }\left({\mathbf{X}}_{1:t}^{\mathrm{task}}\right)\right)$$

(4)

The parser ${\varphi }_{\log }(·)$ yields numeric descriptors per event; the BiGRU concatenates forward and backward hidden states and projects the result to ${\mathbb{R}}^d$ for fusion.

D. Fault Injection Encoder

Fault logs are embedded and passed through a 1D-CNN:

$${\mathbf{h}}_t^{\mathrm{fault}}=\mathrm{Conv}1\mathrm{D}\left(\mathrm{Embed}\left({\mathbf{X}}_{1:t}^{\mathrm{fault}}\right)\right)$$

(5)

Fault codes are mapped to trainable embeddings and convolved with temporal kernels; outputs are pooled and projected to ${\mathbb{R}}^d$.

E. Feature Alignment and Fusion

We use correlation-guided attention to align modalities, motivated by the fact that LightGBM feature importance metrics can identify strong predictors, which in turn inform attention weights. Let ${\rho }_j$ be the Pearson correlation between modality j’s output and the target:

$${\alpha }_j={\displaystyle \frac{exp({\rho }_j/\tau )}{{\sum }_{k}exp({\rho }_k/\tau )}},{\mathbf{z}}_t=\sum _j{\alpha }_j{\mathbf{h}}_t^{\left(j\right)}$$

(6)

In practice ${\rho }_j$ is computed on modality-level summaries over the training set; $\tau$ is a temperature hyperparameter tuned on validation. Each ${\mathbf{h}}_t^{\left(j\right)}$ is linearly projected to common dimension d prior to weighting. These fused embeddings feed both the deep prediction head and the LightGBM learner, ensuring that modality weighting benefits both branches.

F. Prediction and Ensemble Integration

The fused vector ${\mathbf{z}}_t$ enters a Multi-Head Self-Attention (MHSA) block to model inter-metric dependencies:

$${\mathrm{head}}_h=\mathrm{Softmax}\left({\displaystyle \frac{{\mathbf{Q}}_h{\mathbf{K}}_h^{\top }}{\sqrt{d_k}}}\right){\mathbf{V}}_h$$

(7)

Where ${\mathbf{Q}}_h,{\mathbf{K}}_h,{\mathbf{V}}_h$ are linear projections of ${\mathbf{z}}_t$ and $d_k$ denotes key dimension; softmax is applied over the key axis. The MHSA output is processed by a regression MLP to produce ${\widehat{y}}_t^{\mathrm{DL}}$. Simultaneously, ${\mathbf{z}}_t$ and selected engineered features are input to LightGBM to produce ${\widehat{y}}_t^{\mathrm{GBM}}$. The final prediction is:

$${\widehat{y}}_t=\lambda {\widehat{y}}_t^{\mathrm{DL}}+(1-\lambda ){\widehat{y}}_t^{\mathrm{GBM}}$$

(8)

The scalar $\lambda \in [0,1]$ is selected on the validation set (grid search) or learned as a small parameter. Both branch outputs are in ${\mathbb{R}}^M$ before fusion. This fusion capitalizes on LightGBM’s ability to capture decision boundaries in sparse feature spaces, complementing deep learning’s high-capacity nonlinear modeling.

G. Loss Function and Training Strategies

We employ a Focal Mean Squared Error (FMSE) to emphasize harder samples:

$${\mathcal{L}}_{\mathrm{FMSE}}={\displaystyle \frac{1}{MT}}\sum _{m=1}^M\sum _{t=1}^T\left(1+\gamma \left|{\displaystyle \frac{{\widehat{y}}_t^{\left(m\right)}-y_t^{\left(m\right)}}{{\sigma }_m}}\right|\right){\left({\widehat{y}}_t^{\left(m\right)}-y_t^{\left(m\right)}\right)}^2$$

(9)

Here $\gamma$ (>0) controls focal emphasis; ${\sigma }_m$ is the per-metric empirical standard deviation computed on the training set (epsilon added if zero). Training incorporates:

• Curriculum Learning: early training uses shorter sequences for stability.
• Layer-wise Adaptive Learning Rate (LALR): ${\eta }_l={\eta }_0/(1+\beta l)$.
• LightGBM Feature Importance Feedback: periodic feature importance analysis informs neural attention re-weighting.

By embedding LightGBM into every modality’s feature flow rather than as a post-hoc baseline, the model exploits both structured learning and deep representation learning in a single, unified pipeline. Figure 2 illustrates the comprehensive training strategy employed in our framework.

V. Feature Engineering and Preprocessing Enhancements

Effective feature preparation is critical for the MHST-GB framework, as the data exhibits heterogeneity, temporal irregularity, and varying levels of sparsity. We design the feature pipeline to simultaneously serve deep neural encoders and LightGBM, ensuring that both branches receive modality-specific yet complementary inputs are shown in Figure 3

A. Correlation-Guided Feature Selection

To mitigate redundant features post-encoding, we compute Pearson correlation ${\rho }_{x,y}$ with respect to target y:

$${\rho }_{x,y}={\displaystyle \frac{{\sum }_t(x_t-\overline{x})(y_t-\overline{y})}{\sqrt{{\sum }_t{(x_t-\overline{x})}^2}\sqrt{{\sum }_t{(y_t-\overline{y})}^2}}}$$

(10)

Features with $|{\rho }_{x,y}|<\tau$ are pruned. Threshold τ is tuned via cross-validation. This improves LightGBM split quality and regularizes neural branches.

B. Temporal Aggregation and Statistical Summarization

To handle variable-length sequences, we apply aggregations (mean, variance, max, EMA):

$${\mathrm{EMA}}_{\alpha }\left(x_t\right)=\alpha x_t+(1-\alpha ){\mathrm{EMA}}_{\alpha }\left(x_{t-1}\right)$$

(11)

Aggregated features go to LightGBM; full sequences feed neural encoders, ensuring multi-resolution representation.

C. Log-Parsing-Based Numeric Extraction

Task commands are parsed via ${\varphi }_{\log }(·)$ into numerical vectors:

$${\mathbf{e}}_i={\varphi }_{\log }\left(c_i\right)\in {\mathbb{R}}^{d_{\log }}$$

(12)

These features reduce noise and enhance semantic consistency for both LightGBM and neural paths.

VI. Additional Optimization Strategies

Beyond architectural design and feature processing, we incorporate training and inference optimizations to maximize predictive performance while ensuring stability and interpretability are shown in Figure 4

A. Curriculum-Based Multi-Stage Training

Instead of training the entire network on full-length sequences from the start, we employ curriculum learning. Let $T_0$ be the initial sequence length, and at each stage s, we increase it to $T_s=T_{s-1}+\Delta T$. This allows encoders to first learn stable local dependencies before modeling long-range temporal patterns, improving convergence stability.

B. LightGBM Feature Importance Feedback Loop

During training, LightGBM produces a gain-based importance score $I_f$ for each feature f. We normalize these scores and use them to adjust the attention weights ${\alpha }_j$ in the neural fusion layer:

$${\alpha }_j^{\mathrm{new}}={\displaystyle \frac{exp(({\rho }_j+\beta I_j)/\tau )}{{\sum }_{k}exp(({\rho }_k+\beta I_k)/\tau )}}$$

(13)

where $I_j$ is the average importance of features from modality j, and $\beta$ controls the contribution of LightGBM feedback. This cross-model synergy improves alignment between decision boundaries and learned embeddings.

C. Regularization via Mixup and DropConnect

To improve robustness, we apply temporal mixup for sequence data:

$$\tilde{\mathbf{x}}=\lambda {\mathbf{x}}_a+(1-\lambda ){\mathbf{x}}_b,\tilde{y}=\lambda y_a+(1-\lambda )y_b$$

(14)

where $\lambda \sim \mathrm{Beta}(\alpha ,\alpha )$. Additionally, DropConnect is applied to fully connected layers in the deep branch, randomly dropping weights rather than activations, encouraging redundancy and preventing co-adaptation.

These optimizations collectively ensure that MHST-GB not only captures complex temporal-spatial relationships but also retains the practical advantages of gradient boosting in handling mixed and sparse data representations.

VII. Evaluation Metrics

To assess MHST-GB, we adopt four standard metrics covering absolute/relative error and explanatory power:

Weighted Relative Error (WRE) is the main metric:

$$\mathrm{WRE}={\displaystyle \frac{{\sum }_{m=1}^M{w}_m{\sum }_{t=1}^T\left|\frac{{\widehat{y}}_t^{\left(m\right)}-y_t^{\left(m\right)}}{y_t^{\left(m\right)}}\right|}{{\sum }_{m=1}^M{w}_{m}T}}$$

(15)

where $w_m$ denotes the metric weight.

Mean Absolute Error (MAE) measures overall error magnitude:

$$\mathrm{MAE}={\displaystyle \frac{1}{MT}}\sum _{m=1}^M\sum _{t=1}^T\left|{\widehat{y}}_t^{\left(m\right)}-y_t^{\left(m\right)}\right|$$

(16)

Root Mean Squared Error (RMSE) penalizes large deviations:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{MT}}{\sum }_{m=1}^M{\sum }_{t=1}^T{\left({\widehat{y}}_t^{\left(m\right)}-y_t^{\left(m\right)}\right)}^2}$$

(17)

Coefficient of Determination ($R^2$) reflects explained variance:

$$R^2=1-{\displaystyle \frac{{\sum }_{m=1}^M{\sum }_{t=1}^T{\left(y_t^{\left(m\right)}-{\widehat{y}}_t^{\left(m\right)}\right)}^2}{{\sum }_{m=1}^M{\sum }_{t=1}^T{\left(y_t^{\left(m\right)}-{\overline{y}}^{\left(m\right)}\right)}^2}}$$

(18)

Together, these metrics evaluate accuracy, robustness, and generalization.

VIII. Experiment Results

We compare MHST-GB against LSTM-Base, TCN-Attn, and LightGBM-Only. Table 1 and Figure 5 show MHST-GB achieves the best performance across all metrics.

Ablation results (Table 2) show LightGBM integration yields the largest single-component gain (WRE 0.137 without LightGBM vs 0.124 full). Removing correlation-guided attention or the feedback loop degrades performance, confirming that modality alignment and cross-model feedback contribute non-redundant improvements. Error diagnostics (attention heatmaps and GBM importance plots) indicate LightGBM captures threshold-like and sparse predictors while the neural branch reduces temporally structured residuals.

A. Ablation Study

We evaluate the effects of LightGBM, correlation-guided attention, and feedback loop. Table 2 confirms each component contributes, with LightGBM providing the largest benefit.

IX. Conclusion

We proposed the MHST-GB framework, a multi-modal hierarchical spatio-temporal deep ensemble with integrated LightGBM for predicting cloud resource monitoring metrics. By unifying modality-specific neural encoders, correlation-guided fusion, and gradient boosting’s structured feature modeling, our approach achieves superior accuracy and robustness over both neural-only and boosting-only baselines. Ablation results confirm the complementary nature of deep representation learning and decision tree-based modeling in complex cloud workload prediction tasks.

MHST-GB assumes adequate historical coverage per node and representative fault injections; abrupt unobserved distribution shifts may require online GBM updates and domain adaptation. Future work will examine online LightGBM updating and formal calibration of the feedback coefficient under non-stationarity.