GINet-DGC: Structural Inductive Biases and Dynamic Generalization Control for High-Dimensional Small-Sample Tabular Data

1. Introduction

This section presents a general overview of the study by outlining the background, reviewing related works, identifying the research gaps, and summarizing the main contributions of this research.

1.1. Background and Motivation

Tabular data are among the most widely used data forms in scientific computation and industrial analysis because they naturally represent heterogeneous observations through structured variables. These data routinely play an important role in supporting advanced research in high-energy physics [1], computational chemistry [2], and financial forecasting [3]. Beyond these fields, structured variables are particularly crucial for predictive modeling and knowledge discovery in biomedicine and health-oriented artificial intelligence [4,5]. However, across many of these scientific disciplines, informative datasets are difficult to collect because experiments can be costly, time-consuming, or ethically constrained. For example, in experimental particle physics, high-dimensional detector measurements are produced under highly costly acquisition conditions [6].This data scarcity becomes even more severe in biomedical gene-expression analysis. Within typical transcriptomic arrays, a single spreadsheet routinely contains tens of thousands of molecular features against merely dozens or hundreds of clinical patient samples [7]. This severe mathematical discrepancy in scale precisely defines the High-Dimensional Low-Sample Size (HDLSS) nature of the data, creating a heavily asymmetrical dataset. Consequently, researchers must infer reliable patterns under an extreme regime where the feature dimension greatly exceeds the number of available observations [8,9]. In this scenario, limited samples provide insufficient supervision for the massive parameter space of deep learning models. This optimization constraint frequently makes traditional networks vulnerable to stochastic noise, feature redundancy, and unstable gradients, potentially leading to severe overfitting.

Achieving effective inference from such data is thus a pressing challenge in modern machine learning and a key step toward empirical scientific discovery. In this study, we focus on biomedical gene-expression classification, which is widely recognized as a highly typical problem within the HDLSS learning scenario.

1.2. Related Works

This section reviews existing literature to clarify the scope of current paradigms and highlight the open challenges that motivate this study.

In the context of small-sample tabular learning, traditional tree-based ensemble methods and shallow models with regularization remain the most competitive and stable baselines [9].Representative models such as XGBoost [10] and LightGBM [11] achieve good generalization through efficient feature splitting and regularized ensemble learning. However, tree-based models cannot perform end-to-end representation learning, which restricts their ability to capture complex, high-order dependencies among molecular features. To address these limitations, recent deep tabular research primarily focuses on three directions: (1) feature selection-driven representation enhancement, (2) structured parameter generation and conditional weight prediction, and (3) data distribution expansion and prior transfer.

Feature selection–driven representation enhancement methods. The core idea of this class of methods is to dynamically adjust feature contributions at the input or intermediate layers through explicit or implicit feature selection and weighting mechanisms, thereby improving representation capability and alleviating overfitting. For example, AutoInt [12] explores feature interaction and representation enhancement by introducing explicit interaction layers to strengthen feature learning. TabNet [13] employs stepwise attention for dynamic feature selection, achieving a balance between interpretability and performance. Models based on self-attention, such as TabTransformer [14], FT-Transformer [9], and SAINT [15], further enhance feature modeling capacity. FsNet [16] introduces a learnable feature selection module with sparsity constraints to identify key features and reduce model complexity. These approaches have demonstrated performance comparable to or even surpassing gradient boosting models on several small- and medium-scale datasets, highlighting the potential of attention-based and feature-selection techniques under small-sample conditions. However, since they primarily operate at the feature level, they remain susceptible to redundant molecular features and the curse of dimensionality in high-dimensional biomedical spaces, leading to limited stability in practical applications.

Structured Parameter Generation and Conditional Weight Prediction Mechanisms. These methods introduce parameter dependencies or conditional generation networks into model architectures. The goal is to reduce the number of free learnable parameters and incorporate structural priors. As a result, model generalization and stability can be improved. This idea originates from the HyperNetworks framework [17]. In tabular learning, HyperTab [18] generates dynamic weights based on feature subsets. It uses a hypernetwork to reduce parameter redundancy and alleviate overfitting. DietNetworks [19] employ a weight prediction network to obtain a compact parameterization for high-dimensional data. This design improves robustness in small-sample settings. WPFS [20] combines feature embeddings with a weight predictor. It generates the first-layer weight matrix according to structural dependencies among features. More recently, GCondNet [21] integrates implicit inter-sample relationship modeling with conditional parameter generation. It achieves strong generalization performance on high-dimensional small-sample tabular datasets. These results demonstrate the benefits of structure-dependent weight generation. However, most existing methods rely on static architectures. They lack mechanisms to dynamically detect overfitting and adapt model behavior during training. Moreover, in high-dimensional feature spaces, they still face a trade-off between model expressiveness and training stability.

Data distribution expansion and prior transfer strategies. This class of methods aims to alleviate high variance and overfitting under small-sample conditions by expanding the training data distribution or incorporating external prior knowledge. Traditional interpolation-based approaches such as SMOTE [22] and ADASYN [23] generate synthetic minority samples through neighborhood interpolation. In recent years, generative models have become a major direction for tabular data augmentation. For instance, CTGAN [24] designs conditional generation mechanisms for mixed variables, while diffusion-based TabDDPM [25] leverages score-based priors to enhance generation quality. However, in high-dimensional biomedical datasets, synthetic augmentation can potentially amplify stochastic noise or distort dataset-specific molecular distributions. Alternatively, prior transfer strategies, such as TabPFN [26] and large-scale language model transfer [27], attempt to mitigate data scarcity through cross-task knowledge migration. Nevertheless, these strategies remain constrained by task consistency and are frequently prone to prior mismatch across heterogeneous biomedical cohorts and assay conditions.

In summary, while tabular learning has progressed significantly, it still faces severe challenges in high-dimensional, low-sample-size scenarios. The primary bottlenecks include high model complexity, premature overfitting, limited stability under distribution shifts, and a lack of dynamic adaptation to training states.

1.3. Research Gap

Although deep learning has achieved major success across various domains, whether standard deep networks are the optimal choice for high-dimensional, low-sample-size (HDLSS) tabular data remains controversial. On one hand, deep models can learn flexible representations and complex feature interactions. On the other hand, several studies report that their performance frequently degrades when data are small or sparse. For example, Liu et al. [28] observed that standard deep neural networks (DNNs) can fail to converge stably under limited samples because the feature dimension greatly exceeds the available observations. This severe imbalance often induces large gradient variance and poor generalization. Similarly, Margeloiu et al. [20], Margeloiu et al. [21] reported that conventional neural networks frequently fail on HDLSS datasets because noisy and redundant features interfere with optimization. More broadly, Shwartz-Ziv and Armon [8] argued that classical tree-based models such as XGBoost often outperform deep learning methods on tabular tasks, whereas recent studies demonstrate that carefully constrained architectures can be highly competitive under appropriate settings [9]. This ongoing divergence suggests that the central issue is not whether deep learning should be applied, but how its flexibility can be effectively constrained when supervision is scarce.

Despite recent exploration into parameter generation, robust neural modeling for biomedical HDLSS data still faces severe challenges. First, excessive parameter spaces induce premature overfitting. Second, redundant molecular features obscure stable disease-related patterns. Third, static architectures lack the capacity to monitor training trajectories,. These persistent limitations define the central research addressed in this study. Specifically, current networks cannot explicitly limit the feature-interaction search space while adaptively identifying overfitting during optimization.

Therefore, we propose GINet-DGC, a neural framework engineered with structural inductive biases and adaptive trajectory-based early stopping. As illustrated in Figure 1, instead of fitting an unconstrained projection directly from scarce observations, GINet-DGC employs a Structure-Constrained Weight Generation Mechanism. This module synthesizes task-specific weights from multi-view feature embeddings to narrow the interaction search space. Simultaneously, the Dynamic Generalization Control strategy monitors the training trajectory via an Overfitting-aware Index (OFI) to determine adaptive termination. As shown in Figure 2, performance comparisons across multiple benchmarks demonstrate that GINet-DGC achieves competitive and relatively stable results.

1.4. Main Contributions

The main contributions of this research are summarized as follows:

• Framework Proposal: We introduce GINet-DGC, an end-to-end framework tailored with structural inductive biases to resolve the strict sample constraints in biomedical HDLSS tasks.
• Weight Generation Mechanism: We develop a structure-constrained generator that fuses multi-view statistical, topological, and sparse priors to model high-dimensional gene-expression features robustly.
• Adaptive Early Stopping: We implement an Overfitting-aware Index (OFI) alongside an adaptive training strategy to monitor generalization behavior and prevent overfitting during optimization.
• Extensive Empirical Validation: We comprehensively evaluate the proposed framework on eight public gene-expression datasets using repeated stratified $5\times 5$ cross-validation against 17 baseline models, demonstrating competitive stability.

The remainder of this paper is organized as follows: Section 2 presents the materials and methods, Section 3 reports the results, Section 4 discusses the findings, and Section 5 concludes the paper.

2. Materials and Methods

The proposed GINet-DGC framework addresses the challenge of learning from biomedical high-dimensional, low-sample-size data. In these scenarios, molecular features greatly outnumber available clinical observations, meaning that increasing model depth blindly risks severe overfitting. Therefore, our approach introduces structural inductive biases to constrain the parameter solution space. We propose a unified framework that integrates a Structure-Constrained Weight Generator and a Dynamic Generalization Control adaptive stopping strategy. The source code for replication is publicly available at: https://github.com/user123h/Global_Interaction.

Figure 3 outlines the complete methodological workflow of this research, and Section 2.1, Section 2.2, Section 2.3, Section 2.4, and Section 2.5 provide detailed descriptions of the proposed framework, training process, and experimental protocol.

2.1. Model Overview

The framework consists of two core components: the Structure-Constrained Weight Generator, which integrates multi-view structural priors from high-dimensional biomedical features and synthesizes adaptive weight representations; Dynamic Generalization Control, an adaptive early stopping strategy that monitors the training trajectory under some constraints. Figure 3 outlines the overall workflow of the proposed GINet-DGC framework.

2.2. Structure-Constrained Weight Generator

In high-dimensional low-sample-size settings, directly optimizing the first-layer feature interaction matrix as an unconstrained parameter block often results in severe over-parameterization. Prior studies have shown that structured parameter generation and conditional weight prediction can effectively improve generalization by reducing the freedom of model parameters. Inspired by this idea, GINet-DGC introduces multi-view feature-level structural priors to condition the construction of the input interaction matrix. Conceptually, this mechanism resembles a structurally constrained HyperNetwork, as the interaction weights are generated under the guidance of feature descriptors rather than being freely optimized from limited training samples. By incorporating descriptor-based constraints into parameter generation, the proposed design reduces the effective search space of the optimization process and alleviates parameter redundancy, thereby improving robustness in HDLSS scenarios.

We use the following notation throughout this section. Let $X^{\mathrm{raw}}\in {\mathbb{R}}^{N_0\times D}$ denote the original input matrix. Before applying non-negative matrix factorization (NMF), invalid samples are removed. The remaining expression values are then normalized by Min–Max Scaling so that each feature is linearly mapped into the interval $[0,1]$:

$$X_{ij}={\displaystyle \frac{X_{ij}^{\mathrm{raw}}-{min}_i\left(X_{ij}^{\mathrm{raw}}\right)}{{max}_i\left(X_{ij}^{\mathrm{raw}}\right)-{min}_i\left(X_{ij}^{\mathrm{raw}}\right)}},X_{ij}\in [0,1],$$

(1)

where the minimum and maximum are computed over samples for each feature. The resulting preprocessed matrix is denoted as $X\in {\mathbb{R}}^{N\times D}$, with N valid samples and D features, and M denotes the feature-embedding dimension. GINet-DGC then applies NMF directly to decompose the preprocessed non-negative matrix X:

$$X\approx U{V}^{\top },U\in {\mathbb{R}}_{+}^{N\times M},V\in {\mathbb{R}}_{+}^{D\times M},$$

(2)

where U captures the latent sample-specific configurations, and $V={[v_1,v_2,\dots ,v_D]}^{\top }$ represents the feature-wise semantic matrix. The rows of V, denoted as $v_i\in {\mathbb{R}}^{1\times M}$, are extracted directly as the stable latent semantic embeddings for each of the D high-dimensional features. In HDLSS data, this non-negativity constraint encourages additive, part-based feature semantics can prevents arbitrary cancellation among noisy variables. Consequently, the rows of V yield stable local semantic bases, which are inherently easier to regularize than a dense global mapping [29]. Prior work demonstrates that such feature-wise embeddings V significantly improve conditional weight prediction in high-dimensional tabular learning [20].

The objective of the Structure-Constrained Weight Generator is to synthesize the first-layer projection matrix A from the structural priors extracted from V. The generation process is defined as

$$A=G_{\Theta }\left(F\left(V\right)\right)\in {\mathbb{R}}^{D\times H},$$

(3)

where H denotes the hidden width of the first classifier layer, the generator parameters are denoted by $\Theta$, and $F\left(V\right)$ denotes the fused multi-view structural representation. The generated matrix A is subsequently utilized by the task classifier $C_{\Omega }$, whose remaining trainable parameters are denoted by $\Omega$.

GINet-DGC extracts structural priors from four complementary views:

1.

Latent Semantic Prior: It constructs basic subspace constraints.

2.

Global Distributional Prior: It captures dataset-wide statistical stability.

3.

Local Topological Prior: It preserves manifold geometry among features.

4.

Hierarchical Semantic Prior: It provides multi-scale nonlinear abstractions.

Then an adaptive integration mechanism fuses these multi-view priors to form a structure-constrained representation. The generation process is summarized as follows:

$$A=G_{\Theta }\left(\mathrm{Fuse}(F_{\mathrm{lat}}\left(V\right),F_{\mathrm{dist}}\left(V\right),F_{\mathrm{topo}}\left(V\right),F_{\mathrm{hier}}\left(V\right))\right).$$

(4)

Here, $F_{\mathrm{lat}}$, $F_{\mathrm{dist}}$, $F_{\mathrm{topo}}$, and $F_{\mathrm{hier}}$ denote the four prior extraction modules. The generated matrix A strictly constrains the classifier’s first-layer search space. Independently, the Dynamic Generalization Control strategy monitors the training trajectory and terminates optimization when the validation behavior indicates generalization saturation.

2.2.1. Latent Semantic Prior

The Latent Semantic Prior module transforms the feature matrix into a compact subspace representation. Its primary role is to impose a localized structural constraint. For each feature embedding vector $v_i\in {\mathbb{R}}^{1\times M}$ (where $i=1,2,\dots ,D$), the nonlinear mapping is defined as

$$f_{\mathrm{lat}}\left(v_i\right)=\sigma (v_i{B}_{\mathrm{lat}}+b_{\mathrm{lat}})\in {\mathbb{R}}^{1\times M_{\mathrm{lat}}},$$

(5)

where $\sigma (·)$ is the nonlinear activation function, while $B_{\mathrm{lat}}\in {\mathbb{R}}^{M\times M_{\mathrm{lat}}}$ and $b_{\mathrm{lat}}\in {\mathbb{R}}^{1\times M_{\mathrm{lat}}}$ denote the learnable weight matrix and bias vector of this module, respectively. The complete latent representation for all features is stacked as $F_{\mathrm{lat}}\left(V\right)={[f_{\mathrm{lat}}\left(v_1\right),\dots ,f_{\mathrm{lat}}\left(v_D\right)]}^{\top }\in {\mathbb{R}}^{D\times M_{\mathrm{lat}}}$. This projection filters out redundant coordinate noise and maps feature-wise behaviors into a smooth, compact latent semantic space.

2.2.2. Global Distributional Prior

The Global Distributional Prior module captures the macroscopic statistical regularities of the feature-embedding space. Utilizing the embedding matrix $V\in {\mathbb{R}}^{D\times M}$, the module flattens all subspace variables to compute a global descriptor vector $s\in {\mathbb{R}}^{1\times 8}$:

$$s=[\mu ,\mathrm{Med},\sigma ,{\sigma }^2,\mathrm{Skew},\mathrm{Kurt},R,\mathrm{IQR}]\in {\mathbb{R}}^{1\times 8}.$$

(6)

Here, the vector elements denote the global mean, median, standard deviation, variance, skewness, kurtosis, range, and interquartile range of the flattened matrix V. To calibrate the importance of different statistical properties, a learnable weight vector ${\gamma }_{\mathrm{stat}}\in {\mathbb{R}}^{1\times 8}$ is introduced via element-wise multiplication.

The weighted statistical vector is expanded along the feature dimension D to construct an extended prior matrix $S_{\mathrm{ext}}\in {\mathbb{R}}^{D\times 8}$. This matrix is mapped through two linear projection layers to extract cross-descriptor interactions:

$$F_{\mathrm{dist}}\left(V\right)=\sigma (H_{\mathrm{dist}}{B}_{\mathrm{dist},2}+b_{\mathrm{dist},2})\in {\mathbb{R}}^{D\times M_{\mathrm{dist}}},$$

(7)

where $H_{\mathrm{dist}}=\sigma ((S_{\mathrm{ext}}\odot \Gamma )B_{\mathrm{dist},1}+b_{\mathrm{dist},1})\in {\mathbb{R}}^{D\times H_{\mathrm{dist}\_\mathrm{mid}}}$. The matrix $\Gamma \in {\mathbb{R}}^{D\times 8}$ is broadcasted from ${\gamma }_{\mathrm{stat}}$, and ⊙ denotes the Hadamard product. This layout restricts the classifier’s search space based on dataset-wide regularities.

2.2.3. Local Topological Prior

The Local Topological Prior module models the neighborhood geometry and manifold patterns within the latent subspace [30]. It extracts structural descriptors from V via a joint soft-clustering and K-nearest neighbor formulation.

The module maintains $K_c$ learnable cluster centers $C={[c_1,c_2,\dots ,c_{K_c}]}^{\top }\in {\mathbb{R}}^{K_c\times M}$. For each feature vector $v_i\in {\mathbb{R}}^{1\times M}$, the soft assignment $p_{ij}$ to center $c_j$ is defined as

$$p_{ij}={\displaystyle \frac{exp(-\parallel v_i-c_j{\parallel }^2·w_j/\tau )}{{\sum }_{j^{\prime }=1}^{K_c}exp(-\parallel v_i-c_{j^{\prime }}{\parallel }^2·w_{j^{\prime }}/\tau )}},$$

(8)

where $\tau$ is a learnable temperature, and $w_j\in \mathbb{R}$ is the j-th element of the cluster scaling vector $w\in {\mathbb{R}}^{K_c}$. Concurrently, the module extracts local density by computing Euclidean distances to the $K_n$ nearest neighbors:

$$d_{\mathrm{knn}}\left(v_i\right)=[d_{i\left(1\right)},d_{i\left(2\right)},\dots ,d_{i\left(K_n\right)}]\in {\mathbb{R}}^{1\times K_n}.$$

(9)

The soft-clustering assignments and KNN distance profiles jointly form the topological descriptor $S_{\mathrm{topo}}\in {\mathbb{R}}^{D\times (K_c+K_n)}$, capturing both coarse cluster affinity and local neighborhood geometry. The final prior is generated via a linear projection layer:

$$F_{\mathrm{topo}}\left(V\right)=\sigma (S_{\mathrm{topo}}{B}_{\mathrm{topo}}+b_{\mathrm{topo}})\in {\mathbb{R}}^{D\times M_{\mathrm{topo}}},$$

(10)

where $B_{\mathrm{topo}}\in {\mathbb{R}}^{(K_c+K_n)\times M_{\mathrm{topo}}}$ and $b_{\mathrm{topo}}\in {\mathbb{R}}^{1\times M_{\mathrm{topo}}}$ are learnable weights and biases.

2.2.4. Hierarchical Semantic Prior

To capture dependencies across semantic scales, this module employs L parallel shallow subnetworks, each projecting the embedding matrix V into a scale-specific representation:

$$H_l=\sigma (\sigma (V{B}_{l,1}+b_{l,1})B_{l,2}+b_{l,2})\in {\mathbb{R}}^{D\times M_{\mathrm{level}}},l=1,\dots ,L,$$

(11)

where $B_{l,1}\in {\mathbb{R}}^{M\times M_{\mathrm{mid}}}$ and $B_{l,2}\in {\mathbb{R}}^{M_{\mathrm{mid}}\times M_{\mathrm{level}}}$ represent level-specific parameters. All branch outputs are concatenated into a joint tensor $H_{\mathrm{comb}}=[H_1\Vert H_2\Vert \dots \Vert H_L]\in {\mathbb{R}}^{D\times (M_{\mathrm{level}}·L)}$. The hierarchical prior is synthesized via a final fusion layer:

$$F_{\mathrm{hier}}\left(V\right)=\sigma (H_{\mathrm{comb}}{B}_{\mathrm{fusion}}+b_{\mathrm{fusion}})\in {\mathbb{R}}^{D\times M_{\mathrm{hier}}},$$

(12)

where $B_{\mathrm{fusion}}\in {\mathbb{R}}^{(M_{\mathrm{level}}·L)\times M_{\mathrm{hier}}}$ and $b_{\mathrm{fusion}}$ denote the learnable fusion weights and biases.

2.2.5. Adaptive Prior Integration and Weight Generation

The integration layer models the non-linear interactions across the multi-view descriptors. The structural prior fields are concatenated to form a unified tensor $F_{\mathrm{all}}\left(V\right)=[F_{\mathrm{lat}}\left(V\right)\Vert F_{\mathrm{dist}}\left(V\right)\Vert F_{\mathrm{topo}}\left(V\right)\Vert F_{\mathrm{hier}}\left(V\right)]\in {\mathbb{R}}^{D\times M_{\mathrm{all}}}$. To adjust feature-wise contributions dynamically, a multi-head self-attention mechanism [31] maps the contextual dependencies:

$$F_{\mathrm{fuse}}=\mathrm{Softmax}\left({\displaystyle \frac{\left(F_{\mathrm{all}}{B}_Q\right){\left(F_{\mathrm{all}}{B}_K\right)}^{\top }}{\sqrt{d_k}}}\right)\left(F_{\mathrm{all}}{B}_V\right)\in {\mathbb{R}}^{D\times M_{\mathrm{all}}},$$

(13)

where $B_Q,B_K,B_V\in {\mathbb{R}}^{M_{\mathrm{all}}\times M_{\mathrm{all}}}$ are learnable projection weights. This Transformer-based layout enhances global dependency modeling and representational flexibility.

Finally, the weight generator maps the fused representation $F_{\mathrm{fuse}}$ to synthesize the target task projection matrix A:

$$W=\sigma (\sigma (F_{\mathrm{fuse}}{B}_{\mathrm{gen},1}+b_{\mathrm{gen},1})B_{\mathrm{gen},2}+b_{\mathrm{gen},2})B_{\mathrm{gen},3}+b_{\mathrm{gen},3}\in {\mathbb{R}}^{D\times H},$$

(14)

$$A=W^{\top }\in {\mathbb{R}}^{H\times D},$$

(15)

where $B_{\mathrm{gen},1},B_{\mathrm{gen},2},B_{\mathrm{gen},3}$ represent the sequential linear layers, and the transpose operator ensures that the spatial dimensions of A match the first classifier layer exactly.

2.3. Dynamic Generalization Control

In HDLSS tasks, overfitting is not only a final-state problem but also a trajectory problem: the model may continue reducing training loss after validation performance has saturated or begun to deteriorate. Therefore, rather than relying on standard fixed stopping rules [32,33], we introduce an adaptive early stopping strategy termed Dynamic Generalization Control. DGC closely tracks the training trajectory to determine if the optimization process has entered an overfitting regime, where subsequent parameter updates serve primarily to widen the generalization gap.

The stopping signal is the Overfitting-aware Index, which integrates three factors: accuracy discrepancy, loss discrepancy, and validation-performance deterioration. At iteration t, it is defined as

$${\mathrm{OFI}}_t=\alpha ({\mathrm{Acc}}_{\mathrm{train}}^t-{\mathrm{Acc}}_{\mathrm{val}}^t)+\beta ({\mathrm{Loss}}_{\mathrm{val}}^t-{\mathrm{Loss}}_{\mathrm{train}}^t)+\gamma r_t,$$

(16)

where $r_t$ denotes the validation deterioration trend over a recent window of iterations. A low deterioration trend reduces the overfitting warning, whereas a larger deterioration trend increases the warning when it co-occurs with growing train–validation gaps.

To avoid a manually fixed stopping threshold, DGC computes an adaptive threshold from the historical OFI distribution:

$${\tau }_t={\mu }_{\mathrm{OFI}}^t+\kappa {\sigma }_{\mathrm{OFI}}^t,$$

(17)

where ${\mu }_{\mathrm{OFI}}^t$ and ${\sigma }_{\mathrm{OFI}}^t$ are the mean and standard deviation of previous OFI values, and $\kappa$ controls threshold tolerance. Training is terminated only when ${\mathrm{OFI}}_t>{\tau }_t$ for P consecutive validation checks. The OFI-related hyperparameters, including $\alpha$, $\beta$, $\gamma$, $\kappa$, and P, are determined by automatic hyperparameter optimization rather than manual assignment. This patience-based decision separates the trajectory metric from the stopping rule and reduces sensitivity to transient validation fluctuations.

2.4. Training Process

The training objective is to minimize the cross-entropy loss between predictions and true labels:

$$\mathcal{L}(\Theta ,\Omega )=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{c=1}^C}{y}_c^{i}log\left({\widehat{y}}_c^i\right),$$

(18)

where $y_c^i$ is the true label indicator and ${\widehat{y}}_c^i$ represents the predicted probability. The training procedure is summarized in Algorithm 1.

2.5. Experimental Setup

2.5.1. Experimental Data

We evaluated the proposed framework on eight public high-dimensional low-sample size biomedical gene-expression datasets, spanning 50 to 203 samples and 3,312 to 22,283 features. Detailed dataset information is provided in Appendix A.

To ensure reproducibility, we performed repeated stratified 5-fold cross-validation with five independent repeats. Feature embeddings were generated independently within each training fold of the repeated stratified 5-fold cross-validation procedure. For each run, 80% of the data was allocated for development and 20% for testing. Within the development split, 10% of the samples were reserved as a validation subset for hyperparameter tuning and Dynamic Generalization Control. We report the mean balanced accuracy over all 25 test evaluations as the final performance metric.

2.5.2. Evaluation Metrics

We adopt Balanced Accuracy as the primary evaluation metric. Unlike traditional accuracy, balanced accuracy objectively reflects classification capability under class imbalance and small-sample conditions. In high-dimensional datasets, class distribution is often skewed. Standard accuracy can be misleading, as models may bias toward the majority class, obscuring their ability to recognize minority samples.

Balanced accuracy measures performance by averaging the recall of each class:

$$\mathrm{BA}={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{\displaystyle \frac{T{P}_c}{T{P}_c+F{N}_c}}$$

(19)

where C denotes the number of classes, and $T{P}_c$ and $F{N}_c$ represent true positives and false negatives for class c.

This metric provides a stable reflection of discriminative ability, especially in cases of severe imbalance or limited sample size.

2.5.3. GINet-DGC Settings

The classification backbone of GINet-DGC consists of three fully connected layers. The final layer uses Softmax for class distribution. The Structure-Constrained Weight Generator includes four prior extraction modules: a statistical feature extractor(64-dim output), a low-dimensional semantic encoder(64-dim output), a topological clustering module (32-dim output, 8 centers), and a multi-level feature module (64-dim output). The adaptive attention mechanism outputs 128-dimensional fused features. Finally, the generator produces 100-dimensional weights.

For the Dynamic Generalization Control adaptive stopping strategy, the OFI-related hyperparameters were obtained through automatic hyperparameter optimization on the validation subset. The selected trajectory weights in the reported experiments were $\alpha =1.0$ (accuracy gap), $\beta =0.5$ (loss gap), and $\gamma =0.5$ (validation deterioration trend), with threshold tolerance and patience parameters selected by the same validation-search procedure. We employ Batch Normalization [34] and Dropout [35]. LeakyReLU is used for activation to improve stability.

The model is trained using AdamW [36] with a batch size of 8. The maximum training steps are set to 5000.

3. Results

3.1. Baselines

We compare GINet-DGC against 17 representative baselines. These cover traditional machine learning, modern neural architectures, and specialized HDLSS approaches. For fairness, all baseline methods are evaluated on the same eight public datasets and under the same repeated stratified 5×5 cross-validation protocol, and the reported balanced accuracy is averaged over the resulting 25 evaluations. We do not directly copy performance numbers from prior publications when their datasets, generated samples, preprocessing pipelines, or split protocols are not fully aligned with ours.

For traditional methods, we selected Random Forest [37] and LightGBM [11]. These represent bagging and gradient boosting frameworks. They are known for robustness on sparse data.

For deep tabular models, we include standard MLP, TabNet [13], TabTransformer [14], LassoNet [38], DNP [39], and Hypertab [18]. We also include TabPFN [26], a recent foundation model for tabular learning. TabPFN leverages Bayesian priors and conditional sampling, representing a strong small-sample tabular baseline.

For feature generation and structured learning, we evaluate DietNetworks [19], FsNet [16], SPINN [40], and WPFS [20]. We also evaluate GCondNet [21], GCN [41], and GATv2 [42]. These methods enhance representation through mechanisms like conditional weight prediction and implicit relational modeling.

3.2. Benchmark Comparison

Table 1 summarizes the mean balanced accuracy of all evaluated methods across the eight biomedical HDLSS datasets under repeated stratified 5×5 cross-validation.

As shown in Table 1, GINet-DGC demonstrates competitive performance across eight high-dimensional small-sample biomedical datasets. This supports the usefulness of its structural modeling and adaptive stopping strategy for bioinformatics prediction benchmarks. Specifically, GINet-DGC ranks among the top three on all datasets and achieves the highest balanced accuracy on gli, smk, allaml, glioma, toxicity, and lung. Its average rank is 1.50, lower than other recent competitive models such as GCondNet (3.00) and TabPFNTransformer (5.88). This indicates relatively stable performance across the evaluated public biomedical benchmarks.

Regarding model categories, traditional tree-based models remain competitive in certain biomedical tasks. They benefit from robustness to low-order statistics. However, they often underperform compared to deep modeling methods. This is because tree models rely on local greedy partitioning. They struggle to capture global dependencies among high-dimensional molecular variables. This restricts their representation power.

Deep tabular models generally perform stably. Models like DNP and TabPFN show strong generalization. However, they rely on complex priors or pretraining. This may increase sensitivity to distribution shifts between biomedical cohorts or assay conditions. Notably, GINet-DGC achieves competitive performance without external pretraining.

Among feature generation methods, SPINN and GCondNet achieve strong results. This demonstrates the value of explicit relation modeling. However, they rely on strong local assumptions. This makes them sensitive to noise. In contrast, our method models high-order interactions globally while preserving local constraints. This enhances robustness under complex biomedical feature distributions.

Overall, GINet-DGC maintains competitive performance in the evaluated biomedical HDLSS conditions. This supports the usefulness of its structured global interaction design and adaptive early stopping strategy for data-scarce bioinformatics prediction tasks.

3.3. Ablation Study

3.3.1. Multi-View Structural Priors

To quantify the contribution of each prior branch, we conducted an ablation study by systematically removing individual modules. Figure 4 summarizes the resulting performance variations.

The performance degradation exhibits clear task-dependent patterns, reflecting the structural heterogeneity inherent in HDLSS data. Specifically, isolating or removing the hierarchical semantic or local topological prior yields the most pronounced performance drops across the majority of tasks. This confirms that different datasets rely on distinct dominant geometric or categorical regularities. Rather than displaying uniform dominance, the multi-view descriptors offer complementary structural constraints. These findings validate the necessity of our adaptive fusion layer, which dynamically calibrates multi-view representations instead of assuming static, equal contributions across tasks.

3.3.2. Adaptive Prior Integration

To evaluate the attention module, we conducted an ablation by removing it while keeping all feature layers. Figure 5 reports the comparison between the full model and the variant without attention. As shown in Figure 5, the attention mechanism contributes to performance. Removing it leads to a noticeable drop, with the most pronounced decline on the glioma dataset ($>40\%$ decrease). This indicates that attention enables adaptive weighting. It strengthens important channels and suppresses redundancy. This enhances feature fusion.

We also investigated the dependency on multi-view features via a “J-Attention” experiment (attention only, no structural priors). Figure 6 presents the comparison between this attention-only variant and the full GINet-DGC model. As shown in Figure 6, J-Attention performs worse than the full model in the evaluated setting. This indicates that attention alone is limited. It requires structured multi-level features to be effective. Consequently, it fails to capture implicit structures in high-dimensional data without them. Therefore, the attention module operates synergistically with the multi-layer embedding. It is not an isolated factor.

In summary, the structured weight generation strategy improves stability in the evaluated settings. We conducted further comparisons with other initialization strategies (Appendix B), which further support the usefulness of the proposed structure-constrained generation mechanism.

3.4. Analysis of Dynamic Generalization Control

To verify the DGC adaptive stopping strategy, we compared models with and without it on eight datasets. Figure 7 summarizes the effect of DGC on both predictive performance and convergence behavior. As shown in Figure 7, DGC-equipped models achieve an average improvement of ~1% on complex datasets (e.g., smk, glioma). They also reduce convergence steps in some cases (e.g., from 2575 to 1587 in glioma). This indicates that the adaptive OFI threshold can replace fixed early stopping. It allows the stopping decision to adjust based on training dynamics. This avoids underfitting and reduces over-iteration. In summary, DGC enhances generalization and improves efficiency. This validates the practical value of the adaptive early stopping strategy. Applying this mechanism to GCondNet (Appendix B) also yielded gains, demonstrating transferability.

3.5. Training Dynamics

Figure 8 compares the training processes of GINet-DGC and MLP. GINet-DGC exhibits brief early oscillation, followed by rapid stabilization. It maintains higher validation accuracy and lower loss throughout. According to literature [43], early oscillations relate to adaptive parameter adjustment. The network aligns effective feature directions under small-scale initialization. Overall, GINet-DGC demonstrates faster convergence and greater stability than MLP in this example, supporting its robustness in high-dimensional, small-sample biomedical prediction tasks.

4. Discussion

The empirical results demonstrate that GINet-DGC effectively addresses the fundamental challenges of High-Dimensional Low-Sample Size (HDLSS) learning by combining structure-constrained weight generation with adaptive generalization monitoring. Rather than relying on external pretraining or data augmentation, our framework leverages feature-wise structural priors to provide robust mathematical constraints. This methodology successfully regularizes the classification search space when sample sizes are heavily restricted and features are highly redundant.

A key structural finding from the ablation study is that individual prior branches exhibit task-dependent importance, reflecting the distinct geometric or categorical regularities inherent in heterogeneous tabular data. This underscores the necessity of our adaptive fusion layer, which dynamically calibrates multi-view representations instead of assuming static, uniform contributions.

Crucially, the underlying weight generation and trajectory monitoring mechanisms in GINet-DGC hold the potential to extend to broader tabular tasks. The framework could be adapted to other scenarios characterized by high-dimensional, small-sample constraints, ranging from clinical biomarker identification to specific prediction tasks in alternative scientific domains.

Future research will expand in three strategic directions: (1) investigating the direct biological interpretation and visualization of the generated projection weights to map downstream molecular patterns; (2) validating the framework’s scalability on significantly larger, diverse tabular datasets to test its broader generalization limits; and (3) OFI is composed of multiple metric terms whose weighting coefficients are currently determined mainly through validation-based search,future work could explore a dynamic balancing mechanism that adjusts the relative contributions of OFI components during training, rather than relying only on fixed hyperparameters or conventional parameter-search procedures.

5. Conclusions

In this study, we proposed GINet-DGC to address the optimization and over-iteration challenges in High-Dimensional Low-Sample Size learning. By using a Structure-Constrained Weight Generator to synthesize projection weights from multi-view structural priors, the framework effectively restricts the feature-interaction search space under limited supervision. Concurrently, the Overfitting-aware Index enables adaptive training termination before performance saturates. Extensive evaluations across eight typical biomedical datasets demonstrate that our method achieves competitive classification performance and relatively stable generalization behavior. These findings support GINet-DGC as a viable and structurally constrained modeling strategy for data-scarce bioinformatics benchmarks.