Machine Learning Meets Meta-heuristics: Bald Eagle Search Optimization and Red Deer Optimization for Feature Selection in Type II Diabetes Diagnosis

1. Introduction

India faces a significant public health challenge with the escalating prevalence of diabetes. According to the International Diabetes Federation (IDF) Atlas data from 2021, a staggering 77 million adults in India already suffer from this chronic condition. Even more alarming are projections that this number will reach a critical mass of 134 million by 2045. This translates to roughly one in eight adults in India potentially having diabetes [1]. Particularly worrisome is the increasing number of young adults (below 40 years old) being diagnosed with type 2 diabetes.

Several factors contribute to this alarming rise. India’s rapid urbanization is linked to lifestyle changes that heighten diabetes risk. These changes include reduced physical activity levels, increased consumption of processed foods, and rising obesity rates. Additionally, genetic predisposition, stress, and environmental factors all play a role [2]. Unfortunately, many diabetes cases remain undiagnosed until complications arise, highlighting the need for increased awareness and early detection strategies.

The human cost of diabetes in India is immense, with this disease ranking as the seventh leading cause of death. The economic burden is equally concerning. It’s estimated that diabetes drains the Indian economy of approximately $100 billion annually. Clearly, addressing this growing public health crisis requires a multi-pronged approach that focuses on preventive measures for type 2 diabetes, early diagnosis for all types, and proper management of the condition [3].

1.1. Review of Related Works

Early and accurate diagnosis of diabetes is critical for effective management, but traditional methods like blood glucose tests have limitations. Microarray gene technology offers a promising alternative by analyzing gene expression patterns in the pancreas, potentially revealing early signs of the disease [4]. Recent advancements in machine learning algorithms have further enhanced their effectiveness for diabetes detection. A novel Convolutional Neural Network (CNN) architecture specifically designed for diabetes classification using electronic health records (EHR) data [5]. Their model achieved an impressive accuracy of 85.2%, demonstrating the potential of deep learning for diabetes diagnosis from readily available clinical data. The use of Explainable Artificial Intelligence (XAI) techniques with machine learning models for diabetes prediction [6]. Their XAI approach not only achieved high accuracy but also provided insights into the factors most influencing model predictions. This interpretability is crucial for clinicians to understand and trust the model’s recommendations [7]. Certain investigations were made to use of machine learning models that combine various data sources, including gene expression data, clinical information, and lifestyle factors. Their findings suggest that integrating multimodal data sets can lead to more accurate and comprehensive diabetes risk prediction models [8]. Ensemble learning approaches continue to show strong performance. Specific algorithms like Random Forest and Deep Neural Networks have proven effective in various studies[9]. Microarray gene data analysis remains a valuable avenue for exploration. While traditional datasets have dominated research in machine learning for diabetes detection, microarray gene data offers a relatively unexplored avenue[10]. This presents an opportunity to investigate the effectiveness of various performance metrics such as accuracy, F1 Score, Mathew Correlation Coefficient, Jaccard Metrics, Error Rate and Kappa.

2. Materials and Methods

As depicted in Figure 1, this research utilizes a multi-stage approach for accurate diabetic detection. The first stage focuses on extracting relevant features from the microarray gene data. Three techniques are utilized for this purpose such as STFT, RR, PCC. The second stage, which is two roots, one is extracted data is directly given to classifiers to measure the performance and second one is given to feature selection process. In this research, FS used two meta-heuristic algorithms, BESO and RDO. These algorithms can help identify the most informative features from the initial extraction stage, potentially leading to a more efficient classification process. Finally, the third stage utilizes various classification techniques to categorize the data into diabetic and non-diabetic classes by using seven methods as NLR, LR, GMM, EM, LoR, SDC, and SVM-RBF.

2.1. Dataset Details

Our research focused on utilizing microarray gene data from human pancreatic islets to detect diabetes and explore potential features associated with the disease. The data originated from the Nordic islet transplantation program (https://www.ncbi.nlm.nih.gov/bioproject/PRJNA178122) and included gene expression profiles from 57 healthy individuals and 20 diabetic patients. To ensure data quality and facilitate analysis, preprocessing steps were implemented. First, genes with the highest peak intensity per patient were selected, resulting in a dataset of 22,960 genes. Next, a logarithmic transformation (base 10) was applied to standardize the data across samples, ensuring a mean of 0 and a variance of 1. This step helps to normalize gene expression values and account for potential variations between samples. A key challenge in microarray data analysis is dealing with high dimensionality, referring to the large number of genes present. To address this, we employed feature extraction techniques. These techniques aim to reduce the number of features while retaining the most informative content relevant to diabetes detection [11]. Following feature extraction, feature selection techniques were applied to further refine the dataset and potentially improve classification accuracy. Specifically, two optimization algorithms were utilized, BESO [12] and RDO [13]. These algorithms helped identify the most relevant features for diabetic detection, further reducing the dimensionality of the data. Finally, to evaluate the performance and accuracy of diabetes classification, seven different classification algorithms were used. These algorithms will be discussed in detail later in the article.

2.2. Need for Feature Extraction (FE)

Microarray data analysis presents a challenge due to its high dimensionality. This means the data includes a vast number of genes, which can be overwhelming for classification algorithms. A high number of features can lead to a phenomenon called the “curse of dimensionality.” This occurs when the number of features [14] grows exponentially, making it difficult for classification algorithms to learn effective decision boundaries and potentially leading to overfitting. Feature extraction techniques help address this challenge by reducing the number of features while retaining the most informative ones for diabetes detection. This process essentially focuses the analysis on the most relevant aspects of the gene expression data that contribute to differentiating between diabetic and non-diabetic samples. FE acts as a filter, selecting the most informative features from the vast amount of gene expression data. This allows the classification algorithms to work with a more manageable dataset and ultimately leads to more accurate diabetes detection.

2.2.1. Short-Time Fourier Transform (STFT)

The STFT offers a valuable technique for analyzing microarray gene expression data. It excels at capturing frequency domain information within specific time windows. This allows researchers to condense the dataset by extracting key features that represent how gene expression levels change over time and across various frequencies. Like its application in QRS complex detection by [15], STFT provides a time-frequency representation of gene expression data. This visualization facilitates exploration of how gene expression levels fluctuate across time and different frequency components. A key strength of STFT lies in its ability to pinpoint specific time intervals where frequencies are dominant. This allows researchers to identify genes with temporal patterns that activate during distinct time periods within the data. Additionally, STFT contributes to dimensionality reduction by extracting significant genes or groups of genes associated with specific frequency components [16]. This targeted approach streamlines the exploration of gene expression dynamics, ultimately aiding in the discovery of biologically relevant patterns within the data.

$$X\left[m,w\right]={\sum }_{n=-\infty }^{\infty }{x\left[n\right]w\left[n-m\right]e}^{-iwn}$$

(1)

Where $x\left[n\right]$ is the input data where length N represents from 0,1,2,..,N-1; $w\left[m\right]$ is the representation for STFT window with ranges from 0,1,2,…,M-1. Moreover $i=\sqrt{(-1)}$ is the complex variable.

2.2.2. Ridge Regression (RR)

The analysis begins by focusing on smaller groups of samples. For each group, a feature matrix ($X_i$) is created. This matrix includes information about each sample as data point in a row, along with its corresponding outcome in an outcome vector ($Y_i$) [17]. Ridge regression is then applied to this local data to identify patterns. The estimates obtained by applying ridge regression to these local groups are referred to as local ridge regression estimates.

$${\widehat{\beta }}_i={(X_i^T{X}_i+{\lambda }_i I_p)}^{-1} X_i^T{Y}_i$$

(2)

$${\widehat{\beta }}_{dist}={\sum }_{i=1}^q\left({\omega }_i{\widehat{\beta }}_i\right)$$

(3)

the framework of finite sample analysis for linear models. In this context, we consider the standard linear model, where Y represents the outcome variable, X is the feature matrix, b is the coefficient vector, and ε represents the error term. The outcome for each of the ‘n’ independent samples is represented by $Y \in {\mathfrak{R}}^n$ a continuous vector with ‘n’ dimensions. The study utilizes a data matrix, X, to represent the features of the samples. This matrix has dimensions n x p, where ‘n’ represents the number of samples and ‘p’ represents the number of features measured for each sample. We’re also interested in a vector of coefficients, denoted by beta (β). $\beta ={({\beta }_1, {\beta }_2{, \beta }_3, ..., {\beta }_p )}^T \in {\mathfrak{R}}^n$ This vector has ‘p’ dimensions and contains unknown values, which aim to estimate through analysis. These coefficients will ultimately tell how much each feature contributes to the outcome variable we’re trying to predict. This technique is employed for two main purposes: predicting the outcome variable for new samples and accurately estimating the influence of each feature (coefficient) on that outcome. However, it’s important to consider that random errors or noise can significantly impact the outcome vector, $\epsilon ={({\epsilon }_1, {\epsilon }_2, ..., {\epsilon }_n)}^{T }\in {\mathfrak{R}}^n$which represents the actual values that are trying to predict[18].

In linear models, we often assume that the errors in the outcome variable (represented by the vector ε) are independent of each other, have an average value of zero, and a constant variance ${\sigma }^2$. Ridge regression is a popular technique for estimating the coefficients (β). Here the equation for the same is,

$$\widehat{\beta } \left(\lambda \right)={(X^{T}X+n\lambda I_p)}^{-1}{X}^{T}Y$$

(4)

The ridge regression technique incorporates a parameter known as lambda (λ), which plays a crucial role in tuning the model. This parameter offers several advantages when it comes to estimating the coefficients (β). One key benefit is that it helps to ‘shrink’ the coefficients obtained from ordinary least squares regression, often leading to improved estimation accuracy. Now, let’s consider a scenario where our data samples are spread across multiple locations or machines, represented by ‘q’ in total. In this case, we can partition the data for analysis as

$$X=\left[\begin{array}{c}{X}_1\\ ⋮\\ X_q\end{array}\right] and Y=\left[\begin{array}{c}{Y}_1\\ ⋮\\ Y_q\end{array}\right];$$

(5)

In distributed computing environments, where data is spread across multiple machines, approximate solutions for ridge regression are often preferred. This approach involves performing ridge regression locally on smaller subsets of the data ($X_i , Y_i)$ at each machine[19]. One popular technique focuses on a one-shot weighting approach, where the final coefficients are obtained by combining the locally estimated coefficients using weighted sums. This weighting method offers a valuable starting point for more iterative techniques used in distributed regression. Additionally, this approach allows researchers to explore new characteristics of these one-shot weights.

To achieve this, local ridge regression with a regularization parameter ${\lambda }_i$ is performed on each data subset ($X_i , Y_i)$. The formula for these local ridge estimators is as follows:

$${\widehat{\beta }}_i\left({\lambda }_i\right)={(X_i^T{X}_i+n_i {\lambda }_i I_p)}^{-1} X_i^T{Y}_i$$

(6)

Here,

• ${\widehat{\beta }}_i$represents the locally estimated coefficient vector for a specific data subset.
• $X_i$ denotes the design matrix for the ${\mathrm{i}}^{\mathrm{t}\mathrm{h}}$ data subset.
• ${\mathrm{Y}}_{\mathrm{i}}$ represents the outcome vector for the ${\mathrm{i}}^{\mathrm{t}\mathrm{h}}$ data subset.
• ${\lambda }_{\mathrm{i}}$ is the regularization parameter for the local ridge regression on the i-th subset.
• I is the identity matrix with the same dimension as $X_i^T{X}_i$.

By using a weighted one-shot distributed estimation summation, the local ridge estimators from each data subset are combined as:

$${\widehat{\beta }}_{distributed}={\sum }_{i=1}^q\left({\omega }_i{\widehat{\beta }}_i\right)$$

(7)

where,

• ${\widehat{\beta }}_{distributed}$represents the final combined coefficient vector obtained from all data subsets.
• ${\omega }_i$ represents the weight assigned to the local estimator from the ${\mathrm{i}}^{\mathrm{t}\mathrm{h}}$ data subset.
• ${\widehat{\beta }}_i$ represents the locally estimated coefficient vector for the ${\mathrm{i}}^{\mathrm{t}\mathrm{h}}$ data subset (as defined earlier).

Unlike Ordinary Least Squares (OLS), the local ridge regression estimators we defined previously have some inherent bias. Due to this bias, there is no effect of imposing constraints on the weights used to combine them. This approach is particularly well-suited for data matrices (X) with any covariance structure ($\mathsf{\Sigma }$). Assuming we have ‘n’ samples that are equally distributed across different machines, we can compute a local ridge estimator (${\widehat{\beta }}_1$) for each data subset[20]. Additionally, we can estimate local values for the signal-to-noise ratio (SNR, represented by ${\widehat{\sigma }}_i^2$ and the noise level ${\widehat{a}}_i^2$ for each subset. To find the optimal weights for combining these local estimators, we consider three key factors: m, m0, and λ.

2.2.3. Pearson Correlation Coefficient (PCC)

Pearson’s Correlation Coefficient (PCC) [21] serves as a powerful statistical tool for analyzing microarray gene expression data in diabetes detection. It measures the strength and direction of the linear relationship between gene expression levels. High PCC values, positive or negative, indicate that gene expression levels tend to fluctuate together. This co-fluctuation can reveal underlying biological processes. Genes with strong positive PCC may be regulated by similar mechanisms, potentially impacting diabetes development. Conversely, genes with strong negative PCC might play opposing roles in cellular processes, with one potentially protective against diabetes while the other linked to disease progression.

By analyzing PCC values between gene pairs, we can identify potentially valuable genes for diabetes detection. Genes with strong positive correlations may act together to influence the diabetic state. Conversely, genes with strong negative correlations might represent opposing pathways with implications for disease development [22].

PCC analysis helps us understand how gene expression levels co-vary, ultimately aiding in the selection of informative gene sets that can be used to build robust models for accurate diabetes classification.

$$\rho \left(x,y\right)=\frac{cov (x, y)}{{\sigma }_x{\sigma }_y}$$

(8)

$$cov \left(x, y\right)=\frac{\sum _{i=1}^n\left(x_i-\overline{x}\right)(y_i-\overline{y})}{n-1}$$

(9)

$cov(x,y)$ Indicates the covariances between x and y.

$${\sigma }_x=\sqrt{\frac{\sum _{i=1}^n{\left(x_{i-}\overline{x}\right)}^2}{n-1}}$$

(10)

$${\sigma }_y=\sqrt{\frac{\sum _{i=1}^n{\left(y_{i-}\overline{y}\right)}^2}{n-1}}$$

(11)

${\sigma }_x, {\sigma }_y$ indicated the Standard Deviation of x and y.

From the Table 1, analyzing various features extracted from the data, STFT and RR shows minimal class differences in mean of 40.7 and variance is around 11,700 between diabetic and non-diabetic groups, suggesting they might not be strong discriminators for diabetes. Slight variations in Skewness and Kurtosis for most features hint at some distributional differences between the classes. Sample and Shannon Entropy values are identical within each class but differ between them, potentially indicating distinct underlying data patterns. Higuchi Fractal Dimension [23], however, stands out with a clear distinction (around 1.1 for STFT and 2.0 for Diabetic Class data) between classes for all features, suggesting its potential value in diabetes detection. Finally, Canonical Correlation Analysis (CCA) reveals a significant difference between diabetic and non-diabetic classes for the Diabetic Class data (0.4031 vs. 0.0675). This highlights CCA’s potential for identifying features within the Diabetic Class data that are relevant for diabetes classification[24]. Overall, the table emphasizes the importance of analyzing various statistical parameters to understand the characteristics of different features and their potential role in differentiating diabetic and non-diabetic samples. The observed variations in skewness and kurtosis suggest the data may not follow a typical Gaussian (normal) distribution and might exhibit non-linear relationships. This can be further confirmed by visualizing the data using techniques like histograms, normal probability plots, and scatter plots of the feature extraction outputs.

Figure 2 shows the Normal Probability plot of STFT for both diabetic and non-diabetic class, from the data points 1 to 4 as reference points, and from 5 to 8 as upper bounds and from 9 to 12 as clustered variable points.

Figure 3 shows the histograms of the RR FE technique applied to both diabetic and non-diabetic classes. While the distributions appear somewhat bell-shaped, suggesting a possible tendency towards normality, a closer look reveals some deviations. Notably, there is an overlap between the distributions of the diabetic and non-diabetic classes. This overlap, along with potential deviations within each class distribution, suggests a more complex, non-linear relationship between the classes.

Figure 4 displays histogram of the PCC FE technique applied to both diabetic and non-diabetic classes. The markers on the x-axis represent patients, with x(:,1) indicating patient 1 and x(:,10) indicating patient 10. The histograms reveal a skewed distribution of values, suggesting a non-normal data pattern. Additionally, a gap and potential non-linearity are observed in the data distribution for this method.

Fron all the techniques of FE, it is revealed that, non-gaussian, non-linearity and techniques highlight the complexity of the data is observed across the classes and the potential for further feature selection is essential.

3. Feature Selection Method

As mentioned in section 1, there are 2 meta-heuristic algorithms namely BESO and RDO used as feature selection techniques that should be compared with and without FS method from FE techniques to classifiers for enhancement of classifier performance.

3.1. Bald Eagle Search Optimization (BESO)

Bald Eagle Search Optimization (BESO) is a recent addition to the field of Meta-heuristic Algorithms [25]. It draws inspiration from the hunting behavior of bald eagles, specifically their adeptness in locating and capturing prey. During the optimization process, BESO utilizes various search strategies modeled on these predatory behaviors. These strategies allow the search agents to explore the search space effectively, ultimately leading to the identification of optimal solutions for complex real-world problems.

Step 1: Simulating Exploration: Selecting the Search Area

The BESO algorithm mimics the exploration phase of a bald eagle’s hunt through its first stage. In this stage, the algorithm determines a suitable search area for potential solutions, like how a bald eagle might scout a vast region for prey, which is mathematically stated as

$$P_{new}^t=P_{Best}^t+ar(P_{mean}^t-P_i^t)$$

(12)

A control parameter $\left(a\right)$ set between 1.5 and 2 guides the search area’s size, while a random number ($r$) between 0 and 1 introduces an element of chance, preventing stagnation in local optima. This exploration is further influenced by two key factors. The algorithm considers the best solution found so far ${(P}_{Best}^t)$ to guide its search towards promising regions. Additionally, the average position of the current eagle population $\left(P_{mean}^t\right)$ is factored in to ensure exploration extends beyond the immediate vicinity of the best solution and delves into new areas. By combining these elements - a controlled search area size, random exploration, and consideration of both the best-found solution and the average population position – [26] BESO effectively navigates the search space during the first stage, laying the groundwork for identifying optimal solutions in later stages.

Step 2: Intensifying the Search

The second stage of BESO intensifies the search for optimal solutions within the exploration area identified in phase one. This mimics the behavior of a bald eagle that has located a promising hunting ground. Here, the search agents meticulously scan the defined region. They follow a spiral-like pattern, like how a bald eagle might meticulously search a lake for fish. This focused search pattern allows the algorithm to efficiently explore the selected area and identify potential solutions with greater precision. The mathematically expression as,

$$P_{new}^t=P_i^t+y_i X (P_i^t-P_{i+1}^t)+x_i X (P_i^t-P_{mean}^t)$$

(13)

Where,

$$x_i=\frac{x{r}_i}{max\left|xr\right|} , y_i=\frac{y{r}_i}{max\left|yr\right|}$$

$$x{r}_i=r_i X sin\left({\theta }_i\right), y{r}_i=r_i X cos\left({\theta }_i\right)$$

$${\theta }_i=a X \pi X rand, a\in \left(\mathrm{5,10}\right)$$

$$r_i={\theta }_i+R+rand, R\in \left[\mathrm{0.5,2}\right]$$

The two random parameters, denoted as ‘a’ and ‘R’, control the search pattern within the chosen exploration area. These parameters influence the number of spirals undertaken by the search agents and the variation in their coiling shape. This element of randomness helps prevent the algorithm from getting stuck in suboptimal solutions. By introducing variation in the search pattern, BESO encourages exploration of diverse positions within the search area, ultimately leading to the identification of more accurate solutions.

Step 3: Convergence

The final stage of BESO, represents the convergence towards the optimal solution [27]. In this stage, the search agents gravitate towards the most promising position identified so far. This convergence process can be mathematically written as

$$P_{new}^t=rand X P_{Best}^t+{x1}_i X (P_i^t-C_1 X P_{mean}^t)+{y1}_i X (P_i^t-C_2 X P_{Best}^t)$$

(14)

Where,

$${x1}_i=\frac{x{r}_i}{max\left|xr\right|} , {y1}_i=\frac{y{r}_i}{max\left|yr\right|}$$

$$x{r}_i=r_i X sinh\left({\theta }_i\right), y{r}_i=r_i X cosh\left({\theta }_i\right)$$

$$r_i={\theta }_i, {\theta }_i=a X \pi X rand, a\in \left(\mathrm{5,10}\right)$$

Two random parameters ($C_1, C_2$) play a crucial role in this stage. These parameters, ranging between 1 and 2, control the intensity of the search agents’ movement towards the best solution. The algorithm utilizes a polar equation to mathematically model the “swooping” motion of the eagle. This mathematical framework ensures an efficient convergence process. Additionally, the mean position of the search agents can be factored in to introduce more diversity and intensity during the convergence phase, further guiding them towards the optimal solution.

3.2. Red Deer Optimization (RDO)

Red Deer Optimization (RDO) is a recent metaheuristic algorithm inspired by the fascinating mating behavior of red deer during the rut, or breeding season. Introduced in 2016 [28], RDO mimics this natural process to find optimal solutions to complex problems. The algorithm starts with a random population of individuals, representing potential solutions to the diabetic detection problem. These individuals are analogous to red deer. The strongest individuals, like dominant males, are identified. These “stags” compete for influence over the remaining population, like how stags fight for harems of females during the rut. The outcome of this competition determines how solutions are combined to create new offspring, representing the next generation of potential solutions. Stronger “stags” have a greater chance of mating with more “hinds,” leading to a wider exploration of the solution space. Weaker individuals may still contribute, but to a lesser extent. This process balances exploration and exploitation [29] – key aspects of optimization. Through this iterative process of selection, competition, and reproduction, RDO progressively refines the population towards improved solutions for diabetic detection. The final population contains the most promising solutions identified by the algorithm[30].

RDO algorithm.

a) Initialization : Each RD is described by a set of variables, analogous to its genes. The number of variables (XNvar) corresponds to the number of genes. The values of these variables represent the potential contribution of each gene to diabetic detection. For instance, the set XNvar to 50 which means to account for investigations.

Initializing Individual Positions:

The position of each RD (Xi) is defined by this set of variables. Here, Xi would be a vector containing the values for each of the 50 genes (e.g., [θ1, θ2, ..., θ50]). These initial values are assigned randomly.

b) Roaring: In the “roar stage,” each RD representing a potential gene selection for diabetic detection, can explore its surrounding solution space based on the microarray gene data. Imagine each RD has neighboring solutions in this multidimensional space. Here, a RD can adjust its position which means gene selection within this local area. The algorithm evaluates the fitness of both the original and the adjusted positions using a fitness function reflecting how well the gene selection differentiates between diabetic and non-diabetic cases.

An equation is used to update the RD’s position,

$${Male}_{new}=\left\{\begin{array}{c}{male}_{new}+a_1\times \left(\right((UB-LB)\times a_2)+LB, if a_3\ge 0.5\\ {male}_{new}-a_1\times \left(\right((UB-LB)\times a_2)+LB, if a_3<0.5\end{array}\right\}$$

(15)

Here, a1,a2,a3 are the random factors between 0 and 1. ${Male}_{old}$ and ${Male}_{new }$are the old and new positions of RD’s. This process is akin to the RD exploring its surroundings and “roaring” if it finds a better location with a higher fitness value (B*) [31]. Such successful adjustments elevate the RD to commander status, signifying a promising solution for diabetic detection. Conversely, if the exploration yields a position (A*) with a lower fitness value compared to the original, the Red Deer remains a “stag.” These stags still have the chance to contribute in later stages of the algorithm. This roar stage essentially refines the initial population by focusing on RD’s gene selections that demonstrate better potential for differentiating diabetic and non-diabetic cases based on the gene data.

c) Roar

Following the roar stage, commanders engage in a unique form of competition with the remaining stags. This competition doesn’t involve literal combat, but rather an exchange of information about their gene selections.

$${New}_1=\frac{(Com+Stag)}{2}+b_1\times \left(\right((UB-LB)\times b_2)+LB)$$

(16)

$${New}_2=\frac{(Com+Stag)}{2}-b_1\times \left(\right((UB-LB)\times b_2)+LB)$$

(17)

Each commander interacts with a set of randomly chosen stags. The fight’s outcome determines how much information, or details about their gene selections, the commander shares with the stags. Commanders with demonstrably better gene selections will share more information with the stags they compete with. Imagine commanders with promising solutions guiding the stags towards improvement. Here, commanders with demonstrably superior gene selections retain their position as leaders and share more knowledge with the stags. Conversely, if a stag’s gene selection shows potential during the competition, it might surpass the commander’s current selection. In such cases, the stag’s improved solution becomes the new leader, and the information flow is reversed.

d) Creation phase of harems:

Following the roar and fight stages, commanders are rewarded based on their performance. This reward system is reflected in the formation of harems. A harem consists of a commander and a group of hinds as female deer. The number of hinds a commander attracts is directly proportional to his success in the previous stages. Commanders with demonstrably superior gene selections will attract larger harems. Imagine a commander’s “power” being determined by how well its gene selection differentiates between diabetic and non-diabetic cases.

$$\mathrm{N}. {harem}_n=\mathrm{round} \{p_n. N_{hind}\}$$

(18)

Commanders with greater power will attract more hinds to their harems. Stags, on the other hand, are not included in harems. This reward system incentivizes the continued exploration and refinement of promising gene selections for diabetic detection.

e) The mating phase in RDO occurs in three key scenarios: 1. Commander Mating Within Harems - Each commander gets the opportunity to mate with a specific proportion (α) of the hinds within its harem. This mating metaphorically represents the creation of new gene selections based on combinations of the commander’s strong selection and those of the hinds. 2. Commander Expansion Beyond Harems - commanders can potentially mate with hinds from other harems. A random harem is chosen, and its commander gets the chance to “mate” with a certain percentage (β) which lies between 0 and 1) of the hinds in other harems. 3. Stag Mating - Stags also have a chance to contribute. They can mate with the closest hind, regardless of harem boundaries. This allows even less successful gene selections to potentially contribute to the next generation, introducing some level of diversity. By incorporating these three mating scenarios, RDO explores a combination of promising gene selections, ventures beyond established solutions, and maintains some level of diversity through stag mating.

f) Mating phase – New solutions are created. This process combines the strengths of existing gene selections from commanders, hinds, and even stags, promoting a balance between inheritance and exploration. This approach helps refine the population towards more effective gene selections for diabetic detection using your microarray gene data.

g) Building the Next Generation - RDO employs a two-pronged approach to select individuals for the next generation. A portion of the strongest RD is automatically selected for the next generation. These individuals represent the most promising gene selections identified so far. Additional members for the next generation are chosen from the remaining hinds and the newly generated offspring. This selection process often utilizes techniques like fitness tournaments or roulette wheels, which favor individuals with better fitness values.

h) RDO’s stopping criterion to determine the number of Iterations - a set number of iterations can be predetermined as the stopping point. The algorithm might stop if it identifies a solution that surpasses a certain threshold of quality for differentiating diabetic and non-diabetic segregation, a time limit might also be set as the stopping criterion. The parameters of each values involves in this algorithm as described in the following Table 2.

Based on the p-value selection criteria employed in Table 2, features extracted using STFT, RR, and PCC might not be the most informative for diabetes detection. This conclusion is drawn from analyzing p-values obtained through t-tests for various FE methods for two FS techniques as BESO and RDO were revealed that p-values can serve as initial indicators to quantify the potential presence of outliers, non-linearity, and non-Gaussian data distributions within the classes. Further analysis may be necessary to identify the most statistically significant features for accurate diabetes classification.

3.3. Analyzing the Impact of Feature Extraction Methods Using Statistical Measures

Microarray data, with gene expression information, can be overwhelming due to its high dimensionality. Feature extraction techniques help streamline this data. Subsequently, the Pearson Correlation Matrix (PCM) comes into play for in-depth analysis. PCM calculates pairwise correlations between genes, essentially measuring of how gene expression levels varies from low to high change together across different samples. Values closer to 1 (positive correlation) indicate genes that tend to be expressed together, while values closer to -1 (negative correlation) suggest genes with opposite expression patterns[32]. A value around 0 signifies no significant correlation. By identifying clusters of genes with strong positive correlations (potentially sharing similar functions), PCM paves the way for discovering promising diabetic biomarkers. Figure 5 shows weak correlation between the classes. STFT BESO values for diabetic patients range from -0.69 to 0.98, while for non-diabetic patients they range from -0.94 to 0.78. Similarly, Figure 6 also indicates weak correlation between the classes is observed based on RR BESO values. These values range from -0.54 to 0.86 for diabetic patients and -0.69 to 0.68 for non-diabetic patients.

Figure 5. PCM Plots for STFT FE Method.

Figure 5. PCM Plots for STFT FE Method.

Figure 6. PCM Plots for RR FE Method.

Figure 6. PCM Plots for RR FE Method.

4. Classifiers

FE acts like a filter, selecting the most informative data tidbits from the massive count of gene expressions. However, these informative features alone aren’t enough to diagnose diabetes. Here’s where classifiers come in as powerful tools. They analyze the selected features, learning the subtle patterns that differentiate diabetic from non-diabetic samples. These patterns can be complex and non-linear, making them difficult to identify by hand. Classifiers ultimately leverage this knowledge to make automated and accurate classifications of new, unseen diabetic cases.

4.1. Non-linear Regression

While standard linear regression assumes a straightforward, linear relationship between variables, nonlinear regression emerges as a powerful tool for diabetic detection. It tackles the challenge of complex relationships that might exist between features extracted from gene expression data and the presence of diabetes. The core principle involves modeling the relationship between extracted features and the diabetic state using a specific mathematical function that captures the non-linearity [33]. The model strives to minimize the sum of squared errors between the predicted diabetic state and the actual diabetic state for each data point. Imagine finding the best-fitting non-linear curve that represents the data.

Least squares estimation then comes into play to determine the unknown parameters within the chosen non-linear function. These parameters define the shape and characteristics of this non-linear relationship. After training the model with known diabetic and non-diabetic data points, the estimated non-linear function can predict the diabetic state for new, unseen cases based on their extracted features[34]. A threshold probability or value can be established to classify the predicted state as diabetic or non-diabetic. This approach offers a valuable technique for leveraging the complexities within gene expression data for improved diabetes detection.

$$G_m=y(a_m, \theta )+P_{mm}$$

(19)

Nonlinear regression centers on the concept of an expectation function, typically denoted by $y$. This function represents the predicted value of the dependent variable (often signifying the diabetic state) based on the independent variables (also known as features extracted from gene expression data) for each data point. Here’s the key difference between nonlinear and linear models: in nonlinear regression, at least one parameter ($a_m$) for each data point ($m$) must influence one or more derivatives of the expectation function. This allows the model to capture non-linear relationships between the features and the diabetic state. Unlike linear models where the expectation function is a straight line, the nonlinear version can take on various shapes depending on the chosen mathematical formula and its parameters. This flexibility empowers the model to represent the complexities inherent in the data, potentially leading to more accurate classifications of diabetic cases.

4.2. Linear Regression

Linear regression [35], a basis of statistical analysis, excels at modeling linear relationships between variables. However, in the realm of diabetic detection, it falls short as a primary classification tool, linear regression assumes a straight-line connection between the features extracted from gene expression data (independent variables) and the diabetic state (dependent variable). This linearity might not reflect the true complexities of the underlying biology. Diabetic development can involve intricate, non-linear relationships that linear regression might miss, leading to inaccurate classifications. Next, diabetic detection is inherently a binary classification problem. While applying a threshold might seem straightforward, it can be suboptimal for capturing the nuances of the data and the disease itself.

$$Z=p+qX$$

(20)

4.3. Gaussian Mixture Model

Gaussian Mixture Models (GMMs) [36] shine as powerful tools for unsupervised machine learning. Unlike techniques requiring labeled data, GMMs excel at uncovering hidden structures within complex datasets like gene expression profiles. GMMs act like detectives, searching for hidden patterns in the data. They can automatically group similar gene expression profiles into clusters, potentially revealing subgroups within the diabetic or non-diabetic population. Unlike rigid clustering methods, GMMs employ a softer approach. They assign probabilities to each cluster for a given data point, allowing for potential overlap between clusters. This flexibility reflects the inherent complexities of diseases like diabetes, where gene expression patterns might not be perfectly distinct.

GMMs assume that the data arises from a blend of multiple, simpler Gaussian distributions [37]. This allows them to capture the diverse patterns present within gene expression data. By combining these Gaussian components, GMMs create a nuanced model that can effectively represent the underlying biological processes related to diabetes.

$$p(a)=\frac{1}{{\left(2\pi \right)}^{\frac{n}{2}}{\left|\mathsf{\Sigma }\right|}^{\frac{1}{2}}}{e}^{-\left(\frac{1}{2}\right){\left(a-\mu \right)}^T{\mathsf{\Sigma }}^{-1}\left(a-\mu \right)}$$

(21)

Within the Gaussian distribution formula, the symbol $\mu$ denotes the mean vector, representing the average position of the data points in the n-dimensional space. $\mathsf{\Sigma }$ represents the covariance matrix, which captures the relationships between the different features (dimensions) within the data. The determinant of the covariance matrix, denoted by | $\mathsf{\Sigma }$ |, plays a role in calculating the overall spread of the data around the mean. Finally, the $exp\left(\right)$ function is used to calculate the probability density of a particular data point within the Gaussian distribution.

By uncovering hidden structures and capturing the complexities of gene expression data, GMMs provide valuable insights for researchers exploring novel approaches to diabetic detection.

4.4. Expectation Maximum

The EM algorithm emerges as a powerful tool for addressing missing information or hidden factors [38,39]. The EM algorithm acts like a skilled detective in diabetic detection. Two-step process such as:

• Expectation Step: In this initial step, the EM algorithm estimates the missing information, or hidden factors based on the currently available data and the current model parameters.
• Maximization Step: With the estimated missing values in place, the EM algorithm then refines the model parameters by considering the newly completed data.

Figure 5. Flow diagram of EM algorithm.

Figure 5. Flow diagram of EM algorithm.

Through this iterative process of estimating missing information and then improving the model based on those estimates, the EM algorithm gradually uncovers the underlying structure and parameters governing the distribution of gene expression data. This ultimately leads to a more accurate classification of diabetic and non-diabetic cases.

4.5. Logistic Regression

LR establishes itself as a powerful tool for binary classification tasks, proving its worth in decoding the complexities of diabetic detection. Like its role in identifying alcoholic tendencies, logistic regression [40] estimates the probability of an individual developing diabetes based on a collection of factors. The initial phase involves meticulously gathering relevant data on potential risk factors from a sample population. The collected data is then strategically divided into two distinct sets. The training set serves as the foundation for building the logistic regression model, while the testing set acts as an impartial evaluator, assessing the model’s accuracy in predicting new diabetic cases. The LR model [41] meticulously analyzes the training data. It constructs a mathematical function that establishes a relationship between the collected risk factors as independent variables and the likelihood of developing diabetes as dependent variable. Once the model is fully trained and refined, it can be leveraged to predict the probability of diabetes in new individuals based solely on their specific risk factors.

$$Logit\left\{\mathsf{\Pi }\left(x\right)\right\}=\mathrm{}{\upsilon }_0+\mathrm{}{\sum }_{j=1}^q{\upsilon }_j{x}_j^{\mathrm{*}}$$

(22)

The objective is to optimize both the fitness and log-likelihood, which can be achieved by attaining the subsequent function:

$$1\left({\upsilon }_0,\mathrm{}\upsilon \right)=\mathrm{}{\sum }_{j=1}^n\left\{y_i\log \left({\pi }_i\right)+\left(1-{\pi }_i\right)\right\}-\mathrm{}\frac{1}{{2\tau }^2}{‖\upsilon ‖}^2$$

(23)

4.6. Softmax Discriminant Classifier

SDC is an ingenious approach that leverages the power of an individual’s genetic makeup to predict their potential diabetes risk. SDC [42] is meticulously analyzing a person’s genes. It employs a specialized tool – a discriminant function – that acts like a comparison chart. This function compares the genetic profile against those of individuals confirmed to have or not have diabetes. The key strength of SDC lies in its ability to learn and adapt. By analyzing a set of well-defined examples (individuals with confirmed diabetic status), SDC refines its discriminant function. This refinement helps SDC sharpen the distinction between diabetic and non-diabetic gene patterns. The goal is to establish clear boundaries between these two groups, enabling accurate classification of new cases. When presented with a new genetic profile, SDC compares it to the learned patterns. By analyzing the closest match, SDC can estimate the likelihood of an individual developing diabetes.

The process of transforming class samples and test samples in SDC incorporates nonlinear enhancement values, which are calculated utilizing the subsequent equations:

$$h\left(K\right)=\arg \max Z_w^i$$

(24)

$$h\left(K\right)=\arg \underset{i}{\max }\log \left({\sum }_{j=1}^{d_i}\exp (-\lambda {‖v-\mathrm{}{\upsilon }_j^i‖}_2)\right)$$

(25)

In these formulas, $h\left(K\right)$ represents the distinction of the $i^{th}$class, and as ${‖v-{\upsilon }_j^i‖}_2$ converges to zero, $Z_w^i$ is maximized. This characteristic ensures that the test sample is most likely to belong to a particular class.

4.7. Support Vector Machine (Radial Basis Function)

Support Vector Machines (SVMs) are a powerful machine learning technique well-suited for classification tasks like diabetic detection. Unlike linear regression, SVMs can handle non-linear relationships between features extracted from gene expression data and the diabetic state. SVMs [43] can map the extracted features of initially existing in a lower-dimensional space into a higher-dimensional space. This transformation allows the data points to be separated more effectively using a hyperplane, even if the original relationship between features and the diabetic state was non-linear. RBF kernel is a popular choice for SVM classification. It acts like a bridge, calculating the similarity between data points in the higher-dimensional space. This similarity measure is then used by the SVM algorithm to identify the optimal hyperplane that best separates diabetic from non-diabetic data points in this transformed space. Once the optimal hyperplane is established, new, unseen data points can be mapped into the same higher-dimensional space using the RBF kernel. Their position relative to the hyperplane allows the SVM to classify them as diabetic or non-diabetic.

$$\mathrm{RBF}: \mathrm{k}\mathrm{}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)=\exp \left\{\frac{-{\left|{\mathrm{x}}_{\mathrm{i}}-\mathrm{}{\mathrm{x}}_{\mathrm{j}}\right|}^2}{{\left(2\mathrm{*}\mathrm{}\mathsf{\sigma }\right)}^2}\right\}$$

(26)

By effectively handling non-linearities and leveraging the RBF kernel for similarity calculations, SVMs offer a robust approach to classifying diabetic cases based on gene expression data.

4.8. Selection of Classifiers Parameters through Training and Testing

To ensure the model’s generalizability despite limited data, data is employed a robust technique called 10-fold cross-validation. This method splits the data into 10 equal folds, trains the model on 9 folds, and tests it on the remaining fold. This process is repeated for all folds, providing a more reliable estimate of the model’s performance on unseen data compared to a single test/train split. Additionally, the model was trained on a dataset containing 2870 features per patient for 20 diabetic and 50 non-diabetic individuals. This comprehensive training process, combined with k-fold cross-validation, strengthens the reliability of the findings. The training process is measured in the MSE as

$$MSE=\frac{1}{N}{\sum }_{j=1}^N{\left(O_j-T_j\right)}^2$$

(27)

$O_j$ is the actual value observed at a specific time, and $T_j$ is the target value the model predicts for that time.

5. Classifiers Training and Testing

Table 4 represents the confusion matrix for diabetes detection and the parameters are defined as follows,

Table 5 directly compares the Mean Squared Error (MSE) on both the training data it learns from (Train MSE) and unseen test data (Test MSE), with lower MSE indicating better performance. The highest values achieved as Support Vector Machine (SVM) with an RBF kernel, achieving exceptionally low MSE on both training and test data (Train MSE: 1.88× 10⁻⁶, Test MSE: 1× 10⁻⁶). Statistical models like Ridge Regression (Train MSE: 7.29× 10⁻⁶, Test MSE: 3.25× 10⁻⁵), Linear Regression (Train MSE: 1.16× 10⁻⁵, Test MSE: 1.94× 10⁻⁵), and Logistic Regression with L1 penalty (LoR) (Train MSE: 2.7× 10⁻⁵, Test MSE: 3.02× 10⁻⁵) also show promise with consistently low MSE values. Mixture models, Gaussian Mixture Model (GMM) (Train MSE: 1.02× 10⁻⁵, Test MSE: 1.48× 10⁻⁵) and Expectation Maximization (EM) (Train MSE: 5.29× 10⁻⁶, Test MSE: 1.37× 10⁻⁵), demonstrate competitive results. Notably, STFT and PCC have higher MSE values across both Train and Test data, suggesting they may not be as effective for this specific diabetic detection task. It’s note that all models exhibit lower MSE on the training data compared to test data, indicating some degree of overfitting. This underscores the importance of evaluating models using unseen test data to ensure they generalize well to new cases.

In Table 6, SVM with an RBF kernel maintains its dominant position (Train MSE: 2.18× 10⁻⁶, Test MSE: 1.44× 10⁻⁶). It achieves exceptionally low MSE on both the training data it learns from and unseen test data. Statistical models show mixed results. Ridge Regression (Train MSE: 1.44× 10⁻⁵, Test MSE: 2.21× 10⁻⁵) performs consistently, while LR (Train MSE: 3.76× 10⁻⁵, Test MSE: 2.3× 10⁻⁵) delivers slightly higher training MSE but achieves a lower test MSE. Logistic Regression with L1 penalty (Train MSE: 9.97× 10⁻⁶, Test MSE: 1.76× 10⁻⁵) demonstrates the most significant improvement in test MSE compared to training MSE. Mixture models present a similar picture. GMM exhibits a significant jump in Test MSE (3.97× 10⁻⁴) compared to Train MSE (4.51× 10⁻⁵), suggesting potential overfitting. EM shows more balanced performance (Train MSE: 3.14× 10⁻⁴, Test MSE: 1.37× 10⁻⁵). Like the previous table, STFT and PCC remain less competitive in this task, evident from their higher MSE values across both Train and Test data (above 5.00×10⁻⁵ for most cases). An interesting observation is the significant reduction in Test MSE for SDC (Softmax Discriminant Classifier) compared to the previous table (Train MSE: 2.21× 10⁻⁵, Test MSE: 1.6× 10⁻⁵ to Train MSE: 2.81× 10⁻⁶, Test MSE: 2.81× 10⁻⁴). This suggests potential improvements in model generalizability. Overall, SVM with RBF kernel remains the leader, while statistical and mixture models show varying effectiveness depending on the specific model and its ability to generalize to unseen data. The significant reduction in SDC’s Test MSE warrants further investigation into its potential for diabetic detection.

In Table 7, SVM with an RBF kernel maintains its exceptional performance (Train MSE: 4.25× 10⁻⁷, Test MSE: 3.6× 10⁻⁷). It achieves remarkably low MSE on both the training and unseen test data, solidifying its position as the leading contender. Statistical models show further improvement. LoR (Train MSE: 4.85× 10⁻⁵, Test MSE: 1.96× 10⁻⁶) demonstrates a significant decrease in Test MSE compared to the previous table. Logistic Regression with L1 penalty (LoR) (Train MSE: 1.39× 10⁻⁵, Test MSE: 2.25× 10⁻⁶) also exhibits a positive trend with lower Test MSE. However, RR (Train MSE: 6.08× 10⁻⁵, Test MSE: 9× 10⁻⁶) shows a slight increase in Test MSE. Mixture models present a more promising picture in this table. GMM achieves a significant reduction in both Train MSE (9.01× 10⁻⁶) and Test MSE (4.41× 10⁻⁶), suggesting potential improvements in addressing overfitting. EM maintains consistent performance (Train MSE: 3.51× 10⁻⁵, Test MSE: 7.29× 10⁻⁶). Like previous observations, STFT and PCC remain less competitive (Train and Test MSE values above 5.00×10⁻⁶). The most significant improvement is seen with SDC . It achieves exceptionally low Test MSE (1.96× 10⁻⁶) compared to both previous tables (Train MSE: 1.35× 10⁻⁵, Test MSE: 2.89× 10⁻⁶). This dramatic reduction in Test MSE warrants further investigation into the potential of SDC for diabetic detection, especially considering its consistent Train MSE across all three tables.

Overall, SVM with RBF kernel remains the frontrunner. Statistical and mixture models demonstrate ongoing optimization, with Logistic Regression and GMM showing the most notable improvements. The remarkable performance boost of SDC in this table highlights its potential as a viable approach for diabetic detection.

5.1. Selection of Targets

The target value for the non-diabetic class ($T_{Non-Dia}$) ranges from 0 to 1, with emphasis on the lower end of the scale. This range is determined by a specific constraint[44].

$$\frac{1}{N}\sum _{i=1}^N{\mu }_i\le T_{Non-Dia}$$

Here, ${\mu }_i$denotes the mean value of the input feature vectors considered for classification across N non-diabetic features. Similarly, in the case of the diabetic class ($T_{Dia}$) the target value is aligned with the upper range of the 0 to 1 scale. This alignment is determined by the following principle as

$$\frac{1}{M}\sum _{j=1}^N{\mu }_j\le T_{\mathrm{D}\mathrm{i}\mathrm{a}}$$

In this context, ${\mu }_j$ represents the average value of input feature vectors for the M diabetic cases considered in classification. It’s crucial to note that the target value $(T_{Dia}$) is deliberately set higher than the average values of both ${\mu }_i$ and ${\mu }_j$. This choice of target values mandates a minimum discrepancy between them of 0.5, as stated below,

$$\left|\right|T_{\mathrm{D}\mathrm{i}\mathrm{a}}-T_{\mathrm{N}\mathrm{o}\mathrm{n}-\mathrm{D}\mathrm{i}\mathrm{a}}\left|\right|\ge 0.5$$

The target values for non-diabetic ($T_{\mathrm{N}\mathrm{o}\mathrm{n}-\mathrm{D}\mathrm{i}\mathrm{a}}$) and diabetic ($T_{Dia}$) cases are set at 0.1 and 0.85, respectively. After setting the targets, MSE is employed to assess classifier performance. Table 8 illustrates the optimal parameter selection for the classifiers following the training and testing phases.

6. Outcomes and Findings

The evaluation approach ensures robust assessment of the models’ effectiveness in classifying diabetic patients. Researchers employ a widely used technique called “tenfold cross-validation.” Here, the data is meticulously divided into 10 equal sets. Nine of these sets (representing 90% of the data) are used to train the models. The remaining set (10%) serves a crucial purpose - testing the model’s performance on unseen data. This approach helps mitigate overfitting and provides a more reliable picture of how the models would perform in real-world scenarios. To go beyond simple accuracy, we leverage a valuable tool called a confusion matrix. This matrix allows us to calculate a comprehensive set of performance metrics. These metrics include Accuracy (overall percentage of correct predictions), F1 score (balances precision and recall), MCC (considers true positives/negatives for a more robust evaluation), Jaccard Metric (measures the shared proportion of positive predictions between the model and the ground truth), Error rate (percentage of incorrect predictions), and Kappa statistic [45] (measures agreement beyond chance). By analyzing these metrics derived from the confusion matrix, we gain a deeper understanding of the models’ strengths and weaknesses in distinguishing between diabetic and non-diabetic cases. Table 9 which are employed to calculate the performance metrics mentioned. This table provides a transparent view of the evaluation process and allows for further analysis of the models’ configuration.

The result of the above metrics for evaluation of the performance in each classifier is given in the Table 9,

From Table 10, SVM with an RBF kernel solidified its dominance. It achieved the highest overall accuracy (92.8571%), F1 score (85.71%), Matthews Correlation Coefficient (MCC) (0.7979), and Jaccard similarity (75%), demonstrating its exceptional ability to distinguish between diabetic and non-diabetic cases With PCC as FE. Conversely, RR exhibited the lowest performance across most metrics, suggesting it may not be suitable for this task. While the choice of feature extraction technique appears less impactful than the model itself (except for PCC), some interesting observations emerged. The combination of STFT and RR yielded the lowest overall performance (average accuracy of 80%), indicating RR might be less effective here. Excluding these outliers (SVM and Pearson CC), the average accuracy for other models ranged from 81% to 84%, with F1 scores between 72% and 74% and MCC values between 0.60 and 0.65. Overall, SVM with an RBF kernel emerged as the strongest contender for diabetic detection due to its consistent performance across various feature extraction techniques. While other models like NLR and SDC showed promise, further investigation might be necessary to evaluate their generalizability and confirm their effectiveness.

From Figure 6, SVM with an RBF kernel consistently achieved the highest values across all performance metrics, demonstrating its exceptional ability to distinguish diabetic from non-diabetic cases. For instance, we observed exceptional accuracy (around 85%) and F1 score (around 80%) for SVM with RBF compared to other models. While the choice of FE technique seems to have a lesser impact on overall performance compared to the model itself (except for RR), one noteworthy observation emerged. RR, when combined with any feature extraction technique, yielded the lowest performance. Excluding RR and the outliers (SVM and Pearson CC), other models exhibited average accuracy ranging from 80% to 85%, with F1 scores between 70% and 75%. Overall, SVM with RBF kernel stands out as the most promising approach for diabetic detection based on its consistently high performance across various feature extraction techniques. While other models like NLR, SDC, and some with other feature extraction techniques show promise, further investigation might be necessary to evaluate their generalizability and confirm their effectiveness.

Figure 6. Performance analysis of different classifier without FS Method.

Figure 6. Performance analysis of different classifier without FS Method.

From Table 11, SVM with RBF kernel solidified its dominance. It achieved the highest overall accuracy (92.86%), F1 score (87.80%), Matthews Correlation Coefficient (MCC) (0.8280), and Jaccard similarity (78.26%), demonstrating its exceptional ability to distinguish between diabetic and non-diabetic cases with PCC as FE for BESO FS. Conversely, PCC with NLR exhibited the lowest performance across most metrics, with accuracy as low as 57.14%, F1 score of 44.44%, and MCC of 0.1446. Examining the impact of FE techniques revealed some interesting insights. The pairing of STFT with RR yielded a moderate average accuracy of 78%, indicating RR might be less effective here. However, excluding RR techniques, with STFT in NLR, LoR, GMM, EM achieved similar average accuracy (ranging from 80% to 84%) when combined with SVM (RBF) shows high as 91.42%. This highlights the consistent performance of SVM (RBF) across various FE methods. In conclusion, SVM with an RBF kernel emerged as the strongest contender for diabetic detection based on its consistently high performance across different FE techniques.

From Figure 7, it is revealed that SVM with RBF kernel as the strongest classifier for diabetic detection, achieving exceptional performance (accuracy > 92%, F1-score > 87%, MCC > 0.82, Jaccard > 78%). Interestingly, the choice of feature extraction technique had less impact, except for Ridge Regression which yielded the lowest performance across all techniques. While other models like NLR, LR, GMM, EM, and SDC showed promise, SVM with RBF kernel emerged as the leader due to its consistent effectiveness.

From Table 12, it is observed that the diabetic detection models revealed SVM with RBF kernel as the champion, achieving exceptional results (accuracy reached the highest ever as 97.14%, F1-score reaches 95%, MCC exceeding 0.93, Jaccard exceeding 90%). This dominance is evident compared to the lowest performing model, Pearson CC with NLR, which obtained an accuracy as low as 62%, F1-score of 48%, and MCC of 0.22. Excluding these outliers, the average performance across models ranged from 80-85% accuracy, 70-78% F1-score, and 0.55-0.65 MCC. Interestingly, the choice of feature extraction technique seems to have a lesser impact on performance, except for Ridge Regression. It consistently yielded the lowest values (accuracy around 65%, F1-score around 55%, MCC around 0.30) regardless of the technique used. Techniques like NLR, LR, GMM, EM, LoR, and SDC performed similarly well with SVM (RBF), all achieving average accuracy above 80%. This highlights the robustness of SVM with RBF across various feature extraction approaches.

From Figure 8, it is observed that the analysis of diabetic detection models, SVM with RBF kernel emerged as the clear leader, achieving over 97% accuracy, with F1-score exceeding 95% and MCC surpassing 0.93. Interestingly, the choice of feature extraction method had minimal impact on performance, as models like NLR, LR, GMM, EM, LoR, and SDC performed similarly well with SVM (RBF), all achieving an average accuracy of over 80%. However, Ridge Regression consistently underperformed across all metrics, suggesting it may not be the best choice for this task. Overall, SVM with RBF kernel demonstrated exceptional and consistent performance, making it the top contender for diabetic detection.

Figure 9 illustrates the performance of the Jaccard metric and error rate (%) parameters through histograms. It is observed that the maximum error rate stabilizes at 50%, while the maximum Jaccard metric reaches 90%. The histogram representing the error rate is skewed towards the right side of the graph, indicating that regardless of the feature extraction method or feature selection method employed, the classifier’s error rate remains below 50%. On the other hand, the histogram of the Jaccard metric displays sparsity towards the edges and covers a greater number of points in the central area across the classifier.

6.1. Computational Complexity

This study evaluates classifiers based on their Computational Complexity (CC), specifically focusing on how it scales with the size of the input data (denoted as O(n)). Ideally, a classifier should have a low CC, represented as O(1). This indicates that the algorithm’s complexity remains constant regardless of how much data it needs to process. This is a desirable characteristic because it ensures efficient performance even with large datasets. Interestingly, the analysis highlights that the CC in these models is independent of the input size, further emphasizing their efficiency. However, the text also mentions that some models exhibit a logarithmic complexity (O(log n)) as the data size increases. Additionally, the study explores hybrid models that incorporate DR techniques and feature selection methods within their classification process. These techniques can potentially influence the CC of the overall model.

From Table 13, The table reveals a range of complexities across the classifiers. NLR, LR, LoR, and SDC show a complexity of O(n² logn) during training and then increase to O(2n² log2n) or O(2n² logn) for prediction. GMM and EM are slightly more complex, reaching O(n² log2n) in training and O(2n³ log2n) for prediction. However, SVM (RBF) stands out as the most computationally demanding, reaching O(2n⁴ log2n) during training and a slightly lower O(2n² log4n) for prediction. In conclusion, the chosen classifier and FE technique significantly impact the computational cost of your analysis. While all classifiers show increased complexity due to the FE techniques, SVM (RBF) is the most demanding. For large datasets and a balance between efficiency and potentially good performance, consider classifiers like NLR, LR, or LoR with these FE techniques.

Table 14 reveals a range of complexities. NLR, LR, and LoR show a complexity of O(n⁴ logn) during training and then increase to O(2n⁴ log2n) for prediction. GMM and EM are slightly more complex, reaching O(n⁴ log2n) in training and O(2n⁵ log2n) for prediction. However, SVM (RBF) stands out as the most demanding, reaching a staggering O(2n⁶ log2n) during training and a slightly lower O(2n⁴ log4n) for prediction. In conclusion, both the chosen classifier and FE technique significantly impact the computational cost of your analysis. While all classifiers show increased complexity due to the FE techniques, SVM (RBF) is the most challenging.

Table 15 reveals the impact of FE techniques, such as STFT, Ridge Regression, and Pearson CC, on overall complexity with RDO FS techniques. These techniques introduce additional computations, leading to a notable increase in CC across all classifiers. The table provided illustrates a spectrum of complexities for classifiers employed with these FE techniques. NLR, LR, and LoR exhibit a complexity of O(n⁵ logn), while GMM and EM are slightly more complex, reaching O(n⁵ log2n). SDC demonstrates a complexity of O(n⁶ logn). However, SVM (RBF) stands out as the most demanding, with a staggering complexity of O(2n⁷ log2n). Regarding feature selection, it’s crucial to note that this analysis solely focuses on CC based on the chosen FE techniques. Techniques like RDO for feature selection have the potential to reduce the number of features utilized in the classification process, significantly lowering the overall complexity of the model compared to using all features.

6.2. Limitations

Investigated the potential of using microarray gene data to identify type 2 diabetes early and potentially predict associated diseases. The analysis revealed promising classification techniques that could be valuable for screening and identifying diabetes markers, along with potentially linked diseases like strokes and kidney problems. While the findings may be specific to this patient group and require further validation, they pave the way for future research. The methods used, such as microarrays, might not be readily available in all settings due to cost and complexity. However, this study lays the groundwork for developing more accessible and efficient approaches for early diabetes detection and disease management. Overall, the research highlights the potential for early detection of type 2 diabetes and associated diseases, emphasizing the importance of further research to validate these findings and develop more accessible screening methods for improved patient outcomes.

6.3. Conclusions and Future Work

This Analysis explored various FE techniques (STFT, Ridge Regression, and PCC) and feature selection methods (BESO and RDO) for their impact on classifier performance in detecting Type II DM using microarray gene data. While RR with RDO FS yielded lower accuracy, STFT and PCC showcased improved performance metrics, particularly for the SVM (RBF) classifier. Notably, the combination of SVM (RBF) and RDO achieved the highest accuracy (92.85%) even without feature selection. However, employing RDO alongside SVM (RBF) still resulted in the highest accuracy across all FE techniques (95%, 92%, and 97.14% for STFT, RR, and PCC respectively). BESO also yielded promising results, achieving high accuracy values (around 90%) with all three FE techniques. Interestingly, computational complexity remained similar across classifiers with different dimensionality reduction methods, highlighting the crucial role of feature selection in boosting accuracy. In conclusion, this study presents a novel approach for Type II DM detection using microarrays. Future research will explore the application of Convolutional Neural Networks (CNNs), Deep Learning Networks (DNNs), and Long Short-Term Memory (LSTM) networks, along with hyperparameter tuning for further performance optimization.