Machine Learning for Affordable Diagnosis of SARS-COV-2 from Hematological and Biochemical Tests

1. Introduction

On Dec. 31 2019, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causing coronavirus disease (COVID-19) was discovered in Wuhan in China. The virus has spread rapidly around the world and as of Apr. 21 2020 there were more than 2.3 million confirmed cases and a total of 162.000 deaths [1]. Currently, the virus has infected more than 775 million people and caused the deaths of 7 million worldwide [2]. Despite the high number of recorded deaths, the World Health Organization estimates the total number of deaths directly or indirectly from COVID-19 to be 14.9 million people [3], three times higher than recorded.

The symptoms of COVID-19 can be divided into three categories being common symptoms such as fever, cough, tiredness and loss of taste or smell, less common symptoms such as sore throat, headache, myalgia and diarrhea, and the critical symptoms which are dyspnea, loss of speech, mobility or confusion and chest pain [4]. Despite having several symptoms, a large proportion of infected people do not experience them. According to Wang et al. [5] the proportion of asymptomatic people may represent about a third or a fifth of the total number of cases.

Although infected individuals develop mild or moderate disease and recover without hospitalization, a proportion of those infected may have severe symptoms, requiring hospitalization or intensive care unit (ICU) treatment.

To ensure the prevention of COVID-19, it is crucial to implement effective measures of detection, isolation and immediate medical care for infected people, in order to contain the spread of the virus [6,7,8,9]. This approach has proved effective in combating the spread of COVID-19. It is worth noting that, in addition to overloading the health system, the spread of the virus can have considerable economic impacts [10]. The costs associated with treating infected patients put significant pressure on health systems, especially in regions with limited resources [10]. Infected patients may require a diverse range of medical resources, from hospitalization in wards and ITUs to specialized interventions [11,12].

Among the various tests available to detect the disease, the reverse transcription polymerase chain reaction (RT-PCR) test, which detects the genetic material of the virus, is considered the gold standard for detecting COVID-19 and is carried out by collecting nasopharyngeal and/or oropharyngeal secretions [13]. Although RT-PCR has high analytical specificity, its diagnostic sensitivity is limited, especially in the early stages of infection, when the viral concentration may not be high enough to produce accurate results, leading to possible false negative results [14,15,16,17]. In addition, SARS-CoV-2 is an RNA virus prone to mutations, and since the beginning of the spread of the virus we have seen the emergence of several variants with different replication capacities and clinical manifestations [18]. Furthermore, the test has some limitations for testing patients on a large scale, requiring certified laboratories, expensive equipment and trained personnel, as well as the delay in obtaining results [19,20]. These limitations become even more serious in countries with limited resources, such as underdeveloped countries [21].

In addition to the RT-PCR mentioned above, computed tomography (CT) and X-ray have also been proposed to aid in the detection of COVID-19, playing a powerful role in detecting and judging the progression of the disease [22,23,24,25,26,27]. However, these tests are not efficient for the proper management of patients and the control of the spread of the virus, as their sensitivity and specificity are higher only in advanced stages of COVID-19 or other diseases [28,29,30], as these tests detect characteristics associated with COVID-19, such as lesions and lung inflammation, when the disease is already at an advanced stage of lung involvement. This is because the virus needs to replicate and cause significant damage to the lungs before changes are visible on the images. In addition, these exams subject patients to significant doses of ionizing radiation, which can pose major health risks [31,32].

Currently, machine learning has been an important tool to help doctors diagnose various diseases [33,34], including COVID-19 [35,36,37]. Studies have made use of this technology to detect COVID-19, either through images such as x-rays or CT scans, or through laboratory tests [38,39,40], since many clinical studies have shown hematological changes in infected patients [41,42,43,44]. While others focus on analyzing how these parameters vary throughout the course of the disease and their relationship with more serious clinical outcomes [45,46,47]. Given the need for efficient and more accurate diagnoses in the early stages of COVID-19, the relevant attributes of laboratory tests can be used to train machine learning algorithms, making it possible to predict the risk and evolution of the disease, as well as the early classification of infected patients. It is worth noting that laboratory tests have the advantage of being relatively more accessible in terms of cost compared to RT-PCR and imaging tests such as x-rays and CT scans, as well as avoiding exposure to radiation. This approach, based on hematological evaluations, could enable its use on a large scale, with specialist algorithms, which is especially relevant for developing countries with limited resources to carry out RT-PCR tests on the entire population, thus avoiding adverse social and economic impacts.

Several studies in the literature have used machine learning to detect COVID-19 from laboratory tests. One of the first articles, by Wu et al. [35], used the Random Forest (RF) algorithm with data collected from hospitals in China, achieving an AUC-ROC of 0.9926 in predicting patients with COVID-19. However, the model underperformed when tested with external samples. The authors Batista et al. [48] used data collected in a hospital in São Paulo, Brazil, and trained five algorithms for the detection of COVID-19 patients, Support Vector Machine (SVM) and RF were the ones that showed the best results, with an AUC-ROC of 0.847.

Sun et al. [36] employed the SVM algorithm to predict severe or critical cases of COVID-19 with patient data from a clinical center in China, obtaining a sensitivity of 77.5%, specificity of 78.4% and AUC-ROC of 0.9757. Tordjman et al. [49] developed a strategy using binary logistic regression (LR) to estimate the probability of a patient being diagnosed with COVID-19 before RT-PCR, achieving an AUC-ROC of 0.918, with sensitivity and specificity of 79% and 90%, respectively. Yang et al. [50] created a model using routine laboratory tests of patients at a medical center in the United States, and among the models tested, the Gradient Boosting Decision Tree (GBDT) obtained the best performance with an AUC-ROC of 0.854 in the internal validation and 0.838 in the external validation.

Joshi et al. [51] developed a decision support tool using LR to predict COVID-19 negative patients. The model achieved AUC-ROC of 0.78 for Stanford Health Care data, with AUC-ROC ranging from 0.75 to 0.81 at other centers. Goodman-Meza et al. [52] combined the probabilities of several algorithms to detect COVID-19 patients, achieving an AUC-ROC of 0.91 after adding certain tests such as ferritin, CRP and LDH. AlJame et al. [53] proposed ERLX, a model that stacks algorithms such as extremely randomized trees (ET), RF, LR and Extreme Gradient Boosting (XGB) to diagnose COVID-19 patients from blood tests, obtaining an AUC-ROC of 0.9938. Brinati et al. [21] proposed a model capable of predicting which patients were or were not infected with COVID-19, using data from patients admitted to the San Raffaele Hospital in Italy. Among the various algorithms tested, RF obtained the best performance, achieving an AUC-ROC of 0.84, sensitivity of 92% and specificity of 65%.

Alakus and Turkoglu [54] used a dataset containing hematological and biochemical attributes of patients from the Hospital Israelita Albert Einstein in São Paulo, Brazil, to train six deep learning architectures, with the aim of detecting patients with COVID-19. The best performance was obtained with Long Short-Term Memory (LSTM), achieving AUC-ROC of 0.625 in cross-validation and AUC-ROC of 0.90 in hold-out validation. Kukar et al. [55] created Smart Blood Analytics (SBA), a model for diagnosing COVID-19 from blood tests using algorithms such as RF, Deep Neural Network (DNN) and XGB. The best result with an AUC-ROC of 0.97 was obtained by XGB.

Using patient data from the Kepler University Hospital in Linz, Austria, Tschoellitsch et al. [56] applied the RF algorithm to predict the RT-PCR result using blood tests, achieving an AUC-ROC of 0.74. Soltan et al. [57] trained three models to identify COVID-19 patients in emergency departments and verify the need for hospitalization from patient data from Oxford University Hospitals in the UK. The best performance was obtained with XGB, achieving AUC-ROC of 0.939 in detecting patients and 0.94 in predicting hospitalization.

Cabitza et al. [58] evaluated several algorithms, including RF, Naive Bayes (NB), LR, SVM and KNN, to predict COVID-19 patients using data from patients admitted to the San Raphael Hospital in Italy. The models achieved AUC-ROC between 0.74 and 0.90, and during internal-external validation, they obtained AUC-ROC between 0.75 and 0.78. Alves et al. [59] developed a decision tree-based model to aid in the diagnosis of COVID-19 and provide explanations to healthcare teams, achieving AUC-ROC of 0.87, sensitivity of 67% and specificity of 91%. Rahman et al. [60] created the QCoVSML model, which uses stacked machine learning and a nomogram scoring system to detect COVID-19 based on biomarkers. Seven different datasets were used, and the combined model achieved an AUC-ROC of 0.96 and accuracy of 91.45%. There are many other studies that make use of laboratory tests and demonstrate the ability of machine learning algorithms to assist in medical decision-making, speeding up the diagnosis of the disease.

Although the algorithms used have achieved promising results, it is essential to recognize that they have some limitations, especially those based on backpropagation. According to Grossberg [61], these limitations include offline training, catastrophic forgetting, slow learning and sensitivity to outliers. To overcome these problems, Grossberg [62] developed Adaptive Resonance Theory (ART), which is described in detail in Section 2.

Therefore, this article presents the use of neural networks based on Adaptive Resonance Theory (ART) for the classification of COVID-19, using laboratory tests (blood tests) in order to improve the detection of infected patients. Comparisons were made between seven different ART architectures, including online training algorithms. The most successful architecture was subjected to a dataset containing laboratory tests from patients from different countries. This aspect is of paramount importance, as it allows us to evaluate the algorithm’s ability to detect relevant patterns and characteristics in data from different sources, thus increasing its usefulness in a global detection context.

2. Adaptive Resonance Theory

Developed Adaptive resonance theory in order to solve the stability-plasticity dilemma [63], such a dilemma can be understood as the ability of the network to remain able to adapt or group input patterns, and continue learning even with the inclusion of new patterns, without catastrophically forgetting its past knowledge Carpenter and Grossberg [64], this being a fundamental characteristic of ART family networks.

The first architecture based on this theory was developed by Grossberg [62] and named Grossberg Network (GN), in 1987, the ART1 network was created with the purpose of binary grouping patterns and was based on the GN network of Grossberg [64]. In the same year, Carpenter and Grossberg [65] improved ART1 creating ART2. This new network had the purpose of processing patterns with binary or continuous values. Considering the need to solve problems with analog input patterns, in 1991 Carpenter et al. [66], based on ART1 [64], developed the ART Fuzzy neural network (Figure 1), which employs the fuzzy min operator (∧) in place of the intersection operator (∩) present in ART1 network. The first ART network with supervised training was developed by Carpenter et al. [67] and has two interconnected ART modules, the ART${ }_a$ module receives and processes the input pattern. In contrast, the ART${ }_b$ module receives the desired response for the presented pattern, and the Inter-ART module is in charge of performing the “matching” between the two ART modules. The following year Carpenter et al. [68] presented the Fuzzy ARTMAP that also has supervised training. However, its calculations are based on Fuzzy Logic [69].

Different ART network architectures have been tested in order to find the one that best classifies the data. These architectures have different characteristics, whether in their topology (cluster geometry), training method (supervised or unsupervised) or online learning capacity (Continuous Training), these characteristics appear in isolation or combined in the models to optimize data classification. The networks used in this study were:

• Self-expanding Fuzzy ART Neural Network [66]
• Self-Expanding Euclidean ART neural network [70]
• Self-Expanding Fuzzy ARTMAP neural network [68]
• Self-expanding Fuzzy ART neural network with Continuous Training [71]
• Self-expanding Euclidean ART neural network with Continuous Training [71,72]
• Self-expanding Fuzzy ARTMAP neural network with Continuous Training [71]
• Self-expanding Euclidean ARTMAP neural network with Continuous Training [73]

Although the self-expanding mechanism is not present in the original structure of the first three models, it was used because it adds advantages, as will be discussed in Section 2.5. The other characteristics will be detailed in Section 2.1, Section 2.2, Section 2.3, Section 2.4 and Section 2.6.

In these architectures, some parameters play a fundamental role in controlling the functioning of the model, namely $\alpha$, $\beta$ and $\rho$. The $\alpha$ parameter, also known as the learning rate ($\alpha >0$), is crucial, indicating the extent to which new patterns will influence the category when the weights are updated in the next training cycle. The $\beta$ parameter, or training rate ($0<\beta \le 1$), regulates the speed of training, with lower values indicating a slower process and values close to 1 implying faster training. Finally, the $\rho$ parameter, or vigilance parameter ($0<\rho \le 1$), plays a crucial role in determining the degree of similarity required to create a new category. These parameters are essential for adjusting the network’s dynamics and adapting it effectively to the input data.

2.1. ART - Unsupervised

Unsupervised ART models are effective in environments where the training data does not need to be labeled. The model’s independence from labels allows it to autonomously explore the complexity of the data and identify relevant patterns. The basic architecture of an ART network can be seen in Figure 1, and consists of three levels, the F${ }_1$ comparison level, the F${ }_2$ recognition level and the F${ }_0$ level responsible for pre-processing the data, in which the input data is normalized and encoded (Fuzzy Topology). In addition, there is a reset mechanism responsible for controlling the degree of similarity of the patterns associated with the same category.

It is worth noting that, although they adopt an unsupervised approach, labels are assigned to each cluster after the model has been trained. This process is explained in detail in Section 3.4.

Algorithm 1: Algorithm ART.

Input: Input data, parameters [$\beta$, $\rho$, $\alpha$ (Fuzzy)]      Output: Cluster 1: procedure 2:     if Topology for Fuzzy then 3:         Normalize and encode the input data by Equation (1) and 2 4:     end if 5:     while Data for training exists do 6:         Calculate the distance or similarity between the input data and all existing clusters 7:         Equation (3) or Equation (7) 8:         while not finding a cluster to allocate the data do 9:            Select the cluster according to the equation Equation (4) or Equation (8) 10:            if the Equation (5) or Equation (9) is satisfied then 11:                Update the weights according to Equation (6) or Equation (10) 12:            else 13:                Remove the category from the search process 14:            end if 15:            The data will be allocated to the current cluster 16:         end while 17:     end while 18: end procedure

2.2. ARTMAP - Supervised

The ARTMAP model with supervised learning is made up of two ART modules, ART${ }_a$ and ART${ }_b$, as shown in Figure 2. Unlike their unsupervised counterparts, these modules do not operate in the dark. They use both the input data and the corresponding labels, providing guidance during training. Furthermore, ARTMAP doesn’t just passively accept information. It actively searches for meaningful connections. The inter-ART associative memory module is responsible for checking whether there is a match between the input patterns and the output. The vigilant match-tracking mechanism minimizes prediction errors and maximizes the model’s ability to generalize. The weight matrices associated with the ART${ }_a$ (${\mathit{W}}^a$), ART${ }_b$ (${\mathit{W}}^b$) and Inter-ART (${\mathit{W}}^{ab}$) modules are initialized with values equal to 1, indicating the absence of active categories. However, as training takes place, there is resonance between the input patterns and the output, causing specific categories to be activated. This dynamic activation process allows ARTMAP to learn and adapt in a controlled manner.

Although it is a powerful architecture, it is a model sensitive to noise in the labels due to supervised learning, accuracy can be compromised if the labels are inaccurate or noisy.

Algorithm 2: ARTMAP algorithm.

Input: Input data, parameters [$\beta$, $\rho$, $\alpha$ (Fuzzy)]      Output: Cluster 1: procedure 2:     if Topology for Fuzzy then 3:         Normalize and encode the input data by Equation (1) and 2 4:     end if 5:     while Data for training exists do 6:         Module B 7:         Calculate the distance or similarity between the input data and all existing clusters 8:         Equation (3) or Equation (7) 9:         while not finding a cluster to allocate the data do 10:            Select cluster according to Equation (4) or Equation (8) 11:            if the Equation (5) or Equation (9) is met then 12:                Update the weights of module B according to Equation (6) or Equation (10) 13:            else 14:                Remove the category from the search process 15:            end if 16:         end while 17:         Module A 18:         Calculate the distance or similarity between the input data and all existing clusters 19:         Equation (3) or Equation (7) 20:         while not finding a cluster to allocate the data do 21:            Select cluster according to Equation (4) or Equation (8) 22:            if the Equation (5) or Equation (9) is met then 23:                Inter-ART module 24:                if The category found by module A matches the category of module B then 25:                    Update the weights of module A (Equation (6) or Equation (10)) and Inter-ART 26:                else 27:                    Remove the category from the search process and increase the vigilance parameter of module A 28:                end if 29:            else 30:                Remove the category from the search process 31:            end if 32:         end while 33:         The data will be allocated to the current cluster 34:     end while 35: end procedure

2.3. Fuzzy Topology in ART Networks

In the fuzzy topology, ART networks use the fuzzy logic connectives developed by [69]. These connectives are used both to assess the similarity between input patterns and in the vigilance test. Furthermore, in this topology the data needs to be normalized and coded by complement as specified by Equation (1) and (2), in order to avoid the proliferation of categories [68].

$$\overline{\mathit{a}}={\displaystyle \frac{\mathit{a}}{\left|{\displaystyle \sum _i^M}{a}_i\right|}},$$

(1)

$$\mathit{I}=\left[\begin{array}{cc}\overline{\mathit{a}}& {\overline{\mathit{a}}}^c\end{array}\right],$$

(2)

The Equation (3) in turn is responsible for checking the similarity between the input pattern and the existing categories, the winning category will be the one with the greatest similarity to the input data as seen in the Equation (4). The j index refers to the cluster in which the calculations are being performed, while the J index identifies the winning category in this process.

$$T_j={\displaystyle \frac{\left|\mathit{I}\wedge {\mathit{W}}_j\right|}{\alpha +\left|{\mathit{W}}_j\right|}},$$

(3)

$$J=arg\{max\{T_j:j=1,...,\mathrm{number}\mathrm{of}\mathrm{clusters}\},$$

(4)

Resonance occurs if the vigilance criterion given by Equation (5) is met. This process acts as a filter preventing the unbridled creation of categories. It ensures that only significantly different patterns trigger the formation of new categories, preventing the network from becoming full of redundant categories. The $\rho$ parameter sets the threshold for this filter, determining the level of similarity required between an input pattern and the existing categories for the network to accept it or not within the cluster.

$${\displaystyle \frac{|\mathit{I}\wedge {\mathit{W}}_J|}{\left|\mathit{I}\right|}}\ge \rho ,$$

(5)

When resonance is achieved, the network starts a weight adjustment process, as detailed in Equation (6). In this way, the weights associated with the winning category are modified to accommodate the new pattern.

$${\mathit{W}}_J^{new}=\beta \left(\mathit{I}\wedge {\mathit{W}}_J^{old}\right)+(1-\beta ){\mathit{W}}_J^{old},$$

(6)

By using fuzzy logic in the network calculations, together with normalization and complement coding, the clusters take the form of hyperrectangles, giving flexibility to the complexity of the input patterns.

2.4. Euclidean Topology in ART Networks

By adopting Euclidean topology, ART networks use Euclidean distance as a metric to measure the proximity between input patterns and existing categories. As a result, the network generates hyperspheres centered on the cluster representing the class, instead of hyperectangles. This approach eliminates the need to normalize and encode the input vector, thus reducing computational costs [72]. The Equation (7) indicates how the distance between the input pattern and the existing clusters is calculated. The category chosen is the one with the smallest distance according to Equation (8), in contrast to the fuzzy approach, where the choice is based on maximum activation.

$$T_j=\sqrt{{\displaystyle \sum _{i=1}^M}{\left(a_i-W_{ji}\right)}^2},$$

(7)

$$J=arg\{min\{T_j:j=1,...,numberofclusters\}\},$$

(8)

The vigilance criterion will be given by Equation (9) and follows a similar pattern to that used in fuzzy topology, as detailed above in Section 2.3.

$$\frac{T_J}{max\left\{{\displaystyle \sum _{i=1}^M{\left(a_i\right)}^2,\sum _{i=1}^M{\left(W_{Ji}\right)}^2}\right\}}\le \rho ,$$

(9)

Another change is in the Equation (10) responsible for the network’s learning process. Here, the weight matrix (${\mathit{W}}_J$) is updated according to a weighted combination of the input pattern ($\mathit{a}$) and the previous weight matrix (${\mathit{W}}_J^{old}$), controlled by the $\beta$ parameter.

$${\mathit{W}}_J^{new}=\beta \left(\mathit{a}\right)+(1-\beta ){\mathit{W}}_J^{old},$$

(10)

2.5. Self-Expanding Mechanism

The self-expanding mechanism proposed by Moreno [71,72] starts training the network by adopting the first pattern as the center of the first cluster and expands according to the need to create new clusters, i.e. as the new input patterns do not fit into the pre-existing categories, a new cluster is created to allocate that pattern. This modification reduces the processing time and, consequently, the computational expense [71,72], as it only checks the similarity of active clusters, unlike traditional architectures, in which similarity is checked with all clusters, including inactive ones. Furthermore, when inputting data that is incompatible with any of the existing clusters, the network starts a new cluster with the new pattern that is not in resonance with the input data and the empty cluster. These changes do not alter the stability of the network or compromise its ability to recognize new information in the training data.

Algorithm 3: Self-expansion mechanism algorithm.

Require: Initialization of the weight matrix with the first pattern 1: procedure 2:     if The surveillance criterion is not met (Equation (5) or Equation (9)) then 3:         if There are categories to be checked then 4:            Remove the current category from the search process and search again 5:         else 6:            Create a new cluster to allocate the current data 7:            The weight for this cluster will be the input pattern itself 8:         end if 9:     end if 10: end procedure

2.6. Continuous Training in ART Neural Networks

The continuous training strategy enables networks to learn online and continuously so that prediction and training are carried out constantly, making them more efficient. This approach eliminates the need to restart training when incorporating new data, resulting in a network that constantly adjusts to changes in the environment. The inclusion of this module in ART networks is only possible due to their characteristics of stability and plasticity [72]. With online learning, there is no need to restart training when new data becomes available; this module allows new patterns to be added to the network’s memory permanently [72].

In order to incorporate this module, the network needs to have temporary and definitive categories, a necessary process so that data that appears rarely is not incorporated into the network’s memory. In addition, two new parameters are added: the first, called the permanence parameter ($NMIN$), defines the number of patterns needed for a temporary category to become definitive. The second, called the novelty index ($\eta$), checks for the need to update the weights of the definitive category, discarding information that is not very relevant to the network, avoiding overloading the network’s memory [72]. Whenever a pattern belongs to a temporary cluster, resonance will occur; on the other hand, if the category to which the pattern belongs is definitive, resonance will only occur if the distance/similarity between the entry and the center of the category is greater than the parameter ($\eta$) [72].

Thus, Equation (3) and Equation (7) are calculated for their respective topologies, for both temporary and permanent categories. The selection process remains the same. If the chosen category is definitive and passes the surveillance criterion, resonance will only occur if the condition imposed by Equation (11) (Fuzzy) or Equation (12) (Euclidean) is met, i.e. resonance will only occur if the pattern is significantly different.

$$T_J>\eta ,$$

(11)

$$T_J<\eta ,$$

(12)

3. Application of ART Networks in the diagnosis of COVID-19

The simulations were carried out using ten open-access datasets. These datasets contain information on clinical biomarkers collected from patients in hospitals located in Brazil, Spain, Italy and Ethiopia. The number of instances and features contained in each of the datasets can be seen in Table 1.

The methodology used in this article consisted of two stages. In the first stage, the main objective was to select the best ART model from the seven architectures tested. In the second stage, the model selected in the previous stage was used to verify its generalization capacity in relation to data from different countries. Specific details of each stage are described in Section 3.1 and Section 3.2, respectively.

3.1. First Stage

In this stage, three different data sets were selected: Data${ }_1$, Data${ }_2$ and Data${ }_3$. This choice was made considering the diversity of attributes and the fact that they have been subjected to simulations with various algorithms, including deep learning models. This is to make it easier to compare the performance of the ART models with those in the literature. The Table 2 presents in detail the attributes present in each dataset used in this stage, providing an overview of the attributes used during the experiments conducted.

The simulations were conducted using the same attributes as those previously used by other researchers. More specifically, the Data${ }_1$ set underwent the same pre-processing carried out by Brinati et al. [21]. This set contains data from patients who were admitted to the IRCCS San Raffaele Hospital, located in Segrate, in the province of Milan, Italy, between February and March 2020.

The Data${ }_2$ set was used in the same way as in Cabitza et al. [58]. This set includes the results of routine blood tests carried out on 1925 patients admitted to the emergency room of San Raffaele Hospital between mid-February and May 2020.

For its part, the Data${ }_3$ set was made available pre-processed by Alakus and Turkoglu [54]. This set was obtained from data made available by the Hospital Israelita Albert Einstein1. The data was collected from 5644 patients and contains information from 111 laboratory tests carried out on patients treated at the hospital located in São Paulo, Brazil. It is worth noting that all datasets were used without any modification.

3.2. Second Stage

In this stage, Data${ }_2$, Data${ }_4$, Data${ }_5$, Data${ }_6$, Data${ }_7$, Data${ }_8$, Data${ }_9$ and Data${ }_{10}$ were merged to create a new dataset containing hematological information from different countries. This combination was carried out in order to provide more comprehensive information to verify the model’s ability to predict COVID-19, since hematological parameters are influenced by individual factors such as age, gender and lifestyle, as well as population and ecological factors such as ethnic origin, climate, exposure to pathogens and altitude [75,76,77].

The Table 3 shows the averages and standard deviations of the common attributes between all the data sets. The Variance Inflation Factor (VIF) and the Mann-Whitney (MW) test [78] were used to carefully select the attributes for the simulations.

The MW test is a non-parametric statistical technique used to compare the medians of two independent samples [78,79]. The test was used to check for significant differences between the classes in relation to each attribute analyzed. The determination of statistical significance was established at a confidence level of $0.05$, which indicates that values of p below this threshold are considered statistically significant, indicating relevant differences between the medians of the compared groups.

On the other hand, VIF is a statistical measure used to assess multicollinearity between variables. It is obtained from the elements on the main diagonal of the inverse correlation matrix, as expressed in Equation (14) [80].

$${\mathit{R}}_{j\times j}=\left({\displaystyle \frac{1}{i-1}}·{\mathit{W}}_{j\times i}^{\prime }·{\mathit{W}}_{i\times j}\right),$$

(13)

$${\mathbf{V}}_{j\times j}={\left({\mathit{R}}_{j\times j}\right)}^{-1},$$

(14)

Where $\mathit{R}$ represents the correlation matrix, and $\mathit{W}$ is the matrix of centered and reduced data. It is important to note that the $\mathit{R}$ matrix will be invertible if, and only if, the rank ($\mathit{W}$) is complete, i.e. the columns/rows of $\mathit{W}$ are linearly independent [81]. The literature points out that VIF values above 5.0 or 10.0 indicate the presence of multicollinearity [82]. On this basis, it was decided not to include attributes with a VIF above 8.0 in the simulations.

The results obtained after the tests are shown in Table 4. The attributes highlighted were selected for the simulations because they were relevant in both tests. It is worth noting that attributes expressed as percentages were not included in the selection, even though they were indicated as relevant by the tests. The decision to exclude these attributes was based on the consideration that absolute values have clinical significance and provide more complete information than relative values (%) [83].

The selected attributes are part of the white blood series and play a fundamental role in the immune system, helping to fight infections, viruses and bacteria. Analyzing the attributes, it was possible to see some relevant information in relation to patients affected by COVID-19, such as the reduction in cell counts, such as basophils (Figure 3b), eosinophils (Figure 3c) and lymphocytes (Figure 3d). These factors are supported in the literature by authors such as Kermali et al. [84], Samprathi and Jayashree [85] and Keykavousi et al. [86].

After using the combined data set to train and test the model, it was subjected to an external validation process using the Data${ }_1$ and Data${ }_{11}$ sets to assess its ability to generalize on data never before seen by the model. It is worth noting that the same attributes used in training and testing were also used in the external validation phase. Data1 was chosen because it comes from Italy, a country already present in the training data, but with data collected in a different period, while Data11 was selected because it comes from Spain, a different country to those included in the training.

3.3. Evaluation Metrics

To evaluate the performance obtained by each network some metrics frequently present in the literature were used such as accuracy (ACC), sensitivity (Se), specificity (Sp), F${ }_1$-Score (F${ }_1$), Matthews correlation coefficient (MCC) given respectively by (15), (16), (17), (18), and (19) [87,88,89].

$$\mathrm{ACC}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(15)

$$\mathrm{Se}={\displaystyle \frac{TP}{TP+FN}}$$

(16)

$$\mathrm{Sp}={\displaystyle \frac{TN}{TN+FP}}$$

(17)

$${\mathrm{F}}_1={\displaystyle \frac{TP}{TP+\frac{1}{2}\left(FP+FN\right)}}$$

(18)

$$\mathrm{MCC}={\displaystyle \frac{TP·TN-FP·FN}{\sqrt{(TP+FP)·(TP+FN)·(TN+FP)·(TN+FN)}}}$$

(19)

Along with the metrics mentioned above, we use the ROC (Receiver Operating Characteristics) curve to evaluate the quality of the model classification. This curve is generated by plotting the Sensitivity and False Positive Rate ($1-$ Specificity) for different thresholds of the probability estimated by the model. Based on [90] work, the AUC (Area Under the Curve) can be understood as a summary of the ROC curve, the AUC value and the quality of the model can be seen in Table 5.

3.4. Classifier

In the unsupervised learning models, it was necessary to use a classifier to convert the clusters to the respective classes of the problem, “COVID-19” and “non-COVID-19”. The probability required to generate the ROC curves was also extracted in this step, in which the number of patterns contained in each cluster and the class to which it belonged were analyzed. For example, suppose 10 patterns are allocated to a certain cluster during training, of which 9 are from patients with COVID-19. In this case, this cluster will be labeled as COVID-19 by majority vote, the probability of being COVID-19 will be 0.9, and the complementary probability, that is, of the patient being non-COVID-19 will be 0.1, any pattern allocated to this cluster in the test stage will receive this probability. It is worth noting that for the ARTMAP models that have supervised learning it was not possible to extract the probability and consequently generate the ROC curve, due to the model having discrete output.

3.5. Validation

To partition the data, the hold-out cross-validation method was used, randomly dividing the data into training and test sets based on a pre-determined rate. In addition, the best model found during hold-out validation on each data set was subjected to the repeated hold-out method, also known as Monte Carlo cross-validation [91,92], was employed as a way to obtain a more consistent estimate of model performance. The data was partitioned using the 70–30 ratio, that is, 70% of the samples for training and 30% for testing. For Monte Carlo cross-validation, the process was repeated 50 times, with the data split randomly in each repetition.

4. Computational Results

This section will present the results obtained for COVID-19 diagnostics using the ART networks on each dataset.

4.1. Primeita Etapa

4.1.1. Data${ }_1$

The performance of various ART networks was evaluated, and the results are presented in Table 6. The SE Fuzzy ART CT network showed the highest accuracy, reaching 0.871, and also demonstrated the greatest ability in correctly detecting patients with COVID-19, as shown by the sensitivity of 0.926. Notably, fuzzy logic-based networks were found to perform better than Euclidean distance-based networks for this particular dataset.

Considering that the network with the best performance in the simple hold-out validation was the SE Fuzzy ART CT, it was used in the Monte Carlo validation to evaluate the network’s generalization ability. Table 7 shows the mean metrics as well as the 95% confidence interval (CI) for the Monte Carlo validation.

To illustrate the results obtained by the best network in each iteration of the Monte Carlo validation, Figure 5 shows the ACC, Se, and Sp for each iteration. It can be seen that the network’s performance was not constant in all iterations, reaching a maximum accuracy of 0.9294 and a minimum of 0.7176. On the other hand, Figure 6 presents the average ROC curve and the average AUC obtained by the network in this validation. According to [90], an AUC score above 0.8 can be considered a good performance achieved by the network.

4.1.2. Data${ }_2$

Table 8 presents the results obtained by the networks for Data${ }_2$. The tested model’s achieved approximate performance with MCC ≈ 0.5, and the SE Fuzzy ART CT network achieved the best result, almost perfectly detecting positive and negative patients for COVID-19. Although the SE Euclidean ART CT network achieved the best result in specificity, it was not the best model since it correctly detected only 54.7% of the COVID-19 patients.

The SE Fuzzy ART CT Network, which demonstrated the best performance, was subjected to Monte Carlo cross-validation to ensure the reliability of the results, reducing the risk of overfitting. The results are presented in Table 9, where the average results and 95% confidence intervals for the different metrics are presented. The network’s performance was remarkable, with an average accuracy of 0.9348, indicating that it successfully classified the data. In addition, the network obtained an MCC of 0.8721, a vital performance measure that considers the balance between sensitivity and specificity.

Figure 7 illustrates the network’s performance in ACC, Se, and Sp for each iteration of the Monte Carlo validation. The results showed that the network was highly stable performing well in almost all iterations. The network achieved a maximum accuracy of 0.9775 and a minimum of 0.875, and of the 50 iterations, only five were below 0.91 in accuracy. These results suggest that the network is highly robust and can be used effectively for the classification task. It should also be noted that the stability in the network’s performance over the different iterations of the Monte Carlo validation strongly indicates its generalizability and ability to handle different types of input data.

In Figure 8, the ROC curve and the average AUC obtained after Monte Carlo validation are presented. The AUC value obtained was 0.9395, which suggests that the network has excellent discrimination ability, according to [90].

4.1.3. Data${ }_3$

This dataset needs to more balanced, containing 13.33% data from patients with COVID-19 and 86.67% non-COVID-19 patients. As expected, this unbalance impaired the classification ability of the implemented models, resulting in a low predictive performance as can be seen in Table 10.

When working with unbalanced classes, evaluating the performance of a model can be challenging, as traditionally used metrics such as accuracy, sensitivity, and specificity can be misleading and may not provide an accurate representation of model performance [87,88]. Instead, it is recommended to use metrics suitable for unbalanced data, such as the Matthews Correlation Coefficient (MCC) and Area Under the ROC Curve (AUC) [88,89].

In this context, the best performing network for the dataset in question was the SE Euclidean ART CT, achieving an MCC of 0.876. This indicates that the model had a strong correlation between the predicted and actual classes, taking into account both true and false positives and negatives. Table 11 presents the average metrics as well as the 95% confidence interval obtained by the SE Euclidean ART CT network after Monte Carlo validation.

By analyzing the results presented in Table 11, it can be seen that the specificity obtained by the network was very high, with a value of 99.65%. This result was expected since most of the dataset samples were from patients who were not infected with COVID-19. However, the average sensitivity obtained by the network was only 0.4714, which indicates that the network did not perform well in detecting COVID-19 positive cases.

Looking at the MCC, a suitable metric for unbalanced data sets, it is noted that the SE Euclidean ART CT network obtained an average MCC of 0.6195 after Monte Carlo validation. While this is not a high value, it is still considered acceptable. Comparing the results obtained in Table 10 and Table 11, it is evident that there was a reduction in the network’s performance after Monte Carlo validation. The network could not correctly detect positive COVID-19 cases in some iterations of the validation, with the sensitivity falling below 30% in some iterations, as illustrated in Figure 9.

The ROC curve and the AUC can be seen in Figure 10. By analyzing them, we can see that while specificity remained relatively stable at all cut-off points, there was considerable variation in sensitivity, i.e., the ability to detect positive patients correctly. The mean AUC was 0.7458, with a standard deviation of 0.1265, achieving acceptable performance according to [90].

Table 12 Comparison the results obtained by the neural networks used in this paper and the results found in the literature for the same data sets. The table includes metrics such as ACC, Se, Sp, MCC, F${ }_1$, and AUC (ROC).

The results obtained for Data${ }_1$ were quite promising, as the SE Fuzzy ART CT network achieved AUC of 0.8501 and 0.8006 in hold-out and Monte Carlo validation, respectively. These results were better than those obtained by commonly used models such as RF and ET, often used for classification tasks. Although the performance of the SE Fuzzy ART CT network was good, the DNN deep learning model used by [93] obtained even better results, achieving an AUC value of 0.9220.

When examining the results for Data${ }_2$, it can be seen that the SE Fuzzy ART CT network achieved outstanding performance. It obtained an AUC of 0.9481 in the hold-out validation, indicating high accuracy in distinguishing between COVID-19 positive and negative patients. It outperformed machine learning models traditionally used in classification tasks such as ET, KNN, and even the deep learning algorithm DNN, which achieved an AUC of 0.9320. The sensitivity of the ART network was also significantly higher at 97.03%, indicating it is effective in correctly identifying positive patients. It is worth noting that even when the SE Fuzzy ART CT network was subjected to Monte Carlo cross-validation, it maintained its good performance with an AUC of 0.9395, outperforming DNN once again.

For Data${ }_3$, the SE Euclidean ART CT network achieved AUC of 0.8962, a result similar to that obtained with the CNN-LSTM model used by [54], which achieved AUC of 0.9000. However, after performing Monte Carlo cross-validation, a decrease in the performance of the SE Euclidean ART CT network was observed due to a significant imbalance between classes.

The Comparison of the computational time (in seconds) between the two best models implemented in this paper (SE Fuzzy ART CT and SE Euclidean ART CT), with the best model obtained by [93] the Deep Neural Network (DNN), can be seen in Table 13. It is worth noting that the computational time given in the table considers only the training and testing time. Also, the best algorithms for each dataset are highlighted in bold.

When analyzing the computational time in Table 13, it is clear that the ART networks showed significantly faster convergence than the DNN. This is due to the nature of the network and accentuated by the self-expansion mechanism. In addition, it can be seen that the Euclidean distance-based neural network obtained a slightly shorter time than the fuzzy neural network. This difference can be attributed to the fact that the Euclidean network not requiring normalization and encoding of the input data. Consequently, the size of the input data entered into the Euclidean network is halved, leading to faster processing. Another relevant feature of ART networks is that they do not suffer from catastrophic forgetting, unlike those based on the back-propagation algorithm and deep learning models, as reported by [61].

4.2. Second Stage

In this second stage, we sought to compare the performance of the winning model from the previous stage, SE Fuzzy ART CT, with traditional models frequently used in the literature. To do this, the combination of data described in Section 3.2 was used. Algorithms such as K-Nearest Neighbors (KNN), Random Forest (RF), Multi-layer Perceptron (MLP), Logistic Regression (LR), Extra Trees (ET), XGBoost (XGB) and Naive Bayes (NB) were implemented in Python with the Scikit-Learn library [94]. The parameters of each model were optimized using the GridSearchCV method, exploring various combinations of hyperparameters in order to identify the one that resulted in the best performance. The results obtained are presented in the Table 14, highlighting the superiority of the ART network over traditional models, achieving an MCC of 0.9628.

In addition, the Figure 11 shows the ROC curve and AUC for each model. The SE Fuzzy ART CT network obtained the best AUC, with a value of 0.9770, indicating that it has a high ability to distinguish between classes. Among the traditional models tested, the RF algorithm obtained the best performance, with AUC and MCC of 0.8504 and 0.5510, respectively. However, these results are significantly lower than those of the ART network. The RF algorithm’s ability to distinguish between classes is around 12% lower than that of the ART network.

The Table 15 and Figure 12 show, respectively, the metrics and the ROC curve along with the AUC of the models trained with the combined data and validated with Dados${ }_{12}$, i.e. the data collected in Spain.

Although it showed a slight decline in performance when compared to the results presented in Table 15 and Figure 11, the SE Fuzzy ART CT network still outperformed the traditional models, which also showed a reduction in performance when tested with the new data. The model obtained an AUC of 0.9759, a significantly better performance than the traditional models, which obtained AUCs of 0.8413 for XGB and 0.8409 for RF.

When the SE Fuzzy ART CT network trained with the combined data was submitted to the Data${ }_1$ validation set, there was a reduction in its performance compared to the results obtained with the combined data. This reduction was only 1.71% in terms of AUC, from 0.9770 to 0.9606. The traditional models, on the other hand, showed an increase of less than 0.01% in relation to the AUC obtained with the combined data. However, even with this reduction in performance, the ART network still remained the most effective model in distinguishing between patients with and without COVID-19.

4.3. Discussão

The combined analysis of the two stages highlights not only the effectiveness of SE Fuzzy ART CT in detecting COVID-19, but also its ability to handle new patient data. In the first part, it did better than other models mentioned in the literature. In the second stage, when we compared conventional models with SE Fuzzy ART CT, we confirmed its superiority. The network’s results remained consistent even with data from different countries. The online training capacity, combined with the unsupervised training, were the attributes responsible for the excellent performance and places it as an adaptable and promising tool for accurate, real-time classifications.

In addition to its positive results, the study also addresses some limitations listed by other articles. A study by [95], for example, cites the lack of patients from other geographical locations as a limitation. [95] states that it is necessary to consider patients from different demographic groups in order to establish reliable results. This study addresses this limitation by training and testing the models on data from several hospitals in four countries. This allows the results to be more representative of the population.

In addition, the study tests models with unsupervised learning, a limitation pointed out in the work by [96]. Since unsupervised learning allows models to learn to identify patterns in the data without the need for labels.

5. Conclusion

This article presents the results obtained by neural networks based on Adaptive Resonance Theory (ART) for the classification of COVID-19, using laboratory tests (blood tests) in order to correctly classify infected and healthy patients. A comparison was made between seven different ART architectures, including online training algorithms. The most successful architecture was subjected to a dataset made up of laboratory tests from patients in different countries. When comparing the results of the first stage, it can be seen that the self-expanding fuzzy ART network with continuous training (SE Fuzzy ART CT) and the self-expanding Euclidean ART network with continuous training (SE Euclidean ART CT) outperformed the other ART networks implemented. In addition, in some cases, the networks outperformed the DNN and CNN deep learning models frequently used in the literature. The SE Fuzzy ART CT obtained an AUC of 0.8501 for Data${ }_1$, 0.9481 for Data${ }_2$ and 0.8962 for Data${ }_3$. When considering computational time, the self-expanding ART neural networks with continuous training were more efficient in the three data sets tested.

In the second stage, the self-expanding SE Fuzzy ART CT that won out in the first stage was subjected to a data set from various hospitals in four countries. This network was compared with traditional algorithms in the literature, maintaining an excellent performance that significantly outperformed all traditional models. Even during external validation with data from a different country to that included in the training, validating its generalization capacity.

In addition to the superior results compared to most of the models analyzed, the networks used do not face the main problems associated with traditional algorithms, such as slow learning, catastrophic forgetting and off-line training. Thus, these results highlight the potential of networks based on adaptive resonance theory, such as SE Fuzzy ART CT and SE Euclidean ART CT, as highly effective and reliable tools to aid in the detection of COVID-19, reducing the false negative rate of RT-PCR or even as a viable solution for detection in places with limited resources or underdeveloped countries.

Despite the promising results obtained by the ART networks, in future studies we intend to reduce the size of the data sets, selecting the most significant attributes to detect healthy and COVID-19 patients, and we can also add more information to aid classification, such as the use of imaging tests or symptoms. In addition, it is possible to make some adjustments to the models to improve the quality of the classification, such as modifying the normalization used, extending the research to the $\beta$ parameter, using the k-Fold cross-validation method as a way of validating and evaluating the model’s generalization capacity.