Personalized Assessment of Mortality Risk and Hospital Stay Duration in Hospitalized Patients with COVID-19 Treated with Remdesivir: A Machine Learning Approach

1. Introduction

The Coronavirus Disease 2019 (COVID-19) first emerged in late December 2019 in Wuhan, China. The genetic sequencing of the virus revealed it to be a novel beta-coronavirus, named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) [1]. As of December 13, 2023, the World Health Organization (WHO) reports that the global impact of the COVID-19 pandemic has resulted in nearly 7 million deaths [2]. The Omicron variant displaced the Delta variant at the end of 2021. The actual variant is less severe clinically than earlier SARS-CoV-2 variants, likely due to widespread global vaccination [3]. Different treatments have also helped lower hospitalization and death rates [4,5].

COVID-19 is a highly contagious disease with significant risks of morbidity and death. It can cause bilateral pneumonia, severe respiratory failure necessitating invasive mechanical ventilation (IMV) and/or damage to multiple organs, sometimes leading to tragic outcomes [6].

SARS-CoV-2 affects various organs, as seen in autopsy findings (such as the kidneys, heart, intestines, liver, and brain). However, it has a specific tendency to infect the respiratory system [7]. Most individuals infected will experience symptoms such as cough, fever, fatigue, and muscle pain. However, a small percentage will progress to develop a hyperinflammatory state with pulmonary edema and cellular infiltration, leading to acute respiratory distress syndrome (ARDS) [8]. Additionally, it may lead to complications like kidney issues, coagulation disorders, or shock, which can result in mortality rates exceeding 30% [9].

The virus enters cells using structures like angiotensin-converting enzyme 2 and transmembrane serine protease 2, which are present in respiratory tract, cornea, and gastrointestinal cells [10].

In COVID-19, factors contributing to the severity and progression of the infection include advanced age, various health conditions, and changes in lab results. Elevated levels of certain markers like C-reactive protein (CRP), lactate dehydrogenase (LDH), ferritin, procalcitonin, and proinflammatory cytokines such as interleukin (IL)-6, IL-2, IL-1β, TNF-α, and G-CSF have been identified as significant indicators [11]. Furthermore, the neutrophil-to-lymphocyte ratio is often elevated in patients with severe COVID-19 [12].

Presently, the broad vaccination of the population stands out as the most effective public health measure in the ongoing battle against SARS-CoV-2.

Though only a few drugs prove effective in treating severe cases of COVID-19, early diagnosis and prompt initiation of therapy, complemented by nutritional and organ support, can markedly enhance outcomes.

Remdesivir is a bioactive molecule that has demonstrated in vitro activity against SARS-CoV-2, along with prophylactic and therapeutic efficacy in non-clinical models of other viruses, such as SARS-CoV, MERS-CoV, and Ebola [12]. In October 2020, it became the inaugural drug authorized by the Food and Drug Administration (FDA) for the treatment of COVID-19. Based on various clinical trials, its use was approved for adults and pediatric patients aged 12 and older, weighing at least 40 kilograms, for the treatment of COVID-19 requiring hospitalization [13,14,15].

The preliminary findings from the WHO-funded Solidarity trial revealed that lopinavir, hydroxychloroquine, interferon (IFN)-β1a, and remdesivir had little to no effect on hospitalized COVID-19 patients, as indicated by overall mortality, initiation of ventilation, and hospital stay duration [16]. The final results highlight the ineffectiveness of the drugs under investigation, except for remdesivir. While it doesn't significantly impact COVID-19 patients already on ventilation, it does show a modest effect on mortality or progression to ventilation (or both) in other hospitalized patients [17].

Currently, little is known about the predictive factors linked to poor outcomes in hospitalized COVID-19 patients treated with antivirals in general and specifically with remdesivir in clinical practice.

The goal of our study is to use machine learning (ML) models to categorize hospitalized COVID-19 patients undergoing remdesivir treatment based on the risk of mortality and/or hospital stay duration. ML, a subset of artificial intelligence (AI), employs statistical and mathematical algorithms to extract patterns from the data, aiding in making complex decisions [18]. Unlike classical statistical models created for inferences about variable relationships, these models are designed to make accurate predictions using data from a multitude of variables.

AI tools have been implemented in various areas to combat COVID-19, including drug and vaccine discovery or repurposing [19,20].

To our knowledge, this is the first study to develop, compare and validate six supervised ML models predicting factors associated with a high risk of mortality and/or hospital stay duration in patients with SARS-CoV-2 infection undergoing treatment with remdesivir.

2. Materials and Methods

2.1. Data Source

Patient information was systematically gathered from various internal hospital channels, utilizing two primary sources: (1) the electronic medical records (EMR) system, equipped with specialized modules for documenting clinical analysis results, radiological imaging findings, and electronic medical prescriptions, and (2) the intensive care unit (ICU) electronic prescription program. This comprehensive approach facilitated the synthesis of extensive data, enabling the methodical development of a personalized data collection questionnaire (DCQ) for each individual patient.

2.2. Study Design and Population

A retrospective observational study was conducted at a high-complexity tertiary hospital. Initially, 285 patients began remdesivir treatment, but 33 were excluded for not reaching the minimum required dose. Exclusions were based on various criteria, including a glomerular filtration rate < 30 ml/min, mortality or hospital discharge, clinical decision, symptom onset beyond 7 days of starting remdesivir, and a negative diagnostic test. Ultimately, the study focused on 252 patients (58.3% male) admitted with microbiologically confirmed SARS-CoV-2 between January 1 and December 31, 2022, using reverse transcription-polymerase chain reaction (RT-PCR) from nasopharyngeal swabs.

In this study, inclusion criteria involved patients ≥ 12 years old and weighing ≥ 40 kg, admitted to the hospital with a COVID-19 diagnosis. Selected patients met specific requirements for remdesivir administration according to the hospital's internal protocol at the study's outset. These criteria included symptom onset ≤ 7 days before the first remdesivir dose, no need for oxygen therapy or low-flow oxygen therapy, and meeting at least two of the following three conditions: 1) respiratory rate (RR) ≥ 24 breaths per minute; 2) baseline oxygen saturation (SpO₂) < 94% in ambient air; and 3) PAFI index (PaO₂/FIO₂) < 300 mmHg. Patients were required to have received a minimum of three doses of the drug. The approved remdesivir regimen consisted of an initial 200 mg loading dose administered intravenously over 30-120 minutes in 100-250 ml of sterile, pyrogen-free 0.9% sodium chloride solution, followed by maintenance doses of 100 mg for a duration ranging from 5 to 10 days, depending on the patient's level of immunosuppression. Participants provided informed consent before participating in the study, and it was approved by the Ethics Committee of the General University Hospital of Valencia.

2.3. Study Data

The DCQ included data on demographic, clinical, and laboratory variables, organized into 8 sections:

1. Patient profile: This section covered demographic factors like age and sex, as well as clinical aspects including:

a) Weight and height or body mass index.

b) Presence of comorbidities: such as smoking, obesity, hypertension, diabetes mellitus (DM), chronic obstructive pulmonary disease (COPD), asthma, and other chronic respiratory conditions. Details on other serious underlying conditions were provided in an open-text section. The number of comorbidities was categorized (1, 2, 3, and > 3).

c) Pre-admission pharmacological treatment: This included the use of angiotensin-converting enzyme inhibitors/angiotensin receptor blockers, non-steroidal anti-inflammatory drugs, antihistamines, and/or montelukast, as well as information on the type of COVID-19 vaccine received and the number of doses administered.

d) COVID-19 Symptoms: A detailed list of symptoms was recorded, including fever, cough, dyspnea, fatigue, loss of taste or smell, headache, myalgia, sore throat, nasal congestion, rhinorrhea, conjunctivitis, rash, nausea, vomiting, and diarrhea.

2. Initial hospitalization details:

-

Date of admission to the emergency department.

-

Date of admission to the hospital.

-

Date of symptom onset.

-

Date of microbiological confirmation of SARS-CoV-2 infection.

-

Any life support limitations and date implemented.

-

Whether the patient needed ICU admission.

3. ICU admission details:

-

ICU admission date.

-

Mortality risk assessed by the CURB-65 scale.

-

Level of consciousness evaluated using the Glasgow Coma Scale.

-

Other clinical variables in the first 24 hours: fever (≥ 38ºC), RR > 24 breaths/minute, systolic blood pressure < 90 mmHg, SpO2, and number of lung quadrants affected in imaging tests (ranging from 1 to 4).

-

Severity of illness assessed using the APACHE II scale within the first 24 hours of admission

-

Patient's condition evaluated using the SOFA scale during their ICU stay.

4. Analytical and radiological data overview:

This section covers laboratory tests done just after hospital admission (in the emergency department or upon admission) before starting remdesivir treatment, as well as those performed after completing the remdesivir treatment.

The included laboratory parameters were: PAFI index; FIO₂; SpO₂; leukocytes; neutrophils; lymphocytes; monocytes: platelets; mean corpuscular volume; hemoglobin; erythrocyte sedimentation rate; CRP; aspartate aminotransferase; alanine aminotransferase; LDH; blood urea nitrogen; serum creatinine; albumin and/or total proteins; total cholesterol; procalcitonin; lactic acid, bicarbonate and pH; creatine phosphokinase and/or troponin; D-dimer and ferritin.

Additionally, the determination of cycle threshold (Ct) values from the virus RT-PCR at the start of treatment (or the closest available) is included. Ct is a semi-quantitative value inversely related to the amount of RNA in the sample. The SARS-CoV-2 antigenic variant is also identified.

A classification of pulmonary radiological status was included:

-

Affected side (bilateral or unilateral)

-

Type of lung injury (ground-glass opacity, consolidation, or mixed)

-

Density pattern (patchy, confluent, or mixed)

5. Pharmacological treatment during hospitalization: The medications considered included drugs that modulate inflammation and the immune system, such as IL-6 receptor antagonists (e.g., tocilizumab and sarilumab), IL-1 receptor antagonist (anakinra), Janus kinase inhibitors (e.g., baricitinib and tofacitinib), or Bruton's tyrosine kinase inhibitors (e.g., ibrutinib and acalabrutinib). Additionally, other medications like immunosuppressants (e.g., corticosteroids, cyclosporine, and tacrolimus) or immunoglobulins were considered. Antibiotics, vasopressors, and low molecular weight heparin at prophylactic doses were also part of the treatment plan. For each medication, the dosage and duration of treatment were recorded.

The patient's initial condition upon starting remdesivir therapy was categorized into two groups: a) patients not needing extra oxygen and b) patients requiring low-flow oxygen. The patient's clinical status on days 7 and 14 after the first dose of remdesivir was documented as follows: a) discharged from the hospital and resumed normal activities; b) discharged from the hospital but with difficulties resuming normal activities; c) hospitalized without needing extra oxygen; d) hospitalized needing extra oxygen but not IMV; e) hospitalized needing IMV; and f) deceased.

6. Microbiological testing:

We considered the isolated microorganism in every case. Tests included tracheal aspirates, blood cultures, detection of influenza and/or coinfection, as well as tests for pneumococcal and legionella antigens in urine.

7. Medical procedures during hospitalization:

The following procedures were included:

-

Oxygen therapy

-

Non-invasive ventilation (NIV)

-

IMV

-

Prone ventilation

-

Hemodialysis/hemofiltration

-

Extracorporeal membrane oxygenation

8. Patient final outcome:

Complications during hospitalization were noted, including ARDS, sepsis, septic shock, nosocomial pneumonia (non-COVID-19), other nosocomial infections (non-COVID-19, non-pneumonia), acute renal failure, and acute hepatic failure.

The clinical benefit consisted of symptom improvement (fever, cough, etc.), along with improved radiological findings and/or a PAFI index ≥ 300 mmHg or SpO₂ > 93 without oxygen support within the first 5, 14, or 28 days, depending on the length of hospitalization.

Time taken to clear SARS-CoV-2 was considered, and outcomes were divided into hospital discharge or mortality. Collected data included length of hospital stay and date of ICU discharge. Any rehospitalizations within 7 days post-discharge were also documented.

To evaluate remdesivir effectiveness, information on overall in-hospital mortality and duration of hospitalization was recorded. Length of hospital stay was defined as the period from admission to death or discharge.

2.4. Method

2.4.1. Model Development

In this study, the eXtreme Gradient Boost (XGB) method was used as the reference algorithm, thanks to its notable features, such as fast execution, scalability, and high processing capability through parallel computing [21,22]. XGB consistently outperforms other algorithms in accurately solving various data science problems [22,23,24]. Additionally, a comparative analysis was conducted with other supervised ML systems.

Considering a dataset S=xj, yj, the XGB model was formulated using the following:

$$\widehat{y_j}=\sum _{p=1}^P{t}_p\left(x_j\right)$$

(1)

where x_j represents the input vector with m time variables, $\widehat{y_j}$ denotes the predicted output, y_j shows the output, t_p represents a tree with leaf weight w_p and structure u_p, j = 1; 2; ...; n, and P corresponds to the number of trees.

The formulated objective function for the proposed method is expressed in Equation (2). Employing a second-order Taylor expansion is integral to improving prediction accuracy in approximating the XGB objective function [21].

$$R=\sum _{j}r\left(\widehat{y_j},y_j\right)+\sum _p\Psi \left(t_p\right)$$

(2)

In Equation (3), f_p represents the number of leaves on the tree. The R () function penalizes the complexity of the method. The learning rate is denoted by λ and w_p is the vector of leaf scores. To control the complexity weight of the system, a parameter γ is employed. The aim is to optimize Equation (2) [22].

$$\Psi \left(t_p\right)=\lambda f_p+\frac{1}{2}\gamma {‖{\omega }_p‖}^2$$

(3)

In this study, we tested several ML algorithms to evaluate the performance of our proposed method. We chose the top-performing five algorithms from those widely recognized in the scientific community. These include Decision Trees (DT) [25], Gaussian Naive Bayes (GNB) [26], Bayesian Linear Discriminant Analysis (BLDA) [27], K-Nearest Neighbors (KNN) [28], and Support Vector Machines (SVM) [29]. We built the models using the MatLab Statistical and Machine Learning Toolbox (MatLab 2022a; The MathWorks, Natick, Massachusetts, USA). The dataset was split into two parts, with 70% used for training and the remaining 30% for testing, ensuring that patient information was not shared between the sets. To validate the results and prevent overfitting, we conducted 5-fold cross-validation.

Optimizing the ML algorithms involves adjusting various hyperparameters during the training phase. Bayesian techniques were employed in this study to determine optimal hyperparameter values. This optimization method significantly improves the outcomes of the developed methods.

Throughout all simulations, 100 iterations were executed to derive mean and standard deviation values in a uniformly random manner. This systematic approach mitigates the impact of noise, facilitating the calculation of relevant values and ensuring the attainment of statistically valid results [30]. The procedural phases employed in this study are delineated in Figure 1. Initially, subjects for study were selected, followed by the implementation of the database, and subsequent training and validation of ML methods.

2.4.2. Performance Evaluation

In this study, various methods were compared using the following metrics: specificity, precision (positive predictive value), recall (sensitivity), balanced accuracy, degenerate Youden index (DYI), F1-score, Matthew’s correlation coefficient (MCC), Cohen’s Kappa index (CKI), receiver operating characteristic (ROC), and area under the curve (AUC) [30]. The F1 score is defined as:

(4)

MCC was additionally employed to evaluate the performance of the ML methods, and it is defined as:

(5)

where TP denotes the number of true positives, TN represents the number of true negatives, FP is the number of false positives, and FN corresponds to the number of false negatives. CKI was used to assess the overall performance of the system [31].

3. Results

In this section, we discuss the results obtained from patient records used for training and validation to identify predictors of increased in-hospital mortality and hospital stay in COVID-19 patients treated with remdesivir. We compare the performance of our proposed system with various supervised ML classification methods widely accepted in the scientific community.

Table 1 presents the performance outcomes regarding mortality of various classification methods, including DT, GNB, BLDA, KNN, SVM, and our novel XGB system. It is noteworthy that GNB and BLDA-based approaches demonstrate a relatively lower balanced accuracy, falling short of the 82% benchmark. Conversely, DT and SVM techniques exhibit superior classification prowess, nearing a balanced accuracy of 90%, surpassing the effectiveness of GNB and BLDA. In contrast, the KNN method is the one that achieves a result closest to the proposed XGB method resulting in improved predictive capabilities. Particularly, the XGB system achieves an outstanding score surpassing 95%, showcasing remarkable performance in classification tasks.

Table 2 presents the performance results for the hospital stay variable using the same classification methods. As can be observed, the outcomes are similar to those of the mortality variable, with XGB achieving a balanced accuracy exceeding 94%.

KNN and DT stand out as the algorithms that come closest to XGB in terms of precision and recall values, surpassing SVM and notably outperforming BLDA and GNB in results. Moreover, this pattern is reflected in Table 1 and Table 2 when considering the F1 score parameter, where XGB achieves high values, serving as a clear indicator of improved classification performance.

To assess the performance of the proposed XGB system in classifying COVID-19 patients treated with remdesivir with a higher risk of mortality and/or extended hospital stay, we calculated several widely used parameters from the literature, including AUC, MCC, DYI, and the kappa index. Among these, MCC stands out as one of the most reliable statistical indices, as it yields a high score only when predictions have been accurate across all four categories of the confusion matrix.

The results of these four categories (true positives, false negatives, true negatives, and false positives) are directly influenced by the size of positive and negative elements within the dataset. As shown in Table 1 and Table 2, the XGB method achieved MCC values of 84.70% and 83.62%, respectively, clearly surpassing the values obtained by other methods.

In the same vein, when analyzing the kappa index, XGB reached nearly 85% for the final variable of mortality, marking a significant advancement over KNN and DT by 5.4% and 8.2%, respectively. As for the final variable of hospital stay, XGB approached about 84%. This trend persists when considering the AUC and DYI parameters, where XGB surpasses the rest of the methods with higher values. These findings emphasize the superior ability of XGB to accurately categorize COVID-19 patients treated with remdesivir, in terms of mortality and/or hospital stay.

Figure 2 and Figure 3 provide a comprehensive analysis, comparing the performance of the XGB method with alternative classifiers across a variety of essential metrics, for both mortality and hospital stay, respectively. These metrics include balanced accuracy, recall, specificity, precision, F1 score, CKI, MCC, AUC, and DYI.

Furthermore, the ROC curve is a crucial tool for assessing and comparing how effectively the proposed system classifies compared to other machine learning methods. It is constructed by plotting sensitivity against specificity across various threshold values. Figure 4 depict the outcomes of different classification systems, aligning with the primary goal of categorizing patients in the study.

In particular, the XGB method exhibits a significantly larger area under the ROC curve, indicating its enhanced ability to accurately classify the two distinct classes, both in terms of mortality and hospital stay. This is further supported by the specific numerical values provided in Table 1 and Table 2.

For better clarity, we've arranged all metrics for each dataset—both training and validation—and represented them in a radar plot (Figure 5 and Figure 6). In an ideal situation, where the model performs exceptionally well in all metrics, the plot would create a circle covering the entire grid. In our study, the training sets consistently show higher scores across all metrics, whereas the validation sets generally display lower scores.

The radar plots give us a quick look at the model's performance. A bigger circle on the validation set suggests a better predictive method. Figure 5 shows that our suggested XGB system is a good example of a well-balanced model. Importantly, both the training and validation sets have similar radar plots, indicating there's neither overfitting nor underfitting. This boosts the model's ability to work well with new inputs, delivering accurate results effectively.

On the flip side, the GNB method consistently comes in as the least effective performer in all metrics. Based on these findings, we can confidently state that our proposed XGB system excels at categorizing patients in line with the study's goals. It provides high accuracy and automation, making it a valuable tool for clinical practice.

With the proposed XGB method, the predictive factors linked to a poorer outcome in COVID-19 patients treated with remdesivir, in terms of mortality, include: limitation of life support treatment, need for ventilatory support (especially IMV) on day 14 after the first dose of remdesivir, lymphopenia, low levels of albumin and hemoglobin, presence of flu and/or coinfection, and cough. Factors associated with a worse outcome of remdesivir use in terms of hospital stay include: number of doses of the COVID-19 vaccine, patchy lung density, bilateral pulmonary radiological status, number of comorbidities, oxygen therapy, troponin and LDH levels, and asthenia. Figure 7 depicts a bar graph illustrating the weights of predictive variables that significantly enhance the classification accuracy of various ML methods.

The main baseline clinical data of the 252 patients included in the study are presented in Table 3.

4. Discussion

Among the approved medications for COVID-19, remdesivir stands out as the preferred antiviral treatment for hospitalized patients infected with SARS-CoV-2. Additionally, other drugs like tocilizumab and baricitinib show promise, particularly for severe cases, including those requiring IMV [32]. While remdesivir can reduce viral levels and shorten symptom duration, identifying the patients who benefit most from it remains uncertain. This knowledge is crucial for minimizing unnecessary side effects and costs, while optimizing resource use [33].

The European Medicines Agency (EMA) initially granted conditional approval for marketing across the EU on July 3, 2020, which later transitioned to full marketing authorization on August 8, 2022. This medication is approved for use in adults and children as young as 4 weeks old, weighing at least 3 kg, who have pneumonia and need supplemental oxygen (either low- or high-flow oxygen or other non-invasive ventilation at the start of treatment). Additionally, it can be used in adults and children weighing at least 40 kg who do not need supplemental oxygen but are at higher risk of developing severe COVID-19 [34].

On the other hand, AI has been employed to identify genomic sequences of SARS-CoV-2, including antigenic variants, as well as to develop drugs and vaccines for COVID-19 [35]. It has also been used to discover drug combinations against COVID-19 [36].

Drug repurposing trials have aimed at finding potential treatments for COVID-19, including antiviral therapies, anti-inflammatory drugs, antithrombotic agents, and immunomodulators [37]. In a study by Basit SA et al., a deep learning model was employed to predict the effectiveness of different medications, identifying remdesivir as highly effective against COVID-19 with a 95% positive score [38].

Understanding the physiopathology of COVID-19 can help scientists develop effective antiviral drugs by uncovering unknown viral pathways and structures. With the advancements in AI and ML, it's reasonable to use these methods to explore new candidates. Various studies, like the one by Imtiaz F and Pasha MK, have focused on examining the structure of the RNA-dependent RNA polymerase (RdRp) using ML techniques. RdRp is crucial for virus replication and holds potential as a promising target for COVID-19 treatment [39]. Remdesivir, which transforms into an analog of adenosine triphosphate during intracellular metabolism, works mainly by integrating into the developing RNA chain by the RdRp. This disrupts viral replication, a key aspect of its antiviral activity [40]. Monitoring the impact of emerging mutations on viral replication and response to antiviral drugs is essential. Remdesivir shows stability as an RdRp inhibitor compared to other antivirals in the presence of mutations at this level of viral replication [41].

The WHO's Solidarity trial was the first major study to show the limited clinical effectiveness, in terms of mortality, of three repurposed antivirals in hospitalized COVID-19 patients: lopinavir, hydroxychloroquine, and interferon (IFN)-β1a. The remdesivir arm continued, with 4,146 patients receiving remdesivir and 4,129 assigned to the control group. It was observed that remdesivir does not have a significant effect on COVID-19 patients already on ventilation. Among other hospitalized patients, it has a minor impact on mortality or progression to ventilation [42]. Remdesivir may be beneficial in the clinical course for both hospitalized and non-hospitalized patients, but certainty remains limited [43]. There is evidence supporting the clinical benefit of a 5-day regimen of remdesivir in patients with moderate COVID-19 infection (lung infiltrates and SpO₂ in ambient air > 94%)[44]. For patients with severe COVID-19 (SpO₂ ≤ 94% while breathing ambient air and radiological evidence of pneumonia) who do not require mechanical ventilation, there doesn't seem to be a significant difference between a 5-day and a 10-day course of remdesivir [45]. Remdesivir significantly reduces hospitalization days and lowers inflammatory markers compared to standard treatment in patients with moderate to severe COVID-19 [46]. Compared to standard care, remdesivir quickly improves low oxygen levels (reducing the need for ventilatory support) and reduces inflammation (lowering IL-6 levels), leading to a better course of moderate to severe COVID-19 [47]. Patients treated with remdesivir spend less time in the ICU and have better survival rates [48].

Recent meta-analyses confirm that the use of remdesivir can help reduce mortality in COVID-19 patients and shorten the time to clinical improvement [32,49]. Observational studies have demonstrated benefits in hospital mortality with remdesivir therapy [50,51].

Remdesivir appears to lower mortality rates in hospitalized COVID-19 patients who don't require oxygen support or only need standard oxygen therapy. However, it doesn't seem to help patients on mechanical ventilation [52]. A recent meta-analysis by Huang C et al. found that hospitalized adult COVID-19 patients who didn't need extra oxygen or only required low-flow oxygen and were treated with remdesivir had a lower risk of death. However, those needing high-flow oxygen or IMV didn't see the same benefit [53]. Remdesivir also seems to speed up recovery, reduce complications, and might slightly decrease the need for ventilation [32,54,55]. Additionally, a 5-day treatment course appears to provide more benefits with fewer side effects and lower costs for non-ventilated patients compared to a 10-day course [54].

Remdesivir is safe to use, but when combined with corticosteroids, it doesn't seem to offer extra clinical benefits [49]. However, when paired with baricitinib, it's not only safe but also seems to be more effective than using remdesivir alone. This combination can reduce recovery time and speed up clinical improvement in COVID-19 patients, particularly those needing high-flow oxygen or NIV [56].

This study leads the way in developing, comparing, and evaluating six supervised ML methods to predict factors that reduce the effectiveness of remdesivir in hospitalized patients with SARS-CoV-2 pneumonia. We collected data on 133 demographic, clinical, and laboratory variables. Among the ML algorithms tested, XGB stood out as the best performer, achieving the highest balanced accuracy rates for predicting mortality (95.4%) and hospital stay duration (94.2%).

The invasion of respiratory tract cells by SARS-CoV-2 causes cellular damage and triggers a local immune response, leading to the release of proinflammatory substances like IFNγ, IL-1β, IL-6, and TNF-α [57]. IL-6 plays a crucial role in transitioning from mild inflammation to severe hyperinflammatory states, such as cytokine release syndrome (CRS) and ARDS, which can result in high mortality for severely ill COVID-19 patients. ARDS develops due to an unbalanced immune response, intense inflammation, and disruption of blood clotting, with CRS being central to this intricate relationship [58]. This syndrome is the leading cause of death among these patients, with mortality rates exceeding 70% [8]. Therefore, treatment strategies for hospitalized COVID-19 patients mainly focus on regulating the immune response and achieving antiviral, antithrombotic, or anticoagulant effects [8].

In our investigation, we observed a hospital mortality rate of 13.5%, with 3.9% of all patients requiring IMV. These findings align with those of other studies [42,52,59]. The median duration of hospitalization following the administration of remdesivir was 8 days (Interquartile Range, IQR, 5–12).

Our analysis involved testing various ML classifiers, among which the XGB method stood out as the most precise in identifying patients at higher risk of mortality and/or prolonged hospital stay. After thorough examination, we compared the XGB model with several other supervised ML methods commonly found in existing literature, such as BLDA, GNB, DT, KNN, and SVM. It's important to note that in biomedical scenarios, current ML classification techniques consistently outperform unsupervised methods, achieving higher average accuracy rates for both regression and classification tasks [60]. In our study, BLDA and GNB performed the poorest among the methods examined, while KNN's performance closely matched that of XGB. These findings align with previous research on the predictive capabilities of supervised ML algorithms for COVID-19 mortality and hospitalization durations [61,62].

In our study, we use a radar graph to evaluate how well ML models perform during both training and testing. The results clearly show that the XGB model excels, particularly in handling large datasets without overfitting. It outperforms other methods by achieving higher precision, recall, and overall accuracy. The reliable performance of the XGB model makes it incredibly valuable, especially in biomedical applications like predicting cancer stages for patients [63].

In our cohort, 58.3% were male, with a median age of 77 (IQR 66.7–85.2). We found several factors linked to negative outcomes after receiving remdesivir, notably regarding mortality and hospital stay. For mortality, adverse outcomes were associated with life support limitations, the need for ventilatory support (particularly IMV) 14 days after the initial remdesivir dose, lymphopenia, low albumin and hemoglobin levels, flu and/or coinfection, and cough. When it comes to hospital stay, factors associated with a worse outcome with remdesivir use included the COVID-19 vaccine doses, patchy lung density, bilateral pulmonary radiological status, comorbidity count, need for oxygen therapy, high levels of troponin and LDH, and presence of asthenia.

These identified factors provide valuable insights into the potential determinants of adverse outcomes associated with remdesivir use. In the realm of research on factors influencing the response to remdesivir in hospitalized patients with COVID-19, few studies have been conducted [64,65,66].

Previous studies have identified factors that predict the severity and mortality of COVID-19 patients, but these were conducted before standard treatment with remdesivir was introduced. ML techniques have confirmed that demographic factors (like age), clinical factors (such as comorbidities or symptoms), and analytical factors are associated with the severity, mortality, and length of hospital stay of COVID-19 patients, regardless of the specific treatment used [67,68]. Adamidi et al.'s systematic review also found predictors of disease progression and mortality using ML techniques, similar to our study. They emphasized that age, PCR and LDH levels, lymphopenia, and chest X-ray and CT scan findings are commonly linked to adverse outcomes in COVID-19 patients [62]. Additionally, other studies have highlighted the effectiveness of the XGB method in predicting adverse outcomes in hospitalized COVID-19 patients [69].

In Choi YJ et al.'s study, multivariate analysis confirmed that a high National Early Warning Score (NEWS) and Charlson Comorbidity Index (CCI) at admission, along with dyspnea, were independent risk factors for 30-day mortality in COVID-19 pneumonia patients treated with remdesivir and dexamethasone [64]. NEWS is based on a scoring system that combines six key physiological measurements (respiratory rate, SpO2, temperature, systolic blood pressure, heart rate, and level of consciousness) upon hospital admission or monitoring. This tool is highly sensitive and specific in predicting early mortality in prehospital and emergency department settings [70]. It emphasizes that COVID-19 admissions carry a significantly higher mortality risk compared to non-COVID-19 admissions, highlighting the elevated baseline mortality risk associated with COVID-19 [71].

In our study, factors indicating a higher risk of mortality, similar to NEWS and shortness of breath, included the limitation of life support treatment and the need for ventilatory support, especially IMV, 14 days after the initial dose of remdesivir. Regarding hospital stay, a similar factor was the requirement for oxygen therapy. These respiratory parameters and symptoms indicate respiratory failure and suggest a more severe decline in hospitalized COVID-19 patients.

Several studies confirm that the main clinical symptoms in patients admitted with COVID-19 are shortness of breath, cough, and fever [70]. In our study, cough was associated with a higher risk of mortality, while asthenia was linked to prolonged hospital stay.

In the study conducted by Choi YJ et al., similar to our findings, the majority of hospitalized cases of COVID-19 pneumonia occurred in adults aged ≥ 65 years; thus, there were no significant differences in the prognosis based on age. Additionally, within the laboratory parameters, low lymphocyte count, high levels of CRP, and elevated LDH indicated an unfavorable prognosis [64]. In our case, lymphopenia was one of the factors strongly associated with mortality in patients treated with remdesivir, along with low levels of albumin and hemoglobin. Lymphopenia signals an imbalanced T-cell response and weakened adaptive immunity. SARS-CoV-2 infection predominantly affects T lymphocytes, specifically CD4+ and CD8+ T cells, leading to reduced counts. Lymphopenia is more frequently observed in severe cases [72]. Georgakopoulou VE et al., confirmed that low levels of albumin and the C-reactive protein to albumin ratio were predictors of mortality, similar to our findings [66]. For hospital stay, elevated troponin and LDH levels were the most relevant predictors, along with observing bilateral lung involvement and patchy lung density in imaging tests. In the study by Terkes V et al., advanced age, elevated CRP, and the Computed Tomography (CT) score were identified as significant predictors of disease outcome [73]. However, the intense inflammatory response triggered by the infection can lead to alterations in hemostasis and coagulation parameters [74]. In our research, platelet count was not a significant predictor to influence the sought-after final outcome.

On the other hand, comorbidities themselves result from inflammation and can induce a proinflammatory state. The CCI is a simple, easy-to-apply, and valid method for classifying comorbidities and estimating mortality from COVID-19 [75].

Recent studies have taken into account the pre-existing health conditions (comorbidities) of patients infected with SARS-CoV-2 and their association with the progression of the disease in terms of mortality and hospital stay [65].

Aging and pre-existing health conditions can create a state of meta-inflammation, amplifying inflammation in COVID-19 and increasing the risk of mortality. Several studies demonstrate a correlation in patients affected by COVID-19 between age, number of comorbidities and certain laboratory markers [76]. There appears to be a positive link between inflammation biomarkers such as CRP, ferritin, and LDH, and the number of comorbidities in COVID-19 patients. The same pattern is observed in hematological parameters like the neutrophil-to-lymphocyte ratio. Similar to these studies, in our research, the number of comorbidities were associated with worse outcome for the patients and extended hospital stay.

Chronic kidney disease, the incidence of acute kidney injury, and atrial fibrillation have been shown to be comorbidities associated with reduced survival in patients hospitalized for COVID-19 [77]. Other studies confirm that lymphopenia, often observed in cancer patients, is associated with a higher risk of mortality [78]. Hematological disorders are also seen as additional health conditions that can negatively impact the effectiveness of antiviral treatments such as remdesivir [79]. In other studies, hypertension and type 2 diabetes, linked with obesity as metabolic syndrome, are considered significant risk factors for adverse outcomes [80]. In our study, the presence of influenza and/or co-infection constituted the comorbidity associated with higher mortality.

In the recent meta-analysis by Amstutz A et al., neither age, comorbidities, nor the use of corticosteroids had an impact on the effectiveness of remdesivir in terms of mortality [52]. Similarly, the need for increased respiratory support in patients has shown limited effectiveness of remdesivir in reducing mortality, aligning with our own research outcomes [17].

In our investigation, 22.2% of patients developed ARDS according to the latest definition [81]. ARDS is a clinical syndrome of acute hypoxemic respiratory failure due to lung inflammation, not caused by cardiogenic pulmonary edema. Various studies using these techniques have found that ARDS is associated with fatal outcomes in COVID-19 patients making ventilatory support essential, including IMV. [62,68]. In our study, both IMV and oxygen therapy were predictors of poor prognosis.

In contrast to Choi YJ et al.'s study, in ours, a lower number of COVID-19 vaccine doses was linked to clinical deterioration and a longer hospital stay [64]. The study by Georgakopoulou VE et al. confirmed that, regardless of vaccination status, pre-existing comorbidities, age, and gender, patients with a combination of biomarkers indicating acute inflammatory response, cell death, and hypercoagulability – specifically, CRP, LDH, and fibrinogen – reflected the severity of COVID-19 [82]. More recently, Mikulska M et al. demonstrated that receiving fewer doses of the COVID-19 vaccine was a predictive factor for treatment failure with antivirals such as remdesivir [79]. Several studies consistently show that individuals who are not vaccinated or receive fewer vaccine doses are more likely to experience negative outcomes, including the need for mechanical ventilation or death during hospitalization [83]. This aligns with our findings.

In the study by Shimizu H et al., the time intervals between symptom onset, diagnosis, and antiviral treatment were significant predictors of moderate illness [84]. In our study, following the applied protocol, remdesivir treatment was supposed to commence upon microbiological confirmation of SARS-CoV-2 infection and within 7 days of clinical symptom onset.

None of the drug treatments, including early antibiotic use, had a significant effect on the final outcome, similar to findings in other studies [64].

Most current studies using ML techniques confirm that respiratory parameters like SpO₂ and the need for invasive ventilatory support are considered the most important predictors for mortality in hospitalized COVID-19 patients treated with remdesivir. While hypertension and worsening renal function are also considered mortality predictors in these studies, they didn't hold enough significance in our research [85,86]. Kuno T et al. developed a predictive model for in-hospital mortality using ML methods in COVID-19 patients treated with steroids and remdesivir. Variables associated with mortality included age, hypertension, SpO₂, blood urea nitrogen, ICU admission, and endotracheal intubation [85].

On the other hand, evidence indicates the presence of different COVID-19 patient phenotypes showing diverse inflammatory and immune responses, mortality risks, and treatment outcomes [87,88]. In a study by Chen H et al., two COVID-19 phenotypes emerged: hypo-inflammatory and hyper-inflammatory, with the latter marked by heightened pro-inflammatory cytokine levels and increased complication rates. Corticosteroid therapy was linked to lower 28-day mortality (HR, 0.45; 95% CI, 0.25-0.80; p = 0.0062) in the hyper-inflammatory type [88]. The lack of efficacy with remdesivir treatment may lend support to this idea, underscoring the importance of identifying factors for personalized treatments.

Our study has both limitations and strengths. The main limitations come from its retrospective, single-center design and limited sample size. However, these drawbacks are balanced by the use of powerful methodological tools like ML. ML methods have the advantage of being effective even with small datasets, resulting in simple and fast classification for our proposed method. We've also used data augmentation techniques to enhance our analysis [30].

One strength of our study is its inclusion of a diverse patient population, covering individuals with common health conditions often overlooked in standard clinical trials. This wider representation makes our findings applicable to a broader range of patients. Our methodology effectively identifies patients who could benefit from remdesivir, potentially leading to better survival rates and shorter hospital stays. Additionally, similar research using ML techniques has identified factors linked to worse outcomes in severe COVID-19 patients treated with tocilizumab [89]. Comparative studies suggest that ML methods may offer greater accuracy and efficiency compared to traditional logistic regression analysis, particularly with limited sample sizes.

The XGB method is a simple binary classification system that is easy to use and train. As more data is gathered, this algorithm gets better at making predictions.

5. Conclusions

Remdesivir has proven to be beneficial in patients with SARS-CoV-2 pneumonia, especially in those without critical illness criteria. However, a significant number of patients still die or require longer hospital stays despite treatment with remdesivir. Therefore, we utilized ML techniques, which are increasingly important in predicting important events. Among the six supervised ML methods we validated, XGB achieved the highest accuracy in predicting factors associated with worse outcomes, such as mortality and hospital stay length, in hospitalized COVID-19 patients treated with remdesivir. This tool can assist healthcare professionals in making early and effective clinical decisions to optimize remdesivir treatment for COVID-19 patients who meet specific clinical criteria.