Evaluation of Sirs, Mews, and qSOFA Scores to Predict ICU Admission and 14- and 30-Day Mortality in Patients with Hematological Malignancies and Sepsis or Septic Shock

1. Introduction

Sepsis and septic shock result from exacerbated systemic inflammatory response secondary to an infectious process and is an event associated with high rates of morbidity and mortality. Early recognition followed by immediate appropriate therapy reduces unfavorable outcomes which is a global health priority [1,2,3]. The definition of sepsis has changed over the past few decades, (sepsis-1 from 1991 and sepsis-2 from 2001), with the third sepsis diagnostic consensus (sepsis-3) being published in 2016, which focused mainly on identifying cases with greater severity and thus higher mortality risk. The SRIS score was part of the initial screening of sepsis cases (sepsis-1 and sepsis-2) is no longer used. The quick Sequential Organ Failure Assement Score (qSOFA) was introduced as a new tool in screening patients with suspected sepsis. And organ dysfunction, the main marker of life-threatening dysfunction secondary to host response to infection, was now characterized by a 2-point increase in the Sequential Organ Failue Assement (SOFA) score [4].

However, not all medical societies support this definition. In 2017, the Surviving Sepsis Campaign (SSC) adopted the lato senso definition of Sepsis-3, characterizing sepsis as the presence of life-threatening dysfunction due to the presence of dysregulated response to infection. However, the clinical criteria for the definition of organ dysfunction of Sepsis 3 (variation of the SOFA score) were not adopted, keeping the criteria used previously, including hyperlactatemia as one of them. Septic shock was defined by the presence of hypotension unresponsive to fluid use, regardless of lactate values [5,6]. And in 2021, in a new SSC guideline it was contraindicated to use qSOFA as a single screening tool for sepsis or septic shock compared to SIRS, NEWS or MEWS [7].

Patients with hematological malignancies are at high risk to develop sepsis and septic shock due to risk factors such as severe neutropenia (neutrophils ≤ 100 cells/mm3), mucositis, neurological changes among others [8]. Another important point is that fever may be the only clinical sign in the presentation of the infectious picture in view of immunosuppression. Therefore, early identification of septic patients and those at higher risk of mortality will impact their outcomes.

In the service where the study was conducted, since 2016 the MEWS score has been adopted to triage patients with suspected sepsis. The MEWS is a previously validated tool as a simple and user-friendly bedside early detection score for acute clinical deterioration. It is based on the monitoring of physiological parameters such as systemic blood pressure (SBP), heart rate (HR), respiratory rate (RR), axillary temperature, neurological level, and urine output.

Thus, this study seeks to evaluate the prognostic accuracy of qSOFA, SOFA, SIRS and MEWS to predict in-hospital mortality in patients with hematological malignancies and a diagnosis of sepsis or septic shock.

2. Materials and Methods

2.1. Study Design and Setting

Retrospective cohort study that consecutively included patients with hematological malignancies and a diagnosis of sepsis or septic shock, in the period between January 1, 2017 and August 31, 2020. It was performed in a tertiary hospital specialized in patient with hematological malignancies care, in the city of Sao Paulo, Brazil. Data were extracted from electronic medical records. Patients in palliative care and under 18 years of age were excluded from the study.

2.2. Measures and Outcomes

Demographic data (i.e. age, sex), vital parameters (i.e. blood pressure, body temperature, respiratory rate, heart rate, peripheral oxygen saturation, level of consciousness according to Glasgow scale, protocol data (lactate, blood culture, time to antimicrobial therapy, volume, vasopressor, lactate clearance, reevaluation of volemia and perfusion, mortality, organ dysfunction time, time to lactate clearance, adequate antimicrobial), in addition to data regarding the protocol, underlying diagnosis, diagnosis of sepsis or septic shock, neutropenia, chemotherapy, infectious focus, identified microorganisms; antimicrobials used and adequacy according to culture; origin (community or hospital); place of opening of the sepsis protocol (emergency room, ward, intensive care unit, and others).

2.3. Scoring Systems

SIRS criteria were defined as heart rate >90 beats per minute, respiratory rate >20 breaths per minute, temperature <36°C or >38°C, and WBC count <4000/mm3 or >12000/mm3. A positive score is defined as ≥2. The qSOFA criteria are systolic blood pressure ≤100mmHg, respiratory rate ≥22 breaths per minute, and Glasgow Coma Scale score <15. A positive score was defined as ≥2. The MEWS score ranges from 0 to 20 and is based on respiratory rate, temperature, systolic blood pressure, pulse rate and level of consciousness and volume of diuresis. It is used in all sectors of the study hospital, except the ICU, for early detection of acute deterioration and opening of the institutional Sepsis Protocol was associated with a score ≥4, on suspicion or confirmation of infection.

2.4. Statistical Analysis

The sensitivity, specificity, area under the ROC curve (AUROC), positive predictive value (PPV), negative predictive value (NPV), and positive and negative likelihood ratio for predicting 14- and 30-day hospital mortality. In the primary analysis, the predefined positive score thresholds for each scoring system were used. Exploratory analyses then determined the optimal cut-off point for each scoring system, based on the threshold with the highest Youden index. The sensitivity and specificity of the different scoring systems for the primary outcome were compared by McNemar's test and AUROC by DeLong's method. Starting with the conventional significance threshold of P=0.05, applying Bonferroni correction for multiple testing (0.05/9) gave a significance threshold of 0.006. The analyses were performed in Stata V.15.0 (Stata, College Station, Texas, USA).3.

3. Results

3.1. Characteristics of Patients

During the study period, 775 patients had sepsis criteria. Of these, 329 (42.5%) had an underlying diagnosis of hematological malignancies. The mean age of the population was 58.6 years, and males were the most prevalent (49.8%). Among the baseline diagnoses, lymphoma was the most frequent (24.3%), followed by multiple myeloma (19.8%) and others (19.8%).

Among the included patients, the diagnosis of sepsis occurred in 241 (73.3%) and septic shock in 88 (26.7%) of the patients. The main site of infection were pulmonary in 171 (52.0%), intra-abdominal in 58 (17.6%), central venous catheter-associated bloodstream (CLABSI) 25 (7.6%), and urinary tract 16 (4.9%). One hundred and fifty-four (49.5%) patients required ICU admission. The 14-day mortality rate was 27% (n = 89), the 30-day mortality rate was 38.3% (n = 126). All characteristics of patients are described in Table 1.

3.2. Prediction of 14-Day and 30-Day Mortality, and ICU Admission

The AUROCs for the prediction of 14-day mortality were SIRS=0.526 (95%CI 0.469-0.583), MEWS=0.585 (95%CI 0.528-0.640), and qSOFA=0.634 (95%CI 0.578-0.988) (Figure 1). For predicting 14-day mortality, qSOFA outperformed SIRS (p=0.01), but the same was not observed in relation to the MEWS score (p=0.13). For predicting 30-day mortality, qSOFA had higher AUROC (0.632, 95%CI 0.576-0.686), but no statistically significant difference compared to MEWS (0.601, 95%CI 0.544-0.656, p=0.49) and SIRS (0.552, 95%CI 0.495-0.608, p =0.08). qSOFA (0.713, 95%CI 0.659-0.763) outperformed SIRS (0.571, 95%CI 0.514-0.626, p<0.01), and MEWS (0.629, 95%CI 0.573-0.683, p=0.02) for predicting ICU admission. MEWS had a larger area than SIRS for the analyzed outcomes, but without statistical difference, as shown in Figure 1 and Table 2.

3.3. Exploratory Analysis Using the Youden Index

Using Youden's J statistic, qSOFA≥2 and SIRS≥2 were the optimal cut-off points for predicting 14- and 30-day mortality and ICU admission. The sensitivity SIRS≥2 for 14- and 30-day mortality and intensive care unit admission was 91.01% (83.1-96.0%), 86.51% (79.3-91.9%), and 90.64% (85.3-94.6%), respectively. However, the specificity found was low for all three outcomes, as shown in Table 2. The qSOFA≥2 score showed lower sensitivity but higher specificity for 14-day mortality (67.1%, 60.7-73.0%), 30-day mortality (69.5%, 62.6-75.7%), and ICU admission (77.2%, 69.9-83.5%).

The MEWS≥4 score, used at the institution for opening the sepsis protocol, showed sensitivity for 14-day mortality of 84.3% (75.0-91.1%), 30-day mortality of 80.2% (72.1-86.7%), and ICU admission of 80.1% (73.3-85.8%), but with low specificity, 47.1% (40.6-53.6%), 50.3% (43.2-57.3%), and 58.9% (50.8-66.6%), respectively. According to Youden's J index, the best cutoff to predict the three outcomes analyzed in the study would be scores ≥6. With this new cutoff, the sensitivity of the MEWS for 14-day (43.8%, CI95% 33.3-54.7%), and 30-day (40.5%, CI95% 31.8-49.6%) mortality, and ICU admission (41.5, CI95% 34.0-49.3%) was lower, with higher specificity, 71.3% (65.1-76.9%), 71.9% (65.2-78.0%), and 76.6 (69.2-82.9%), respectively.

4. Discussion

In our study, we observed that no score had high sensitivity and specificity for predicting ICU admission and 14- and 30-day mortality in patients with hematological malignancies and sepsis. Regarding AUROC, we did not observe a statistically significant difference between MEWS and qSOFA in predicting 14-day and 30-day mortality, although qSOFA had a higher AUROC among the three scores. MEWS showed similar sensitivity to SIRS for predicting the outcomes evaluated in the study, while we observed lower sensitivity for qSOFA.

Despite changes in the definition of sepsis and recommendation of the use of qSOFA for triage of suspected patients in the publication of SEPSIS-3 [4], Surviving Sepsis Campaign (2021) recommend against using qSOFA compared with SIRS, NEWS, or MEWS as a singlescreening tool for sepsis or septic shock. And its main limitation is the low sensitivity demonstrated in most studies, despite the high specified [7]. A meta-analysis that compared qSOFA, SIRS and the NEWS score found similar results to our study. qSOFA showed higher overall prognostic accuracy than SIRS and NEWS by comparing AUROC, but also with lower sensitivity [9]. Garbero (2019) also demonstrated that qSOFA did not perform well for predicting poor prognosis in the emergency department, showing 56.8% sensitivity for death and 41.4% for need for ICU [10]. We found sensitivity similar to that described in this study, demonstrating that qSOFA is not a good tool for sepsis screening also in patients with hematologic malignancies.

Studies demonstrated that early warning scores performed better than qSOFA and SIRS for sepsis in predicting adverse outcomes associated with sepsis [11]. One justification is that these scores have weighted score points and incorporate more physiological components. Like others early warning scores, MEWS is based on clinical parameters that are easily accessed at the bedside. Despite this, MEWS has been poorly evaluated in literature in relation to sepsis outcomes. A study that evaluated 5 early deterioration scores frequently used in healthcare services showed similar performance in patients with suspected infection of the MEWS to NEWS, a more widely studied score that has already demonstrated better performance relative to qSOFA and SIRS in the general population [15].

In the exploratory analysis of the best cut-off point, Youden's J-index demonstrated that MEWS score ≥6 would be the best score to predict the three outcomes analyzed in the study. However, we would observe important reduction in the sensitivity of the score. The qSOFA and SIRS maintained the same score, ≥2, according to Youden's J index.

The main limitation is that this is a retrospective study in a single tertiary center. Nevertheless, data regarding the scores were collected by the sepsis team promptly during review of the protocols opened at the institution, thus decreasing the bias of the chart review used to collect clinical data for the study. Although the study population was patients with hematological malignancies, it was not a homogeneous group of patients. These neoplasms are known to be directly related to the risk of infection and mortality associated with them, and may vary widely according to their status and stage of treatment. Therefore, this may influence the performance of these scores, making it even more difficult to identify a better one.

5. Conclusions

In conclusion, MEWS has a higher sensitivity than qSOFA and higher AUROC than SIRS for prediction of ICU admission and 14- and 30-day mortality in patients with hematological malignancies and sepsis, thus, appearing to be a better choice of score in the study population.