Comparative Analysis of the Impact of Severe Acute Respiratory Syndrome Coronavirus 2 Infection on the Performance of Clinical Decision‐Making Algorithms for Pulmonary Embolism

1. Introduction

Pulmonary embolism (PE) is an important complication of coronavirus disease 2019 (COVID-19) [1,2,3]. Strategies for diagnosing PE in the emergency department are based on several clinically validated algorithms developed to safely limit the use of radiation-based imaging modalities, specifically computed tomography pulmonary angiography (CTPA), which is considered the gold standard [4]. These clinical algorithms combine pretest probability with D-dimer results to classify the risk and guide the indication for CTPA. The Wells and Geneva scores, used in conjunction with D-dimer measurement, are widely used in PE diagnosis and are the preferred tools for assessing pretest clinical probability [5,6]. These scores determine the likelihood of PE upon assessment of symptoms, medical history, and risk factors as well as provide guidance for excluding PE or determining the need for further diagnostic testing when used in combination with D-dimer testing. New algorithms, including age-adjusted D-dimer (AADD), YEARS, and pulmonary embolism graduated D-dimer (PEGeD), have been introduced to safely limit the use of CTPA. [7,8,9]

Although it is well established that COVID-19 increases the risk of venous thromboembolism (VTE) in the general population, it is unclear whether this increase poses a higher risk in emergency department patients with suspected PE. Because PE and COVID-19 have similar signs and symptoms, it led to significant diagnostic challenges for emergency physicians during the COVID-19 pandemic [10]. The hyperinflammatory state and higher D-dimer levels in patients with COVID-19 made it difficult to rule out PE without CTPA, leading to increased CTPA requests [11,12,13,14]. Moreover, several studies have suggested that clinical prediction models used to determine the probability of PE in the general population may not be fully applicable to patients with COVID-19 [15,16,17].

In this study, we primarily aimed to comprehensively investigate the diagnostic performance of several clinical algorithms for PE diagnosis during the COVID-19 pandemic. Moreover, we aimed to compare the effectiveness of these algorithms in patient groups that tested positive and negative for COVID-19. We also analyzed the diagnostic performances of different D-dimer thresholds used in PE prediction.

2. Materials and Methods

The Ethics Committee of Health Sciences University Fatih Sultan Mehmet Training and Research Hospital approved this retrospective cohort study (Ethics Approval Protocol Number: 2023/112, Approval Date: 12.10.2023). This study included patients aged ≥18 years who presented to the emergency department of Fatih Sultan Mehmet Training and Research Hospital between May 01, 2020, and June 01, 2021, and underwent CTPA for suspected PE. We excluded patients with incomplete or inadequate medical records, those without D-dimer test, and those with inconclusive CTPA from the study. As per further exclusion criteria, we excluded patients with known pregnancy, those with genetic thrombotic disease, and those with therapeutic anticoagulant use for indications other than VTE or suspected VTE from the study.

We collected data on patient background, demographics, clinical findings, D-dimer levels, and risk factors for PE (immobilization or a history of surgery in the last month, a history of previous deep vein thrombosis (DVT) or PE, malignancy, hemoptysis, symptoms and findings of DVT, CTPA findings, and coronavirus disease 2019 (COVID-19) reverse transcription polymerase chain reaction (RT-PCR) test results. An investigator blinded to the CTPA reports collected demographic, clinical, and laboratory data.

2.1. Scores and Evaluation of Algorithm

In the present study, we retrospectively calculated the components of diagnostic prediction rules based on the clinical data records at the time of CTPA request. We excluded components for any undocumented score.

We identified the item “PE is the most likely diagnosis/alternate diagnosis is less likely” in the patient’s medical record based on the following criteria:

• The physician stated that PE was the most likely diagnosis in the medical record, or
• No other YEARS criteria were specified in the medical record, but a CTPA procedure was performed when the D-dimer level was <1000 μg/L within 24 h of D-dimer measurement.

We rated this subjective item as 0 if the criteria were not met [18].

The Wells score is 0–12.5 points and based on certain criteria, including signs and symptoms of DVT, PE being the most likely diagnosis, previously diagnosed PE/DVT, heart rate >100 beats/min, immobilization or recent surgery, malignancy, and hemoptysis [9]. In the present study, we categorized patients into low- (<4 points), moderate- (4.5–6 points), and high-risk (≥6.5 points) groups [19].

The revised Geneva score considers certain factors, including previously diagnosed PE/DVT, unilateral lower extremity pain, heart rate, active malignancy, hemoptysis, age >65 years, and pain on extremity palpation, when assessing the risk for PE [9]. Based on this score, we categorized patients into low- (0–3 points), moderate- (4–10 points), and high-risk (≥11 points) groups [19].

In the standard protocol, patients at high risk of PE as per the Wells or Geneva scores are evaluated with CTPA, and patients at low- and moderate-risk are evaluated with CTPA when D-dimer levels were above 500 ng/mL or above age-adjusted cutoff values. The AADD threshold was defined as 10 times the age in patients older than 50 years [20].

The YEARS algorithm assesses PE by D-dimer levels, considering hemoptysis, signs of DVT, and whether PE is the most likely diagnosis into consideration. PE was excluded in patients with D-dimer levels below 1000 ng/mL and in those who meet 0 YEARS criteria, whereas CTPA was required in patients, who meet one or multiple YEARS criteria and D-dimer levels above 500 ng/mL [7].

PEGeD algorithm excludes PE in patients with a low pretest probability and a D-dimer level below 1000 ng/mL or an intermediate pretest probability and a D-dimer level below 500 ng/mL; all other patients, including the high-risk group, are evaluated with CTPA [9]. In this algorithm, the pretest clinical probability is based on the Wells score.

2.2. COVID-19 Assessment

We evaluated the patients based on COVID-19 RT-PCR test results obtained within the last month from the date of imaging. We considered patients who tested positive for COVID-19 RT-PCR in the emergency department or had a positive test result within 30 days before admission as COVID-19-positive. We also considered patients with COVID-19-specific typical viral pneumonia findings on CTPA imaging, with clinical symptoms consistent with COVID-19 infection, and on COVID-19 treatment as COVID-19-positive. This method enabled a broader identification of patients with COVID-19 based on clinical and radiological findings, rather than relying solely on PCR results.

2.3. CTPA Protocol

CTPA was performed by administering intravenous 50–55 mL injection of iodinated contrast medium followed by the procedure using a 128-slice multidetector CT scanner (GE Healthcare Goldseal Optima CT 660). We diagnosed PE based on the presence of filling defects in at least two consecutive axial sections of the pulmonary artery. An emergency room physician who was blinded to the clinical information reviewed CTPA scans of the patients included in the study. CTPA results were recorded as positive or negative for PE.

2.4. Statistical Analysis

Descriptive statistics were summarized as counts and percentages for categorical variables; and median (interquartile range) for others. Intergroup differences by categorical variables were compared by using chi-squared test or Fisher’s Exact test. Differences between two groups for continuous variables, which did not meet normal distribution hypothesis, and ordinal variables were analyzed by Mann–Whitney U test. Receiver operating characteristic (ROC) curves were used to describe the performance of diagnostics value of the alternative tests. The area under the corresponding curves was calculated as described by Hanley and McNeil [21]. Furthermore, sensitivity, specificity, positive and negative predictive values, likelihood ratios, and odds ratio were calculated with a confidence interval of 95%. A p-value of less than 0.05 was considered statistically significant.

3. Results

In this study, we analyzed data from 2430 patients who underwent CTPA for suspected PE between May 01, 2020, and June 01, 2021. We excluded 1007 patients because of missing data and inconclusive examinations and included the remaining 1423 patients in the analyses (Figure 1).

3.1. Patient Demographics

The median age of the patients was 63.0 years (IQR: 49.0–76.0). Regarding the distribution by sex, 54.7% of patients were female (779 patients) and 45.3% were male (644 patients). Of all the patients, 69.2% (994 patients) and 30.8% (429 patients) were positive and negative for COVID-19, respectively; 7.3% of patients (104 patients) were diagnosed with PE. There were no significant differences in PE rates between patients with and without COVID-19 (p = 0.886).

There was a significant difference in the median Wells score between patients who were and were not diagnosed with PE; the former group had a median Wells score of 1.5 (IQR: 0.0–3.0) and the latter had a median Wells score of 1.0 (IQR: 0.0–1.5) (p < 0.001). Similarly, patients diagnosed with PE had a median Geneva score of 6.0 (IQR: 4.0–8.0), whereas patients not diagnosed with PE had a median Geneva score of 5.0 (IQR: 3.0–6.0); this difference was statistically significant (p < 0.001) (Table S1). Additionally, certain factors, including age >65 years, a previous history of DVT/PE, clinical DVT findings, a history of surgery or fracture in the last 1 month, immobilization, unilateral leg edema, and unilateral leg pain, were significantly more prevalent in patients diagnosed with PE (Table 1). However, we found no intergroup difference in terms of malignancy, heart rate >100 bpm, and hemoptysis rates.

3.2. Score and Algorithms

3.2.1. Diagnostic Performance of Wells and Geneva Scores

The cutoff value of 500 ng/mL D-dimer with Wells score exhibited 96.15% (95% CI: 92.46%–99.85%) sensitivity and 5.16% (95% CI: 3.96%–6.35%) specificity, with an area under the curve (AUC) of 0.507 (95% CI: 0.480–0.533). The Wells score with AADD exhibited 95.19% (95% CI: 91.08–99.30%) sensitivity, 8.11% (95% CI: 6.64%–9.59%) specificity, and 0.517 (95% CI: 0.490–0.543) AUC. The Geneva score with 500 ng/mL D-dimer had a 96.15% (95% CI: 92.46%–99.85%) sensitivity, 5.08% (95% CI: 3.89%–6.26%) specificity, and 0.506 (95% CI: 0.480–0.532) AUC, whereas the Geneva score with AADD had a 95.19% (95% CI: 91.08%–99.30%) sensitivity, 8.04% (95% CI: 6.57%–9.50%) specificity, and 0.516 (95% CI: 0.490–0.542) AUC (Figure 2) (Table S2). We found no significant differences in sensitivity and specificity between the Wells score and Geneva algorithm (p > 0.05).

3.2.2. Diagnostic Performance of YEARS and PEGeD Algorithms

The YEARS algorithm had a sensitivity of 85.58% (95% CI: 78.82%–92.33%), a specificity of 30.93% (95% CI: 28.44%–33.43%), and an AUC of 0.583 (95% CI: 0.556–0.608). The PEGeD algorithm had a sensitivity of 85.58% (95% CI: 78.82%–92.33%), a specificity of 32.22% (95% CI: 29.70%–34.74%), and an AUC of 0.589 (95% CI: 0.563–0.615) (Figure 2) (Table S2). The Wells and Geneva scores showed higher sensitivity than YEARS and PEGeD (p < 0.001), but YEARS and PEGeD algorithms were superior in terms of specificity (p < 0.0001).

3.2.3. Performance of Algorithms by COVID-19 Status

With the Wells score and 500 ng/mL D-dimer, there was a sensitivity of 93.75% and a specificity of 5.54% in patients without COVID-19 and a sensitivity of 97.22% and a specificity of 4.99% in patients with COVID-19. With AADD, these rates were 90.63% for sensitivity and 8.82% for specificity in patients without COVID-19 and 97.22% for sensitivity and 7.81% for specificity in patients with COVID-19. Similar to the Geneva score, there was a sensitivity of 93.75% and a specificity of 5.29% in patients without COVID-19 and a sensitivity of 97.22% and a specificity of 4.99% in patients with COVID-19. With AADD, there was a sensitivity of 90.63% and a specificity of 8.56% in patients without COVID-19 and a sensitivity of 97.22% and a specificity of 7.81% in patients with COVID-19. There was no significant difference between patients with and without COVID-19 by sensitivity, specificity, PPV, NPV, accuracy, and AUC values between Wells and Geneva scores and D-dimer cutoff values (p > 0.05) (Table 2).

With YEARS algorithm, there was a sensitivity of 84.38% and a specificity of 26.70% in patients without COVID-19 and a sensitivity of 86.11% and a specificity of 32.75% in patients with COVID-19. The accuracy rate was 31.00% in patients without COVID-19 and 36.62% in patients with COVID-19. There were significant differences in specificity (p = 0.029) and accuracy (p = 0.041) between patients with and without COVID-19 (Table 3). The PEGeD algorithm had a sensitivity of 84.38% and specificity of 27.96% in patients without COVID-19 and a sensitivity of 86.11% and a specificity of 34.06% in patients with COVID-19. The accuracy rates in patients without and with COVID-19 were 32.17% and 37.83%, respectively. There was also a significant difference for the PEGeD algorithm by specificity (p = 0.030) and accuracy (p = 0.041) (Table 3).

3.3. Diagnostic Performance of D-Dimer Cutoff Values

In the present study, we reviewed the diagnostic performance of different cutoff values based on patients’ D-dimer levels. The 500 ng/mL cutoff value had a sensitivity of 96.15%, a specificity of 5.08%, and an NPV of 94.37%. With this cutoff value, 67 unnecessary CTPAs were avoided, but 4 PE diagnoses were missed. At a cutoff value of 1000 ng/mL, sensitivity decreased to 81.73%, specificity increased to 33.13%, and NPV was 95.83%; accordingly, 437 CTPAs were avoided and 19 PE diagnoses were missed. A cutoff value of 2390 ng/mL reduced the sensitivity to 52.88% and increased the specificity to 73.77%; therefore, at this level, 973 CTPAs were avoided and 49 PE diagnoses were missed (Table 4).

4. Discussion

In this study, we compared the diagnostic performance of the most commonly used clinical decision-making algorithms for PE diagnosis during the COVID-19 pandemic. Our results indicated that all algorithms exhibited similar specificity and sensitivity to those of the prepandemic period. The algorithms that used Wells and Geneva scores with D-dimer threshold values performed similarly in patients with and without COVID-19. The YEARS and PEGeD algorithms performed significantly better in patients with COVID-19 than in patients without COVID-19 in terms of specificity and accuracy. The Wells and Geneva scores provided higher sensitivity when used with different D-dimer thresholds, but low specificity limited the diagnostic performance of these algorithms. Higher D-dimer thresholds increased specificity; however, the decrease in sensitivity increased the risk of missed PE cases.

The results of PEPCOV international retrospective study indicated that COVID-19 did not increase the likelihood of PE diagnosis in the emergency department [22]. The PEPCOV study reviewed the PE diagnosis rates between patients with and without COVID-19 and reported that COVID-19 was not associated with an increased risk of PE and that there was no significant effect of COVID-19 on PE risk on multivariate logistic regression analysis (adjusted OR = 0.98, 95% CI = 0.76–1.26). This result also applied to patients who underwent CTPA, particularly during the pandemic, indicating that COVID-19 did not increase the risk of PE. We found no significant difference in the rate of PE diagnosis between patients with and without COVID-19 in the present study, which is consistent with the results of the PEPCOV study and supports studies that reported that COVID-19 was not associated with an increased likelihood of PE diagnosis in the emergency department.

In the present study, area under the ROC (AUROC) values remained below 0.6, and the ability to predict PE was limited when the clinical pretest probability assessed using Wells and Geneva scores was combined with D-dimer measurement. In a study by Kirsch et al., the AUROC curve of the Wells score in patients with COVID-19 was reported to be 0.54, demonstrating its limited diagnostic capacity [23]. Another study of patients hospitalized because of COVID-19 reported that the modified Wells score offered limited accuracy in PE diagnosis. In this study, although the modified Wells score was significant for the diagnosis of PE with an AUROC value of 0.611, its diagnostic performance was considered poor [24]. Most studies that investigated the prevalence of PE in patients with COVID-19 included the performance of Wells or Geneva scores in their analyses but not PE diagnostic algorithms. Silva et al. used the method most similar to that of our study [17]. They compared the standard approaches based on Wells and Geneva scores with the AADD, YEARS, and PEGeD algorithms, and these clinical algorithms had limited discriminatory power in patients with COVID-19 and did not have adequate performance in predicting PE. Furthermore, AUROC values as low as 0.520 emphasized the limited diagnostic capacity of the algorithms in patients with COVID-19 [17]. The results of the present study are consistent with those of Silva et al. Nevertheless, only patients with COVID-19 were included in this study. Another study by Silva et al. investigated the performance of the same algorithms by excluding patients with COVID-19 [25]. In this study, the AADD strategy increased specificity in patients older than 70 years and safely reduced CTPA requests without significantly decreasing sensitivity, whereas the YEARS and PEGeD algorithms provided higher specificity in all age groups. To the best of our knowledge, the present study is the first to simultaneously investigate clinical algorithms for PE in patients with and without COVID-19 and, thus, makes an important contribution to the understanding of the effect of COVID-19 on the performance of these algorithms. Our results provide valuable insights by suggesting that the algorithms performed similarly in both patient groups.

In a study by Chassagnon et al., it was suggested that PE exclusion strategies in patients with COVID-19 should not differ from those in patients without COVID-19, but the D-dimer level was the most important predictor [26]. The results of the present study indicated that clinical algorithms based on Wells and Geneva scores provided higher sensitivity but lower specificity and limited discriminative capacity in patients with COVID-19, which was consistent with the results of previous studies [17,26,27]. Another study that investigated the combined use of Wells score and D-dimer levels in the diagnosis of DVT and PE in patients with COVID-19 similarly reported that the combination of Wells PE score and D-dimer provided higher sensitivity but lower specificity [27]. The study reported that COVID-19-specific factors (e.g., intubation and severe systemic inflammation) were important predictors of PE occurrence and suggested that PE was usually asymptomatic and not significantly associated with classical risk factors. Conversely, in the present study, classical thromboembolic risk factors were more prevalent in patients diagnosed with PE. The relatively larger population in the present study and the inclusion of patients without COVID-19 along with those with COVID-19 may explain this difference. The present study provides a comprehensive perspective on PE diagnosis in patients with and without COVID-19, demonstrating the need to reassess the importance of classical risk factors.

International guidelines recommend the YEARS and PEGeD algorithms for the diagnosis and management of PE as an alternative to standard methods for excluding PE [19]. Although these algorithms have the potential to reduce unnecessary CTPA requests, the increase in specificity may lead to a decrease in sensitivity with an increased risk of missed PE cases. A previous study reported that the YEARS algorithm provided high specificity with a cutoff value of 500 ng/mL and the use of AADD, but this might increase the risk of missed PE diagnosis [17]. Another study that used YEARS and Wells scores with AADD cutoff value in patients with COVID-19 reported that the risk of missed PE diagnosis persisted upon introduction of these algorithms and that new clinical assessment strategies might be needed in patients with COVID-19 [18]. The present study indicated that the YEARS and PEGeD algorithms provided higher specificity than the Wells and Geneva scores, particularly regarding patients with COVID-19. Nevertheless, the reduced sensitivity of these algorithms may put some cases of PE at risk of being missed. These results suggested that the YEARS and PEGeD algorithms had the potential to improve specificity for the diagnosis of PE in patients with COVID-19 but require careful evaluation.

It is widely accepted in previous studies that D-dimer levels are an important predictor of VTE risk in patients with COVID-19. Various studies have reported that different D-dimer cutoff values yield different results in terms of sensitivity and specificity. For example, Kampouri et al. reported that a combination of D-dimer ≥3000 ng/mL and Wells score ≥2 provided high specificity in determining VTE risk (91.6% specificity) [28]. This result suggested that the combined assessment of high D-dimer levels and clinical scores might reduce unnecessary imaging requests. Similarly, Ventura-Díaz et al. suggested the D-dimer cutoff value for the diagnosis of PE in patients with COVID-19 as 2903 ng/mL and reported that this value had a sensitivity of 81% [29]. This study suggested that higher D-dimer cutoff values might decrease sensitivity but increase specificity, contributing to the avoidance of unnecessary investigations. Brem et al. also showed that D-dimer levels were a strong predictor of PE; specifically, levels >2590 ng/mL were significantly associated with an increased risk of PE [30]. Notwithstanding the above, it was also reported that high cutoff values may lead to an increased number of missed PE cases due to reduced sensitivity. The results of the present study are consistent with the results of previous studies. Increasing the D-dimer cutoff value increased specificity but decreased sensitivity, and therefore, some PE cases might be missed. A study by Silva et al. similarly reported that high D-dimer cutoff values were not reliable as a strategy to exclude PE because sensitivity was severely reduced [17]. Increasing the D-dimer cutoff value might be useful to avoid unnecessary investigations, but it may increase the risk of overlooking PE cases. In patients with COVID-19, assessing D-dimer levels along with clinical scores and patient characteristics can provide a more reliable and effective diagnostic strategy.

4.1. Limitations

The present study has some limitations. First, we designed this study as a retrospective, single-center chart review, which led to clinical assessments based on recorded notes rather than direct patient observation. This might have affected the accuracy of the clinical decisions. Furthermore, the fact that only patients who underwent CTPA in the emergency department and had D-dimer test results available were included in the study might affect the generalizability of the results to a larger emergency department population with suspected PE. The present study did not investigate the likely effects of COVID-19 vaccination.

5. Conclusions

Based on the study results, the COVID-19 pandemic did not significantly affect the performance of clinical decision-making algorithms used in the diagnosis of PE and the performance of algorithms using Wells and Geneva scores with different D-dimer threshold values was not affected by being positive or negative for COVID-19. Additionally, YEARS and PEGeD algorithms showed significantly better performance in terms of specificity and accuracy in patients with COVID-19 than those in patients without COVID-19, indicating that they could be prioritized over other algorithms, particularly in this patient group. However, different D-dimer predictive values determined using the stepwise increase strategy can offer higher specificity but may increase the risk of missed PE cases due to decreased sensitivity.