The use of clinical prediction rules without echocardiographic evaluation in Staphylococcus aureus bacteraemia may not be enough to exclude infective endocarditis and be associated with increased mortality

: Background . It is unclear whether the use of clinical prediction rules is sufficient to rule out infective endocarditis (IE) in patients with Staphylococcus aureus bacteremia (SAB) without an echocardiogram evaluation, either transthoracic (TTE) and/or transesophageal (TEE). Our primary purpose was to test the usefulness of PREDICT, POSITIVE and VIRSTA scores to rule out IE without echocardiography. Our secondary purpose was to evaluate whether not performing an echocardiogram evaluation is associated with higher mortality. Methods. We conducted a unicentric retrospective cohort including all patients with a first SAB episode from January 2015 to December 2020. IE was defined according to modified Duke criteria. We predefined threshold cut-off points to consider that IE was ruled out by means of the mentioned scores. To assess 30-day mortality, we used a multivariable regression model considering performing an echocardiogram as covariate. Results. Out of 404 patients, IE was diagnosed in 50 (12.4%). Prevalence of IE within patients with negative PREDICT, POSITIVE and VIRSTA scores was: 3.6% (95% CI 0.1-6.9%), 4.9% (95% CI 2.2-7.7%), and 2.2% (95% CI 0.2-4.3%), respectively. Patients with negative VIRSTA and negative TTE had an IE prevalence of 0.9% (95% CI 0-2.8%). Performing an echocardiogram was independently associated with lower 30-day mortality (OR 0.24 95%CI 0.10-0.54, p = 0.001). Conclusion. PREDICT and POSITIVE scores were not sufficient to rule out IE without TEE. In patients with negative VIRSTA score, it was doubtful if IE could be discarded with a negative TTE. Not performing an echocardiogram was associated with worse outcomes, which might be related to presence of occult IE. Further studies are needed to assess the usefulness of clinical prediction rules in avoiding echocardiographic evaluation in SAB patients.


Introduction
Staphylococcus aureus (SA) bacteremia (SAB) is one of the most frequent causes of positive blood cultures (BC) and has a high mortality (1,2). One of the most serious complications in patients with SAB is the development of infective endocarditis (IE), which worsens the prognosis (2,3). IE in patients with SAB can occur in the absence of risk factors and with structurally normal heart valves (4).
The diagnosis of IE in patients with SAB is of vital importance because it modifies the antibiotic selection and duration (5) and cardiac surgery may be necessary (6,7). Some authors have estimated that not reaching a diagnosis of IE in patients with SAB conveys a 15-20% increase in mortality (8). Therefore, systematic echocardiography has been classically recommended, with transesophageal echocardiography (TEE) being preferred to transthoracic echocardiography (TTE). This recommendation appears in the current guidelines (9) with a grade of evidence A-II.
However, this increased mortality should be weighed against the risks induced by routine TEE. Several studies have shown that serious complications of the performance of TEE is around 0.1%, in addition to the economic and personnel costs involved (10)(11)(12)(13).
Consequently, there have been several attempts to define subsets of patients with SAB at low risk for IE, who could be safely managed omitting the performance of TEE. Recently, three clinical prediction scores have been published, PREDICT (14), VIRSTA (15) and POSITIVE (16). These scores can identify patients at low risk in which TEE could be deferred. To date, the scarce validation studies available have yielded controversial results (17)(18)(19). Moreover, all these studies have an important limitation: the low rate of echocardiography assessment, especially TEE in groups classified as low risk. It has been shown that the proportion of IE diagnosed increases with the number of patients with SAB who receive echocardiographic assessment (20,21). Hence, it is likely that some patients had undiagnosed IE in these studies. This underdiagnosis could have an influence on mortality.
Despite this controversial results and limitations, some authors recommend forgoing echocardiographic evaluation (even TTE) in patients with no identifiable risk factor (22,23). Furthermore, a recent scenario-based survey showed that approximately 15-20% of experts would recommend not performing TTE in patients with negative VIRSTA score (24). Therefore, our primary objective was to evaluate the PREDICT, VIRSTA and POSI-TIVE scores, and to test their usefulness in safely ruling out IE without the need for TEE or TTE. The secondary objective was to determine, in patients without IE, the effect of withholding echocardiographic assessment on SAB mortality, in particular in those identified as low risk for IE, considering that missing an IE diagnosis could potentially cause an excess mortality.

Materials and Methods
We conducted a retrospective single-center study from January 2015 to December 2020. Our center is a third level university hospital with 613 beds in Madrid and a target population of 550,000 inhabitants.
Through a microbiology database, we identified all patients older than 18 years with a first (index) BC positive for SA during the study period. Patients with previous episodes of SAB were excluded. Index and follow-up BCs were obtained at discretion of the attending physician. BC were processed using the BD BACTEC FX system (Becton Dickinson, Sparks, Maryland, USA). When BC were positive, the strain was identified by MALDI-TOF (Bruker DaltonicTM). All systems were applied according to the manufacturer's instructions.
Data were collected retrospectively from the electronic medical record, including demographic, comorbidities, clinical, microbiological, echocardiographic and outcome data.
The study was approved by the hospital ethics committee. Since this was a retrospective, non-interventional study and only required collection of previously generated and anonymized data, informed consent was not required.

Definitions
IE was defined according to modified Duke criteria for definite IE (6). Bacteremia was considered persistent when follow-up BC were positive at least 48 hours after the extraction of the index BC despite appropriate antibiotic therapy, accordingly to the scores development studies definition (14)(15)(16). Complicated bacteremia was defined according to current guidelines (9). Primary bacteremia was defined as that in which the original source could not be determined. Sepsis and septic shock were defined according to current guidelines (25). Relapse was defined as the appearance of a new SAB at least 15 days after negative follow-up BC or 30 days after the extraction of the index BCs and initial clinical resolution in the absence of follow-up BCs.

Predefined Ruling out IE thresholds
IE was considered ruled out when the risk of IE was low enough for the patient not to benefit from echocardiographic assessment. In defining cutoff points for this low IE risk, we considered recent publications that estimate the risk of IE beyond which a patient does benefit from such assessment (8,26). We also considered the usefulness of the negative likelihood ratio (NLR) to predict this risk, as previously reported (27). In accordance with the aforementioned studies, we predefined the following cut-off points: 1-If the risk of IE was less than 1% and the NLR was less than 0.05, IE would be considered ruled out without the need for any echocardiographic assessment; 2-If the risk of IE was between 1-2% and the NLR was less than 0.10, IE would be considered ruled out with the use of TTE without TEE. 3-If the risk of IE was between 2-5% and the NLR was less than 0.20, it would be considered uncertain if IE may possibly be ruled out with a negative TTE without performing TEE. 4-If the risk of IE was greater than 5% or the NLR was greater than 0.20, TEE would be considered necessary to rule out IE. Additionally, we considered that if the prevalence of endocarditis in low-risk patients with negative TTE was higher than 1.1%, these patients would benefit from TEE, in accordance with other authors (8).

Primary and secondary objectives
Our primary purpose was to assess the prevalence of IE in patients identified as low risk by means of PREDICT, VIRSTA and POSITIVE score, as well as determine the NLR of these scores. Our secondary objective was to evaluate the association between not performing an echocardiogram and 30-day mortality.

Statistical analysis
Quantitative variables are presented as median and interquartile range (IQR) and qualitative variables are presented as percentages and absolute values.
For the primary objective, the PREDICT, VIRSTA and POSITIVE scores were validated by calculating their sensitivity, specificity, likelihood ratios, predictive values, and their area under the receiving operating curve (AUC). The percentage of patients identified as low risk by these scores and who finally had IE is provided, including its 95% confidence interval (CI). Sensitivity analyses were performed including different populations according to whether different echocardiogram modalities were performed. A score was considered valid to rule out IE, with or without ETT, when the percentage of IE in patients identified as low risk and the score`s NLR were below the mentioned cut-off points.
For the secondary objective, patients with IE diagnosed were excluded. In order to mitigate survivor bias, we excluded those patients who died within 48 hours of index BC extraction. Univariate analysis of 30-day mortality was performed using chi-square for qualitative variables (or Fisher exact test) and Mann-Whitney U for quantitative variables. Multivariate logistic regression models were developed, including echocardiography assessment (TTE, TEE or either) and those clinically relevant variables identified as statistically significant in the univariate analysis. Bilateral p-values of less than 0.05 were considered statistically significant. All statistical analyses were performed using SPSS version 25 statistical software (SPSS Inc, IBM, Chicago, Illinois, United States).

POSITIVE score evaluation
Using the POSITIVE score, 60.3% of patients (245/404) were identified as low risk. Of these, 4.9% had IE (Table 2), with a 95% CI exceeding 5% (2.2-7.7%). The validation of the score is shown in table 3 and figure S1. POSITIVE score had a NLR of 0.37. According to the predefined cut-off points, using the POSITIVE score would not be sufficient to rule out IE without a TEE. Table 4 shows sensitivity analyses with the percentage of IE in patients with negative POSITIVE score in different population according to echocardiogram evaluation. Of the low-risk patients with a negative TTE, 2.8% (4/141) were diagnosed with IE.

VIRSTA score evaluation
Using the VIRSTA score, 45.6% of patients (185/404) were identified as low risk. Of these, 2.2% had IE (Table 2), with a 95% CI not exceeding 5% (2.2-4.3%). The validation of the score is shown in table 3 and figure S1. VIRSTA score had a NLR of 0.16. According to the predefined cut-off points, it would be uncertain if using the VIRSTA score in conjunction with a negative TTE would allow IE to be safely ruled out without a TEE.  Table 4 shows sensitivity analyses with the percentage of IE in patients with negative VIRSTA score in different population according to echocardiogram evaluation. Of the low-risk patients with a negative TTE, 0.9% (1/103) were diagnosed with IE.
Supplementary Table S1 shows the patient-level data for SAB episodes identified as low risk by the VIRSTA score and finally diagnosed as IE. Only one patient had an identifiable risk factor (community-acquired bacteremia).

Echocardiographic assessment and 30-day mortality
After excluding patients diagnosed with IE and those who died within 48 hours after index BC extraction, performing an echocardiographic assessment was independently associated with lower 30-day mortality in a multivariate logistic regression model, including adjustment for VIRSTA score (OR 0.24 95%CI 0.10-0.54). The model is shown in Table 5. Univariate analysis for the selection of variables included is shown in supplementary Table S2. 30-day mortality in patients classified as low risk by VIRSTA score was lower if they had undergone TTE (5.1% vs 15.7%, p = 0.031).

Discussion
In our study, we evaluated whether the application of different scores for predicting IE risk in patients with SAB, with or without TTE, could safely rule out IE without TEE. Our main conclusion is that neither the PREDICT nor the POSITIVE scores would be sufficient to rule out IE without TEE, even with negative TTE. It was uncertain whether a negative VIRSTA score together with a negative TTE could allow to rule out IE without the need for TEE.
The PREDICT score was the first to be published (14). Its high negative predictive value was ratified in a validation study performed by the same group (28). However, other authors have found that this score does not identify low-risk patients with sufficient accuracy (18,19). These results are in line with ours. It should be noted that PREDICT score does not include among its variables an important set of risk factors for IE (such as prosthetic heart valve), which could reduce its sensitivity (29). In summary, the application of the PREDICT score to avoid performance of TEE in SAB should not be recommended.
The POSITIVE score (16) is largely based on the shorter time to positivity of BC in the case of IE. In our study, as well as in other previous work (30), despite being a risk factor for IE, the association of time to positivity with IE was not as strong as that presented by the POSITIVE score authors. In line with our findings, a recent study (19) also failed to validate this score. In summary, this score does not identify patients at low risk with enough precision to avoid performing a TEE, so that it should not be recommended for this purpose. The VIRSTA score (15) incorporates the largest number of variables as compared to the previous. Consequently, some studies have shown a higher sensitivity (18,19). Hence, a recent survey showed that 15-20% of experts would recommend not performing TTE, nor TEE, evaluation in patients identified as low risk this score. Yet, in our data, the VIRSTA score was not sufficient to rule out IE without performing an echocardiogram evaluation (incidence of IE in low-risk patients 2.2%, with NLR 0.16), and it was doubtful if a negative VIRSTA score plus a negative TTE would be able to rule out IE without performing a TEE. It should be noted that most of the patients eventually diagnosed with IE that had been identified as low risk by the VIRSTA score did not have any identifiable risk factors that would have raised IE suspicion. Ruling out IE in patients with negative VIRSTA and a negative TTE would allow to avoid TEE in approximately half of patients with SAB (15,18,19). However, well-designed studies are needed to confirm the usefulness of this approach.
A potential restriction when applying these scores is that both their development and validation studies have been performed in cohorts without universal echocardiographic assessment, and only a small proportion of patients have undergone TEE. Several authors have demonstrated that carrying out an echocardiogram in a larger proportion of patients implies a higher rate of IE diagnoses (20,21,31,32). It has even been proposed that performing an echocardiogram in these patients may be associated with lower mortality (33). In our study, we observed an increased 30-day mortality among patients with SAB who did not undergo an echocardiogram. We even found a higher mortality among patients identified as low risk for IE who had not undergo TTE. The fact that the association persists after adjusting for multiple variables, including comorbidity and severity of the bacteremia, supports the likelihood that this worse outcome was caused by the presence of undiagnosed occult IE, at least partially. Therefore, we decidedly believe that prospective studies validating these scores in a SAB cohort with systematic TTE and TEE is mandatory before assuming that they allow to avoid unnecessary echocardiography in low-risk patients.
Our study has some limitations. Firstly, as a single-center retrospective study, it presents the limitations of external validity that this entails. However, our population is comparable to that described in recent cohorts (34). Secondly, our predefined cut-off points are not prospectively validated. Nevertheless, both the IE risk percentages selected and the use of NLR have previously been proposed and accepted by several authors (8,24,26,27,32). Thirdly, we cannot exclude that the higher 30-day mortality found in patients with no echocardiogram is caused to uncontrolled confounding variables. However, the fact that this association persist after adjusting for several factors increases the likelihood that this worse outcome is ascribable, at least partially, to an undiagnosed IE. Finally, as mentioned, not all patients underwent echocardiography, and the percentage of patients lacking an echocardiogram was higher in low-risk patients. This absence of an echocardiogram, common in other studies as well, prevents the exclusion of occult IE in some of these patients, a fact that may limit the accuracy of the scores.

Conclusions
Neither the PREDICT nor the POSITIVE score were sufficient to exclude IE without concurrent TEE. In patients with negative VIRSTA score, it was uncertain whether IE could be discarded only with a negative TTE, without undergoing TEE. Moreover, we found an association between not performing echocardiographic assessment and an increased 30-day mortality, even in patients at low risk for IE, which could be related to the presence of occult IE. Hence, prospective well-designed studies with systematic performance of TTE and TEE are needed to verify whether these tests can be safely avoided in any subgroup of patients.
Supplementary Materials: The following supporting information can be downloaded at: www.mdpi.com/xxx/s1, Table S1: Individual characteristics of patients identified as low risk of IE by means of VIRSTA score but who finally were diagnosed of IE. Table S2: Univariate analysis of 30-day mortality in patients with no diagnosed with infective endocarditis and who survived the first 48 hours after the index blood culture extraction. Institutional Review Board Statement: The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Hospital Puerta de Hierro (protocol code PI 191/21).
Informed Consent Statement: Since this was a retrospective, non-interventional study and only required collection of previously generated and anonymized data, informed consent was not waived.
Data Availability Statement: The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest.