Evaluation of Hospital Antimicrobial Stewardship Programs: Process, Implementation, Impact and Outcomes. Review of Systematic Reviews

1. Introduction

The primary goals of Public Health (PH), are to improve population wellbeing and outcomes through promotion of health , prevention of diseases and facilitation of fair access to healthcare [1]. Fundamental to the concept of PH is the provision of safe and quality care which is equitable and cost effective [2]. To achieve such targets, efforts should be directed towards reliable mechanisms for evaluation of healthcare programs interventions [3]. Historically, aims were directed towards efficacy and outcomes, however it has been argued that achieving intended objectives does not always equate delivering quality care since according to Linford et al; outcomes measures are “blunt instruments for judging performance” [4]. To bridge that gap , quality experts recommend provision of appropriate evaluation methods to facilitate delivery of aimed objectives [4]. Therefore, programs evaluation is of utmost importance primarily to appraise effectiveness and desired outcomes, as well as address challenges [5]. According to the Centers for Disease Control and Prevention (CDC) of USA , the process is defined as : “a systematic method for collecting, analyzing, and using data to examine the effectiveness and efficiency of programs and, importantly, to contribute to continuous program improvement”[6].

Alarmed by concerns regarding inappropriate and over-prescribing of antimicrobials in healthcare that is directly linked to patients adverse events as well as being indirectly associated with escalating antimicrobial resistance (AMR) , at the turn of the 21^th century , Western healthcare authorities across the Atlantic introduced the concept of Antimicrobial Stewardship Programs (ASPs) following a justifiable appeal from the field experts [7] . According to the CDC, at inpatient hospital settings, more than 50 % of prescribed antimicrobials are not consistent with recommended practice while the majority of common infections are over-treated [6]. The National Institute for Healthcare and Excellence (NICE) in UK , defines the process as: “the organizational or healthcare-system-wide approach to promote and monitor the judicious use of antimicrobials to preserve their future effectiveness” [8]. The program entails collaboration of a wide range of healthcare professionals led by infection and antimicrobial specialists with prime aim to oversee appropriate and optimal prescribing of antimicrobials through provision of guidance, awareness in addition to monitoring of outcomes [9].

Almost twenty-five years following its implementation, ASPs became widely accepted and subsequently embraced at national and international levels and eventually promoted by the WHO to the point of being hailed as one of the major 21^st century public health interventions [10,11,12] .Nevertheless, because of differences in healthcare settings and population diversity , there have been uncertainties regarding conclusive and generalizable evidence to answer raised questions such as efficacy, and outcomes [12,13]. For example, there are implementation challenges because of local epidemiology or healthcare settings [14,15], conflicting differences for optimal programs elements [13,16], key role of antimicrobial pharmacists [17], paucity of reporting of microbiological outcomes [18] and challenges in surveillance processes [13]. Similarly, program outcomes such as targeted mortality has been conflicting with reports citing lower rates while others showing no differences [19,20]. Conversely, despite multiple studies concluded that ASPs can reduce AMR [21,22,23] , others were not conclusive [24,25]. Equally, the economic impact of the program is confounded by difficulties in adopting consensus models [10,26] . Lastly and importantly, although the program has major components, some studies looked at additional IT reinforcing concepts such as use of smart applications which merits further evaluation [27,28].

For all these reasons, it is prudent to have an umbrella overview to evaluate ASPs process measures, impact and outcomes as well as assess any specific service improvement concepts. Distinctively, although other systematic reviews, meta- analysis and overviews examined different concepts of ASPs repeatedly, they only focused on specific aspects rather than the overall aspects [25,29,30].

2. Methods and Search Strategies

A systematic literature review was conducted guided by the framework of Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) [31]. We included only peer reviewed primary systematic reviews (SRs) in ASPs that reported any implementation process, impact, or outcomes measures specifically at inpatients hospital settings with no age restrictions. Five major databases were searched: OVID-Medline, PubMed, Embase, Cochrane databases and Google Scholar. The search was conducted and updated up to 28^th August 2022 restricted for the last 10 years focusing on humans with no language restrictions. Studies that were conducted at primary, ambulatory or long-term facilities were excluded together with studies that focused on singular countries or pathogens. Similarly, studies identified mainly as reviews or scoping reviews with non-congruent systematic reviews methodology were excluded. Since the COVID-19 pandemic was still unfolding, related data on the subjects were excluded. Guided by inclusion and exclusion criteria, two separate primary investigators screened titles, abstract and methods and agreed on final output when selected studies were read in depth including ranking for critical appraisal.

3. Search Outcomes

The search outcome is depicted as recommended in PRISMA Figure 1 and details of search protocols are provided at Appendix 1. Out of 265 identified SRs, 202 studies were excluded leaving the remaining 63 studies for final analysis.

4. Synthesis of Evidence

Since studies were heterogenous in concepts and methods examining wide aspects of the program, thematic categorization of studies was performed to produce subclassifications according to published reporting period, origin of affiliated institutions, global or regional focus of studies, studied population and more importantly, objective concepts. The six recognized WHO geographical criteria were used to group regional countries accordingly [32]. From each examined themes, specific conclusions were extracted together with relevant recommendations.

5. Critical Appraisal for Quality of Evidence

Critical appraisal of the examined SRs was conducted by two investigators using the updated online version of: A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) [33]. Studies were ranked from high to critically low depending on the overall non-numerical assessment. It must be emphasized that since the concept of ASPs is complex, with clear heterogeneity that is frequently not tested through desired evidence-based practice including randomized trials or meta-analysis with no rigorous safeguarding for biases , it wasn’t surprising that the stringent AMSTAR -2 critical appraisal criteria rated most SRs as low quality with the exception of two Table 1 [24,34].

6. Results

Out of 63 selected systematic reviews, the majority were conducted in European region (EUR : 31) and regions of Americas (AMR: 14) at a total of 71 % (46/63) , while the remaining regions constituted 29 % ( 18/63) detailed as : Western Pacific region (WPR:8) , Eastern Mediterranean Region (EMRO: 7) , Southeast Asian Region (SEAR: 2) and the African Region (AF:1) as outlined in Figure 2 . Most of the SRs (84 %) focused on global objectives while only 16 % reported regional data. Predominant SRs (51/63) included all age groups, while adults were specified in 6, children in 5 and infants in only 1 study. The ASPs were evaluated through 25 different concepts dominated by efficacy and safety [9] and AMR as outcomes [8], while economic impact was evaluated in [5] studies, role of information technology in [5], as well as the role of pharmacy [4] and behavioral factors [4] as outlined in Table 2. Out of the 63 SRs , 22 (35 %) examined solely implementation process while 17 (27 %) focused on outcome measures and the two concepts were examined in 38 %.

7. Overview of Evaluation Process

Examining the spectrum of global studies, most SRs focused on Western population with paucity of data from developing countries Figure 2. In their large and high rated SR focusing on efficacy and safety, Davey et al study encompassed 221 articles included 58 randomized controlled trials and 163 that wasn’t: studies were from North America [96] and Europe [87] while the remaining were from Asia [19], South America [8], Australia [8], and the East Asia [3] [24]. Table 1 and Table 2.

8. Studied Population

The ASPs systematic reviews identified studied population as adults, children, and infants mainly from high income Western countries [24,35,36]. Most of data from the SRs were extracted from adults’ population while data from the pediatric cohorts are scarce specially regarding the implementation measures Table 2 [18,36]. Despite the limited available data from the children population, the ASPs managed to implement objectives without compromising safety even at pediatric intensive care units (PICUs) with limited data regarding healthcare associated infections and antimicrobial resistance [36].

9. Healthcare Settings

Although the majority of secondary care facilities in developed countries are small and medium sized hospitals (200- 500 beds) , the majority of data were extracted from studies conducted at large size hospitals (> 500 beds) albeit antimicrobial consumption might be similar but the implementation barriers are more pronounced [10,37]. Conversely, despite the highlighted benefits of the program at different part across the globe, there are equally outlined implementation and outcome challenges in developing regions leading to strong demands to strengthen fundamental programs aspects [11,14,15,38] .Therefore, to improve global delivering of ASPs objectives , healthcare services in developed countries should focus their attention towards small and medium sized hospitals while developing countries should cement ASPs core elements.

10. Process Implementation Interventions

The main examined interventions during program implementation include empirical therapy according to guidelines, timely de-escalation therapy, switch from intravenous to oral, therapeutic drug monitoring and preauthorization through restricted antimicrobials lists . From multiple SRs, there is evidence of successful implementation of these interventions aligned with outcomes [15,24,25,29]. When comparing enablement objectives with or without ASPs interventions, there was no observed mortality differences [24]. Efficacy without compromising safety has been similarly observed in vulnerable population at critical units both at adults and pediatrics settings through effective program interventions of audit of prescribing, feedback, education, and persuasion [20,24,39].

11. Epidemiology and Surveillance

Although the importance of surveillance and epidemiological reports in guiding ASPs has been clearly overstated, however, from multiple SRs, challenges are evidence both in developed and developing countries with variable representation [13,14,15]. Moreover, although ASPs have been implemented with anticipations to curb AMR as an outcome, SR studies described challenges in accurate reporting in healthcare affecting accuracy [25,40]. Additionally, there were paucity of studies that report accurate microbiological results linked to ASPs which is the hallmark for monitoring AMR [18]. To bridge such gaps, it is advocated to conduct proper AMR surveillance methods in all healthcare facilities that should guide ASPs implementation process and not vice versa [13,41] .

12. Efficacy and Safety

Since the program concepts are based upon restricting antimicrobials as well as reducing the duration of therapy, there have been some concerns that it might impact on patients’ safety. From multiple SRs, ASPs proved to be efficacious in reducing antimicrobial consumption and inappropriate prescribing without compromising patients’ safety both for adults and pediatrics populations [19,35,42,43]. The feared prime concern was directed towards critical care where highest levels of hospital antimicrobial consumption is usually reported combined with critical nature of patients’ cohort. Nevertheless, SRs in such important aspect affirmed safety and efficacy although the evidence is more pronounced for adults when compared to pediatrics population because of limited data [20,39,42,44]. However, the picture is not fully clear since counter argument that restricting antimicrobials at critical care settings might be protective through limiting unfavorable adverse events or might be confounded by the plausible practice of reducing interventions in less critical patients [44].

13. Influencing Prescribing Behavior

The foremost definition of ASPs accentuates “appropriate and judicious prescribing of antimicrobials” which entails focusing efforts to influence the behavior and attitude for antimicrobial prescribing directed towards healthcare professionals [24]. Determinants factors imported from behavioral studies such as psychosocial theory of planned behavior advocates how developed cognition influence decision making and behavior [45,46]. In ASPs, the main positive attitude determinants that influences prescribing behavior has been identified as education and training as well as audit and feedback aimed to alter conceived prescribing cultures [24]. Despite the conceivable association between the influence of cognitive behavioral and previous experience on prescribing cultures, some SRs pointed towards gaps in targeting described attitude particularly lack of qualitative studies to address raised barriers hence exploring prescribing attitude and practice particularly behavior changed techniques is advocated both in developed and developing countries [24,46,47].

14. Quantifying Metrics

Appropriate quantifying metrics (QM) must be adopted to closely monitor consumption as well as estimate associated costs. As the concepts of ASPs were introduced without defining relevant QMs, ensuing challenges were to evaluate exiting as well as develop new ones [48]. Amongst the existing QMs were the Defined Daily Dose (DDD) which was developed in the late 1970s , standardized through patients’ total hospital days depicted per 1000s days (DDD-1000 days) [49]. In their SR regarding QMs, Stanic Benic et al identified 12 common measurements that include the commonly used DDDs, Days of Therapy (DOT) as well as total antimicrobial cost. The recommended practice is a combined metric approach using at least two of the prime measures [50]. Despite there have been many cited limitations for using QMs such as DDDs because of variations in relation to geographical locations or hospital setting , role of weight adjustments, adjusted doses in renal impairment as well as combined therapy , however it has been widely adopted across the globe in the adults population as well as being recommended by the WHO [51]. In the pediatric population, identified metric challenges concluded no optimal recommended measures although DOT are generally preferred when compared to DDDs [52].

15. Role of Pharmacy

Few SRs examined the roles of antimicrobial pharmacist during implementation of ASPs demonstrating effectiveness interventions in form of education-based interventions, compliance with guidelines and reduction in the duration of antimicrobial therapy calling for further empowerment [53]. Similarly, the role of pharmacists has been emphasized at the emergency department as well as small and medium sized hospitals where pharmacists play a major role in ensuring appropriate prescribing and guarding against misuse [17,54] . Uniquely, Daniels and Weber looked at the role of hospital-based pharmacists in verifying hospital discharge antimicrobials highlighting positive outcomes in validating antibiotic choice, duration, frequency, and directed therapy in line with ASPs objectives [55].

16. Evaluation of Outcome Measures

16.1. Interface of ASP and Its Impact on Antimicrobial Resistance

The global and secondary care burden of antimicrobial resistance on morbidity , mortality and cost of management has been previously outlined in major studies [56,57]. Despite the plausible link between antimicrobials consumption and the development of AMR, the causal relationship proved difficult to establish. In their SR encompassing 32 studies and almost 9 million reviewed reported cases, Baur et al reported that ASPs reduces hospital infection and colonization with multidrug resistance organisms (MDRO) and clostridium Difficile infections by a factor of almost 50 %, a similar outcome of reduction of infections caused by MDROs was shared by Karanika et al adding that reduced antimicrobial consumption was not associated with observed adverse outcomes [29,58] . This view is contradicted by Tacconelli et al highlighting flaws with epidemiological reports , while Chatzopoulou et al pointed that the plausible projected hypothesis is supported by poor evidence as well as Bertollo et al citing paucity of randomized control trials and reliance on observational and quasi experimental studies [13,40,59] . The lack of conclusive evidence to support direct correlation has been affirmed by a wider meta- analysis by Schuts et al [25].

Trying to delve into potential explanations for the lack of supporting evidence despite the coherent assumption, there are major confounding factors which are impossible to control. For example, potential factors that is might directly affect propagation of AMR include, local environment and various healthcare settings , demographics of affected populations including susceptibilities to infections either from acquired or genetic factors or colonization with resistant strains , previous exposure to antimicrobials , dominance of certain resistant clones, access to critical care including the use of invasive devices as well as local practices of infection control and prevention [13,60,61]. Hence, to bridge the obvious gap between implementation of ASPs and reduction of AMRs at healthcare settings, it is conceivable to advocate a reliable and functional epidemiological reporting and surveillance systems which must be continuously improved and developed [13] .

16.2. Economic Impact and Cost Effectiveness

Although it wasn’t declared amongst the main goals of ASPs at its inception , it was soon realized that there are observed secondary benefits based upon economic savings [62]. From reciprocated SRs, the economic benefit of the ASPs is evident mainly through three main aspects: direct saving from restricting more expensive broad-spectrum antimicrobials, reduction of duration of therapy as well as reducing patients’ length of hospital stay [10,26,29,63] . Nevertheless, there were conclusive remarks that there are non-uniform agreements for calculating economic parameters, quality of extracted data, and heterogeneity of studies as in pediatric population [36,64,65].

16.3. Quality Assessment and Development of ASPs

In healthcare, quality has been defined as “the systemic process to improve healthcare delivery” which is integral to provision of continuously developed ASPs [66]. In their SR, Schweitzer et al concluded the overall studies in ASPs are of low quality and did not improve over time, additional they expressed concerns of lack of microbiological and clinical data as an outcome calling to improve of studies methodology [64]. To converge towards consensus quality indicators for ASPs, about 50 QIs have been outlined to serve towards future standardized measures [67]. As quality challenges in reporting and surveillance has been outlined calling for standardized methods both in developed and developing countries [13,15]

17. Role of Modern Information Technology

Since the ASPs heavily relies on updated knowledge, accurate prescribing, applied and retrieved data, it is plausible to explore the role of information technology (IT) towards quality improvement of the program [68]. The concept has been explored by Baysari et al identifying computerized decision support system (CDSSs), computerized antimicrobial approval system and surveillance methods concluding positive outcomes confounded by the absence of comparative studies [27]. Although the modern role of CDSSs became more evident in ASPs, Rawson et al expressed concerns it was not designed properly to guide antimicrobial prescribers calling for better models [69]. Furthermore, Micallef et al advocated secondary use of information technology and hospital electronic systems to support delivering of effective ASPs [70]. Lastly, although not supported by much updated evidence, the use of specific ASPs smartphone applications to improve knowledge, skills and attitude towards antimicrobial prescribing demonstrated a promising platform that worth exploring [28] .

18. Limitations

One of the fundamental limitations of the evaluation of the SRs regarding ASPs, that the prime concept is relatively new at different parts of the world with less than 25 years of experience with continuously evolving standards. From the review, most of the data are driven from established Western healthcare facilities with limited comparators from global developing countries. Additionally, from multiple SRs, extracted evidence is mainly from observational studies rather than the gold standards of randomized controlled or interventional trials with no comparators, dwarfing generated evidence and conclusions. Even upon review of studies examining shared concepts such as AMR or economic burden, there are clearly highlighted heterogeneity of studies, objectives and methodology and no consensus of agreed measurements making combining evidence synthesis a daunting task. These limitations were undoubtedly reflected on the quality of evidence extracted from SRs examined through AMSTAR critical appraisal tool which demonstrated uniformly low-quality standards. Furthermore, from reviewing the concept of ASPs, there are multiple co-dependent confounding factors that can affect implantations and outcome processes. For example, facilities related infection prevention and control programs practices can directly impact upon both implantation processes such as increased antimicrobial consumption or outcomes by spreading AMR. Additionally, local population characteristics, healthcare settings, resources, and facilities as well as professional practice culture have direct impact on ASPs.

19. Summary and Conclusions

The review of the SRs evaluating ASPs implementation process, impact, and outcome delineated that the concepts are strongly cemented in Western healthcare facilities but evolving in developing countries. The studied population is mainly adults from Western countries with limited data from children and infants. Implantation effectiveness process consists of multiple connected concepts ranging from guidelines, restricted antimicrobials as well as role of IT and pharmacists to influencing prescribers’ behaviors. For safety and efficacy, objective interventions has been met in all evaluating studies including vulnerable population such as children as well as patients at critical care. As for efficiency, evaluation of the economic impact of ASPs as a secondary outcome, demonstrated proven benefit mainly through reduction in expenditure of antimicrobial therapy as well as reduction of patients’ length of hospital stay. Although one of the fundamental aims of the ASPs is to reduce the mounting scale of AMR , evaluated evidence did not conclusively supported that objective opening the gates for conducting further research in that field with better designed studies . Similarly, there is lack of consensus regarding quality indicators that set the objectives for the program standards for future evaluation. Correspondingly, with the advances on reliance on IT in healthcare, there are promising but not sufficiently explored opportunities to expand that aspect for the better delivery of ASPs aimed objectives .