Evaluating Performance of the US Surveillance Systems for Monitoring Antimicrobial-Resistant Gonorrhea: An Agent-Based Modelling Study

1. Author Summary

A lot of effort has been made in the recent years to stop the spread of antimicrobial-resistant (AMR) gonorrhea. In the US, one of the initiatives was establishment of the enhanced Gonococcal Isolate Surveillance Project (eGISP) in 2017 which includes the non-urethral isolates in addition to the urethral ones utilized in the original Gonococcal Isolate Surveillance Project (GISP). However, the actual positive impact of this new initiative remained unknown. We evaluated the performance of both surveillance systems using the novel simulation model of gonorrhea transmission among the US male homosexual population. Our results demonstrate that eGISP system is more accurate and more effective for stopping the spread of AMR gonorrhea than the original GISP. Also, it was determined that the type of isolates included in the surveillance plays more important role than the number of isolates used, and that rectum and pharynx contribute to the spread of resistance significantly.

2. Introduction

The rise of AMR gonorrhea is an emerging problem in many countries. The N. gonorrhoeae bacteria has already developed resistance to many classes of antibiotics used for its treatment [1]. Some of the reasons for this growth are the inappropriate use of antibiotics, mutations and low quality of the antibiotics used. The infections outside of the genital area are also believed to contribute to it as those infections are mainly asymptomatic and the bacteria exchanges genetic material with other organisms in these parts of the body [2,3].

Untreated gonorrhea can lead to a number of health problems, such as neonatal eye infections, infertility, ectopic pregnancy, increased ability to give and receive HIV etc. [4] Also, this growth of resistance threatens our ability to treat other infections which are being cured with the same antibiotics. However, the development of new antibiotics has been rather slow over the recent years as it is not profitable for pharmaceutical companies [5]. There are few antibiotics in clinical trials [6], but none of them has reached the market yet. Therefore, accurate surveillance of AMR gonorrhea is essential as it allows to make right decisions at the right time regarding switching to a different antibiotic so that the spread is prevented but also that the precious antibiotics are not introduced earlier than necessary.

In the US, the Gonococcal Isolate Surveillance Project (GISP) is a sentinel surveillance system to monitor the trend in AMR gonorrhea. GISP estimates the percentage of cases which are resistant to different antibiotics used for the treatment of gonorrhea. Once that value for a current first-line drug reaches 5% (the switching time recommended by WHO), a decision to switch to a different antibiotic is being made.

Until 2017, only isolates from the first 25 men who were diagnosed with urethral gonorrhea in a number of sites were included [7]. In 2017, eGISP was established which also included pharyngeal, rectal, and endocervical isolates. This was done in order to understand whether rectum and pharynx can serve as anatomic niches that foster resistance and whether the antibiotic susceptibility pattern differ between men and women [8]. Since then, a number of sites of each systems has been established each year [9]. In particular, in 2017 there were 27 GISP and 12 eGISP sites [9]; 33 GISP and 9 eGISP sites in 2021 [8]; and 31 GISP and 9 eGISP sites in 2023 [10]. So, despite the potential of eGISP to improve the monitoring of AMR gonorrhea, not many sites have been funded over the years and their number has not been increasing. Also, there are states with low prevalence of gonorrhea where no single GISP or eGISP site has been funded [10]. This also affects the monitoring of the spread of AMR gonorrhea negatively.

We aim to evaluate to which extend the inclusion of the rectal and pharyngeal isolates has improved surveillance of AMR gonorrhea. In order to do this, we modelled the transmission of gonorrhea among the US MSM population as well as the two surveillance systems and evaluated their performance.

3. Methods

3.1. Model Description

We developed a continuous-time agent-based model of gonorrhea transmission among the US MSM, which accounts for susceptible and resistant strains of N. gonorrhoeae, symptomatic and asymptomatic infection and various routes of transmission between different anatomical sites.

It was assumed that all the contacts are casual, i.e. one-time, and that the number of people reaching the sexually active age equals to the number of people who leave this population group. Sexual practices included in the model are kissing, oral sex, anal sex, rimming and docking. As the result, the following eight transmission routes were modelled: pharynx to pharynx, pharynx to urethra, urethra to pharynx, urethra to rectum, rectum to urethra, pharynx to rectum, rectum to pharynx and urethra to urethra.

Infection can occur at one site or it can occur at two sites at the same time. Urethral infection can be both symptomatic and asymptomatic, while rectal and pharyngeal infection was assumed to be always asymptomatic as symptomatic cases are rare [3]. Individuals can get infected with susceptible or resistant strain of gonorrhea. In case of a multi-site infection, the strains can be the same or different at two sites.

Asymptomatic individuals can either get detected during the screening at healthcare facilities and receive the first-line treatment (ceftriaxone), or they can recover naturally. The treatment fails if individuals are infected with resistant strain of gonorrhea. Symptomatic individuals seek the first-line treatment at healthcare facilities. If case of the treatment failure due to the strain being resistant, these individuals are being re-treated with the second-line drug (ertapenem). In some cases, the infection can develop resistance to the first-line drug while an individual infected with susceptible strain is being under treatment which results in the treatment failure.

The algorithm of the model can be found in the supplementary material. The model was developed using the simulation modelling tool AnyLogic (version 8.8.1 University) which is Java-based and provides the graphical interface which significantly simplifies the development of simulation models.

3.2. Calibration

The model was calibrated using the Bayesian calibration approach. Prevalence of gonorrhea among the US MSM at different anatomical sites, prevalence of gonorrhea resistant to ceftriaxone among the US MSM at different anatomical sites and incidence of gonorrhea (the reported rate of gonorrhea cases per 100,000 US MSM population) were used as the calibration targets.

The current first-line gonorrhea treatment is ceftriaxone, which has become the first-line treatment in the US in 2007 [11]. However, until 2012 cefixime was also used [12]. We could not find any data which could give us an estimate of the percentage of cases which were treated with each drug during that time period. So, for the modelling purpose, we assumed that ceftriaxone was prescribed for each case. The available data on the prevalence of gonorrhea resistant to ceftriaxone in different anatomical sites [13] was based on the data collected from the beginning of 2018 until the end of 2019. Therefore, we initiated the model in 2007, assigned the initial prevalence of gonorrhea resistant to ceftriaxone to zero and ran the model for 12 years. Further in the text, gonorrhea resistant to ceftriaxone is referred simply as AMR gonorrhea. The details of the calibration procedure can be found in the supplementary material.

3.3. Surveillance

Since only a limited number of isolates is being tested for drug susceptibility each year, there is the estimation error. Therefore, we modelled a surveillance system as the sum of the percentage of diagnosed cases resistant to ceftriaxone and the estimation error which was modelled as:

$${\epsilon }_t~N(0,\sqrt{\frac{y_t(1-y_t)}{K}}),$$

where $K$ is the number of isolates collected in a year and $y_t$ is the percentage of diagnosed cases resistant to ceftriaxone in year t. The details of the estimation of K are provided in Section 3 of the supplementary material.

We ran the model for 35 years and recorded the year when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% and when that percentage is detected under GISP and eGISP surveillance systems. For each surveillance system, we calculated accuracy, sensitivity and specificity. The analysis was conducted in Python (version 3.11.5). The additional details can be found in the supplement.

3.4. Sensitivity Analysis

It was investigated whether the number of submitted isolates, the set of selected trajectories, the simulation duration and the percentage of isolates that came from the MSM attending eGISP clinics (used for computation of the number of isolates collected in a year ($K$)) affected the conclusions that we obtained. The details are provided in the supplementary material.

4. Results

The calibration results for 35 years of simulation (2007-2042) of the 100 selected trajectories are shown in Figure 1.

It is evident that all the calibration targets have been reached. Therefore, our model is capable of capturing the dynamics of gonorrhea transmission among the US MSM and can be used for conducting investigations. The posterior distributions of the parameters can be found in Tables C-E in the supplement. The histograms of the posterior distributions are shown in Figure A in the supplement.

Our results indicate that the percentage of diagnosed cases that are symptomatic fluctuates just slightly during the years of simulation. The results for the percentage of diagnosed cases resistant to ceftriaxone outline the variety of scenarios that can take place due to the numerous combinations of many factors. For some trajectories, the resistant remains low for many years, which is then followed by a sharp rise; for the other ones, it starts to rise fairly quickly, reaches high level and stabilizes; while for few of them, the resistance never develops (remains at very low level throughout the entire simulation).

We also simulated prevalence of gonorrhea among the US MSM at a single site and at a combination of two sites (Figure 2) as well as the number of patients seeking treatment per 100,000 US MSM population at different anatomical sites and the number of treatment failures per 100,000 US MSM population (Figure 3). The simulation results for prevalence of AMR gonorrhea among the US MSM at a single site and at a combination of two sites as well as the percentage of detected cases among the US MSM at different anatomical sites are provided in Figures B and C in the supplementary material.

The highest prevalence of gonorrhea was observed at rectum only, followed by prevalence at pharynx only and then by prevalence at urethra only. As expected, prevalence of gonorrhea at urethra and some other site was very low (between 0.01% and 1%) due to the fact that urethral infection is mainly symptomatic [19,20,21] and is being treated promptly.

Our results indicate that, starting from some point, much more patients seek treatment for infection at rectum and urethra than for infection at pharynx and urethra. From Figure 2 it is evident that prevalence at rectum only also grows much faster starting from some point than prevalence at pharynx only. Since urethral infection is mainly simptomatic, it is expectable that people who have infection at rectum and urethra are being diagnosed and treated, so prevalence at rectum and urethra and prevalence in pharynx and urethra remains at about the same very low level (Figure 2). As expected, the number of treatment failures is raising over the years as the resistance to ceftriaxone develops for most of the trajectories (except for the scenarios where the resistance does not develop).

The results for comparison of performance of GISP and eGISP surveillance systems are presented in Figure 4.

Our results show that the accuracy of GISP is 87.9% (55%, 100%), while for eGISP it is 97.8% (83.5%, 100%). The specificity of the enhanced surveillance system is 11% lower than the specificity of the original one (89% vs. 100%). GISP demonstrates a low sensitivity of 34%, while for eGISP it is as high as 94%. When GISP does not determine the year of switch correctly, it is always later than the actual moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5%. This results in additional infections of AMR gonorrhea and contributes to the spread of the resistance. eGISP determines the year of switch later than the actual moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% in about half of the cases of incorrect detections. When the opposite happens, this results in a slightly reduced antibiotic lifespan. But mostly eGISP determines the year of switch correctly.

Through the sensitivity analysis (Figure D – Figure G in the supplement) we have shown that out conclusions are not sensitive to the number of submitted isolates, the set of simulated trajectories used to calibrate the model, the simulation duration and the percentage of isolates that came from the MSM attending the eGISP clinics.

5. Discussion

This study evaluates and compares performance of GISP and eGISP surveillance systems. Our results demonstrate that incorporation of the rectal and pharyngeal isolates has increased the accuracy of monitoring the spread of AMR gonorrhea among the US MSM population by nearly 10%. Also, eGISP has significantly higher sensitivity than GISP. Most of time, the original surveillance system detects the moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% later than it is actually reached which leads to the additional spread of AMR gonorrhea due to the continued prescription of an antibiotic which is already meant to be replaced. Through the sensitivity analysis, by varying the number of isolates, we have shown that including the extragenital isolates is more important for accurate monitoring of the spread than using a large number of isolates. These findings support the assumption made in the recent years that rectum and pharynx play an important role in the growth of antimicrobial resistance [2,3].

To the best of our knowledge, this work is the first agent-based study of gonorrhea transmission where all the parameters were found through the calibration. This can provide the new insights. Also, our simulation results can shed some light on the prevalence of gonorrhea and prevalence of AMR gonorrhea at different anatomical sites (as well as at their combination) and on the number of patients seeking treatment for infection at different sites (as well as at their combination).

These findings should be interpreted in the context of a number of limitations. In particular, ceftriaxone started to be used as the first-line treatment of gonorrhea in the US in 2007 [11]. But until 2012 cefixime was also used [12], so the results that we obtained for the percentage of diagnosed cases resistant to ceftriaxone are slightly higher than the actual ones. They are also expected to be slightly higher due to the fact that for the AMR calibration targets we used the data from the STD clinics (as they were the only ones available) which tend to overestimate prevalence [22,23].

In addition, the formation of partnerships and condom usage has not been included in the model. However, since MSM population in general has high rates of partner numbers and high percentage (80%) of casual partners [24], the current assumption of all the contacts being casual is expected to reflect the reality well. Also, a rare situation when a person develops infection in all three sites [3] or a rectal or pharyngeal infection being symptomatic has not been accounted for. While conducting this research, we assumed a full adherence to the guidelines for the first-line treatment. The recent data indicate that around 80% of male patients received the recommended first-line antibiotic therapy [25]. However, all the assumptions mentioned above are not expected to have major impact on the conclusions, especially, given the comparative nature of our research.

There are few additional points to consider when a decision on the number of sites to establish is being made. In the recent years, there were the cases of “super gonorrhea”, when urethral symptoms have disappeared after the treatment, but the pharyngeal specimen remained positive to Neisseria gonorrhoeae and the resistance to the first-line treatment used has been indicated [26]. In case of a clinic participating in eGISP instead of GISP and the pharyngeal infection being asymptomatic, those cases of great concern would be more likely identified and treated. When it comes to the non-urethral isolates, currently many isolate collection attempts fail due to a number of factors such as poor viability, growth problems or transport problems [9]. However, technical assistance with improving culture yields can resolve this problem [9].

The outcomes of this work can be used by the policymakers who decide how many GISP and eGISP sites to allocate each year, which would improve the monitoring of the spread of AMR gonorrhea in the US and assist in the global fight against the growing resistance of N. gonorrhoeae.