Prerpints.org logo
Preprint
Article

This version is not peer-reviewed.

Evaluating Efficiency of the US Surveillance Systems for Monitoring Antimicrobial-Resistant Gonorrhea: An Agent-Based Modelling Study

Submitted:

12 October 2023

Posted:

12 October 2023

Read the latest preprint version here

Abstract
We aim to evaluate efficiency of two American surveillance systems for monitoring the spread of antimicrobial-resistant (AMR) gonorrhea among men who have sex with men (MSM) using the novel continuous-time agent-based model of gonorrhea transmission. The model was developed using the simulation modelling tool AnyLogic and accounts for susceptible and resistant strains of N. gonorrhoeae, symptomatic and asymptomatic infection and various routes of transmission between different anatomical sites. The model was calibrated using a Bayesian calibration approach. The surveillance systems are the Gonococcal Isolate Surveillance Project (GISP) and the enhanced Gonococcal Isolate Surveillance Project (eGISP). We calculated accuracy, sensitivity, specificity and estimation error for each surveillance system based on the number of isolates submitted in 2018. We also varied that number to see its effect on the outcomes. Our results show that the accuracy of eGISP was between 66% and 92%, while GISP demonstrates low accuracy between 44% and 48%. We also determined that increasing the number of isolates results in improved performance for eGISP, while GISP is not particularly sensitive to it.
Keywords: 
Agent-based modelling
;  
antimicrobial-resistant gonorrhea
;  
surveillance systems
Subject: 
Computer Science and Mathematics  -   Mathematical and Computational Biology

Introduction

The rise of AMR gonorrhea is an emerging problem in many countries. The N. gonorrhoeae bacteria has already developed resistance to many classes of antibiotics used for its treatment [1]. Some of the reasons for this growth are the inappropriate use of antibiotics, mutations and low quality of the antibiotics used. The infections outside of the genital area are also believed to contribute to it as those infections are mainly asymptomatic and the bacteria exchanges genetic material with other organisms in these parts of the body [2,3].
Untreated gonorrhea can lead to a number of health problems, such as neonatal eye infections, infertility, ectopic pregnancy, increased ability to give and receive HIV etc. [4] Also, this growth of resistance threatens our ability to treat other infections which are being cured with the same antibiotics. However, the development of new antibiotics has been rather slow over the recent years as it is not profitable for pharmaceutical companies [5]. There are few antibiotics in clinical trials [6], but none of them has reached the market yet. Therefore, accurate surveillance of AMR gonorrhea is essential as it allows to make right decisions at the right time regarding switching to a different antibiotic so that the spread is prevented but also that the precious antibiotics are not introduced earlier than necessary.
In the US, the Gonococcal Isolate Surveillance Project (GISP) is a sentinel surveillance system to monitor the trend in AMR gonorrhea. GISP estimates the percentage of cases which are resistant to different antibiotics used for the treatment of gonorrhea. Until 2017, only isolates from the first 25 men who were diagnosed with urethral gonorrhea in a number of sites were included [7]. In 2017, GISP was expanded to also include pharyngeal, rectal, and endocervical isolates. The impact of these changes on the accuracy of monitoring the spread of AMR gonorrhea has not been studied yet.
In this study we investigate the efficiency of GISP and enhanced GISP (eGISP) in monitoring this spread among men who have sex with men (MSM) in the US. In order to do this, we simulated the original surveillance system and the improved version and recorded the moment when the percentage resistant reaches 5% (the switching time recommended by WHO) with the real-life situation. We then calculated accuracy, sensitivity, specificity and estimation error. Since we limit our research to the US MSM population, only urethral, rectal and pharyngeal estimates were included in the simulation of eGISP. We conduct this research using the novel agent-based model of gonorrhea transmission.

Methods

  • Model description
This is a continuous-time agent-based model of gonorrhea transmission among the US MSM, which accounts for susceptible and resistant strains of N. gonorrhoeae, symptomatic and asymptomatic infection as well as various routes of transmission between different anatomical sites. The model was developed in AnyLogic (version 8.8.1 University) which is computationally fast and has a number of other advantages. The model can be viewed at the AnyLogic Cloud directory1.
Sexual practices included in the model are kissing, oral sex, anal sex, rimming and docking. As the result, the following eight transmission routes were modelled: pharynx to pharynx, pharynx to urethra, urethra to pharynx, pharynx to rectum, rectum to pharynx, urethra to rectum, rectum to urethra, and urethra to urethra.
It was assumed that all the contacts are casual. Individuals can get infected in pharynx, rectum, urethra or they can get infected in two sites. A situation when a person develops infection in all three sites is rare [8] and has not been accounted for so far. Rectal and pharyngeal infection is mainly asymptomatic [3], so we only include these cases in the current version of the model. However, in case of urethral infection both symptomatic and asymptomatic cases are included. Individuals can get infected with susceptible or resistant strain of gonorrhea. In case of infection in multiple sites, individuals can get infected with the same strain or with different strains in two sites.
In case of asymptomatic infection, infected individuals get detected during the screening and receive treatment, or they recover naturally. In case of symptomatic infection, infected individuals seek treatment. If symptomatic individuals infected with resistant strain receive the first-line treatment (ceftriaxone) and their symptoms persist, they are being re-treated with the second-line antibiotic. When asymptomatic individuals infected with resistant strain receive the first-line treatment, they remain infectious. Individuals can also develop resistance while being under treatment.
Partnership formation, condom usage and aging of population have not been included in the model so far.
The model was calibrated using a Bayesian calibration approach against the estimates of total prevalence of gonorrhea at each site (rectum, pharynx and urethra), prevalence of gonorrhea at each site only, prevalence of AMR gonorrhea at each site, incidence of gonorrhea and the number of symptomatic cases.
The Initialization algorithm and transmission and recovery algorithm can be found in the supplementary material.
  • Calibration
The model was calibrated using the Bayesian calibration approach. A total of 8000 runs were performed.
The simulations were stopped if any of the conditions below were met:
  • Incidence of gonorrhea is less than 2000 or greater than 15000.
  • Prevalence of rectal gonorrhea is greater than 12%.
  • Prevalence of pharyngeal gonorrhea is greater than 12%.
  • Prevalence of urethral gonorrhea is greater than 12%.
  • Prevalence of gonorrhea at rectum only is greater than 10%.
  • Prevalence of gonorrhea at pharynx only is greater than 12%.
  • Prevalence of gonorrhea at urethra only is greater than 10%.
  • Prevalence of rectal AMR gonorrhea is greater than 2%.
  • Prevalence of pharyngeal AMR gonorrhea is greater than 2%.
  • Prevalence of urethral AMR gonorrhea is greater than 2%.
  • Percentage of symptomatic cases is less than 50%.
Ceftriaxone started to be used for the treatment of gonorrhea in the US in 2007. Since 2012, the combination of ceftriaxone and azithromycin was the only recommended treatment by CDC [17]. The clinical trial on monitoring AMR gonorrhea [18] was run from the beginning of 2018 until the end of 2019. Therefore, we initiated the model in 2007, assigned the initial prevalence of AMR gonorrhea to zero and ran the model for 12 years. However, until 2012 cefixime was also used, so the results that we obtained for the percentage of resistant cases are higher than the actual ones.
For each simulated trajectory, eleven functions (likelihoods) were computed according to the formula:
L i = m i ln ⁡ p i + ( M i − m i ) ln ⁡ 1 − p i ,
i = 1 , . . 11 .
The parameters for each calibration target are provided in Table 1. Most of the targets were informed by [12] as it is the only study that we found that provides prevalence data on a single side. Those data were obtained from the STD clinic and the gay men’s community health center. For the first two targets we combined those data with the data from a study which tested MSM attending community venues [19] for a better accuracy as trials based on data from STD clinics and health centers tend to overestimate prevalence.
For data that we have for incidence of gonorrhea no confidence interval was reported. We assumed that it was calculated with 20% error and used the approach described in [22] to calculate the values of M and m.
After that, we calculated the function L T o t a l as:
L T o t a l = ∑ i = 1 7 L i .
Then we selected one hundred trajectories for which the value of L T o t a l was the highest.
  • Surveillance
The surveillance system estimates the percentage of cases which are resistant to the antibiotics used for the treatment of gonorrhea. Once that value reaches 5%, a decision to switch to a different antibiotic is being made. We model surveillance of resistance to the current first-line antibiotic (ceftriaxone).
Since only a certain percentage of cases is included in the monitoring, there is an estimation error:
ε t ~ N ( 0 ,   y t ( 1 − y t ) K ) ,
where K is the number of isolates and y t is the percentage of diagnosed cases resistant to ceftriaxone at year t (percentage resistant).
In 2018, a total of 8628 isolates were submitted [23]. We used this data as this is the most recent ones we were aware of. Based on the number of isolates provided by GISP and eGISP participating clinics [7], we assumed that 0.25% of those 8628 isolates came from GISP and the remaining 0.75 came from eGISP. In [24], the data on the percentage of the urethral isolates that come from MSM attending the STD clinics participating in GISP are provided (about 37% in 2018). We assumed that the percentage of isolates that come from MSM attending the clinics participating in eGISP is the same. Therefore, we assumed that 2394 isolates were used in eGISP and 796 isolates were used in GISP.
A surveillance system was modeled as the sum of the percentage resistant and the estimation error. We recorded the year when the 5% percentage resistant is detected under GISP and eGISP surveillance systems. We also recorded when the percentage resistant reaches 5% in real life.
After that we compare the accuracy of the two surveillance systems. For each trajectory, we calculated it as number of correct predictions divided by total number of predictions.
Also, for each trajectory, we computed the estimation error as:
E = | t s - t r | ,
where t r is the year when the percentage resistant reached 5% in real life and t s is the year when the 5% percentage resistant is detected under a surveillance system. Then the average estimation error was computed.
We also calculated sensitivity (the probability that when the percentage of resistant cases passes 5% a surveillance system correctly detects that) and specificity (the probability that if the percentage of resistant cases is below 5% a surveillance system correctly informs that) for both systems.

Results

1. 
Calibration results
The results for 25 years of simulation of the selected 100 trajectories along with the calibration targets and confidence intervals are presented in Figure 1.
In Figure 2, percentage of detected cases in different anatomical sites is shown.
The histograms of the posterior distributions are presented in the supplementary material.
2. 
Surveillance results
We calculated the accuracy of GISP and eGISP surveillance systems using 8628 isolates. We ran the simulations for 25 years. For the analysis, we only used the trajectories for which the percentage resistant was reached both in real life and under both surveillance systems. The results are shown in Table 2.
The accuracy of eGISP is 41% higher than the accuracy of GISP.
We also varied the number of samples to see its effect. In particular, we simulated the situation when 1000 isolates and 15000 isolates were submitted. The results are presented in the Table 3 and Table 4. In case the percentage resistant was not reached for some of the trajectories used for the analysis under GISP or eGISP, the longer simulations (e.g. 35 years) were run.
The results are presented graphically in the Figure 3.
As expected, including isolates from the non-urethral sites has increased the accuracy, sensitivity and specificity of the original GISP surveillance system and reduced the estimation error.
Increasing the number of isolates improves all the criteria for eGISP. Changing that number from 1000 to 8628 has increased the accuracy by 22%, while increasing it further to 15000 has resulted in just 3% increase. So, the number of estimates used in 2018 appear reasonable.
However, for the original GISP we can see a different trend. Initially, the accuracy increased, but later it decreased again. It remains between 44% and 48% regardless of the number of samples used. The estimation error declines though as the number of isolates increases. Sensitivity decreases slightly as we increase the number of samples, while specificity increases and remains at 97% level for 8628 and 15000 isolates.

Discussion

In this paper we presented a novel agent-based model of gonorrhea transmission among the US MSM population, which was employed to evaluate the performance of several American surveillance systems for monitoring the spread of AMR gonorrhea. Our results show that incorporation of the non-urethral isolates increases the accuracy of monitoring the spread of AMR gonorrhea among the US MSM population. The accuracy of eGISP ranges between 66% and 92%, while for GISP it is between 44% and 48%. We also determined that increasing the number of estimates results in improving all the criteria for eGISP, while for GISP there is no particular trend.
While conducting this research, we assumed a full adherence to the guidelines for the first-line treatment, which might not always be the case. Also, we could not find the actual data on the percentage of isolates that come from MSM attending the clinics participating in eGISP. In addition, in the current version of our model all the contacts were assumed to be casual. Incorporating formation of partnerships as well as condom usage and aging of population would improve the accuracy of forecasting. In terms of future research directions, it would be interesting to evaluate the performance of the two surveillance systems also taking into account the costs involved.

Acknowledgment

A special thanks goes to Dr. Reza Yaesoubi from Health Policy and Management, Yale University for his helpful advises.

References

  1. A Dutescu, I.; A Hillier, S. Encouraging the Development of New Antibiotics: Are Financial Incentives the Right Way Forward? A Systematic Review and Case Study. Infect. Drug Resist. 2021, 14, 415–434. [Google Scholar] [CrossRef]
  2. D. Lewis, Will targeting oropharyngeal gonorrhoea delay the further emergence of drug-resistant Neisseria gonorrhoeae strains? Sexually transmitted infections 2015, 91, 234–237. [Google Scholar] [CrossRef]
  3. P. A. Chan et al. Extragenital infections caused by Chlamydia trachomatis and Neisseria gonorrhoeae: a review of the literature. Infectious diseases in obstetrics and gynecology 2016, 2016. [Google Scholar]
  4. J. C. Dombrowski Chlamydia and gonorrhea. Annals of Internal Medicine 2021, 174, ITC145–ITC160. [Google Scholar] [CrossRef]
  5. Alanis, A.J. Resistance to Antibiotics: Are We in the Post-Antibiotic Era? Arch. Med. Res. 2005, 36, 697–705. [Google Scholar] [CrossRef]
  6. Butler, M.S.; Henderson, I.R.; Capon, R.J.; Blaskovich, M.A.T. Antibiotics in the clinical pipeline as of December 2022. J. Antibiot. 2023, 76, 431–473. [Google Scholar] [CrossRef] [PubMed]
  7. C. f. D. C. a. Prevention, Gonococcal Isolate Surveilance Project (GISP) and Enhanced GISP (eGISP), 2022.
  8. Spicknall, I.H.; Mayer, K.H.; Aral, S.O.; Romero-Severson, E.O. Assessing Uncertainty in an Anatomical Site-Specific Gonorrhea Transmission Model of Men Who Have Sex With Men. Sex. Transm. Dis. 2019, 46, 321–328. [Google Scholar] [CrossRef] [PubMed]
  9. Donovan, L.C.; Fairley, C.K.; Aung, E.T.; Traeger, M.W.; Wright, E.J.; A Stoové, M.; Chow, E.P.F. The Presence or Absence of Symptoms Among Cases of Urethral Gonorrhoea Occurring in a Cohort of Men Taking Human Immunodeficiency Virus Pre-exposure Prophylaxis in the PrEPX Study. Open Forum Infect. Dis. 2021, 8, ofab263. [Google Scholar] [CrossRef] [PubMed]
  10. Ong, J.; Fethers, K.; Howden, B.; Fairley, C.; Chow, E.; Williamson, D.; Petalotis, I.; Aung, E.; Kanhutu, K.; De Petra, V.; et al. Asymptomatic and symptomatic urethral gonorrhoea in men who have sex with men attending a sexual health service. Clin. Microbiol. Infect. 2017, 23, 555–559. [Google Scholar] [CrossRef]
  11. Martín-Sánchez, M.; Ong, J.J.; Fairley, C.K.; Chen, M.Y.; Williamson, D.A.; Maddaford, K.; Aung, E.T.; Carter, G.; Bradshaw, C.S.; Chow, E.P.F. Clinical presentation of asymptomatic and symptomatic heterosexual men who tested positive for urethral gonorrhoea at a sexual health clinic in Melbourne, Australia. BMC Infect. Dis. 2020, 20, 1–9. [Google Scholar] [CrossRef]
  12. Kent, C.K.; Chaw, J.K.; Wong, W.; Liska, S.; Gibson, S.; Hubbard, G.; Klausner, J.D. Prevalence of Rectal, Urethral, and Pharyngeal Chlamydia and Gonorrhea Detected in 2 Clinical Settings among Men Who Have Sex with Men: San Francisco, California, 2003. Clin. Infect. Dis. 2005, 41, 67–74. [Google Scholar] [CrossRef]
  13. H. H. Handsfield et al., Asymptomatic gonorrhea in men: diagnosis, natural course, prevalence and significance. New England Journal of Medicine 1974, 290, 117–123. [Google Scholar] [CrossRef]
  14. K. M. Wall, R. Stephenson, and P. S. Sullivan Frequency of sexual activity with most recent male partner among young, Internet-using men who have sex with men in the United States. Journal of Homosexuality 2013, 60, 1520–1538. [Google Scholar] [CrossRef]
  15. Duan, Q.; Carmody, C.; Donovan, B.; Guy, R.J.; Hui, B.B.; Kaldor, J.M.; Lahra, M.M.; Law, M.G.; Lewis, D.A.; Maley, M.; et al. Modelling response strategies for controlling gonorrhoea outbreaks in men who have sex with men in Australia. PLOS Comput. Biol. 2021, 17, e1009385. [Google Scholar] [CrossRef]
  16. https://www.nhs.uk/conditions/gonorrhoea/treatment/#:~:text=It's%20sometimes%20possible%20to%20have,or%20testicles%20to%20disappear%20completely.
  17. S. S. Cyr et al., Update to CDC's treatment guidelines for gonococcal infection, 2020. Morbidity and Mortality Weekly Report 2020, 69, 1911. [CrossRef]
  18. Quilter, L.A.S.; Cyr, S.B.S.; Hong, J.; Asbel, L.; Bautista, I.; Carter, B.M. (.; Casimir, Y.A.; Denny, M.; Ervin, M.M.; Gomez, R.M.; et al. Antimicrobial Susceptibility of Urogenital and Extragenital Neisseria gonorrhoeae Isolates Among Men Who Have Sex With Men: Strengthening the US Response to Resistant Gonorrhea and Enhanced Gonococcal Isolate Surveillance Project, 2018 to 2019. Sex. Transm. Dis. 2021, 48, S111–S117. [Google Scholar] [CrossRef]
  19. M. L. J. Jones et al., Extragenital chlamydia and gonorrhea among community venue–attending men who have sex with men—five cities, United States, 2017. Morbidity and Mortality Weekly Report 2019, 68, 321. [Google Scholar] [CrossRef] [PubMed]
  20. Newman, L.M.; Dowell, D.; Bernstein, K.; Donnelly, J.; Martins, S.; Stenger, M.; Stover, J.; Weinstock, H. A Tale of Two Gonorrhea Epidemics: Results from the STD Surveillance Network. Public Heal. Rep. 2012, 127, 282–292. [Google Scholar] [CrossRef] [PubMed]
  21. C. f. D. C. a. Prevention, Sexually Transmitted Disease Surveillance, 2017.
  22. Yaesoubi, R.; Cohen, T.; Hsu, K.; Gift, T.L.; Cyr, S.B.S.; Salomon, J.A.; Grad, Y.H. Evaluating spatially adaptive guidelines for the treatment of gonorrhea to reduce the incidence of gonococcal infection and increase the effective lifespan of antibiotics. PLOS Comput. Biol. 2022, 18, e1009842. [Google Scholar] [CrossRef] [PubMed]
  23. Kersh, E.N.; Pham, C.D.; Papp, J.R.; Myers, R.; Steece, R.; Kubin, G.; Gautom, R.; Nash, E.E.; Sharpe, S.; Gernert, K.M.; et al. Expanding U.S. Laboratory Capacity for Neisseria gonorrhoeae Antimicrobial Susceptibility Testing and Whole-Genome Sequencing through the CDC's Antibiotic Resistance Laboratory Network. J. Clin. Microbiol. 2020, 58. [Google Scholar] [CrossRef]
  24. C. f. D. C. a. Prevention, Sexually Transmitted Disease Surveillance 2020, 2020.
  25. V. B. Bowen et al., “Sexually transmitted disease surveillance 2018,” 2019.
  26. Earnest, R.; Rönn, M.M.; Bellerose, M.B.; Gift, T.L.; Berruti, A.A.; Hsu, K.K.; Testa, C.B.; Zhu, L.; Malyuta, Y.; Menzies, N.A.; et al. Population-level Benefits of Extragenital Gonorrhea Screening Among Men Who Have Sex With Men: An Exploratory Modeling Analysis. Sex. Transm. Dis. 2020, 47, 484–490. [Google Scholar] [CrossRef] [PubMed]
1
https://cloud.anylogic.com/model/bb6b7cfa-880b-4345-8913-f21d5e6d4ec6?mode=SETTINGS&tab=GENERAL
Preprints 87630 g001
Figure 1. The results for 25 years of simulation of the selected 100 trajectories along with the calibration targets and confidence intervals.
Figure 1. The results for 25 years of simulation of the selected 100 trajectories along with the calibration targets and confidence intervals.
Preprints 87630 g001
Preprints 87630 g002
Figure 2. Percentage of detected cases in different anatomical sites.
Figure 2. Percentage of detected cases in different anatomical sites.
Preprints 87630 g002
Preprints 87630 g003
Figure 3. Evaluating efficiency of GISP and eGISP for different number of isolates.
Figure 3. Evaluating efficiency of GISP and eGISP for different number of isolates.
Preprints 87630 g003
Table 1. Calibration parameters.
Table 1. Calibration parameters.
p M, m Data Source
Prevalence of rectal gonorrhea 5458, 223 6.49% [5.9%, 7.6%] [12,19]
Prevalence of pharyngeal gonorrhea 5507, 309 9% [8.1%; 10%] [12,19]
Prevalence of urethral gonorrhea 3435, 148 4.31% [3.7%; 5%] [12]
Prevalence of gonorrhea at rectum only 3435, 121 3.52% [3%; 4.2%] [12]
Prevalence of gonorrhea at pharynx only 3435, 209 6% [5.3%; 6.9%] [12]
Prevalence of gonorrhea at urethra only 3435, 86 2.5% [2%; 3.1%] [12]
Prevalence of rectal AMR gonorrhea 1553, 2 0.13% [0.03–0.51] [18]
Prevalence of pharyngeal AMR gonorrhea 1049, 7 0.67% [0.32–1.4] [18]
Prevalence of urethral AMR 3974, 7 0.18% [0.08–0.37] [18]
Percentage of symptomatic cases 686, 466 67.9% [64.4%, 71.4%] [20]
Incidence of gonorrhea 1736, 91 5241.8 (4193, 6290) [21]
Table 2. Accuracy of GISP and eGISP surveillance systems when 8628 isolates were submitted.
Table 2. Accuracy of GISP and eGISP surveillance systems when 8628 isolates were submitted.
Simulation experiment Average time when percentage resistant reaches 5% (years) Average time when 5% percentage resistant is detected (years) Estimation error (years) Accuracy (%) Sensitivity (%) Specificity
(%)
Real life 15.4 (8.1, 24)
eGISP 15.3 (7, 24) 0.16 (0, 2) 89 96.5 92
GISP 16.24 (8.1, 24) 0.85 (0, 3.4) 48 48 97
Table 3. Accuracy of GISP and eGISP surveillance systems when 1000 isolates were submitted.
Table 3. Accuracy of GISP and eGISP surveillance systems when 1000 isolates were submitted.
Simulation experiment Average time when percentage resistant reaches 5% (years) Average time when 5% percentage resistant is detected (years) Estimation error (years) Accuracy (%) Sensitivity (%) Specificity
(%)
Real life 15.4 (8.1, 24)
eGISP 15.35 (7, 24) 0.43 (0, 2) 66 80.7 79
GISP 16 (7, 24.4) 0.94 (0, 3.4) 44 49 82
Table 4. Accuracy of GISP and eGISP surveillance systems when 15000 isolates were submitted.
Table 4. Accuracy of GISP and eGISP surveillance systems when 15000 isolates were submitted.
Simulation experiment Average time when percentage resistant reaches 5% (years) Average time when 5% percentage resistant is detected (years) Estimation error (years) Accuracy (%) Sensitivity (%) Specificity
(%)
Real life 15.4 (8.1, 24)
eGISP 15.38 (7, 24) 0.11 (0, 1.4) 92 97 95
GISP 16.2 (8.1, 24) 0.8 (0, 3.4) 46 47 97
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
bsky
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

Privacy Settings

© 2025 MDPI (Basel, Switzerland) unless otherwise stated