Prerpints.org logo
Preprint
Article

Evaluating Performance of the US Surveillance Systems for Monitoring Antimicrobial-Resistant Gonorrhea: An Agent-Based Modelling Study

This version is not peer-reviewed.

Submitted:

16 May 2024

Posted:

17 May 2024

Read the latest preprint version here

Abstract
In this study we evaluate performance of two American surveillance systems for monitoring the spread of antimicrobial-resistant (AMR) gonorrhea: the Gonococcal Isolate Surveillance Project (GISP) and the enhanced Gonococcal Isolate Surveillance Project (eGISP) which includes the non-urethral isolates in addition to the urethral ones utilized in the original surveillance system. A continuous-time agent-based model of gonorrhea transmission among the US men who have sex with men (MSM) population was developed and used for this purpose. Our results show that the accuracy of eGISP was nearly 10% higher than the accuracy of GISP (97.8% (95% uncertainty interval: 83.5%, 100%) vs. 87.9% (55%, 100%)). Most of time, GISP detects the moment when the resistance to the first-line antibiotic reaches the switch threshold recommended by the WHO later than when it was actually reached. This leads to the additional spread of AMR gonorrhea. We also found that including the extragenital isolates in the surveillance is more important for the accurate monitoring than increasing the number of isolates. Finally, our results demonstrate that rectum and pharynx can serve as a niche for fostering antimicrobial resistance. These findings provide a scientific basis which can be used to inform the decision made by the policymakers on how many GISP and eGISP sites to establish each year.
Keywords: 
agent-based modelling
;  
gonorrhea
;  
antimicrobial resistance
;  
surveillance systems
Subject: 
Computer Science and Mathematics  -   Mathematical and Computational Biology

supplementary.docx (696.93KB )

1. Author Summary

A lot of effort has been made in the recent years to stop the spread of antimicrobial-resistant (AMR) gonorrhea. In the US, one of the initiatives was establishment of the enhanced Gonococcal Isolate Surveillance Project (eGISP) in 2017 which includes the non-urethral isolates in addition to the urethral ones utilized in the original Gonococcal Isolate Surveillance Project (GISP). However, the actual positive impact of this new initiative remained unknown. We evaluated the performance of both surveillance systems using the novel simulation model of gonorrhea transmission among the US male homosexual population. Our results demonstrate that eGISP system is more accurate and more effective for stopping the spread of AMR gonorrhea than the original GISP. Also, it was determined that the type of isolates included in the surveillance plays more important role than the number of isolates used, and that rectum and pharynx contribute to the spread of resistance significantly.

2. Introduction

The rise of AMR gonorrhea is an emerging problem in many countries. The N. gonorrhoeae bacteria has already developed resistance to many classes of antibiotics used for its treatment [1]. Some of the reasons for this growth are the inappropriate use of antibiotics, mutations and low quality of the antibiotics used. The infections outside of the genital area are also believed to contribute to it as those infections are mainly asymptomatic and the bacteria exchanges genetic material with other organisms in these parts of the body [2,3].
Untreated gonorrhea can lead to a number of health problems, such as neonatal eye infections, infertility, ectopic pregnancy, increased ability to give and receive HIV etc. [4] Also, this growth of resistance threatens our ability to treat other infections which are being cured with the same antibiotics. However, the development of new antibiotics has been rather slow over the recent years as it is not profitable for pharmaceutical companies [5]. There are few antibiotics in clinical trials [6], but none of them has reached the market yet. Therefore, accurate surveillance of AMR gonorrhea is essential as it allows to make right decisions at the right time regarding switching to a different antibiotic so that the spread is prevented but also that the precious antibiotics are not introduced earlier than necessary.
In the US, the Gonococcal Isolate Surveillance Project (GISP) is a sentinel surveillance system to monitor the trend in AMR gonorrhea. GISP estimates the percentage of cases which are resistant to different antibiotics used for the treatment of gonorrhea. Once that value for a current first-line drug reaches 5% (the switching time recommended by WHO), a decision to switch to a different antibiotic is being made.
Until 2017, only isolates from the first 25 men who were diagnosed with urethral gonorrhea in a number of sites were included [7]. In 2017, eGISP was established which also included pharyngeal, rectal, and endocervical isolates. This was done in order to understand whether rectum and pharynx can serve as anatomic niches that foster resistance and whether the antibiotic susceptibility pattern differ between men and women [8]. Since then, a number of sites of each systems has been established each year [9]. In particular, in 2017 there were 27 GISP and 12 eGISP sites [9]; 33 GISP and 9 eGISP sites in 2021 [8]; and 31 GISP and 9 eGISP sites in 2023 [10]. So, despite the potential of eGISP to improve the monitoring of AMR gonorrhea, not many sites have been funded over the years and their number has not been increasing. Also, there are states with low prevalence of gonorrhea where no single GISP or eGISP site has been funded [10]. This also affects the monitoring of the spread of AMR gonorrhea negatively.
We aim to evaluate to which extend the inclusion of the rectal and pharyngeal isolates has improved surveillance of AMR gonorrhea. In order to do this, we modelled the transmission of gonorrhea among the US MSM population as well as the two surveillance systems and evaluated their performance.

3. Methods

3.1. Model Description

We developed a continuous-time agent-based model of gonorrhea transmission among the US MSM, which accounts for susceptible and resistant strains of N. gonorrhoeae, symptomatic and asymptomatic infection and various routes of transmission between different anatomical sites.
It was assumed that all the contacts are casual, i.e. one-time, and that the number of people reaching the sexually active age equals to the number of people who leave this population group. Sexual practices included in the model are kissing, oral sex, anal sex, rimming and docking. As the result, the following eight transmission routes were modelled: pharynx to pharynx, pharynx to urethra, urethra to pharynx, urethra to rectum, rectum to urethra, pharynx to rectum, rectum to pharynx and urethra to urethra.
Infection can occur at one site or it can occur at two sites at the same time. Urethral infection can be both symptomatic and asymptomatic, while rectal and pharyngeal infection was assumed to be always asymptomatic as symptomatic cases are rare [3]. Individuals can get infected with susceptible or resistant strain of gonorrhea. In case of a multi-site infection, the strains can be the same or different at two sites.
Asymptomatic individuals can either get detected during the screening at healthcare facilities and receive the first-line treatment (ceftriaxone), or they can recover naturally. The treatment fails if individuals are infected with resistant strain of gonorrhea. Symptomatic individuals seek the first-line treatment at healthcare facilities. If case of the treatment failure due to the strain being resistant, these individuals are being re-treated with the second-line drug (ertapenem). In some cases, the infection can develop resistance to the first-line drug while an individual infected with susceptible strain is being under treatment which results in the treatment failure.
The algorithm of the model can be found in the supplementary material. The model was developed using the simulation modelling tool AnyLogic (version 8.8.1 University) which is Java-based and provides the graphical interface which significantly simplifies the development of simulation models.

3.2. Calibration

The model was calibrated using the Bayesian calibration approach. Prevalence of gonorrhea among the US MSM at different anatomical sites, prevalence of gonorrhea resistant to ceftriaxone among the US MSM at different anatomical sites and incidence of gonorrhea (the reported rate of gonorrhea cases per 100,000 US MSM population) were used as the calibration targets.
The current first-line gonorrhea treatment is ceftriaxone, which has become the first-line treatment in the US in 2007 [11]. However, until 2012 cefixime was also used [12]. We could not find any data which could give us an estimate of the percentage of cases which were treated with each drug during that time period. So, for the modelling purpose, we assumed that ceftriaxone was prescribed for each case. The available data on the prevalence of gonorrhea resistant to ceftriaxone in different anatomical sites [13] was based on the data collected from the beginning of 2018 until the end of 2019. Therefore, we initiated the model in 2007, assigned the initial prevalence of gonorrhea resistant to ceftriaxone to zero and ran the model for 12 years. Further in the text, gonorrhea resistant to ceftriaxone is referred simply as AMR gonorrhea. The details of the calibration procedure can be found in the supplementary material.

3.3. Surveillance

Since only a limited number of isolates is being tested for drug susceptibility each year, there is the estimation error. Therefore, we modelled a surveillance system as the sum of the percentage of diagnosed cases resistant to ceftriaxone and the estimation error which was modelled as:
ε t ~ N ( 0 ,   y t ( 1 − y t ) K ) ,
where K is the number of isolates collected in a year and y t is the percentage of diagnosed cases resistant to ceftriaxone in year t. The details of the estimation of K are provided in Section 3 of the supplementary material.
We ran the model for 35 years and recorded the year when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% and when that percentage is detected under GISP and eGISP surveillance systems. For each surveillance system, we calculated accuracy, sensitivity and specificity. The analysis was conducted in Python (version 3.11.5). The additional details can be found in the supplement.

3.4. Sensitivity Analysis

It was investigated whether the number of submitted isolates, the set of selected trajectories, the simulation duration and the percentage of isolates that came from the MSM attending eGISP clinics (used for computation of the number of isolates collected in a year ( K )) affected the conclusions that we obtained. The details are provided in the supplementary material.

4. Results

The calibration results for 35 years of simulation (2007-2042) of the 100 selected trajectories are shown in Figure 1.
It is evident that all the calibration targets have been reached. Therefore, our model is capable of capturing the dynamics of gonorrhea transmission among the US MSM and can be used for conducting investigations. The posterior distributions of the parameters can be found in Tables C-E in the supplement. The histograms of the posterior distributions are shown in Figure A in the supplement.
Our results indicate that the percentage of diagnosed cases that are symptomatic fluctuates just slightly during the years of simulation. The results for the percentage of diagnosed cases resistant to ceftriaxone outline the variety of scenarios that can take place due to the numerous combinations of many factors. For some trajectories, the resistant remains low for many years, which is then followed by a sharp rise; for the other ones, it starts to rise fairly quickly, reaches high level and stabilizes; while for few of them, the resistance never develops (remains at very low level throughout the entire simulation).
We also simulated prevalence of gonorrhea among the US MSM at a single site and at a combination of two sites (Figure 2) as well as the number of patients seeking treatment per 100,000 US MSM population at different anatomical sites and the number of treatment failures per 100,000 US MSM population (Figure 3). The simulation results for prevalence of AMR gonorrhea among the US MSM at a single site and at a combination of two sites as well as the percentage of detected cases among the US MSM at different anatomical sites are provided in Figures B and C in the supplementary material.
The highest prevalence of gonorrhea was observed at rectum only, followed by prevalence at pharynx only and then by prevalence at urethra only. As expected, prevalence of gonorrhea at urethra and some other site was very low (between 0.01% and 1%) due to the fact that urethral infection is mainly symptomatic [19,20,21] and is being treated promptly.
Our results indicate that, starting from some point, much more patients seek treatment for infection at rectum and urethra than for infection at pharynx and urethra. From Figure 2 it is evident that prevalence at rectum only also grows much faster starting from some point than prevalence at pharynx only. Since urethral infection is mainly simptomatic, it is expectable that people who have infection at rectum and urethra are being diagnosed and treated, so prevalence at rectum and urethra and prevalence in pharynx and urethra remains at about the same very low level (Figure 2). As expected, the number of treatment failures is raising over the years as the resistance to ceftriaxone develops for most of the trajectories (except for the scenarios where the resistance does not develop).
The results for comparison of performance of GISP and eGISP surveillance systems are presented in Figure 4.
Our results show that the accuracy of GISP is 87.9% (55%, 100%), while for eGISP it is 97.8% (83.5%, 100%). The specificity of the enhanced surveillance system is 11% lower than the specificity of the original one (89% vs. 100%). GISP demonstrates a low sensitivity of 34%, while for eGISP it is as high as 94%. When GISP does not determine the year of switch correctly, it is always later than the actual moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5%. This results in additional infections of AMR gonorrhea and contributes to the spread of the resistance. eGISP determines the year of switch later than the actual moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% in about half of the cases of incorrect detections. When the opposite happens, this results in a slightly reduced antibiotic lifespan. But mostly eGISP determines the year of switch correctly.
Through the sensitivity analysis (Figure D – Figure G in the supplement) we have shown that out conclusions are not sensitive to the number of submitted isolates, the set of simulated trajectories used to calibrate the model, the simulation duration and the percentage of isolates that came from the MSM attending the eGISP clinics.

5. Discussion

This study evaluates and compares performance of GISP and eGISP surveillance systems. Our results demonstrate that incorporation of the rectal and pharyngeal isolates has increased the accuracy of monitoring the spread of AMR gonorrhea among the US MSM population by nearly 10%. Also, eGISP has significantly higher sensitivity than GISP. Most of time, the original surveillance system detects the moment when the percentage of diagnosed cases resistant to ceftriaxone reaches 5% later than it is actually reached which leads to the additional spread of AMR gonorrhea due to the continued prescription of an antibiotic which is already meant to be replaced. Through the sensitivity analysis, by varying the number of isolates, we have shown that including the extragenital isolates is more important for accurate monitoring of the spread than using a large number of isolates. These findings support the assumption made in the recent years that rectum and pharynx play an important role in the growth of antimicrobial resistance [2,3].
To the best of our knowledge, this work is the first agent-based study of gonorrhea transmission where all the parameters were found through the calibration. This can provide the new insights. Also, our simulation results can shed some light on the prevalence of gonorrhea and prevalence of AMR gonorrhea at different anatomical sites (as well as at their combination) and on the number of patients seeking treatment for infection at different sites (as well as at their combination).
These findings should be interpreted in the context of a number of limitations. In particular, ceftriaxone started to be used as the first-line treatment of gonorrhea in the US in 2007 [11]. But until 2012 cefixime was also used [12], so the results that we obtained for the percentage of diagnosed cases resistant to ceftriaxone are slightly higher than the actual ones. They are also expected to be slightly higher due to the fact that for the AMR calibration targets we used the data from the STD clinics (as they were the only ones available) which tend to overestimate prevalence [22,23].
In addition, the formation of partnerships and condom usage has not been included in the model. However, since MSM population in general has high rates of partner numbers and high percentage (80%) of casual partners [24], the current assumption of all the contacts being casual is expected to reflect the reality well. Also, a rare situation when a person develops infection in all three sites [3] or a rectal or pharyngeal infection being symptomatic has not been accounted for. While conducting this research, we assumed a full adherence to the guidelines for the first-line treatment. The recent data indicate that around 80% of male patients received the recommended first-line antibiotic therapy [25]. However, all the assumptions mentioned above are not expected to have major impact on the conclusions, especially, given the comparative nature of our research.
There are few additional points to consider when a decision on the number of sites to establish is being made. In the recent years, there were the cases of “super gonorrhea”, when urethral symptoms have disappeared after the treatment, but the pharyngeal specimen remained positive to Neisseria gonorrhoeae and the resistance to the first-line treatment used has been indicated [26]. In case of a clinic participating in eGISP instead of GISP and the pharyngeal infection being asymptomatic, those cases of great concern would be more likely identified and treated. When it comes to the non-urethral isolates, currently many isolate collection attempts fail due to a number of factors such as poor viability, growth problems or transport problems [9]. However, technical assistance with improving culture yields can resolve this problem [9].
The outcomes of this work can be used by the policymakers who decide how many GISP and eGISP sites to allocate each year, which would improve the monitoring of the spread of AMR gonorrhea in the US and assist in the global fight against the growing resistance of N. gonorrhoeae.

Acknowledgement

A special thanks goes to Dr. Reza Yaesoubi from Health Policy and Management, Yale University for his helpful advises.

References

  1. Dutescu, I.A. and S.A. Hillier, Encouraging the development of new antibiotics: are financial incentives the right way forward? A systematic review and case study. Infection and drug resistance, 2021: p. 415-434. [CrossRef]
  2. Lewis, D., Will targeting oropharyngeal gonorrhoea delay the further emergence of drug-resistant Neisseria gonorrhoeae strains? Sexually transmitted infections, 2015. 91(4): p. 234-237.
  3. Chan, P.A., et al., Extragenital infections caused by Chlamydia trachomatis and Neisseria gonorrhoeae: a review of the literature. Infectious diseases in obstetrics and gynecology, 2016. 2016.
  4. Dombrowski, J.C., Chlamydia and gonorrhea. Annals of Internal Medicine, 2021. 174(10): p. ITC145-ITC160.
  5. Alanis, A.J., Resistance to antibiotics: are we in the post-antibiotic era? Archives of medical research, 2005. 36(6): p. 697-705.
  6. Butler, M.S., et al., Antibiotics in the clinical pipeline as of December 2022. The Journal of Antibiotics, 2023: p. 1-43.
  7. Prevention, C.f.D.C.a., Gonococcal Isolate Surveilance Project (GISP) and Enhanced GISP (eGISP). 2022.
  8. St Cyr, S., et al., Gonococcal Isolate Surveillance Project (GISP) and Enhanced GISP (eGISP) Protocol. 2021.
  9. Kersh, E.N., et al., Expanding US laboratory capacity for Neisseria gonorrhoeae antimicrobial susceptibility testing and whole-genome sequencing through the CDC's Antibiotic Resistance Laboratory Network. Journal of Clinical Microbiology, 2020. 58(4): p. e01461-19.
  10. Sancta St. Cyr, K.K., Myriam Bélanger, Matthew Schmerer, Gonococcal Isolate Surveillance Project (GISP) and Enhanced GISP (eGISP) 2023.
  11. Workowski, K.A. and S.M. Berman, Sexually transmitted diseases treatment guidelines, 2006. 2006.
  12. Workowski, K.A. and G.A. Bolan, Sexually transmitted diseases treatment guidelines, 2015. MMWR. Recommendations and reports: Morbidity and mortality weekly report. Recommendations and reports, 2015. 64(RR-03): p. 1.
  13. Quilter, L.A.S., et al., Antimicrobial Susceptibility of Urogenital and Extragenital Neisseria gonorrhoeae Isolates Among Men Who Have Sex With Men: Strengthening the US Response to Resistant Gonorrhea and Enhanced Gonococcal Isolate Surveillance Project, 2018 to 2019. Sexually Transmitted Diseases, 2021. 48(12S): p. S111-S117. [CrossRef]
  14. Sullivan, P.S., et al., Understanding racial HIV/STI disparities in black and white men who have sex with men: a multilevel approach. PloS one, 2014. 9(3): p. e90514. [CrossRef]
  15. Jones, M.L.J., et al., Extragenital chlamydia and gonorrhea among community venue–attending men who have sex with men—five cities, United States, 2017. Morbidity and Mortality Weekly Report, 2019. 68(14): p. 321.
  16. Morris, S.R., et al., Prevalence and incidence of pharyngeal gonorrhea in a longitudinal sample of men who have sex with men: the EXPLORE study. Clinical Infectious Diseases, 2006. 43(10): p. 1284-1289. [CrossRef]
  17. Bowen, V.B., et al., Sexually transmitted disease surveillance 2018. 2019.
  18. Yaesoubi, R., et al., The Impact of Rapid Drug Susceptibility Tests on Gonorrhea Burden and the Life Span of Antibiotic Treatments: A Modeling Study Among Men Who Have Sex With Men in the United States. American Journal of Epidemiology, 2023: p. kwad175. [CrossRef]
  19. Donovan, L.C., et al. The presence or absence of symptoms among cases of urethral gonorrhoea occurring in a cohort of men taking human immunodeficiency virus pre-exposure prophylaxis in the PrEPX study. in Open Forum Infectious Diseases. 2021. Oxford University Press US. [CrossRef]
  20. Ong, J., et al., Asymptomatic and symptomatic urethral gonorrhoea in men who have sex with men attending a sexual health service. Clinical Microbiology and Infection, 2017. 23(8): p. 555-559. [CrossRef]
  21. Martín-Sánchez, M., et al., Clinical presentation of asymptomatic and symptomatic heterosexual men who tested positive for urethral gonorrhoea at a sexual health clinic in Melbourne, Australia. BMC infectious diseases, 2020. 20: p. 1-9. [CrossRef]
  22. Torrone, E.A., et al., Prevalence of sexually transmitted infections and bacterial vaginosis among women in sub-Saharan Africa: an individual participant data meta-analysis of 18 HIV prevention studies. PLoS medicine, 2018. 15(2): p. e1002511.
  23. Dandona, L., et al., Is the HIV burden in India being overestimated? BMC public health, 2006. 6: p. 1-10.
  24. Pines, H.A., M.Y. Karris, and S.J. Little, Sexual partner concurrency among partners reported by MSM with recent HIV infection. AIDS and Behavior, 2017. 21: p. 3026-3034. [CrossRef]
  25. Sittig, K.R., S.M. Collin, and R. Rosa, Factors associated with non-guideline-adherent treatment for gonorrhea and chlamydia among outpatient prescriptions in the Unites States. International journal of STD & AIDS, 2022. 33(7): p. 694-700. [CrossRef]
  26. Eyre, D.W., et al., Gonorrhoea treatment failure caused by a Neisseria gonorrhoeae strain with combined ceftriaxone and high-level azithromycin resistance, England, February 2018. Eurosurveillance, 2018. 23(27): p. 1800323. [CrossRef]
Preprints 106504 g001
Figure 1. The calibration results for 35 years of simulation (2007-2042) of the 100 selected trajectories along with the calibration targets and 95% uncertainty intervals. The black dots and bars in panels A-G represent the estimates the model is calibrated against and the 95% uncertaity intervals: prevalence of rectal gonorrhea among the US MSM (5.01% [4.2%, 6%] [14,15]), prevalence of pharyngeal gonorrhea among the US MSM (5.08% [4.5%; 5.8%] [15,16]), prevalence of urethral gonorrhea among the US MSM (5.08% [4.5%; 5.8%] [14]), prevalence of rectal AMR gonorrhea among the US MSM (0.13% [0.03%; 0.51%] [13]), prevalence of pharyngeal AMR gonorrhea among the US MSM (0.67% [0.32%; 1.4%] [13]), prevalence of urethral AMR gonorrhea among the US MSM (0.18% [0.08%; 0.37%] [13]), incidence of gonorrhea (6508 [5206, 7809] reported cases per 100,000 US MSM population [17,18]).
Figure 1. The calibration results for 35 years of simulation (2007-2042) of the 100 selected trajectories along with the calibration targets and 95% uncertainty intervals. The black dots and bars in panels A-G represent the estimates the model is calibrated against and the 95% uncertaity intervals: prevalence of rectal gonorrhea among the US MSM (5.01% [4.2%, 6%] [14,15]), prevalence of pharyngeal gonorrhea among the US MSM (5.08% [4.5%; 5.8%] [15,16]), prevalence of urethral gonorrhea among the US MSM (5.08% [4.5%; 5.8%] [14]), prevalence of rectal AMR gonorrhea among the US MSM (0.13% [0.03%; 0.51%] [13]), prevalence of pharyngeal AMR gonorrhea among the US MSM (0.67% [0.32%; 1.4%] [13]), prevalence of urethral AMR gonorrhea among the US MSM (0.18% [0.08%; 0.37%] [13]), incidence of gonorrhea (6508 [5206, 7809] reported cases per 100,000 US MSM population [17,18]).
Preprints 106504 g001
Preprints 106504 g002
Figure 2. Prevalence of gonorrhea among the US MSM at a single site and at a combination of two sites for 35 years of simulation (2007-2042).
Figure 2. Prevalence of gonorrhea among the US MSM at a single site and at a combination of two sites for 35 years of simulation (2007-2042).
Preprints 106504 g002
Preprints 106504 g003
Figure 3. Number of patients seeking treatment per 100,000 US MSM population at different anatomical sites and number of treatment failures per 100,000 US MSM population for 35 years of simulation (2007-2042).
Figure 3. Number of patients seeking treatment per 100,000 US MSM population at different anatomical sites and number of treatment failures per 100,000 US MSM population for 35 years of simulation (2007-2042).
Preprints 106504 g003
Preprints 106504 g004
Figure 4. Performance of GISP and eGISP surveillance systems. The bars represent 95% uncertainty intervals.
Figure 4. Performance of GISP and eGISP surveillance systems. The bars represent 95% uncertainty intervals.
Preprints 106504 g004
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

Downloads

435

Views

676

Comments

0

Subscription

Notify me about updates to this article or when a peer-reviewed version is published.

Email

Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2025 MDPI (Basel, Switzerland) unless otherwise stated