1. Introduction
The RV144 vaccine efficacy trial tested a canarypox-protein HIV vaccine regimen (ALVAC-HIV and AIDSVAX B/E gp120) and is the only HIV trial to date to demonstrate partial vaccine efficacy, with a modest 31.2% estimated protection at 42 months [
1]. An initial analysis showed that IgG responses binding to the variable loops 1 and 2 (V1V2) and antibody-dependent cellular cytotoxicity (ADCC) responses in the presence of low anti-envelope IgA responses were correlated with reduced risk of infection [
2]. A follow-up study also indicated that decreased risk of infection was associated with higher response rates and magnitudes of IgG3 V1V2-specific antibodies [
3].
HIV remains a significant health burden in southern Africa. In 2023, southern and eastern Africa accounted for 35% (450,000) and 41% (260,000) of new infections and HIV-related deaths, respectively [
4]. Consequently, there is a significant focus on a protective vaccine specific for this region. In contrast to Thailand, where the circulating recombinant strain CRF01_AE is most prevalent, southern Africa has very high levels of HIV-1 subtype C. Accordingly, several vaccine concepts have been developed using subtype C immunogens. HVTN702 (NTC02968849) was a phase IIb/III trial designed to test a subtype C specific regimen analogous to the RV144 regimen (ALVAC-C prime with an MF59-adjuvanted bivalent subtype C gp120 boost) but did not show any efficacy and so was stopped early [
5]. Similarly, the phase IIb Imbokodo vaccine efficacy trial (HVTN705; NTC03060629), which consisted of an adenovirus-26 mosaic HIV vector and a subtype C gp140 boost, did not show any efficacy [
6]. Both vaccines elicited lower magnitude of V1V2 responses than RV144 but, in separate correlates analyses, this response was still associated with decreased risk of HIV-1 acquisition in participants with high V1V2 binding antibodies [
7,
8].
A non-human primate (NHP) vaccine study which followed up on the results of the RV144 trial with an ALVAC prime and a pentavalent B/E/E/E/E gp120 boost previously showed that improved coverage of V2 epitopes in the vaccine can improve protection (55%) against a heterologous tier 2 subtype E/C SHIV challenge, compared to the RV144 ALVAC prime-B/E gp120 boost vaccine regimen which showed no protection [
9]. Given that the improved coverage of V2 resulted in higher levels of protection together with the substantially higher diversity of subtype C viruses in southern Africa compared to CRF01_AE strains in Thailand (Shen et al. submitted), there is a stronger rationale to increase the diversity of immunogens included in subtype C vaccines.
In this study, we performed an NHP vaccine study to determine if an ALVAC prime with gp120 boost immunogens, computationally designed to maximize subtype C V1V2 coverage and including signatures associated with sensitivity to broadly neutralizing antibodies (Shen et al. submitted), can improve protection of NHPs from a heterologous subtype C SHIV challenge compared to ALVAC prime-gp120 boost vaccines similar to the RV144 regimen and the P5 (Pox-Protein Public Private Partnership) subtype C version tested in HVTN100 and HVTN702 [
5,
10]. Humoral immunogenicity of the four vaccine regimens was defined and five immune parameters were pre-selected for their correlation with protection. We then used a systems serology approach to determine if immune markers could predict protection from infection regardless of vaccine regimen.
2. Methods
2.1. Ethics Statement
Rhesus macaques (M. mulatta) were housed at Bioqual, Inc. (Rockville, MD), in accordance with the standards of the American Association for Accreditation of Laboratory Animal Care. The protocol was approved by Bioqual’s Institutional Animal care and Use Committee under OLAW Assurance Number A-3086-01. Bioqual is IAAALAC accredited. This study was carried out in strict accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health (NIH) and with the recommendations of the Weatherall report: ‘The use of non-human primates in research’. All procedures were performed under anesthesia using ketamine hydrochloride, and all efforts were made to minimize stress, improve housing conditions and to provide enrichment opportunities (for example, social housing when possible, objects to manipulate in cage, varied food supplements, foraging and task-oriented feeding methods, interaction with caregivers and research staff). Animals were euthanized by sodium pentobarbital injection in accordance with the recommendations of the panel on Euthanasia of the American Veterinary Medical Association.
Human PBMCs from HIV-1-negative individuals were collected with Institutional Review Board approval by the Duke Medicine Institutional Review Board for Clinical Investigations (Protocols Pro00006526, Pro00000873, Pro00009459) All subjects were consented following 45 CFR 46 and written informed consent was obtained by all participants. No minors were recruited into this study.
2.2. Vaccine Design and Production
The vaccine consisted of two primes with ALVAC-HIV (vCP2438) (previously described in [
10]) and five boosts of ALVAC-HIV with different combinations of gp120 proteins.
Three natural subtype C Env proteins were selected for delivery, based on providing the best complementary coverage of the V2 region among known subtype C sequences (described in Shen et al. submitted). These were combined with proteins used in the P5 (HVTN100 and HVTN702 trials; 1086c and TV1.21) [
10] and RV144 Thai trial (A244_AE) [
1]. Sequences for the three newly selected strains are available in GenBank as CAP260: JN681228, CAP174: JN967791, and Ko224: JN681240. Gp120 recombinant proteins used for immunization were produced by the Duke Human Vaccine Institute Protein Production Facility, Duke University Medical Center as previously described [
16].
2.3. Immunization and SHIV Challenge of Rhesus Macaques
Forty Indian origin male and female rhesus monkeys (Macaca mulatta) were genotyped and selected as negative for the protective major histocompatibility complex class I allele Mamu-A*01. Monkeys were housed at Bioqual, Rockville, MD. The animals were maintained in accordance with the National Institutes of Health and Harvard Medical School guidelines and all studies were approved by the appropriate Institutional Animal Care and Use Committee. Selected animals were randomly assigned to each vaccine arm. Monkeys were vaccinated by the intramuscular (IM) route with 1x10
8 pfu ALVAC (vCP2438; Sanofi Pasteur) vector alone twice and then boosted five times with 100 µg total purified Env gp120 protein in ALFQ adjuvant. ALVAC was delivered IM into one leg and the protein was delivered IM into the other leg. Animals were then challenged six times weekly by the intrarectal route with a SHIV-CH505.375H.dCT challenge stock. The virus stock was grown from the infectious molecular clone in rhesus peripheral blood mononuclear cells (PBMCs) and the stock was titrated in rhesus macaques to select the appropriate dilution. Sequence diversity of the viral stock was determined by single-genome amplification [
17].
SHIV plasma viral RNA measurements were performed at the Immunology Virology Quality Assessment Center Laboratory Shared Resource, Duke Human Vaccine Institute, Durham, NC as described [
17].
2.4. HIV-1 Specific Binding Antibody Multiplex Assay (BAMA)
HIV-1 specific IgG antibodies to gp120 proteins and V1/V2 scaffolds were measured at time of challenge (week 143) by an HIV-1 BAMA as previously described [
2,
18] with the exception that the detection antibody was biotinylated goat anti-monkey IgG (Rockland Immunochemicals, Inc.). Positive controls included a HIVIG and CH58 monoclonal antibody IgG titration. Negative controls included in every assay were blank (uncoupled) and MulVgp70_His6 (empty gp70 scaffold) coupled beads, a blank well on each assay plate, as well as HIV-1 negative sera. To control for antigen performance, we used the preset criteria that the positive control titer (HIVIG) included on each assay (and for assays with V1V2 antigens, CH58 monoclonal antibody [
3]), had to be within +/- 3 standard deviations of the mean for each antigen (tracked with a Levey-Jennings plot with preset acceptance of titer and calculated with a four-parameter logistic equation, SigmaPlot, Systat Software). Antibody measurements were acquired on a Bio-Plex 200 instrument (Bio-Rad, Hercules, CA) using 21CFR Part 11 compliant software and the readout is in Median Fluorescent Intensity (MFI). All samples were tested at a screening 1:80 dilution and post baseline samples’ testing were followed by a 5-fold serial dilution for a total of six dilution points (1:80 to 1:250000). Area under the curve of titrations (AUC) was calculated for each sample and antigen using the trapezoidal rule, where the x-axis is log10 of dilution and y-axis is truncated MFI with negative values set to 0 for the calculation. If one or more dilutions were missing due to incomplete titration or failing QC, AUC was not calculated. Positivity of plasma binding response was determined at the screening dilution (1:80) with the following positivity criteria: 1) MFI> antigen-specific cutoff (which is 95th percentile MFI of all baseline plasma samples, and 100 minimum); 2) MFI> 3-fold MFI of the matched baseline sample before and after subtracting the negative control bead MFI.
2.5. Magnitude-Breadth Score Calculation
The magnitude-breadth of responses was evaluated for binding to 14 heterologous V1V2 antigens designed to calculate the breadth of V1V2 responses [
19]. Two of the V1V2 breadth antigens were not included as they matched vaccine proteins. A magnitude-breadth score is calculated for each animal as the area under the magnitude-breadth curve (AUC-MB), where the breadth represents the proportion of V1V2 antigens for which the animal had a positive vaccine-induced response at each magnitude interval, and AUC-MB is equivalent to the mean of the log10 scaled response magnitudes over the V1V2 antigen panel and available when all panel antigens’ data were available. The plots of partial MB curves included both individual subject specific and group specific curves.
2.6. ADCC Against gp120-Coated Target Cells
The GranToxiLux assay was used to detect ADCC activities of NHP plasma samples directed against CEM.NKRCCR5 CD4. T cells (NIH AIDS Reagent Program Division of AIDS, NIAID, NIH from Alexandra Trkola) coated with recombinant gp120 as described [
20]. ADCC activities of four-fold serial plasma dilutions starting at 1:100 were measured against cells coated with gp120 isolates representing vaccine immunogens A244.AE, 1086.C, CAP260.C, CAP174.C and Ko244.C and the challenge virus, CH505.C. Cryopreserved human PBMCs from an HIV-seronegative donor (Duke University) with the heterozygous 158F/V genotype for Fc-gamma receptor IIIa were used as the source of effector cells [
21]. Data were reported as the maximum proportion of cells positive for proteolytically active granzyme B out of the total viable target cell population (maximum %GzB activity) after subtracting the background activity observed in wells containing effector and target cells in the absence of plasma. ADCC end point titers (ADCC titer) were determined by interpolating the dilutions of plasma that intercept the previously established positive cutoff for this assay (8% GzB activity) using GraphPad Prism, version 6.0f software (GraphPad Software, Inc.). ADCC assays were performed in duplicate for each animal at each time point.
2.7. Antibody-Dependent Cellular Phagocytosis Assay
Antibody-dependent phagocytosis was assessed by the measurement of the uptake of antibody-opsonized, antigen-coated fluorescent beads by the monocytic THP-1 cell line (ATCC; #TIB-201) [
22]. Briefly, THP-1 cells were purchased from ATCC and cultured as recommended. Antibody-mediated phagocytosis assay was performed as described [
23], with the following modifications. Briefly, 9x10
5 beads coupled with CH505 gp120 Env was mixed with 10 µl of a 1:50 dilution of rhesus serum in a 96-well round-bottom plate and incubated at 37°C for 2 hours. During the incubation, blocking of CD4 on the cells was achieved by pretreating the cells at 10x10
6 cells/ml with 20 µg/ml anti-human CD4 antibody (clone SK3) (Biolegend) for 15 min at 4°C. After incubation at 37°C for 2 hours, 2.5x10
4 THP-1 cells were added to each well with final volume 100 µl each, then spinoculated at 1,200xg for 1 hour at 4°C. Following spinoculation, immune complexes and cells were incubated at 37°C for 1 hour to initiate phagocytosis internalization. Cells were then fixed in 2% paraformaldehyde. Phagocytosis score was calculated by sample specific percentage of cells positive x mean MFI, normalized to the corresponding result for the no-antibody control. A background level of phagocytosis was determined based on the mean+3 s.d. of non-HIV-specific antibodies. Assays were performed in duplicate for each animal, and the average phagocytosis score over the replicates, was reported.
2.8. Monoclonal Antibody Competition ELISAs
Monoclonal antibody competition assays were performed as described previously [
9,
24]. In brief, plates were coated with HIV-1 envelope protein, washed, and blocked. Non-biotinylated monoclonal antibodies were serially and incubated in triplicate wells for 90 min, followed by addition of biotinylated monoclonal antibodies at sub-saturating concentrations and a 1-hour incubation. The same antibody, in non-biotinylated form, was used to block itself as a positive control. An anti-influenza antibody CH65 was included as a negative was included as a negative control. Binding of biotinylated monoclonal antibodies was determined with horseradish peroxidase (HRP)-conjugated streptavidin. Binding of the biotinylated monoclonal antibody to HIV-1 envelope in the presence and absence of competing antibody was compared to calculate percent inhibition of binding. Historical data on the negative controls of the assay were tracked, and an assay was considered valid if the positive control antibodies blocked greater than 20% of the biotinylated antibody binding.
2.9. Neutralization
Neutralization activities of animal plasma and purified antibodies were determined by the TZM-bl-cell-based neutralization assay [
25]. Tier phenotyping of the pseudoviruses was assayed by sensitivity to a pool of HIV-infected serum as described [
26]. Neutralization assays are performed in technical triplicate for all animals at each serum time point or antibody concentration.
2.10. Machine Learning
2.10.1. Missing Data
V1V2 MB-AUC values were missing for four animals. Values were imputed using the average value for the respective group to ensure all immune features could be used in the model construction.
2.10.2. Dimensionality Reduction
Dimensionality reduction using principal component analysis (PCA) was performed on the measured immune features (n=36). Immune features were scaled and centered, and PCA performed using R package ‘stats’ [
27]. The R package ‘ggplot’ [
28] was used to make PC biplots.
2.11. Group Classification
A 5-fold cross-validated random forest algorithm was trained and tested on the immune feature dataset (n=36) using R package ‘randomForest’ [
29]. Training and testing were repeated 100 times with different splits of animals. As a control, permutation testing, in which vaccine group labels were shuffled randomly and the same 5-fold cross-validated random forest model was used for training and testing was performed and repeated 100 times to generate a null result distribution. Model accuracy was measured using Mathew’s Correlation Coefficient (MCC) and performance differences between the actual and permuted models were evaluated using statistical significance (Kolmogorov-Smirnov Test) and effect size (Cliff’s delta), using package ‘stats’1 and ‘effsize’ [
30]. To evaluate the outcome of the classification model, a confusion matrix was constructed by converting the predictions into percentages across the 100 repeats. Feature contributions were averaged and reported along with standard deviation across the 100 repeats.
2.12. Classification of Animals According to Susceptibility to Infection
Based on the number of challenges to infection, animals were divided into two groups: (i) Susceptible to infection (≤4 challenges, n=21), and (ii) Resistant to infection (>4 challenges, n=14). As described above, a 5-fold cross-validated random forest algorithm was trained and tested on the immune feature dataset.
2.13. Survival Analysis Using Predictive Models
As previously reported [
9,
31,
32], a multivariate survival analysis approach was used to train a Cox proportional hazards model to predict risk of infection at each challenge timepoint using Fc array features at 143 weeks post challenge. Steps involved in the analysis were as follows:
Model training and risk prediction: Cox models were trained using the R package ‘survival’ [
33]. Models were trained and tested using 10-fold cross-validation to predict risk of infection of each animal relative to the mean risk observed over all animals. Model performance was evaluated by Concordance index [
34]. The cross-validation was repeated 100 times with different splits of animals. As a control, permutation testing was performed using randomly shuffled challenge data and following the same exact procedure to filter features and train and test models. Performance differences between the actual and permuted data were characterized by measuring statistical significance and effect size (Cliff’s delta) on the Concordance index distributions of models tested actual and permuted data.
Representative model predicted risk of infection for each animal: A representative run of 10-fold cross-validated modeling was performed using the features that appeared in at least 50% of the models obtained over the repeated cross-validation. The representative run enabled plotting of predicted risk for each animal when part of test set.
‘Risk group’ model to compare survival probability of the predicted and susceptible groups: For this purpose, a ‘risk group’ model was constructed. The survival probability for the resistant and susceptible groups were estimated by making prediction of mean values of features of animals in each group i.e., ‘mean’ animal for the group. The predicted and actual Kaplan-Meier curves for the two groups were compared by log-rank statistic. A high p-value favors the null hypothesis i.e., there is no statistical evidence of difference between the actual and predictive curves.
2.14. Statistical Analyses
The number of NHPs for each group was calculated as the minimum required for at least 80% power using a log-rank test for comparison of survival curves. Investigators were blinded to the vaccine arm of the samples at the time of assay but were not blinded when analyzing the data. The Kaplan-Meier curves based on observed data were compared between groups using the log-rank test. Cox proportional hazards models were used for evaluating potential correlates of protection.
Group comparisons were made using exact Wilcoxon–Mann–Whitney tests. All statistical analysis was performed using SAS v9.4 (SAS Institute, Inc., Cary, NC). GraphPad Prism version 6.01 was used for graphical representation.
2.15. Systems Serology Analysis and Code Availability
Systems serology analysis was performed using R version 4.2.3 in software Rstudio version 2023.12.1+402. The code used for the machine learning analysis will be made available on Github at the time of publication. The data that support the findings of this study are available from the corresponding authors upon request.
3. Results
3.1. ALVAC Prime with Subtype C gp120 Trivalent Boost Protects from Repeated Intrarectal SHIV Challenge
To assess if immunogens computationally selected to provide additional V1V2 coverage could improve vaccine efficacy against a heterologous SHIV, we immunized four groups (n=9) of Indian origin rhesus macaques (RM;
Macaca Mulata) with an ALVAC vCP2438 (ALVAC C) prime and gp120 protein boosts in ALFQ adjuvant consisting of i) A244.AE, 1086.C, CAP260.C, Ko244.C and CAP174.C (group 1; Pentavalent), ii) TV1.C and 1086.C (group 2; P5 bivalent), iii) A244.AE and 1086.C (group 3; RV144 bivalent) or iv) CAP260.C, CAP174.C and Ko244.C (group 4; trivalent) (
Figure 1A). RM were primed with ALVAC twice at 0 and 4 weeks and then initially boosted three times at weeks 13, 21 and 47. Due to the COVID-19 pandemic, the study was halted and reinitiated a year later when two additional boosts were given at weeks 101 and 139. This was followed by a weekly low-dose intrarectal challenge with the heterologous tier 2 transmitted/founder SHIV-CH505, starting at week 143 (
Figure 1B). Of note, 66.7% (6/9) of RM from the Trivalent group remained uninfected after five challenges, whereas only 37.5% (3/8) from the Pentavalent group, 2/9 (22.2%) from the P5 Bivalent group, and 1/9 (11.1%) from the RV144 Bivalent group remained uninfected. All animals in the control group became infected by the 4th immunization. This resulted in overall statistically significant protection per exposure compared to the control group (p=0.0225, log-rank test) (
Figure 1C). When comparing groups individually to the control arm, protection remained significant in the Trivalent and Pentavalent groups (p=0.0041 and p=0.0260, respectively) (
Figure S1 D,A) but not in either bivalent group (
Figure S1 B,C). Among animals that became infected, there was no impact on peak viral load compared to the control animals (
Figure 1D).
3.2. Different gp120 Combination Boosts Elicit Similar Magnitudes of gp120 and Breadth of V1V2 Binding Antibodies
To understand how the different combinations of gp120 immunogens impacted responses in the different groups, we evaluated the immunogenicity of each vaccine at the time of challenge. We first measured binding antibodies to the six gp120 immunogens and a gp120 protein representing the Envelope of the challenge virus, CH505TFD7gp120. Because of the limited number of animals, we did not adjust for multiple comparisons throughout but FDR-corrected p-values are reported in supplementary
Table S1. Responses to the gp120s were generally similar and of high magnitude among all four groups (
Figure 2A). Without adjusting for multiple comparisons, the Trivalent group had significantly higher responses to several antigens, including higher responses to CAP174D11gp120 and CAP260D11gp120 than the RV144 Bivalent group (p=0.0315 and 0.0056, respectively) and to A244D11gp120 and CAP260D11gp120 than the P5 Bivalent group (p=0.0315 and p=0.0106, respectively). The Pentavalent group also had significantly higher binding to A244D11gp120 than the P5 Bivalent group (p=0.0055).
Breadth of V1V2 responses elicited by each vaccine at the time of challenge were also measured against a panel of 14 heterologous V1V2 gp70 proteins, representative of global HIV-1 diversity [
11] which were used to generate area-under-magnitude and breadth curves (AUC-MBs) (
Figure 2B). Comparing the AUC-MBs revealed similar breadth of responses across the four vaccine groups, except for the RV144 group, which had significantly lower V1V2 AUC-MBs than the P5 Bivalent group (p=0.0055).
3.3. Vaccines Elicit Strong Functional Antibody Responses
To understand the functional capacity of the antibodies elicited by each vaccination at the time of challenge, we tested the ability of antibodies in sera at the time of challenge to mediate Fc effector functions against, or neutralize, the immunogen or challenge Envelopes. Among the groups, antibody dependent cellular cytotoxicity (ADCC) responses tended to be highest in the Pentavalent group: of the six Envelopes tested, significantly different responses were detected against 1086C.gp140, A244D11gp120, CAP174D11gp120, and Ko244D11gp120. Against the 1086C.gp120, the Pentavalent group (median pAUC=4.8) had significantly higher levels of ADCC than the Trivalent (median pAUC=0; p=0.0137) and RV144 Bivalent (median pAUC=0; p=0.0123) groups. The Pentavalent group (median pAUC=3.51) had significantly higher levels of ADCC against A244D11gp120 than the Trivalent group (median pAUC=0; p=0.0097). Against the CAP174D11gp120, the Pentavalent group and P5 Bivalent groups (median pAUCs=4.94 and 4.82, respectively) had significantly higher levels of ADCC than the RV144 bivalent group (median pAUC=3.60; p=0.0027 and 0.0196, respectively). Lastly, the Pentavalent group (median pAUC=4.54) had significantly higher levels of ADCC against Ko244D11gp120 than the RV144 Bivalent group (median pAUC =4.12, p=0.0283) (
Figure 3A). There were no differences in magnitudes of ADCC against the Envelope representing the challenge virus, CH505TFD7gp120 (median pAUCs 3.94-4.84) (
Figure 3A). While antibody dependent cellular phagocytosis (ADCP) responses against CH505TFD7gp120 at the time of challenge were similar (median ADCP scores: 17-18.95), the Trivalent group was significantly higher than the P5 Bivalent group (p=0.0379) (
Figure 3B).
Neutralization was only observed against easy-to-neutralize tier 1 Envelopes: the CRF01_AE Envelope TH023.6 and the subtype C Envelope TV1C8.2 (
Figure 3C). Neutralization of TH023.6 was significantly lower in the P5 Bivalent group (median ID
50 titer: 432) than the other three groups (median ID
50 titers: 3581-4462, p-values: 0.0315-0.0010) and all four groups had similar neutralizing antibody responses to TV1C8.2 (median ID
50 titers: 1450-2383). Very low (ID
50<35) to no neutralization of the Tier 2 challenge virus, SHIV.CH505.375H.dCT, or the other five Tier 2 Envelopes tested, were detected (
Table S2).
We next assessed the capacity of antibodies in sera to block binding of soluble CD4 or several monoclonal antibodies to the gp120 Envelopes A244D11gp120 (CD4, A32, CH58, PG9, CH01 and 2G12), 6240 (CD4, PG9) and 902114B2gp140 (CH01). The only parameters with significant differences between groups were blocking of CD4 binding to A244D11gp120, where blocking by antibodies in the P5 Bivalent group (median = 81.6%) was significantly lower than the other three groups (medians = 97.6-100%; p=values 0.0002-0.0077), blocking of the anti-V2 non-neutralizing monoclonal antibody CH58 binding to A244D11gp120, where blocking by antibodies in the Trivalent group (median = 24.1%) was significantly lower than the other three groups (medians = 90.2-95.9%; p=values 0.00008-0.0005) (
Figure 3D) and blocking of 2G12 to A244D11gp120 where the Pentavalent group (median = 36.2%) was significantly higher than the P5 Bivalent group (median = 28.1%, p=0.0037) (
Figure S2). Interestingly, blocking of the V2-targeting broadly neutralizing antibody CH01 also trended higher in the trivalent (median = 63%) and pentavalent (median = 56.9%) groups compared to the bivalent RV144 (median = 44.5%) and P5 Bivalent (median = 39.2%) groups (
Figure 3D).
3.4. Immune Responses Induced by the Different Vaccine Arms
As an unbiased way to evaluate differences in immune responses in immunized animals, we performed unsupervised dimensionality reduction using Principal Component (PC) analysis. Because V1V2 MB-AUCs for four animals were missing, and these analyses cannot be completed without the full data set, values for those animals were imputed by inserting the average MB-AUC value of the relevant groups. PC biplots demonstrated that animals in the Trivalent group, the group that showed the greatest degree of protection, were distinct in PC2 values, as compared to the animals in the Pentavalent, P5 Bivalent and RV144 Bivalent groups (
Figure 4A). Blocking CH58 from binding to the CRF01_AE gp120 A244D11gp120 and ADCC against Ko224D11gp120- and A244D11gp120-coated cells were the top three features that made strongest contribution to PC2 (
Figure S3), in line with the fact that the Trivalent group was the only group not to include the A244.AE/1086.C V1V2 linear epitope. While these features differentiate the immunological profiles of the vaccine groups based on our test reagents, it worth emphasizing that group differentiation driven by the evaluation of a CRF01_AE virus as a test reagent (A244D11gp120) may have limited implications for vaccine breadth within Southern Africa. Group 4, which ranked highest in terms of protecting against a C clade heterologous challenge, did not include A244 or any other AE vaccine in the vaccine cocktail.
To identify signatures of the responses that were specific to each of the different vaccine arms, we then constructed a repeated 5-fold cross-validated supervised classifier using the random forest algorithm. Model accuracy for test data across folds and replicates was calculated using Mathew’s Correlation Coefficient (MCC) and demonstrated high accuracy in classifying animals according to vaccine arm. Comparison of MCC values yielded for models trained on actual versus permuted class labels indicated these predictions were robust: the high quality of classification results was unlikely to be observed at random and demonstrated a large effect size, as measured by Cliff’s delta (
Figure 4B). To explore classification accuracy observed for each individual class, a confusion matrix was plotted, demonstrating that the model accurately classified animals according to vaccine arms with an overall accuracy of ~ 67% (
Figure 4C). The Trivalent group (group 4), which showed distinct immunological responses from the other groups, was classified correctly >79% of time while the Pentavalent and two bivalent groups (groups 1-3) were classified with similar accuracy (57-68%). The 25 immune features that contribute the most to group classification using to the random forest model included blocking of CH58 and CD4 binding to A244D11gp120 and breadth of V1V2 binding antibody responses (
Figure 4D). Overall, supervised models identified characteristic differences in immunogenicity data between immunization groups.
3.5. Immune Correlates of Decreased Infection Risk
We then divided animals into two groups based on susceptibility to infection, to study protection from infection irrespective of vaccine immunogen design. Animals that were infected in ≤4 challenges were defined as susceptible (n=21), and those infected at challenge 5 or those that remained uninfected were defined as resistant (n=14). In PC space, susceptibility groups showed some evidence of distinct profiles (
Figure S4A), and when a repeated 5-fold cross-validated random forest classifier was trained and tested, accurate and robust classification was observed (
Figure S4B). The confusion matrix demonstrated accurate classification of animals belonging to resistant and susceptible groups >66% of the time (
Figure S4C). The 25 immune features contributing the most to the random forest model included blocking of several proteins (CH58, CH01 and CD4) binding to various gp120s, IgG binding to CAP174D11gp120 and ADCC against cells coated with the challenge protein (CH505TFD11gp120) (
Figure S4D).
To define immune responses associated with reduced risk of infection with a greater degree of resolution than afforded by classification, Cox proportional hazards (Cox) models were trained and tested following aggressive filtering and in the context of repeated cross-validation. Model robustness was demonstrated by both reproducible results across repeated cross-validation replicates and as compared to predictions made following risk class label permutation testing (
Figure 5A). The agreement of predictions to observed risk of infection, as defined by Concordance Index was significantly greater for models trained on actual than on permuted risk data as demonstrated by the Kolmogorov Smirnov Test and high Cliff’s delta. Moreover, as demonstrated in a representative run, prediction of susceptibility to infection of individual animals was also accurate, with animals infected at earlier challenges exhibiting higher predicted risk (
Figure 5B). When the animals were grouped based on the relative risk groups defined previously, animals in the susceptible group had higher predicted risk as compared to animals in the resistant group (
Figure 5C). Indeed, when modeled based on group-wise immunogenicity profiles, the risk of infection for resistant and susceptible groups predicted by the model closely mirrored the actual infection risk as represented in Kaplan-Meier curves (
Figure 5D). The final model employed a set of seven immune features that predicted susceptibility to infection (
Figure 5E). Of those, blocking of PG9 binding to the CRF01_AE gp120 A244D11gp120 was significantly associated with susceptibility to infection and blocking of CH01 to the subtype B gp140 902114B2gp140 and ADCC against 1086Cgp140-coated cells were significantly associated with resistance to infection. Overall, these results indicate robust relationships exist between immunogenicity data and challenge outcomes, and sera antibodies that were able to block different V2 monoclonal antibodies from binding contributed to prediction of either susceptibility or resistance to infection.
4. Discussion
In this study, we tested combinations of HIV immunogens computationally selected to improve the coverage of subtype C V1V2 diversity in a rigorous NHP challenge model to determine if improved coverage could increase vaccine efficacy against a subtype C heterologous challenge. All groups elicited generally similar levels of humoral responses pre-challenge and low to no detectable tier 2 neutralizing antibodies. However, the Trivalent and Pentavalent groups were both able to significantly protect animals from infection with the difficult-to-neutralize (Tier-2) challenge SHIV-CH505TF.375H.dCT, with survival rates of 66.7% and 37.5%, respectively. A combined analysis of all groups together identified only ADCC responses against CH505TFD7gp120 as a correlate of protection at the time of challenge (hazard ratio: 0.793; 95%CI: 0.648-0.970, p=0.0241). Applying a machine learning systems serology approach showed that levels of blocking of V2 broadly neutralizing antibodies to different Envelopes were able to predict susceptibility or resistance to infection across all groups.
The failure of the phase IIb/III HVTN702 vaccine efficacy trial (the southern African follow-up trial to the RV144 trial) to provide any protection in southern Africa was disappointing for the field [
5]. There are several hypotheses for this failure, including the significantly increased diversity of subtype C viruses in southern Africa relative to CRF01_AE viruses in Thailand (Shen et al. submitted). To address this issue, we designed vaccine regimens to maximize the coverage of diversity observed in the highly variable V1V2 region of subtype C viruses given the practical constraints of using no more than three natural variants in a vaccine cocktail (Shen et al. submitted for consideration to this special issue). Here we compared vaccine responses in NHPs using the three optimized variants we selected for C clade coverage (group 4); a bivalent vaccine that included 2 C clade variants that were not particularly selected with respect to natural C clade diversity coverage (group 2); a bivalent vaccine that included one C clade but also the original CRF01 AE variant A244.AE to address the possibility that this Env might intrinsically favor V1V2 directed responses and this may have contributed to the protection observed in the RV144 trial (group 3); and one pentavalent vaccine that combined groups 3 and 4. The groups boosted with the computationally selected Envelopes were significantly more protected from a heterologous challenge than the groups that did not include them (66.7% and 37.5% versus 22.2% and 11.1%), suggesting this approach succeeded in this study.
Here, a Cox proportional hazards primary correlates analysis associated ADCC responses against the challenge Envelope at the time of challenge with protection from infection. ADCC was a correlate with decreased risk of infection in RV144 [
2]. In RV144, genetic signatures in the V2 were associated with increased vaccine efficacy [
12] and antibodies targeting a heterogenous site in the V2 were isolated from sera of vaccinees [
13]. In addition, a follow-up NHP study that aimed to improve on the results of RV144 [
9] and a DNA-protein coadministration NHP study [
14] also found ADCC correlated with increased resistance to infection. These studies suggest that while broad neutralizing antibodies are the gold standard for an HIV-1 vaccine, effector functions of antibodies targeting this region may still play a role in protection.
Importantly, while ADCC was the primary correlate of protection in the traditional Cox analysis, the systems serology analysis revealed other factors that contributed to protection. In particular, blocking of V2 broadly neutralizing antibodies that recognize conformational epitopes binding to gp120s was associated with both increased susceptibility and resistance to infection: blocking of PG9 binding to A244D11gp120 was associated with susceptibility to infection while blocking of CH01 to 902114B2gp140 was associated with resistance to infection and blocking of CH01 to A244D11gp120 was highest in pentavalent and trivalent groups, suggesting subtle differences in the epitopes of monoclonal antibodies elicited by the different vaccine arms. PG9 and CH01 both bind to conformational epitopes on the V2 loop but CH01 binds a distinct epitope/approaches the epitope from a different angle to PG9 [
15]. Antibodies targeting a similar conformational epitope to CH01 but not PG9 may have played a role in protection in this study.
Analysis of the blocking of various proteins binding to A244D11gp120 allows some deconvolution of the responses developed in the Trivalent group. Sera in all groups were able to block CD4 binding to A244D11gp120, suggesting the development of antibodies that could recognize A244D11gp120 in all four groups regardless of the presence of the gp120 in the vaccine arm. However, sera from animals in the Trivalent group blocked binding of CH58, which recognizes a linear V2 epitope [
13], significantly less than the other three groups. This was the feature that most separated the immune response in the Trivalent group from the other three groups. Instead, sera in this group were able to block CH01 binding to the highest extent, suggesting immunogens in the Trivalent group were able to focus B cell maturation and antibody development to preferentially target conformational V2 epitopes—a desired outcome of an HIV vaccine.
This study elicited antibodies that were able to neutralize tier 1 Envelopes but not the heterologous tier 2 Envelope used in the challenge or the five other tier 2 Envelopes representing the vaccine immunogens, suggesting only weak neutralizing antibodies were elicited. This was not surprising given the use of gp120 proteins as immunogens. Despite this, the systems serology suggests responses may be targeting a relevant epitope and alternative forms of the immunogen (i.e., trimeric forms of the Envelope instead of monomers) may be able to elicit neutralizing antibodies targeting conformational V2 epitopes. However, further analysis of the B cell repertoire and monoclonal antibody isolation are needed to confirm the epitopes of antibodies elicited by these vaccines.
One interesting and unintentional feature of this study was the delayed period between the fifth and sixth vaccination due to the interruption caused by the COVID-19 pandemic. To our knowledge, this is the third NHP challenge study that included a significantly longer interval during the boost administration. In the ALVAC-AE pentavalent gp120 boost study by Bradley et al.
[9], there was a 41-week interval between the fourth and fifth doses, after which animals were challenged with SHIV-1157(QNE)Y173H; while in a DNA+protein coadministration study by Felber et al., there was a 20-week interval between the fifth and sixth doses after which animals were challenged with SHIV-CH505.375H [
14]. Both studies also showed high levels of protection (55% and 67%, respectively). Collectively, these studies raise questions about the optimal period between boosts and suggest a delayed boost may improve the quality of humoral immune responses. This needs to be explored further in future studies.
In conclusion, we have demonstrated that a vaccine with computationally selected immunogens to maximize coverage of the subtype C V1V2 region is able to protect NHPs from a heterologous SHIV challenge. Delayed boosting with computationally defined subtype C immunogens should be further investigated in clinical trials as a potential mechanism to induce protective antibody responses.
Supplementary Materials
The following supporting information can be downloaded at the website of this paper posted on
Preprints.org.
Author Contributions
Designing the study—LC, GDT, BK, BFH, XS, GF ; Conducting the experiments—RP, LDT, TP, NE, DG, AZ, PR, SSO, TD, DCM; Analyzing data—DM, MT, NK, YW, WR, SS, LZ, SJ, MA, XS, GF; Providing reagents—SS, GM, ZB, MR, JA; Project administration and supervision—GDT, BFH, XS, GF; Writing the manuscript—DM, MT, NK, MA, GF; Funding acquisition—GDT, BFH, XS.
Funding
This work was supported by the National Institutes of Health (NIH), National Institute of Allergy and Infectious Diseases (NIAID) Center for HIV/AIDS Vaccine Immunology (AI067854) and the Duke Center for HIV/AIDS Vaccine Immunology and Immunogen Discovery (AI100645), 1R56AI165448-01A1, 1P01AI162242-03, the Duke Center for AIDS Research (P30AI064518), the External Quality Program Oversight Laboratory (EQAPOL) program (75N93024D00003), the Nonhuman Primate Core Humoral Immunology Laboratory for AIDS Vaccine Research and Development (HHSN27201100016).
Acknowledgements
We would like to thank Jennifer Peele for technical assistance with assays and Jim Tartaglia and Sanjay Phogat (Sanofi) for kindly providing ALVAC-C for this study.
Conflicts of Interest
None of the authors have a conflict of interest.
References
- Rerks-Ngarm, S.; Pitisuttithum, P.; Nitayaphan, S.; Kaewkungwal, J.; Chiu, J.; Paris, R.; Premsri, N.; Namwat, C.; de Souza, M.; Adams, E.; et al. Vaccination with ALVAC and AIDSVAX to Prevent HIV-1 Infection in Thailand. N Engl J Med 2009, 361, 2209–2220. [CrossRef]
- Haynes, B.F.; Gilbert, P.B.; McElrath, M.J.; Zolla-Pazner, S.; Tomaras, G.D.; Alam, S.M.; Evans, D.T.; Montefiori, D.C.; Karnasuta, C.; Sutthent, R.; et al. Immune-Correlates Analysis of an HIV-1 Vaccine Efficacy Trial. N Engl J Med 2012, 366, 1275–1286. [CrossRef]
- Yates, N.L.; Liao, H.-X.; Fong, Y.; deCamp, A.; Vandergrift, N.A.; Williams, W.T.; Alam, S.M.; Ferrari, G.; Yang, Z.; Seaton, K.E.; et al. Vaccine-Induced Env V1-V2 IgG3 Correlates with Lower HIV-1 Infection Risk and Declines Soon after Vaccination. Sci Transl Med 2014, 6, 228ra39. [CrossRef]
- UNAIDS 2024 Global AIDS Report — The Urgency of Now: AIDS at a Crossroads.
- Gray, G.E.; Bekker, L.-G.; Laher, F.; Malahleha, M.; Allen, M.; Moodie, Z.; Grunenberg, N.; Huang, Y.; Grove, D.; Prigmore, B.; et al. Vaccine Efficacy of ALVAC-HIV and Bivalent Subtype C Gp120-MF59 in Adults. N Engl J Med 2021, 384, 1089–1100. [CrossRef]
- Gray, G.E.; Mngadi, K.; Lavreys, L.; Nijs, S.; Gilbert, P.B.; Hural, J.; Hyrien, O.; Juraska, M.; Luedtke, A.; Mann, P.; et al. Mosaic HIV-1 Vaccine Regimen in Southern African Women (Imbokodo/HVTN 705/HPX2008): A Randomised, Double-Blind, Placebo-Controlled, Phase 2b Trial. Lancet Infect Dis 2024, 24, 1201–1212. [CrossRef]
- Moodie, Z.; Dintwe, O.; Sawant, S.; Grove, D.; Huang, Y.; Janes, H.; Heptinstall, J.; Omar, F.L.; Cohen, K.; De Rosa, S.C.; et al. Analysis of the HIV Vaccine Trials Network 702 Phase 2b-3 HIV-1 Vaccine Trial in South Africa Assessing RV144 Antibody and T-Cell Correlates of HIV-1 Acquisition Risk. J Infect Dis 2022, 226, 246–257. [CrossRef]
- Kenny, A.; van Duijn, J.; Dintwe, O.; Heptinstall, J.; Burnham, R.; Sawant, S.; Zhang, L.; Mielke, D.; Khuzwayo, S.; Omar, F.L.; et al. Immune Correlates Analysis of the Imbokodo (HVTN 705/HPX2008) Efficacy Trial of a Mosaic HIV-1 Vaccine Regimen Evaluated in Southern African People Assigned Female Sex at Birth: A Two-Phase Case-Control Study. EBioMedicine 2024, 108, 105320. [CrossRef]
- Bradley, T.; Pollara, J.; Santra, S.; Vandergrift, N.; Pittala, S.; Bailey-Kellogg, C.; Shen, X.; Parks, R.; Goodman, D.; Eaton, A.; et al. Pentavalent HIV-1 Vaccine Protects against Simian-Human Immunodeficiency Virus Challenge. Nat Commun 2017, 8, 15711. [CrossRef]
- Bekker, L.-G.; Moodie, Z.; Grunenberg, N.; Laher, F.; Tomaras, G.D.; Cohen, K.W.; Allen, M.; Malahleha, M.; Mngadi, K.; Daniels, B.; et al. Subtype C ALVAC-HIV and Bivalent Subtype C Gp120/MF59 HIV-1 Vaccine in Low-Risk, HIV-Uninfected, South African Adults: A Phase 1/2 Trial. Lancet HIV 2018, 5, e366–e378. [CrossRef]
- Yates, N.L.; deCamp, A.C.; Korber, B.T.; Liao, H.-X.; Irene, C.; Pinter, A.; Peacock, J.; Harris, L.J.; Sawant, S.; Hraber, P.; et al. HIV-1 Envelope Glycoproteins from Diverse Clades Differentiate Antibody Responses and Durability among Vaccinees. J Virol 2018, 92. [CrossRef]
- Rolland, M.; Edlefsen, P.T.; Larsen, B.B.; Tovanabutra, S.; Sanders-Buell, E.; Hertz, T.; deCamp, A.C.; Carrico, C.; Menis, S.; Magaret, C.A.; et al. Increased HIV-1 Vaccine Efficacy against Viruses with Genetic Signatures in Env V2. Nature 2012, 490, 417–420. [CrossRef]
- Liao, H.-X.; Bonsignori, M.; Alam, S.M.; McLellan, J.S.; Tomaras, G.D.; Moody, M.A.; Kozink, D.M.; Hwang, K.-K.; Chen, X.; Tsao, C.-Y.; et al. Vaccine Induction of Antibodies against a Structurally Heterogeneous Site of Immune Pressure within HIV-1 Envelope Protein Variable Regions 1 and 2. Immunity 2013, 38, 176–186. [CrossRef]
- Felber, B.K.; Lu, Z.; Hu, X.; Valentin, A.; Rosati, M.; Remmel, C.A.L.; Weiner, J.A.; Carpenter, M.C.; Faircloth, K.; Stanfield-Oakley, S.; et al. Co-Immunization of DNA and Protein in the Same Anatomical Sites Induces Superior Protective Immune Responses against SHIV Challenge. Cell Rep 2020, 31, 107624. [CrossRef]
- Bonsignori, M.; Hwang, K.-K.; Chen, X.; Tsao, C.-Y.; Morris, L.; Gray, E.; Marshall, D.J.; Crump, J.A.; Kapiga, S.H.; Sam, N.E.; et al. Analysis of a Clonal Lineage of HIV-1 Envelope V2/V3 Conformational Epitope-Specific Broadly Neutralizing Antibodies and Their Inferred Unmutated Common Ancestors. J Virol 2011, 85, 9998–10009. [CrossRef]
- Liao, H.-X.; Sutherland, L.L.; Xia, S.-M.; Brock, M.E.; Scearce, R.M.; Vanleeuwen, S.; Alam, S.M.; McAdams, M.; Weaver, E.A.; Camacho, Z.; et al. A Group M Consensus Envelope Glycoprotein Induces Antibodies That Neutralize Subsets of Subtype B and C HIV-1 Primary Viruses. Virology 2006, 353, 268–282. [CrossRef]
- Santra, S.; Tomaras, G.D.; Warrier, R.; Nicely, N.I.; Liao, H.-X.; Pollara, J.; Liu, P.; Alam, S.M.; Zhang, R.; Cocklin, S.L.; et al. Human Non-Neutralizing HIV-1 Envelope Monoclonal Antibodies Limit the Number of Founder Viruses during SHIV Mucosal Infection in Rhesus Macaques. PLoS Pathog 2015, 11, e1005042. [CrossRef]
- Tomaras, G.D.; Yates, N.L.; Liu, P.; Qin, L.; Fouda, G.G.; Chavez, L.L.; Decamp, A.C.; Parks, R.J.; Ashley, V.C.; Lucas, J.T.; et al. Initial B-Cell Responses to Transmitted Human Immunodeficiency Virus Type 1: Virion-Binding Immunoglobulin M (IgM) and IgG Antibodies Followed by Plasma Anti-Gp41 Antibodies with Ineffective Control of Initial Viremia. J Virol 2008, 82, 12449–12463. [CrossRef]
- Yates, N.L.; deCamp, A.C.; Korber, B.T.; Liao, H.-X.; Irene, C.; Pinter, A.; Peacock, J.; Harris, L.J.; Sawant, S.; Hraber, P.; et al. HIV-1 Envelope Glycoproteins from Diverse Clades Differentiate Antibody Responses and Durability among Vaccinees. J Virol 2018, 92. [CrossRef]
- Pollara, J.; Hart, L.; Brewer, F.; Pickeral, J.; Packard, B.Z.; Hoxie, J.A.; Komoriya, A.; Ochsenbauer, C.; Kappes, J.C.; Roederer, M.; et al. High-Throughput Quantitative Analysis of HIV-1 and SIV-Specific ADCC-Mediating Antibody Responses. Cytometry A 2011, 79, 603–612. [CrossRef]
- Koene, H.R.; Kleijer, M.; Algra, J.; Roos, D.; von dem Borne, A.E.; de Haas, M. Fc GammaRIIIa-158V/F Polymorphism Influences the Binding of IgG by Natural Killer Cell Fc GammaRIIIa, Independently of the Fc GammaRIIIa-48L/R/H Phenotype. Blood 1997, 90, 1109–1114.
- Ackerman, M.E.; Moldt, B.; Wyatt, R.T.; Dugast, A.-S.; McAndrew, E.; Tsoukas, S.; Jost, S.; Berger, C.T.; Sciaranghella, G.; Liu, Q.; et al. A Robust, High-Throughput Assay to Determine the Phagocytic Activity of Clinical Antibody Samples. J Immunol Methods 2011, 366, 8–19. [CrossRef]
- Tay, M.Z.; Liu, P.; Williams, L.D.; McRaven, M.D.; Sawant, S.; Gurley, T.C.; Xu, T.T.; Dennison, S.M.; Liao, H.-X.; Chenine, A.-L.; et al. Antibody-Mediated Internalization of Infectious HIV-1 Virions Differs among Antibody Isotypes and Subclasses. PLoS Pathog 2016, 12, e1005817. [CrossRef]
- Hulot, S.L.; Korber, B.; Giorgi, E.E.; Vandergrift, N.; Saunders, K.O.; Balachandran, H.; Mach, L. V; Lifton, M.A.; Pantaleo, G.; Tartaglia, J.; et al. Comparison of Immunogenicity in Rhesus Macaques of Transmitted-Founder, HIV-1 Group M Consensus, and Trivalent Mosaic Envelope Vaccines Formulated as a DNA Prime, NYVAC, and Envelope Protein Boost. J Virol 2015, 89, 6462–6480. [CrossRef]
- Sarzotti-Kelsoe, M.; Bailer, R.T.; Turk, E.; Lin, C.; Bilska, M.; Greene, K.M.; Gao, H.; Todd, C.A.; Ozaki, D.A.; Seaman, M.S.; et al. Optimization and Validation of the TZM-Bl Assay for Standardized Assessments of Neutralizing Antibodies against HIV-1. J Immunol Methods 2014, 409, 131–146. [CrossRef]
- Seaman, M.S.; Janes, H.; Hawkins, N.; Grandpre, L.E.; Devoy, C.; Giri, A.; Coffey, R.T.; Harris, L.; Wood, B.; Daniels, M.G.; et al. Tiered Categorization of a Diverse Panel of HIV-1 Env Pseudoviruses for Assessment of Neutralizing Antibodies. J Virol 2010, 84, 1439–1452. [CrossRef]
- R: A Language and Environment for Statistical Computing. 2023.
- Wickam, H. Ggplot2: Elegant Graphics for Data Analysis 2016.
- Liaw, A.; Wiener, M. Classification and Regression by RandomForest. R News 2002, 2/3, 18–22.
- Torchiano, M. Effsize: Efficient Effect Size Computation 2020.
- Ackerman, M.E.; Das, J.; Pittala, S.; Broge, T.; Linde, C.; Suscovich, T.J.; Brown, E.P.; Bradley, T.; Natarajan, H.; Lin, S.; et al. Route of Immunization Defines Multiple Mechanisms of Vaccine-Mediated Protection against SIV. Nat Med 2018, 24, 1590–1598. [CrossRef]
- Pittala, S.; Bagley, K.; Schwartz, J.A.; Brown, E.P.; Weiner, J.A.; Prado, I.J.; Zhang, W.; Xu, R.; Ota-Setlik, A.; Pal, R.; et al. Antibody Fab-Fc Properties Outperform Titer in Predictive Models of SIV Vaccine-Induced Protection. Mol Syst Biol 2019, 15, e8747. [CrossRef]
- Therneau, T.; Grambsch, P. Modeling Survival Data: Extending the Cox Model; Springer: NY,NY, 2010;
- Harrell, F.E.; Lee, K.L.; Mark, D.B. Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors. Stat Med 1996, 15, 361–387. [CrossRef]
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).