Changes in Antigen-Reactive Antibody Level at Various Time Spans after COVID-19 Vaccination Using Machine Learning

1. Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the novel coronavirus strain causing Coronavirus Disease 2019 (COVID-19) [1]. On March 11, 2020, COVID-19 was eventually classified as a pandemic by the World Health Organization (WHO) [2]. More than 6.3 million people have died from COVID-19 globally, according to the WHO, and more than 500 million cases have been confirmed. Additionally, more than 11 billion doses of the vaccine have been distributed [3]. Fever, sore throat, dry cough, and pneumonia symptoms are among the clinical manifestations of COVID-19 [4]. The Omicron variant, which evolved from the Alpha variant, is currently the most prevalent variant of SARS-CoV-2 and is characterized by increased infectivity [5]. Increased infectiousness and antibody evasion have been linked to the mutations of SARS-CoV-2 spike protein [6].

Scientists have developed COVID-19 vaccines to combat the pandemic. To date, several types of vaccines against SARS-CoV-2 have been developed and widely used worldwide, such as RNA-based type, non-replicating viral vector type, and protein-based type [7]. BNT16b2, mRNA-1273, Ad26.COV2.S, CIGB-66 Abdala, and other common vaccines require one to three doses, depending on the type [7-10]. BNT162b2 contains mRNA encoding a full-length stable S glycoprotein that elicits dose-dependent SARS-CoV-2 neutralizing antibody titers [11]. Two doses of BNT162B2 exhibit approximately 95% protection against COVID-19 [9,12-15]. All current vaccines have high levels of efficacy in reducing COVID-19 severe cases and death while low levels of efficiency in controlling viral infection and mild symptoms [9,10,16,17]. Vaccine coverage must be extended to all countries while maintaining and improving public health control mechanisms to control COVID-19 morbidity and mortality worldwide.

However, the efficacy of the BNT162b2 mRNA vaccine against SARS-CoV-2 decreases over time [11,18]. Similarly, declining immunity only months after immunization has been reported [19-21]. Whether the decline in vaccine protection is linked to a decrease in virus resistance remains unclear. Vaccines trigger a complicated immunological response that includes B and T cells, with B cells producing antibodies [18,21,22]. Spike (S), membrane (M), nucleocapsid (N), and envelope (E) are the four structural proteins encoded by SARS-CoV-2 [23-25]. Most of the antibodies generated by vaccination are directed against the S protein, specifically the receptor-binding domain (RBD) [7,20]. A recent study of antibody alterations following two doses of inactivated COVID-19 vaccine, separated into three groups based on immunization duration, revealed that the levels of antibodies (anti-Spike IgG) decrease with time [26]. However, the SARS-CoV-2-related antibody profiles after vaccination have not been elucidated.

In the present study, we investigated the influence of vaccines on antibody synthesis and monitored changes in antibody levels in the body over time following vaccination. Data on blood antibody levels in a cohort of volunteers vaccinated for COVID-19 vaccines were obtained from Gene Expression Omnibus (GEO) and measured using antigen microarray [27]. The volunteers were examined for their reaction before receiving the mRNA vaccine (Pfizer or Moderna), shortly after receiving the first and second doses, and up to 6 months later. Vaccine-induced antibodies are mainly directed against the S1 and RBD domains of the S protein and to a lesser extent against the S2 domain. Antibody levels are increasing significantly 2 months after vaccination and begin to decline after 6 months. 73 antigens and 1373 volunteer records were involved in the study of Hosseinian et al. [27]. In the present study, 1373 samples were classified into four groups according to the time of vaccination: before vaccination, within 60 days of vaccination, 60–180 days after vaccination, and over 180 days after vaccination. Multiple machine learning methods were integrated to identify key antigen-reactive antibodies that changed after COVID-19 vaccination over time and to establish quantitative rules for accurate prediction. Several essential antigen-reactive antibodies and classification rules were obtained, some of which were extensively analysed. The results of this study could serve as a basis for developing effective vaccines with long-lasting protection and elucidating the defense mechanisms of COVID-19 vaccines.

2. Materials and Methods

2.1. Data and Preprocessing

Individualized antibody reactivity levels for SARS-CoV-2 antigens induced by mRNA vaccines were quantified using coronavirus antigen microarray (CoVAM) following the procedure described by Hosseinian et al. [27] Data were downloaded from the GEO database under accession number GSE199668. The samples were divided into four classes according to the time of vaccination: 104 healthcare workers before vaccination, 534 healthcare workers within 60 days after vaccination, 594 healthcare workers between 60 and 180 days after vaccination, and 141 healthcare workers over 180 days after vaccination [27]. In terms of features, the CoVAM contained 10 SARS-CoV-2 antigens, including nucleocapsid protein (NP) and several varying fragments of the S protein, as well as 4 SARS, 3 MERS, 12 Common CoV, 8 influenza, and 36 other antigens. In terms of feature naming, the virus name was placed at the beginning to distinguish between the different sources of antibodies, followed by the protein name, and the specific tag name followed after the protein name. The normalized fluorescence intensity was used to characterize the expression levels of antigen-reactive antibodies in blood. Above features and four classes comprised a classification problem. By investigating such problem, essential features can be obtained.

2.2. Feature Selection Methods

Several features were adopted to represent samples. Some of them were important to classify samples into different classes, whereas others were not. In machine learning, the important features can be extracted by feature selection methods. To date, lots of such methods have been proposed. It is challenge to select a proper one to process a given dataset. Generally, one single method can only output a part of essential features as each method has its limitations. In this study, we adopted four feature selection methods: least absolute shrinkage and selection operator (LASSO) [28,29], light gradient boosting machine (LighGBM) [30], Monte Carlo feature selection (MCFS) [31] and maximum relevance minimum redundancy (mRMR) [32]. These methods was designed following different principles, meaning that they can overview the given dataset from different aspects. Thus, more essential features can be obtained by applying them to the same dataset. Their brief descriptions are as follows.

Least Absolute Shrinkage and Selection Operator. The LASSO was introduced by Robert Tibshirani in 1996. It is a statistical method used for regularization and feature selection [28,29]. This method reduces the regression coefficients of the redundant features to zero. The feature selection phase occurs after the reduction, where non-zero-valued features are sorted in a list by the absolute value of their coefficients. This study adopted the LASSO program obtained from Scikit-learn [33], which was run with default parameters.

Light Gradient Boosting Machine. The LightGBM is a free and open-source distributed gradient boosting framework for machine learning that was created by Microsoft [30]. It performs regression and classification by transforming weak decision tree (DT) classifiers into strong learners. In addition to regression and classification, LightGBM ranks features based on their importance, quantified by the number of times they are selected for building DTs. A high ranking is given to features that are used frequently. LightGBM was implemented through a Python module, which can be obtained at https://lightgbm.readthedocs.io/en/latest/. It was also performed with default parameters.

Monte Carlo Feature Selection. The MCFS is a useful tool for selecting informative features according to their relative importance in building DTs [31,34-36]. Subsets of features are randomly constructed for many times. For each subset, some samples are randomly selected for training, and the others are for testing. For instance, a DT is built based on 2/3 of the samples that are randomly selected, and the rest is used for testing, which is also repeated for many times. The relative importance (RI) of each feature can then be estimated by considering the number of times they are used to construct the DTs, the information gain of the features, and the weighted accuracy of the DTs. Finally, features can be sorted according to their RI scores. The MCFS program used in this study was retrieved from http://www.ipipan.eu/staff/m.draminski/mcfs.html. Also, it was executed using default parameters.

Maximum Relevance Minimum Redundancy. The mRMR is a classic and powerful feature selection method [32]. It evaluates the importance of features according to their relevance to class variable and redundancy to other features. The relevance and redundancy are all measured by mutual information (MI). Similar to above methods, mRMR also generates a feature list to indicate the importance of features. At first, such list is empty. Then, a loop procedure is executed. In each round, one feature with maximum relevance to class variable and minimum redundancy to features in the current list is selected from all remaining features and appended to the current list. The loop procedure stops until all features are in the list. The mRMR program adopted in this study was obtained from http://home.penglab.com/proj/mRMR/ and it was run with the default settings.

Above four feature selection methods were applied to the dataset mentioned in Section 2.1, resulting in four feature lists, which were called LASSO, LightGBM, MCFS and mRMR feature lists.

2.3. Incremental Feature Selection

Although the feature selection methods can sort features in lists, it still exits a gap for extracting essential features. It is not easy to determine how many top features should be selected. In view of this, incremental feature selection (IFS) was employed in this study [37]. It can find out the optimal number of features for building the classifiers with best performance [38-40]. In the present study, one step interval was applied to each feature list in the IFS method. Under this setting, a series of feature subsets were constructed from a given feature list in a way that the first subset contained the first feature in the list, the second one contained the top two features in the list, and so forth. For each subset, a classifier was built based on one classification algorithm and samples represented by features in this subset. All classifiers were evaluated by tenfold cross-validation [41]. According to the evaluation results, the classifier with the highest performance can be obtained. Such classifier was termed as the optimal classifier and the corresponding feature subset was called the optimal feature set.

2.4. Synthetic Minority Oversampling Technique

As mentioned in Section 2.1, the sizes of four classes were quite different. The classifier directly built on such dataset may produce bias. It should be processed by some advanced computational method. Here, we selected synthetic minority oversampling technique (SMOTE) [42-44]. The idea of this method is to generate synthetic samples for each minority classes, thereby balancing the dataset. In detail, SMOTE randomly chooses a sample from a minority class and determines its k nearest neighbors of the same class. One of its neighbors is randomly selected and a synthetic sample is generated by the linear combination of the sample and its chosen neighbor. This new generated sample is put into the minority class, thereby enlarging its size. This procedure can be performed several rounds until the size of the minority class is same as that of the majority class. Herein, we used the SMOTE tool from https://github.com/scikit-learn-contrib/imbalanced-learn with default parameters.

2.5. Classification Algorithms

In IFS method, one classification algorithm is necessary for building classifiers. This study adopted four classification algorithms: DT [45], K-nearest neighbor (KNN) [46], support vector machine (SVM) [47], and random forest (RF) [48]. These algorithms have wide applications in tackling various medical or biological problems [49-55]. DT uses a tree-like model to build classifiers, which can be extended by maximizing Gini index or information gain in each tree node [45]. The KNN algorithm finds the nearest neighbors of a new sample and categorizes the new sample into one that is shared by most of its nearest neighbors [46]. The SVM can map samples into a high-dimensional space and finds a hyperplane that distinctly classifies samples in different classes. The test samples are then mapped into the same space and the category to which they belong are predicted based on which side of the hyperplane they fall [47]. A RF consists of a large number of individual DTs that operate as an ensemble [48]. Each decision tree in an RF generates class predictions on a test sample, and the class with the most votes is taken as the prediction result.

2.6. Performance Evaluation

The weighted F1 was selected as the major measurement to evaluate the performance of the classifier. To calculate such measurement, the F1-measure for each class should be computed in advance. It is defined as the harmonic mean of other two measurements: recall and precision, where recall is the proportion of correctly predicted positive samples among all positive samples, precision is the proportion of correctly predicted positive samples among all predicted positive samples. The weighted F1 is the weighted average of all F1-measure values on different classes, where the weight for one class is defined as the proportion of samples in this class.

In addition, other measurements were also employed to fully display the performance of classifiers. Macro F1 is another form to integrate the F1-measure values on different classes, which is defined as the average of all F1-measure values. Prediction accuracy (ACC) is the most classic measurement to assess the performance of classifiers. It is defined as the proportion of correctly predicted samples. However, ACC is not accurate when the dataset is imbalanced. Matthew correlation coefficients (MCC) [56] is a more balanced measurement than ACC. Two matrices are used to calculate MCC. One is to store the true class of each sample and the other one is to store the predicted class of each sample. MCC assesses the relationship between these two matrices.

2.7. Extraction of Essential Features for Each Class

Based on IFS method, some essential features can be obtained. However, it is not clear which class they are highly related to. In view of this, we reconstructed the original dataset and applied above feature selection methods on it. For one class, one dataset was generated, in which samples in this class were termed as positive samples and other samples were considered as negative samples. Then, four feature selection methods (LASSO, LightGBM, MCFS, and mRMR) were adopted to investigate this dataset, resulting in four feature lists. From each list, top 20 features were picked up, thereby obtaining four feature subsets. By investigating the overlap of these feature subsets, some essential features that occurred in multiple subsets can be obtained, which were deemed to be highly related to the given class.

3. Results

In this study, a dataset on the antibody reactivity levels for SARS-CoV-2 antigens induced by mRNA vaccines was investigated. The overall computational framework is shown in Figure 1. The results of each step are presented in this section.

3.1. Results of Feature Selection Methods

LASSO, LightGBM, MCFS, and mRMR were used to rank the 73 antigens according to the degree to which they contributed to the classification. These lists are provided in Table S1. For easy descriptions, they were called LASSO, LightGBM, MCFS and mRMR feature lists.

3.2. IFS Results and Feature Intersection

As mentioned above, four feature lists were obtained. Each list was fed into IFS method one by one. Four classification algorithms (DT, KNN, RF, and SVM) were adopted in the IFS method. The performance of each classification algorithm under some top features in each list is listed in Table S2. Using the weighted F1 as the major measurement, we compared the performance of the classifiers based on the same classification algorithm and feature list. Several IFS curves were generated by plotting the weighted F1 on the y-axis and the number of features on the x-axis, as shown in Figure 2 and Figure 3.

For the LASSO feature list, the IFS curves of four classification algorithms are shown in Figure 2(A). It can be observed that when top 47, 73, 21 and 73 features in such list were used, the DT, KNN, RF and SVM can yield the highest weighted F1 values of 0.702, 0.711, 0.735 and 0.733, respectively. Accordingly, the optimal DT, KNN, RF and SVM classifiers can be built with corresponding top features. Their detailed performance, including ACC, MCC, macro F1 and weighted F1, is listed in Table 1. Evidently, the optimal RF classifier was superior to other three optimal classifiers.

For the LightGBM feature list, the obtained four IFS curves are illustrated in Figure 2(B). From this figure, four optimal classifiers can be obtained, which adopted top 40, 18, 31 and 35 features in this list. They produced the weighted F1 values of 0.717, 0.744, 0.742 and 0.758. Table 1 also lists the performance of these optimal classifiers. Clearly, the optimal SVM classifier was a little better than other three optimal classifiers.

For the MCFS feature list, the IFS curves for four classification algorithms are shown in Figure 3(A). It can be observed that the optimal DT/KNN/RF/SVM classifier adopted the top 17/20/23/41 features in this list. The detailed performance of these optimal classifiers is provided in Table 1. Evidently, the optimal SVM classifier was best among four optimal classifiers, which yielded the weighted F1 of 0.765.

As for the last mRMR feature list, Figure 3(B) shows the IFS curves of four classification algorithms. The highest weighted F1 values for DT, KNN, RF and SVM were 0.728, 0.737, 0.745 and 0.758, respectively. Such performance was obtained using top 14, 24, 26 and 30 features in the list. Thus, the optimal DT, KNN, RF and SVM classifiers can be constructed using these features. Their detailed performance is available in Table 1. The optimal SVM classifier provided better performance than other three optimal classifiers.

Based on above results, we can find that the best classifiers on four feature lists. In detail, the best classifier on the LASSO feature list was the optimal RF classifier, whereas it was the optimal SVM classifier on other three lists. The optimal feature subsets were picked up for further investigation. A Venn diagram was plotted for these subsets, as shown in Figure 4. The intersection results of these optimal feature subsets are available in Table S3. The antigens appearing in several feature subsets suggest that they were identified to be essential by multiple feature selection methods. They may play important roles in differentiating healthcare workers at different time spans after vaccination. The biological significance of some antigens (features) will be discussed in Section 4.

3.3. Essential Features for Each Class

The essential features obtained above may not be highly related to one class. To extract the essential features for each class, four datasets corresponding four classes were constructed, as described in Section 2.7. Then, LASSO, LightGBM, MCFS and mRMR were applied to each dataset. Four feature lists were obtained. Top 20 features were picked up and taken intersection. A Venn diagram was plotted for each class, as shown in Figure 5. The specific antigen names are listed in Table S4. For the first class, namely, unvaccinated healthcare workers, antigens such as SARS.CoV.2.S1.RBD.mFc and SARS.CoV.S1.HisTag were identified by all four feature selection methods. For the second class, namely, healthcare workers within 60 days after vaccination, SARS.CoV.2.S1.mFcTag and HuIgM.0.30 were deemed to be important by all feature selection methods. For the third class, namely, healthcare workers between 60–180 days after vaccination, three features (SARS.CoV.2.S1.mFcTag, HuIgM.0.30, and SARS.CoV.2.S1.RBD.mFc) were identified to be essential. For the fourth class, namely, healthcare workers over 180 days after vaccination, MERS.CoV.S1.RBD.367.606.rFcTag, Flu.B_Mal/.HA1, and a-HuIgG_0.03 were screened out by all methods. The discussion on the importance and functionality of some features will be provided in detail in Section 4.

3.4. Classification Rules

It can be observed from Table 1 that the optimal DT classifier was generally inferior to the other three optimal classifiers on the same feature list. However, DT classifier has a great merit that was not shared by other three classifiers. It can provide a group of classification rules, which made the classification procedures completely open. The optimal DT classifiers on four feature lists adopted top 47, 40, 17 and 14, respectively, features in the corresponding lists. All healthcare workers were represented by above features, respectively. And four trees were built by DT, from which four rule groups were established. These rules are provided in Table S5. 190, 183, 202, and 226 classification rules, respectively, were contained in four groups. Each rule is composed of antigen features and their associated fluorescence intensity values, which explains how the feature’s high or low fluorescence intensity influences the capacity to identify the classes of samples. A detailed discussion of some quantitative rules can be found in Section 4.

4. Discussion

We identified a set of antigen-reactive antibodies as potential features that could reveal the effect of COVID-19 vaccines on anti-viral immune activation and reflect changes in antibody levels in the body over time after vaccination by using data on serum antibody levels in volunteers after receiving COVID-19 vaccines. This confirms the potential of such features to contribute to the development of effective vaccines with long-lasting protection. Recent publications have found that some identified features, as well as the relevant quantification rules, are linked to vaccine-induced anti-viral immune activation and duration.

4.1. Key Features for Identifying the Effect of COVID-19 Vaccines on Antibody Production

Using the computational methods, we discovered a set of unique viral antigens-reactive antibodies selected by at least three methods. The antigens we analyzed were distinguished according to virus type (CoV-2, MERS, and other common human coronaviruses), expression system (HEK cells, Baculovirus), and protein fusion tags (such as His and Fc). These top-specific antibodies are closely related to the components of various COVID-19 vaccines, suggesting the protective effect of these vaccines. In the present study, we analyzed 13 specific antibodies, listed in Table 2. In this section, we compared the changes in significant viral antigens-reactive antibodies in the serum of vaccinated and unvaccinated individuals. We also discussed the plausibility and cross-immunization of important antibodies (including non-SARS-CoV-2 antibodies) induced by COVID-19 vaccines.

The top eight features identified were from SARS-CoV-2: S1+S2, S1.mFcTag, S1.HisTag, S1, S2, Spike.RBD.His.Bac, Spike.RBD.rFc, and S1.RBD.mFc. The composition of COVID-19 vaccines is listed in a recent paper comparing these vaccines [7]. The S protein of SARS-CoV-2 was chosen as a promising target by the majority of COVID-19 vaccines because blocking the interaction between the RBD of echinocandin and human angiotensin-converting enzyme 2 (ACE2) is effective in preventing infection [57,58]. In addition, the RBD is part of the S protein’s S1 subunit [59,60]. Suthar et al. highlighted that the S protein of SARS-CoV-2, particularly RBD, stimulates the production of neutralizing antibody NAbs [61]. Similarly, an animal study revealed that RBD-specific IgG accounts for half of the antibody responses induced by S proteins. As a result, given that popular COVID-19 vaccines such as BNT162B1 encode the S protein of SARS-CoV-2, they can stimulate the production of S protein (including S1 and S2 subunits) and RBD-specific antibodies.

SARS.CoV.S1.HisTag and SARS.CoV.S1.RBD.HisTag are top features from SARS-CoV. SARS-CoV and SARS-CoV-2, both belonging to β-B coronavirus, share 79% of their gene sequences [62,63], and the S protein shares 76% of its amino acid identity [64]. SARS-CoV-2 and SARS-CoV share the same host cell receptor ACE2 and are structurally similar; thus, they may exhibit some degree of cross-immunity [58]. These data suggest the effectiveness of SARS-CoV-reactive antibodies against SARS-CoV-2. These results were further confirmed by Wec et al., who isolated several antibodies from a SARS survivor that neutralized coronaviruses such as SARS-CoV-2[65]. Min et al. identified several monoclonal antibodies against SARS-CoV S protein or RBD that are cross-immunoreactive with SARS-CoV-2 [20], which agrees with our predicted features.

MERS.CoV.S1.RBD.367.606.rFcTag from MERS-CoV was the next feature identified. MERS-CoV also belongs to β coronavirus and shares a 50% sequence similarity to SARS-CoV-2 [62], a coronavirus with a high lethality rate. The S protein of MERS-CoV and the RBD in it share some similarities to SARS-CoV-2, suggesting that the cross-immunity of the RBD-specific antibody to the S protein of MERS-CoV against SARS-CoV-2 is less than that of the SARS-CoV-specific antibody but still exists.

The last two identified features, hCoV.HKU1.NP and hCoV.229E.S1, are antigens from β coronavirus hCoV-HKU1 and α coronavirus hCoV-229E, respectively. Cross-immunization with SAR-CoV-2 is possible due to the close relationship. HCoVs are composed of proteins called spike (S), membrane (M), envelope (E), and nucleocapsid (N) [66]. In addition to the S protein, the N protein is an important antibody target [61,67], implying that hCoV.HKU1.NP-specific antibodies contribute to SARS-CoV-2 prevention. Although hCOV-228E is less closely related to SARS-CoV-2 than the other coronaviruses we mentioned above, the potential preventive effect of its specific antibodies against COVID-19 cannot be ruled out. However, given that hCoV-HKU1 and hCoV-229E are common coronaviruses, the detection of these antibodies in the sera of volunteers may attribute to their previous infection.

Research on pan-coronavirus vaccines has attracted increasing attention to prevent novel SAR-CoV-2 variants. Some studies reported that conserved regions on the inner surface of the RBD are potential targets for pan-coronavirus vaccines [68]. New studies of mRNA vaccines against a variety of the more common coronaviruses are underway [69]. In summary, the positive serum test for non-SARS-CoV-2 antigens is could be to the ability of certain antibodies induced by COVID-19 vaccines to act on other coronaviruses. Therefore, the non-SARS-CoV-2 antigens we mentioned above can be seemed as useful features.

4.2. Features Related to Time since Vaccination for Determining the Duration of Specific Antibodies after COVID-19 Vaccination

The essential antigen-reactive antibodies were identified using the computational method and divided into four classes based on vaccination time. The top features from each subclass were selected for discussion. Figure 6 shows the values of these top features in each of the four classes to visualize the changes in the antibodies that target specific antigens over time. Unlike the previous section, this section focuses on the changes in important antibodies at different periods after vaccination according to subclasses, including unvaccinated cases.

The S protein of SARS-CoV-2 is currently the antigen targeted by a majority of COVID-19 vaccines [16,19,26,70,71]. The top features we identified are contained in the S protein of SARS-CoV-2, and antibodies against them all change significantly over time after vaccination.

As shown in Figure 6(A), the first identified feature was SARS.CoV.2.S1+S2. Based on the overall structure of the S protein of SARS-CoV-2 [72], the specificity of the SARS.CoV.2.S1+S2-reactive antibodies was the lowest among the four selected features. As shown in Figure 6 (B)–(D), the second, third, and last identified features were SARS.CoV.2.S1.mFcTag, SARS.CoV.2.S2, and SARS.CoV.2.Spike.RBD.His.Bac, respectively.

According to the changes in the value of each feature in class 1, SARS.CoV.2.S1+S2 and SARS.CoV.2.S2 showed elevated levels, whereas SARS.CoV.2.S1.mFcTag and SARS.CoV.2.Spike.RBD.His.Bac were almost undetectable in serum. Thus, antibodies against the S2 subunit of the S protein were produced earlier after immunization and resulted in relevant specific protection. However, volunteers infected with SARS-CoV-2 before COVID-19 vaccination may also increase the levels of SARS.CoV.2.S1+S2 and SARS.CoV.2.S2.

Comparison of the levels of the four features in class2 revealed that SARS.CoV.2.S1.mFcTag showed the most significant increase, and the values were relatively concentrated within a month after vaccination. The values of SARS.CoV.2.S2 increased less significantly and were less consistent than those of SARS.CoV.2.S1.mFcTag. A study of healthcare workers found a 14-day boost in serum anti-S antibodies, followed by a significant drop in anti-S antibody levels until 42 days after vaccination [73]. Therefore, the levels of other antigens contained within the S protein of SARS-CoV-2 can also elevate antibodies against them within 42 days after vaccination, which agrees with the results of the present study.

Based on the trend from class2 to class4, the values of all features showed varying degrees of decline after 60 days. Among them, the values of SARS.CoV.2.Spike.RBD.His.Bac and SARS.CoV.2.S1.mFcTag declined slower than those of the other features and stimulated some stable antibodies that existed for a longer period. By contrast, the levels of SARS.CoV.2.S1+S2 and SARS.CoV.2.S2 decreased more rapidly, suggesting that the S2 subunit is less ideal as an antibody target than the S1 subunit and RBD after COVID-19 vaccination. Similarly, Previous studies reported that the antibodies identified in the serum following immunization are predominantly anti-S or anti-RBD antibodies [9,10,14] appear to support this hypothesis.

The levels of features in class4 were maintained at high levels, except for SARS.CoV.2.S, which was lower among them. This result indicates that the features found after COVID-19 immunization can persist for more than 6 months (180 days). The immunogenicity of mRNA-1273 lasts for at least 3 months [74], whereas that of BNT162b2 lasts for at least 2 months [12]. The varied compositions based on the type of vaccines can lead to variation in the duration of specific antibody presence. However, the four features identified imply that the S-protein and RBD-specific antibodies are present in the serum for long periods in general.

4.3. Rules for Quantitative Time after COVID-19 Vaccination and Antibody Levels

In addition to the qualitative features, a set of quantitative rules for accurate classification at the time after COVID-19 vaccination were established. All criteria were linked to specific antibody levels, and they were selected using at least two sorting methods. Some top features have been validated as having the ability to classify samples. In the present study, we selected the most typical rules for each time group for further discussion. Table 3 lists all of the rules, followed by a comprehensive analysis.

The first rule (Rule 0) uses four criteria to identify samples that have not received COVID-19 vaccination. The first and second criteria were SARS.CoV.2.S1.mFcTag and SARS.CoV.2.S1.HisTag. However, the former value was less than 5354.39, and the latter value was more than -383.87, implying that the anti-S1 antibody levels were low without vaccination. A recent study has found that a single dose of vaccination is sufficient to produce a strong anti-S1/2 antibody response in SARS-CoV-2-infected people [75]. Another study revealed that antibody responses is not immediate after a single dose of vaccine [13], confirming the credibility and accuracy of the range used for differentiation. As for the third criterion, it was SARS.CoV.2.S1.RBD.mFc. According to rule 0, its level in serum should be between -414.30 and 3773.83, which is relatively low in unvaccinated individuals. Vaccination raises anti-RBD IgG levels in the body [76]. Thus, the level interval anticipated by this rule can aid to rule out vaccinated people. The final criterion in Rule 0 is hCoV.OC43.HE, an antigen from a common coronavirus that causes similar symptoms to the common cold, should be more than 414.54 in the serum. The rise in hCoV.OC43.HE levels may be due to the sample participants having been previously infected with hCoV.OC43. It could also be due to a brief period of vaccine-induced cross-reactive antibodies that bind to other HCoVs [77], considering that vaccinations encourage the creation of more precisely targeted antibodies over time [18], which can help exclude people who have been vaccinated for more time.

The second rule (Rule 1) has three criteria that can be used to identify people from 0 to 60 days after COVID-19 vaccination. The first criterion, which appeared in the previous rule, was SARS.CoV.2.S1.mFcTag, and should contain less than 54010.17 in this rule. Anti-S/RBD antibodies have been identified at high levels 8 weeks after the second dose of mRNA-1273 or BNT162b2 vaccination [14]. Most vaccines stimulate antibody responses against S proteins, and the S1 subunit is included in S proteins; thus, an increase in anti-S1 antibodies should be expected after COVID-19 vaccination. Due to the restricted number of antibodies created by vaccines[78], the maximum value in this rule should be limited; this value should theoretically be highest between 0 and 60 days following immunization[9]. The second and third criteria are SARS.CoV.2.S2 and SARS.CoV.2.S1+S2, whose serum levels should be higher than 37653.75 and 48882.58, as specified in Rule 1. As the S1 and S2 subunits are included in the S protein, changes in the level of S1+S2 specific antibodies should have a strong correlation with anti-S antibodies. A recent study has reported that the levels of anti-S antibodies in serum significantly increase 14 days after vaccination [73], indicating that high levels of SARS.CoV.2.S1+S2 in this rule are useful for differentiation. As for anti-S2 antibodies, Lange et al. measured that the levels of antibodies against S2 are much higher after vaccination [79], whereas Wheeler et al. noted that the intensity of S2-reactive antibody responses is lower than that of anti-S1 and anti-RBD responses after vaccination [13]. These results confirm that the high value of SARS.CoV.2.S2 facilitates the differentiation while the lowest value of SARS.CoV.2.S2 in Rule 1 can be lower than that of SARS.CoV.2.S1+S2.

The next rule (Rule 2) involves three criteria that help us identify individuals 60–180 days after receiving the COVID-19 vaccine. As previously stated, the first and second criteria are SARS.CoV.2.S1.mFcTag and SARS.CoV.2.S1.RBD.mFc, and Rule 2 states that their serum levels must be higher than 5354.39 and between 3773.83 and 33656.48, respectively. The protective ability of the vaccine is proportional to the number of antibodies, and a recent study has shown that the efficiency of COVID-19 vaccines declines from 1 to 6 months following COVID-19 vaccination [80]. Thus, the levels of antigen-reactive antibodies also decline. S1 and RBD-specific antibodies are the primary antibodies induced by COVID-19 vaccines; thus, the levels of SARS.CoV.2.S1.mFcTag and SARS.CoV.2.S1.RBD.mFc should be reduced. Although no study has yet demonstrated the range of levels predicted by our rule, we can still reasonably predict that SARS.CoV.2.S1.mFcTag levels are lower than the range predicted by Rule 1 and that SARS.CoV.2.S1.RBD.mFc levels are higher than the range predicted by Rule 0. The next criterion is SARS.CoV.S1.HisTag, which differs from the first two criteria in Rule 2 in that it is an antigen from SARS-CoV rather than SARS-CoV-2. Given the high similarity of SARS-CoV and SARS-CoV-2 sequences [81], the presence of cross-recognition non-specific epitopes was considered to make SARS.CoV.S1.HisTag predicted by us as a criterion of Rule 2. Lv et al. reported that some SARS-CoV-2-infected individuals can create cross-reactive antibodies that bind to the RBD of SARS-CoV [82], implying that the COVID-19 vaccination can stimulate similar cross-reactive antibodies in individuals.

The last rule (Rule 3), which is used to distinguish people who have been vaccinated for more than 180 days, involves two criteria. In specific, SARS.CoV.2.S1.mFcTag and SARS.CoV.2.S1.RBD.mFc should be between 5354.39 and 34194.92 and above 3773.83, respectively, in serum. The values predicted in Rule 3 were similar to the content predicted in Rule 2 probably because the vaccine-induced production of S1- and RBD-reactive antibodies has dropped to its lowest level within 180 days [83,84], after which their values remain stable in the serum. However, unlike in Rule 2, SARS.CoV.2.S1.mFcTag levels are limited to a maximum value in this rule, indicating that levels have decreased overall, which would help exclude individuals within 180 days of COVID-19 vaccination. Similarly, SARS.CoV.2.S1.mFcTag and SARS.CoV.2.S1.RBD.mFc were predicted to be higher in this rule than in rule 0, implying that the vaccine induces the production of stably present anti-S1/RBD antibodies, which can effectively exclude unvaccinated populations.

5. Conclusions

Combining data on serum antibody levels in volunteers after COVID-19 vaccination and advanced machine learning methods, a set of antigen-reactive antibodies were extracted, which could reveal the effect of the vaccine on antiviral immune activation and reflect changes in antibody levels in the body over time after vaccination. In the computational framework, four efficient feature selecting algorithms, namely, LASSO, LightGBM, MCFS, and mRMR, were performed to rank the features according to their contributions to the classification. Then, through the IFS method, the optimal features for four classification algorithms (DT, KNN, RF, SVM) in each feature list were confirmed. Subsequently, the overlapping features by taking the intersection of the optimal feature subsets corresponding to the four feature selection algorithms were identified, such as SARS.CoV.2.S1.mFcTag, SARS.CoV.2.Spike.RBD.His.Bac, and SARS.CoV.2.S1+S2. Meanwhile, we determined the specific features that were highly related to one class. In addition, the classification rules were constructed, which can explain quantitatively the important roles of features in the classification.