Role of Machine and Deep Learning in Predicting Protein Modification Sites: Review and Future Directions

1. Introduction

Protein synthesis follows the central dogma of genetics and involves three primary processes: replication, transcription, and translation. Protein post-translational modification (PTM) generally occurs during translation. After synthesis, proteins undergo various modifications like phosphorylation [1], acetylation [2], methylation [3], ubiquitination [4], and glycosylation [5]. These modifications expand the functional diversity of proteins and increase their complexity significantly. PTMs play crucial roless in cellular and organismal functions, impacting processes such as cell differentiation, apoptosis, protein degradation, protein-protein interactions, and gene expression and regulation. Furthermore, PTMs are closely linked to human diseases, with current targeted therapies involving regulatory enzymes associated with these modifications. Thus, studying protein PTMs is essential for advancing our understanding of biological processes.

Experimental methods are adept at accurately identifying protein modification sites, with mass spectrometry[6] being the predominant approach, complemented by liquid chromatography[7], and radiochemical methods[8]. However, as sequencing technologies continue to advance, an increasing number of protein sequences have been discovered, rendering traditional experimental methods insufficient for managing the vast scale of data. In this context, computational methods emerge as viable alternatives for analyzing protein sequences and identifying the corresponding modification sites. Machine-learning methods have been successfully used in the field of modification-site identification. However, current prediction methods still need to be improved, such as the simplicity of features and classification methods and the unreliable sequence in the training set[9]. Deep learning methods have been increasingly used in PTM site recognition research to address the limitations of machine learning methods, yielding promising results[10]. The information on modification sites provided by computational prediction is merely speculative, and their biological authenticity must be ultimately confirmed through experimental validation.

Machine learning-based computational identification methods typically consist of six key steps. First, data were obtained from established databases. Second, the acquired data were pre-processing. Third, sequence or structural features are all derived from protein sequences. These features include sequence position information, amino acid physicochemical properties, protein structure information, and so on. Fourth, redundant or insignificant features are eliminated using feature dimension reduction or selection methods. Fifth, a suitable model was chosen for training. Finally, the performance of the model is assessed using a test set. Most machine-learning-based computational identification methods adhere to this fundamental process, as illustrated in Figure 1. This manuscript summarizes the research according to the machine learning process to better understand its application in PTM identification and discusses potential future research directions to enhance the sufficiency of identifying modification sites.

2. Datasets and Data Pre-Processing

2.1. Dataset

With continuous advancements in sequencing technology and proteomics, researchers have developed numerous PTM databases that can be used by other researchers. The following section provides an overview of several popular databases, with additional information in Table 1.

2.1.1. Uniport

UniProt [11,12] is a comprehensive database that offers protein structure, sequence information, functional annotations, Gene Ontology (GO) annotations, subcellular location data, PTM information, similar proteins, and more. UniProt database is an authoritative repository of protein sequence and functional information, systematically integrating annotations of PTMs. These annotations, after manual curation, are stored in Swiss-Prot entries and are centrally displayed in the dedicated “PTM/Processing” module, covering various modification types such as phosphorylation and glycosylation, with specific amino acid residue sites of modification clearly annotated. The data sources include published experimental evidence and reliable computational predictions, often linked to relevant literature. Moreover, PTM information does not exist in isolation but is deeply interconnected with modules such as “Function”, “Disease and Variants”, and “Sequence”, collectively elucidating the biological significance of PTMs in regulating protein activity, localization, interactions, and stability. Simultaneously, UniProt provides cross-references to specialized PTM databases like PhosphoSitePlus and GlyGen, serving as a comprehensive PTM information hub that guides users in further exploration.

2.1.2. dbPTM

dbPTM [13,14] is a comprehensive resource for PTMs of proteins. The database contains 2,235,664 experimental PTM sites and over 70 PTM types integrated into more than 40 databases and includes 30 benchmark datasets. In addition, dbPTMs offer information on the association between modification sites and diseases, which can be valuable for disease research. Researchers can select specific modification sites for data download by clicking on the download bar. Instead of providing the entire protein sequence, the database offers protein fragments with details of the modification site, each fragment being 21 amino acids long.

2.1.3. CPLM 4.0

The Compendium of Protein Lysine Modifications 4.0 (CPLM 4.0) [15] is a comprehensive data resource that builds on previous versions of CPLA [16], CPLM [17] and PLMD [18]. CPLM 4.0 focuses on protein lysine modification. This database includes a significant number of modification events encompassing a wide range of unique sites on various proteins. In total, 105,673 proteins were included in CPLM 4.0, with data pertaining to up to 29 different types of protein lysine modifications across 219 different species [15].

2.2. Data Pre-Processing

Data pre-processing involves three primary steps. First, the protein sequence was segmented to generate fragments. The next step was to collect trustworthy negative data for building the dataset. Finally, the problem of imbalanced datasets was investigated to mitigate the potential adverse effects. The data pre-processing workflow is illustrated in Figure 2.

2.2.1. Sequence Slice

In current studies on PTM, most researchers have chosen the peptide representation method outlined in Equation (1).

$$S=P_{-\epsilon }\dots P_{-2}{P}_{-1}{P}_0{P}_1{P}_2\dots P_{\epsilon }$$

(1)

$P_0$ signifies the central amino acid in focus for PTM site recognition studies, aiming to determine if these amino acids undergo modifications. For instance, in methylation recognition studies, $P_0$ represents either lysine or arginine. The $P_{-\epsilon }\dots P_{-2}{P}_{-1}$ represents the upstream $\epsilon$-th amino acid from the central amino acid $⨂$, whereas$P_1{P}_2\dots P_{\epsilon }$ denotes the downstream $\epsilon$-th amino acid from the central amino acid $P_0$. Therefore, the length of the peptide is $2\epsilon +1$. It is customary to use ‘-’ or ‘X’ as placeholders when there are insufficient upstream or downstream amino acids.

The lengths of the peptides used in PTM site studies vary from one research project to another. Lai et al. [32] developed an auto-machine learning method to predict lysine lactylation sites in 51 amino acids. Wei et al. [33] used 11 residues to predict the methylation sites. Li et al. [34] used peptides with 31 amino acids. Nie et al. [35] trained on 27-peptide sequence segments and tested 20 sequences. Lyu et al. [36] segmented proteins into 35-residue segments with cysteine at the center. Auliah et al. [37] used a local sliding window of 57 residues to predict pupylation sites. Bao et al. [38] created 27-tuple peptides for K-PTMs. The peptide lengths available in dbPTM and PhosphoSitePlus were 21 and 15 bp, respectively. The choice of peptide length may affect the prediction results, leading researchers to explore various lengths. Khalili et al. [39] investigated window sizes ranging 7–35 and found that a window size of 13 yielded the best performance in their models.

2.2.2. Sequence Redundancy

CD-HIT, a method proposed by Li et al. [40], has been widely used to eliminate homologous protein data. CD-HIT uses a clustering approach to identify and remove similar protein sequences.

The algorithm is as follows [40,41]: The algorithm sorts sequences according to their length. The longest sequence is designated as the representative of the first cluster. Next, the remaining sequences are compared with representatives in the existing class [42]. If the similarity exceeds a predefined threshold, a new sequence is added to the existing class [43]. Otherwise, a new class was created. CD-HIT is widely used in PTM site studies to eliminate the homologous protein sequences and residual fragments. Some studies aimed to remove redundant protein sequences [34,44], whereas others focused on eliminating fragmented residue sequences [45,46].

2.2.3. Selected Reliable Negative Sequences

Although databases commonly provide information on modification sites, they do not provide non-modification site information. Peptides without annotation information (known as unlabeled data) can be categorized into two groups. One is that the site has not been identified as a modification site and the other is that additional research is required to verify whether the site has indeed been modified. Therefore, we obtained reliable positive sequences; however, there may be potential issues with the negative data. The modification site prediction problem can be described as a positive-unlabeled (PU) problem [9,47,48]. Two methods can be used to address the PU problem in modification site prediction.

Segments without modification information are considered negative samples by default [32]. The proposed method is straightforward and manageable. Several studies have used this method to build models; however, it ignores the potential for modifications. Gao et al. [49] established three criteria for identifying non-phosphorylated sites: (1) the fragment must not be labeled as a positive site, (2) the fragment should be within the sequence containing a positive site, and (3) the negative site must be solvent-inaccessible. In [39,50], all residues from a protein with a minimum of three confirmed positive sites were considered negative sites.

Another method aims to build models using limited positive data and large amounts of unlabeled data. Ning et al. [51] used semi-supervised learning and a support vector machine (SVM) to select reliable negative data. Jiang et al. [48] introduced the PUL-PUP algorithm for acquiring negative data. PUL-PUP initially used similarity to identify negative data distant from positive data. Subsequently, PUL-PUP iteratively trains the SVM to expand the reliable negative data.

2.2.4. Balanced Dataset

In PTM studies, the amount of positive data is less than that of negative data, leading to a data imbalance. Imbalanced datasets may have detrimental effects on model training. Various methods have been proposed, such as data-based, algorithm-based, and hybrid-based methods.

(1) Data based methods

Data-based methods can be divided into over-sampling [52], under-sampling [53], and hybrid sampling [54,55,56]. Random over-sampling (ROS) randomly duplicates minority samples to achieve a balanced dataset. The synthetic minority over-sampling technique (SMOTE) [52] generates new samples by considering the k-nearest neighbors of each minority class sample, and is widely used to address imbalanced datasets such as SulSite-GTB [57]. Adaptive synthetic sampling (ADASYN) [53] is an advanced method that can generate samples based on the learning difficulties of individual minority samples. Balanced datasets can also be achieved through undersampling, which involves reducing the number of majority samples. Random under-sampling (RUS) selects sequences from the majority of subsets and is widely used in PTM prediction. NearMiss [58] selects a subset of the majority class samples closest to the minority class samples as representatives. ENN [59] is a KNN-based method that avoids interference samples. Hybrid sampling methods such as SMOTETomek [60,61] and SMOTEENN combine oversampling with under-sampling.

(2) Algorithm-based method

Algorithm-based methods solve imbalanced datasets by shifting the focus toward minority class samples, including cost-sensitive learning [62], ensemble learning [63,64], and one-class classification [64,65]. Cost-sensitive learning uses cost functions to build classifiers by minimizing the misclassification cost, and adjusts the cost of misclassified samples based on a cost matrix [66,67,68]. Commonly used methods include weighted SVM [69], fuzzy SVM [70,71], and cost-sensitive neural networks [72]. Ensemble learning can reduce the bias caused by a single learner and enhance the model efficiency. Bagging, boosting, stacking, and hybrid models are the most representative ensemble-learning models. RUSBOOST [73] trains weak classifiers by constructing multiple balanced datasets by using a combination of RUS and AdaBoost. Jia et al. [74] used an ensemble method to predict the O-GlcNAcylation sites. One-class learning, or novelty detection, is a useful approach for handling significant imbalances between positive and negative samples. This technique builds models for the minority class, for example, a One-class SVM [75].

(3) Hybrid-based methods

Based on the aforementioned methods for handling imbalanced datasets, researchers have proposed hybrid methods that combine different balancing strategies to further enhance model performance. For instance, Islam et al. [76] used undersampling and K-nearest neighbors (KNN) to balance data. In [73] a combination of RUS and AdaBoost was used to balance a dataset. Reference [54] implemented hybrid sampling and a bagging classifier to address imbalanced learning.

2.2.5. Data Splitting

To train a model and objectively assess its generalization capability, the dataset must be divided into training and test sets following specific guidelines. The hold-out method, a widely used and straightforward approach, involves splitting data according to a predefined ratio, such as 70% for training and 30% for testing. Alternatively, a temporal partitioning strategy may be employed, using data collected before 2022 for training and data from 2022 to 2025 for testing. Regardless of the chosen method, it is crucial to ensure that the training and test sets have similar data distributions and that no test data is included in the training set.

3. Feature Engineering

3.1. Feature Extraction

Machine learning cannot recognize sequence data directly. Therefore, researchers must design algorithms to complete feature extraction. Feature extraction methods are categorized into three types: (1) sequence-based methods, (2) physicochemical-based methods, and (3) annotation-based methods. With the development of deep learning, language models have been used to predict PTM sites.

Sequence-based Feature

Sequence-based methods commonly use the composition, position, and other relevant information on amino acids to achieve a numerical representation of protein sequences. In sequence-based method, placeholder (‘-’ or ‘X’) is considered a particular type of amino acid. Therefore, the total number of amino acid residues was 21.

Amino acid composition (AAC) [77,78,79,80] is a common feature extraction method based on the frequency of amino acids in sequence segments. AAC can yield 21 features, including 20 amino acids and one placeholder. The composition of k-spaced amino acid pairs (CKSAAP) uses the frequency of amino acid pairs with the separation of k spaces to represent the protein sequence, where the value of k is available. In particular, CKSAAP has 441 features (from AA, AC, to XX) when k is 0. One-hot encoding [79] commonly used in deep learning, in which each amino acid is converted into a vector of length 21. Finally, a protein fragment of length L was depicted as a two-dimensional (2D) matrix of size L×21. The use of machine-learning methods to extract features is an innovative approach. The K nearest neighbor (KNN) Score [81,82,83] was used to characterize the fragments.

In addition to the methods discussed above, there are several other sequence-based feature-extraction methods. For instance, conjoint triad descriptor (CTriad) [80,84], dipeptide composition (DPC) [80,85], amino acid pair composition (AAPC) [77,79,86], pair potential [49,87], four-body statistical pseudo-potential [49,88], local structural entropy [49,89], information of proximal PTMs [51], position-special amino acid propensity (PSAAP) [51,90,91], enhanced amino acid pair (EAAC), enhanced group amino acid pair (EGAAC) [79,92], and position weight amino acid composition (PWAAC) [57].

3.1.1. Physicochemical Properties

Numerous studies have demonstrated variations in physicochemical properties between sites that undergo PTMs and those that do not. The physicochemical properties of amino acids not only reveal the biological characteristics of PTM sites, but can also aid in predicting the development of identified models. In physicochemical properties, placeholders (‘-’ or ‘X’) are either disregarded or assigned a default value such as 0.5.

The AAindex database, comprising 566 amino acid properties [24,80,93,94,95], can be integrated with computational techniques such as grey models, principal component analysis, and clustering to enhance feature extraction efficiency. Secondary structure (SS) [80,96] is determined using SPIDER2, converting fragments into a 63-dimensional vector that includes probability scores for α-helix, β-helix, and coil for each amino acid (with placeholders). The composition, transition, and distribution (CTD) method [80,97,98] introduced by Dubchak et al. categorized 20 amino acids into three groups based on eight properties, ultimately transforming the fragments into a 188-dimensional vector.

There are some commonly used feature extraction methods, such as backbone torsion angles (BTA) [80,94,99], accessible surface area (ASA) [49,80,100,101,102], physio-chemical properties (PCPs), other binding sites for any chemical groups [94], positively charged amino acid composition (PCAAC), discorded regions by DISOPRED2 [94,103], BioJava [94,104], disorder [49,105], grey pseudo amino acid composition[51], and encoding based on grouped weight (EBGW) [57].

3.1.2. Annotation Information

Protein annotation information typically encompasses basic, structural, and functional details as well as other pertinent information. These data aid in comprehending the structure, function, and significance of proteins within organisms and are frequently used to describe protein fragments.

Numerous annotation-based methods exist, for example, position-specific scoring matrix (PSSM)[76,77,79], evolutionary-based profile bigrams [76,106,107,108], gene ontology (GO) [94,109], InterPro [94,110], KEGG [94,111], Pfam [94,112], STRING [94,113], functional domain [94], active site [94], natural variants [94], BLOSUM62 scoring matrix (B62) [77,114], evolutionary conservation score [49,115,116], and pseudo-position specific scoring matrix (PsePSSM)[57,117].

The PSSM is the most popular method. The PSSM is a $20\times L$ matrix, where $L$ represents the length of the fragment. Each column corresponds to a residue position in the protein sequence and each row represents one of the 20 possible amino acids. Each element (i, j) in the matrix represents the probability or score of the j-th position in the protein sequence being mutated into the i-th amino acid during evolution. This score typically reflects the degree of conservation and the preference for a specific amino acid at that position.

3.1.3. Network-Based Feature

Deep learning is extensively employed in PTM site recognition research, serving dual purposes: as a classifier for prediction, and as a tool for extracting features from network structures. Convolutional neural networks (CNNs) extract features and reduce dimensionality via convolutional and pooling layers, while recurrent neural networks (RNNs) capture sequence context. Some studies integrate these approaches for hybrid feature extraction.

Natural language processing (NLP) has developed rapidly in recent years. The protein sequences are similar to those of natural languages in several respects. First, both datasets were sequential. Second, both contained contextual information. Several studies have used language models to extract the features of protein fragments. Bidirectional encoder representations from transformers (BERT) are commonly used to predict PTM sites. Alkuhlani et al. [118] used six protein language models, ProtBERT-BFD [119], ProtBERT [119], ProtALBERT [119], ProtXLNet [119], ESM-1b [120], and TAPE [121] to identify PTM sites based on BERT [122], Albert [123], and XLNet [124]. Qiao et al. used BERT to build a novel predictor, BERT-Kcr, for protein Kcr sites prediction [10]. Lyu et al. [36] used word embedding to encode protein fragments, whereas Wang et al. [125] predicted plant ubiquitination using the word2vec feature extraction method.

Post-translational modification is intrinsically linked to the enzyme, with enzyme-substrate relationships deducible from the physical and chemical properties of modification sites through feature engineering. Deep-PLA [126] demonstrates the effective integration of enzyme-specific constraints into deep neural network architectures, offering a pertinent case study for this topic.

3.2. Feature Reduction

We introduce the four types of feature-extraction methods in detail. Several studies have used multiple feature extraction methods to obtain comprehensive feature sets. However, ensemble method often leads to the challenge of an excessive number of feature dimensions. Redundant and nonessential features may reduce the efficiency of the predictor. Important features are retained through feature reduction, whereas those with lower importance are eliminated. Consequently, the final feature vector exhibits a lower dimensionality yet higher relevance. Feature-reduction approaches can be divided into two types: feature selection, which only reduces the number of features, and feature transformation or dimensionality reduction, which focuses on decreasing complexity by transforming existing features. In modification site prediction studies, these two methods are commonly used to optimize feature sets.

Auliah et al. [37] used the chi-squared test to perform feature selection. The chi-square test is a widely used hypothesis testing method that is important in statistics. The chi-square test was used to examine whether the two variables were independent. Maximal-relevance-maximal-distance (MRMD) was used to rank the importance of features in [33]. Li et al. [127] combined analysis of variance (ANOVA) with incremental feature selection (IFS) to find the most vital feature subset. Minimum redundancy maximum relevance (mRMR) [128,129,130,131] was often used to select optimal features from the entire feature set. He et al. [132] proposed a feature selection method called MRMD3.0, which consisted of two steps. The first step contained nine feature rank methods (tree importance, ANOVA, variance threshold, chi-squared, linear model method, mutual information, minimum-redundancy-maximum-relevance, max-relevance-max-distance, and recursive feature elimination) and four-link analysis strategies (PageRank, Trust Rank, Leader Rank, and HITS). The second step uses IFS to select the best feature subset. Ensemble methods are commonly used for feature selection. Yu et al. [133] selected the features via XGboost [134]. Principal component analysis (PCA) is widely used to reduce dimensionality. This method can describe existing high-dimensional feature sets using fewer comprehensive features. Another standard method for feature dimensionality reduction is singular value decomposition (SVD).

In contrast to machine learning, deep learning methods can automatically learn feature information from input data without manual feature extraction. In deep learning, feature-dimensionality reduction is typically not treated as a distinct step. However, in the model structure, some processes are similar to feature selection, such as the pooling layers in convolutional neural networks.

4. Classifiers

In this step, machine-learning methods are trained using the feature set obtained from the previous stage. Currently, popular machine-learning methods in the field of PTM site prediction include support vector machines, naïve Bayes, and decision trees. As research continues to advance, ensemble and deep learning are increasingly being applied in the study of modification site recognition.

4.1. Machine Learning Classifier

Support vector machines (SVM) separate protein fragments by creating an optimal hyperplane for classification, which is particularly effective with small sample data. Xu et al. [135] used SVM to identify protein lysine glycation using sequences. Bao et al. [38] successfully used SVM and multilayer neural networks to predict various PTM sites. Auliah et al. [37] used multiple classifiers to assess the recognition efficiency of PUP-Fuse. Decision Trees use if, then judgment rules for protein fragment classification. K-Nearest Neighbors (KNN) predicts unknown protein fragments based on the nearest samples. Ning et al. [136] used KNN as a classifier to identify formylation sites and explored the impact of different values of K on the experimental results. Artificial Neural Network (ANN) is popular classifiers in bioinformatics that mimic the structures and functions of biological neural networks [137]. Several studies used ANN to predict PTM sites [138,139].

Ensemble methods may enhance the prediction accuracy. Ensemble methods can be categorized into three types: Bagging, Boosting, and Stacking. Bagging resembles voting. Basic classifiers have been used to predict protein fragments, resulting in various outcomes. The category of unknown protein fragments was determined based on the most frequent category. Random Forest (RF) [140,141] is a representative bagging algorithm commonly used in PTM site prediction. Hasan et al. [142] used RF to predict S sulfenylation sites. Cascade Forest [143,144] uses a layered approach comprising multiple forest structures, where the input for each layer is derived from the output feature information of the preceding layer. This methodology facilitated incremental feature extraction and allowed adaptive adjustments to the complexity of the model. Qian et al. [145] proposed a novel predictor, SUMO-forest, based on cascade forests. Boosting refines the base learner by adjusting the data sample weights, and ultimately determines the segment classes through weighted voting. Gradient tree boosting (GTB) [146,147] is a popular boosting method using multiple decision tree (DT) with excellent performance, which has been used in multiple fields. Wang et al. [57] proposed SulSite-GTB to predict the S-sulfenylation sites based on GTB. The stacking method uses multiple-base learners to identify protein fragments and generate classification results. These classification results were then used as features for another learner to learn and produce the final classification results. He et al. [148] used stacking ensemble layers to build a predictor in which the base learners were convolutional neural with different specifications. In addition to the aforementioned ensemble methods, other hybrid methods also exist. Zhang et al. [149] integrated five classifiers–RF, SVM, GBDT, KNN and Logistic Regression–to predict lysine malonylation sites.

4.2. Based on Deep Learning

Deep learning has been extensively used to predict PTM sites. Unlike traditional machine learning techniques that require manual feature design and selection, deep learning models can autonomously learn data feature representations without human intervention.

CNN consists of convolution, pooling, and fully connected layers. The convolution layer extracts features from the input data, and the pooling layer selects these features. The fully connected layer classifies unknown protein fragments. Wang et al. [150] used a CNN to predict multiple PTMs. Zhao et al. [151] used a CNN to predict Kcr. CNN-SuccSite [79] was developed as a CNN model for predicting lysine succinylation sites, comprising an input layer, two convolution layers, two max-pooling layers, two fully connected layers, and an output layer. Wei et al. [152] created a one-dimensional (1D) CNN to predict Kcr sites. However, RNN is advantageous for sequential data due to its ability to handle sequences of varying lengths and capture temporal dependencies. RNN provide rich contextual information and include two common variants: long short-term memory (LSTM) [153] and gated recurrent unit (GRU). Lyu et al. [36] constructed a five-layer LSTM model featuring an input layer, word embedding layer, LSTM layer, dense layer, and output layer to predict cysteine sulfophenylation sites. Li et al. [154] proposed a transfer learning model based on LSTM to predict lysine propionylation, whereas Mul-SNO [155] combined bidirectional long short-term memory (BiLSTM) and bidirectional encoder representations from transformers (BERT) to predict S-nitrosylation sites. Yu et al. [156] used a CNN-LSTM hybrid network for feature extraction and prediction sites. Currently, some studies have integrated deep learning with machine learning to enhance prediction efficiency. Ning et al. [157] combined 4-layer DNN and penalized logistic regression for succinylation site prediction. PROSPECT [158], proposed by Chen et al., integrates two CNNs and an RF to predict phosphorylation sites.

Transformer[159] is a deep learning model that utilizes the self-attention mechanism. Its primary innovation is the ability to capture global dependencies among all elements in a sequence through parallel computation, which enhances the model’s capability to contextualize information within that sequence. The Transformer architecture comprises an encoder and a decoder, with each layer featuring multi-head self-attention mechanisms and feed-forward neural networks. This design enables the model to dynamically assess the significance of each amino acid in the input sequence. Meng et al. [160] proposed TransPTM, a transformer-based neural network model for non-histone acetylation site predication. Liang et al. [161] proposed an effective model named DeepMM-Kcr, which is based on multiple features and multi-head self-attention mechanism.

The core idea of transfer learning is to apply the knowledge, including model parameters and feature representations, acquired from solving one task (the source task) to another related but distinct new task (the target task). This application enhances the learning efficiency and performance of the new task. In some instances, data for certain modification sites may be limited, potentially resulting in insufficient annotated data to train a high-performance model. Utilizing transfer learning methods can effectively address this challenge. Xu et al. [162] developed DTL-NeddSite, a convolutional neural network-based predictor that leverages deep transfer learning and one-hot encoding. The model was first trained on a large dataset of lysine post-translational modification sites, and then fine-tuned using neddylation site data to construct the target model. Soylu et al. [163] developed the DEEPPTM model, integrating a protein embedding approach using ProtBERT with an attention-based Vision Transformer (ViT) to enhance modification prediction accuracy and elucidate the relationships between modification types and protein sequences.

5. Measurement

The prediction of PTM sites is a binary classification problem. The modified sites were divided into positive and unlabeled data. Typically, unlabeled data are considered negative samples. Researchers typically use accuracy (ACC), Matthews correlation coefficient (MCC), F-measure, and area under the receiver operating characteristic curve (AUC) to assess classifier performance. The formulas for these metrics are as follows:

$$\mathrm{A}\mathrm{C}\mathrm{C}={\displaystyle \frac{TP+TN}{TN+TP+FN+FP}}$$

(2)

$$\mathrm{M}\mathrm{C}\mathrm{C}={\displaystyle \frac{\left(TP\times TN\right)-\left(FN\times FP\right)}{\sqrt{\left(TP+FN\right)\left(TN+FP\right)\left(TP+FP\right)\left(TN+FN\right)}}}$$

(3)

$$\mathrm{F}1={\displaystyle \frac{2\times \frac{TP}{TP+FP}\times \frac{TP}{TP+FN}}{\frac{TP}{TP+FP}+\frac{TP}{TP+FN}}}$$

(4)

$$Sn={\displaystyle \frac{TP}{TP+FN}}$$

(5)

$$Sp={\displaystyle \frac{TN}{TN+FP}}$$

(6)

where TP denotes the number of correct classifications in the positive dataset. The TN represents the number of correct classifications in a negative dataset. FN is the number of false-negative results. where FP is the number of false positives.

The receiver operating characteristic (ROC) curve was originally used for radar-signal detection to differentiate between signals and noise. Subsequently, the researchers adopted it for the model evaluation. The horizontal axis of the ROC curve represents the false positive rate (FPR), and the vertical axis represents the true positive rate (TPR). Owing to the curved nature of the ROC, assessing the quality of the model can be challenging. Therefore, in practical applications, the area under the curve (AUC) serves as a measure of the model performance and is particularly beneficial for handling unbalanced data.

6. Summary of Predictors

With advancements in machine and deep learning, there has been a surge in research focusing on predicting PTM sites. Table 2 summarizes various studies, including PTM types, datasets, window sizes, feature extraction methods, prediction models, and web servers.

7. Challenges and Future Directions

Currently, significant progress has been made in the identification of PTM sites using machine learning and deep learning techniques. However, there are still several noteworthy aspects that warrant further attention.

7.1. Data Limitations

The study of PTM sites based on machine learning and deep learning requires a large and accurate dataset for model training. As previously mentioned, there are three main issues concerning the data on modification sites: (1) the absence of completely reliable negative examples; (2) the significant imbalance present in the datasets; and (3) current modification site databases reveal an underrepresentation of certain modification types. For example, the dbPTM database lists only 194 O-palmitoleoylation sites, encompassing both experimentally validated and predicted instances. It is insufficient to support the training requirements of deep learning or machine learning.

Several studies have proposed effective solutions to these issues. However, most of these solutions rely on sampling-based methods for dataset acquisition, often addressing only a single aspect of the problem without thoroughly examining the construction of the dataset. The construction of comprehensive and representative datasets remains a critical challenge in the field of PTM recognition. Negative instance data, which is typically more abundant than positive instance data, is often unreliable and contributes to an imbalanced dataset. To address this issue, further investigation into the application of semi-supervised learning and one-class learning approaches in dataset construction and model building processes is warranted. Transfer learning is a machine learning technique that leverages knowledge gained from one task to improve performance on a related but distinct task. By pre-training a model on a large dataset and then fine-tuning it on a smaller, task-specific dataset, transfer learning can mitigate the challenges posed by limited training data, thereby addressing the issue of underfitting that may arise when working with insufficient data for certain modified sites.

7.2. Interpretability

In recent years, deep learning techniques have shown considerable application value in bioinformatics, particularly in the prediction of protein modification sites. However, these models typically utilize complex nonlinear network architectures, which leads to a lack of interpretability regarding their internal mechanisms. This ‘black box’ nature not only undermines researchers’ trust in the predictive outcomes of these models but also ignites discussions concerning the reliability of algorithmic decisions in biomedical applications.

Establishing interpretable models is crucial in the study of modification sites. Interpretability methods aim to transform the opaque prediction processes of black-box models into specific biological explanations, elucidating the reasons behind the emergence of modified sites. These approaches seek to identify key sequence features that determine modification sites, indicating not only where modifications occur but also explaining why they occur at specific locations. Furthermore, developing interpretable methods that correlate sequence-level importance with three-dimensional protein structures can elucidate the spatial and physicochemical constraints on modifications. This includes explaining why certain sites are more readily recognized and bound by modifying enzymes, often due to their exposure on protein surfaces or within specific domains. Ultimately, the core value of interpretability analysis lies in converting model predictions into experimentally verifiable scientific hypotheses, thereby exploring the relationship between modification sites and biological activity. These models are categorized into post-hoc and ante-hoc interpretability. Post-hoc interpretability encompasses methods such as example-based, attribution-based, latent semantics-based, and rule-based approaches. In contrast, ante-hoc interpretable learning prioritizes interpretability as a fundamental objective during model design and training. This involves adopting a transparent model architecture or imposing interpretability constraints during training. When developing interpretable models, it is essential to design self-transparent neural networks that maintain model performance and efficiency while selecting appropriate interpretability evaluation metrics.

7.3. Interplay Between Modification Sites and Processes

Protein sequences may contain multiple modification sites, which play a crucial role in the functionality and stability of proteins. PTM crosstalk plays a crucial role in organisms and can be categorized into crosstalk within the same protein (intra- protein) and crosstalk between different proteins (inter-protein). Currently, some studies are committed to finding PTM crosstalk pairs[178,179,180]. However, there is limited research on PTM crosstalk mechanisms and crosstalk interaction networks.

Research has shown that changes in certain modification sites may lead to diseases [181]. Therefore, studying the relationship between modification sites and diseases is crucial for a deeper understanding of disease progression, intervention mechanisms, and the development of precision therapeutic drugs. However, there is currently limited research utilizing machine learning or deep learning methods to explore the relationship between modification sites and diseases, making this direction worthy of further exploration.

8. Conclusion

The accurate identification of PTM is crucial for various applications, including drug development, disease diagnosis, and the understanding of molecular processes. Although traditional biological experimental methods are accurate, they are resource intensive. Machine learning can efficiently process large datasets. However, its prediction accuracy may be influenced by various factors. Machine learning methods include traditional methods that require feature extraction and deep learning. This manuscript reviews current PTM site predictors based on machine learning, including datasets, feature extraction, classifiers, and evaluation metrics. This manuscript summarizes the existing methods and explores future research directions. Data play an important role in machine learning. Thus, future research should explore ways to solve imbalanced data and PU problems. The use of few-shot and transfer learning addresses the data scarcity and model complexity. Multilabel learning is conducive to further exploring protein modifications. The association between modification sites and diseases is worth exploring. We hope that our review and analysis will assist research related to PTM.