Deep Learning Guided Prediction Modeling of Dengue Virus Evolving Serotype

1. Introduction

Understanding the evolutionary dynamics of a virus is crucial for discerning its origin, focusing on key characteristics such as structure, classification and evolution. This knowledge plays a pivotal role in unraveling the fundamental biological mechanisms, thereby advancing vaccine and drug development. Despite the ongoing discoveries related to viruses, the potential existence of unidentified viruses remains a constant concern. The rapid and widespread dissemination of viruses has become robust, presenting formidable challenges in controlling and predicting their expansion. Thus leads to epidemics and pandemics, underscoring the unpredictable risks associated with these agents [1].

Dengue infection is transmitted through the bite of an Aedes mosquito carrying the ~10kb genome-size dengue virus (DENV). Clinically, identifying dengue fever has historically been challenging due to the prevalence of other infections with similar syndromes in tropical environments. The ambiguity in distinguishing dengue from various viral diseases, ranging from yellow fever to tropical influenza has persisted. Additionally, labeling the historical dengue outbreaks as chikungunya particularly in the 1800s is complicated due to the inconsistent and conflicting reporting information. Despite these challenges, the global public health system has remained engaged in addressing the persistent threats posed by the dengue virus [2]. In the landscape of genomic analysis, deep learning (DL) has emerged as a powerful tool, particularly for extracting features and patterns from complex genomic data. Similarly, in the context of infectious diseases like dengue, machine learning applications have gained prominence, leveraging epidemiological and clinical data for predictive modeling. Rachata et al., notably utilized weather data and feature selection algorithms to forecast dengue incidences, employing Artificial Neural Networks (ANNs) and Support Vector Machines (SVMs) to predict number of cases based on weather and gene expression data [3,4]. Despite the success of SVMs using word2vec representation in specific tasks, their efficiency waned when directly applied to nucleic acid sequences. There are multiple methods for viral genome classification, employing both alignment and machine learning approaches. In the alignment based approach, detection of viral sequences is carried out using tools such as USEARCH, SCUEAL [5] and REGA [6], relying on alignment scores for genome classification. However, these alignment based methods have limitations, notably their performance dependence on selection of initial alignments. However, in the machine learning approach, various methods have been proposed for the classification of viral genome sequences. But, these methods face limitations in their ability to detect viral genome contigs and the challenge of extracting useful hidden information. Moreover, those trained exclusively on nucleotide sequence data, further constraints their utility in comprehensive genome analysis.

The deep learning models such as Recurrent Neural Networks (RNNs) have demonstrated their effectiveness in the field of natural language processing. The applications of deep learning in computational biology mainly concentrates in genome analysis and sequencing. However, the RNN are considered black boxes due to the complexity of the model in interpreting the hidden state of the model. Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN) have been used to detect viral genome sequences using training of pattern and frequency of branches. Whereas, the Feedforward Neural Network (FNN), a type of CNN model, is adept at evaluating visual patterns while accommodating the inherent heterogeneity within the data. This network is designed to map fixed length inputs to a fixed size output and is trained using a backpropagation algorithm. Consisting of multiple layers, the CNN effectively stores and updates information in filter weights as it learns the intricate relationship between input and output.

This study aims to predict the genomic sequences of emerging serotype to deepen our understanding of dengue virus evolution and also classifies predicted as well as unknown dengue virus genome sequences respective to their serotypes. Utilizing the RNNs trained on complete genome, the model focuses on learning patterns in tokens rather than representing character meanings. LSTM, chosen for genomic prediction tasks, excel in handling sequential data and mitigating the vanishing gradient problem inherent in traditional RNNs.

2. Materials and Methods

2.1. Data Collection and Preprocessing

A dataset comprising 2000 complete genome sequences of dengue virus were assembled from the NCBI [7] and BV-BRC databases [8]. The datasets incorporated 500 sequences for each of the four serotypes of dengue virus. Before further analysis, a preprocessing step was performed to ensure that the data were in suitable format. Subsequently, four distinct datasets representing serotypes separately were built, each containing sequences spanning 10,273 base pairs. All the sequences were concatenated into a single dataset following a labeling process. All nucleotides were selected as features and DNV1, DNV2, DNV3 and DNV4 as serotype labels.

2.2. DL-DVE Architecture and Working for Sequence Generation

The genomic sequences of DENV were extracted from FASTA files using the Biopython library. A tokenization approach was used, treating the sequences at the character level, and an n-gram strategy was employed to generate the input sequences for the model from a FASTA file. To ensure uniformity in the sequence length, the data underwent padding. The generative model, implemented as a sequential model in Tensorflow Keras, comprised an embedding layer [9], two LSTM layers for capturing sequential patterns, and a dense layer for output. The model was compiled using sparse categorical cross entropy loss and the Adam optimizer. The sequential model designed for sequence processing with a specific focus on capturing patterns in the sequences related to the classification task.

For training, the model involved utilization of prepared predictors and labels over 15 epochs to efficiently capture underlying patterns in the data. Additionally, a function was used to generate a new sequence based on a given seed text and the trained generative model. The sequence generation process involved predicting the next word with a controlled level of randomness to introduce diversity while preserving the patterns in the entire data [10] enhancing the generative capabilities of the model. A detailed architecture of DL-DVE model for sequence classification and generation is shown in Figure 1.

2.2.1. Comparison of Generative Models2.2.2. ConV1d

We used ConV1d, a CNN based model implemented with an embedding layer to represent words in a continuous vector space. A single Conv1D layer with 128 filters and a kernel size of 5 captured local patterns, GlobalMaxPooling1D reduced the dimensionality, and a dense layer with a softmax activation generated a probability distribution over the entire network.

2.2.3. GRU

The model consisted of an embedding layer for word representation, followed by two GRU layers with 100 units, capturing the sequential patterns. The final dense layer outputs a probability distribution.

2.2.4. Simple RNN

A simple RNN model was implemented using an embedding layer converting nucleotides into 50-dimensional vectors, followed by two simple RNN layers with 100 units each. The first layer returned sequences, capturing the temporal patterns, while the second provided a condensed representation. The final dense layer outputs probabilities making its suitability for our sequence generation task. We further analyzed the generated sequences using BLAST search and Pairwise2 algorithm to assess the most similar type of sequence and similarity scores with other serotypes using Biopython. Furthermore, we used MEME Suite (version 5.5.5) to enrich our understanding of the conserved patterns and motifs in the sequences [11].

2.3. DL-DVE Architecture and Working for Sequence Classification

The Biopython library was used to extract genomic sequences of DENV from FASTA files. Sequence parsing facilitated the extraction of sequences and associated virus types. The unique nucleotide characters within the sequences were identified to assess dataset diversity. To enable the deep learning models utilization, nucleotide characters were mapped to numerical values and vice versa. One- hot encoding transformed the sequences into numerical representations [12]. Dengue virus types underwent conversion to numerical labels using scikit-learn’s LabelEncoder for multi-class classification modeling [13]. The dataset was divided into training and testing sets, with 80% of the data allocated for training and 20% for testing ensuring the model’s ability to generalize to the unseen data.

A FNN model was deployed using Tensorflow Keras library. The model consists of a flattening layer, a dense hidden layer employing Rectified Linear Unit (ReLU) activation, and an output layer utilizing softmax activation for effective multi-class classification. The pivotal role of non-linear activation function is highlighted post-convolution, particularly in comprehending CNN dynamics. Among the most commonly employed activation functions, namely ReLU, Sigmoid, and Tanh, ReLU demonstrates accelerated learning. The outer layer employed Softmax activation, enabling the assessment of class probabilities in prediction scenarios. The FNN integrated multiple filters traversing a one-hot encoded binary vector representing the sequence.

The model underwent compilation utilizing the categorical cross-entropy loss function and the Adam optimizer. During training, the Adam optimizer dynamically adjusted weights and biases, while the sparse categorical cross entropy loss function quantified the dissimilarity between predicted probabilities and true labels. The training spanned 10 epochs with a batch size of 32, incorporating 10% of the training data for validation purposes. Evaluation on the testing set gauged the model’s accuracy in predicting virus type, with predictions made on a subset of testing data compared against actual virus types to assess performance. To comprehensively evaluate the model’s effectiveness, various classification metrics, including accuracy, precision, recall, and F1 score were computed. A user-friendly interface was developed to facilitate the input of new viral sequences. The provided sequence underwent preprocessing and was fed to the trained model to predict the associated virus type. The workflow and architecture of classification and generative model is shown in Figure 1.

3. Results

Complete genome sequence data of DENV from the four existing serotypes was employed in our LSTM model with the aim of generating sequences that exhibit the probability of emerging as a new serotype. Initially, we employed a range of models including ConV1d, GRU, Simple RNN and our proposed LSTM model, aiming to ascertain the most effective approach. Notably, at 10 epoch, our LSTM model demonstrated 37% accuracy, outperforming ConV1d, GRU and Simple RNN. Moreover, when trained at 10 epochs, our LSTM model generated sequences exceeding 10kb in size, which distinguished it from the other models that generated sequences of less than 10kb.

Recognizing the potential for enhancement, we extended the training duration of our LSTM model to 15 epochs. This adjustment yielded a substantial increase in accuracy, reaching 93%. This improvement underscored the efficacy of prolonged training in refining the model’s predictive capabilities as shown in Table 1.

To further analyze the origin of generated sequences, we employed a multi-class classification approach using the FNN model for genomic sequence classification. Our study contributes to the landscape of genomic analysis, employing one-hot encoding, a proven method for next generation sequencing reads and phenotype label abstraction. Our CNN based FFN classification model trained on DENV complete genome data achieved an accuracy of approximately 99.00%. Across the 10 training epochs, the model consistently improved, achieving a final validation accuracy of 98.12% (Figure 2).

The model demonstrated reliable results in achieving high-quality predictions across multiple evaluation metrics with an overall accuracy, precision, recall, and F1 score of 99.00% as shown in Figure 3.

The model’s performance was assessed through the ROC-AUC curve. The loss function appeared to decrease significantly and the area under the ROC-AUC on the validation set showed fluctuations, ultimately stabilizing around 0.818. This robust accuracy, coupled with the low training loss, suggests the model's effectiveness in accurately classifying instances as shown in Figure 4.

We investigated the model’s performance by inserting unknown sequences of the dengue virus genome. The model defined a serotype for that sequence. Which was further cross-checked the predicted serotype with the actual serotype confirmed from its source. The FNN model worked efficiently on the unknown dengue virus sequence data and predicted the actual classes or serotypes of dengue virus Figure 5.

Using our DL-DVE generative and classification model, the generated sequences were successfully classified as belonging to the DENV-4 serotype. We assessed the similarity between the sequences generated by our LSTM model and DENV-4 serotype. Surprisingly our investigation revealed a similarity of approximately 66%, suggesting that the generated sequences did not align closely with DENV-4 serotype. We checked the similarity of generated sequences with other serotypes and the approximate similarity observed was 63 to 66% suggesting that the generated sequences did not align closely with any known serotype as shown in Figure 6.

The generated sequences met the specified sequence length requirement and emphasize the effectiveness of our model in accurately predicting and classifying DENV serotypes.

The MEME Suite analysis contributed to an enhanced comprehension of conserved patterns shedding light on motifs within all serotypes and the emerging serotype as shown in Figure 7.

4. Discussion

With the advent of next generation sequencing, predictions have become feasible or at least possible using complete genome sequence data [14]. Genomic data have been used to classify COVID-19 variants and other viruses using deep learning approaches [15,16,17]. CNNs and LSTMs have been more frequently used for prediction of dengue cases [3,18,19,20,21,22,23,24]. That underscores the importance of neural networks combined with genomic sequences as a futuristic method capable of revolutionizing virus studies [25,26,27].

Machine learning models go beyond human reasoning and build prediction models from a number of complex combinations. The DL models such as LSTM, GRU and CNN have been used for sequence classification and generative tasks [28]. Complete genome sequencing data was used to detect HCV variants that showed resistance to direct-acting antivirals. And the identified variants were incorporated into machine learning algorithms for assessment of effectiveness of the predictive model [29]. The LSTM model is considered effective in capturing the complex patterns in data and multiple features to make accurate predictions [30]. Being a subtype of RNNs, these models possess an enhanced capability to learn information from distant points in time. While traditional RNNs encounter the vanishing gradient problem impeding their ability to capture changes that occurred in data long ago. LSTMs overcome this challenge through a gating mechanism where the gates open and close based on values learned from each input. This mechanism enables LSTMs to accumulate information over an extended period by dynamically learning to forget certain aspects of information. This aspect of sequence length carried significant implications for the model’s predictive capacity and biological relevance.

In the realm of biological sequence analysis, machine learning and deep learning using CNNs have demonstrated high precision for binary or multi-class classification [31]. Nucleotide sequence based studies have typically employed one-hot encoding vectors to represent each nucleotide and with all unknown nucleotides represented as all zero vectors. Chaos game representation (CGR), particularly Frequency CGR (FCGR) has shown promise in encoding sequences in image format and has been applied to predict drug resistance [32]. The identification of novel genomic regions in viral pathogens using CNNs and LSTMs have emerged as compelling areas of exploration among researchers [1]. In the CNN framework, the initialization of filter weights involves random uniformness, and these weights are subsequently refined through the backpropagation process to minimize the loss or cost function. The iterative learning process allowed the network to adapt and optimize its performance enhancing its ability to discern meaningful patterns and features within the given data.

An approximate sequence similarity of 65% among DENV serotypes has been demonstrated in multiple studies [33,34,35]. The exploration of motifs with statistical significance was integral in our study of complete genome sequence data encompassing DENV 1 to 4 serotypes, juxtaposed with the generated sequences [36].

5. Conclusions

Our study demonstrates the effectiveness of utilizing sequential models for classifying and generating DENV genomic sequences resulting in the generation of sequences resembling a potential emerging serotype. Our predictive model classified the generated sequences as belonging to DENV-4 serotypes, showing a close resemblance to DENV-4 with a 65.99% sequence similarity while diverging significantly from existing serotypes. The intra-serotype divergence was characterized by a sequence similarity of at least 90%, affirming the distinctiveness of generated sequences within the DENV serotype landscape; further exploration includes the analysis of conserved motifs. Considering the complete genome sequence size of DENV genome is approximately 10kb, our trained model is limited to predict DENV sequences of similar size. Nonetheless, in future directions, predictive modeling applied well in advance of mutations holds promise for informing vaccine design and the development of diagnostic kits.