Ensemble of Deep Learning Classifiers and Source Fusion for Improved Malaria Cell Classification

1. Introduction

Malaria is a dangerous disease spread by mosquitoes. On average, the malaria virus is more common worldwide in tropical climates. With over 200 million cases internationally and more than 400,000 fatalities annually, malaria represents a colossal danger to public health. In 2021, the WHO African Region accounted for around 95% of malaria cases and 96% of malaria-related deaths. Over half of malaria deaths globally occurred in four African countries: Nigeria (31.3%), the Democratic Republic of the Congo (12.6%), the United Republic of Tanzania (4.1%), and Niger (3.9%) [1]. Vulnerable groups include newborns, pregnant women, and visitors. Consequently, information technology may be heavily exploited in areas lacking specialist workers and infrastructure. In addition, as the rate of errors in traditional approaches is higher, the classification analysis implemented may be utilized to reduce human stress and aid in making an accurate diagnosis [2]. Few techniques are used to detect malaria disease such as rapid diagnostics test (RDT) [3], clinical diagnosis, polymerase chain reaction (PCR) [4], and microscopic diagnosis. Conventional diagnosis procedures such as clinical diagnosis and PCR depend on human abilities. Currently, RDT and microscopic diagnosis are the most efficient approaches for reducing malaria [5]. Implementing modern information technology plays a vital role in the fight against this fatal and widely spread illness. Specifically, deep learning is used due to its high accuracy in classifying huge quantities of data [6].

Malaria is a dangerous illness that affects millions globally, particularly in impoverished nations. Early identification and treatment are crucial to avoid complications and lower mortality rates. Many studies on the malaria virus have been carried out in the literature using various deep learning models and architectures. Using the transfer learning method, Vijayalakshmi et al. proposed a novel neural network model to identify infectious malaria parasites. They combined the support vector machine with the VGG network to develop a novel neural network model. They claimed that the classification accuracy of their created network was 93.1% [7]. Pan et al. observed that a deep convolutional network based on LeNet-5 achieves good classification accuracy for automated malaria detection. They investigated the performance impact of the data set by running their algorithm on data sets with varying amounts of images [8].

Models that were used in a study by Rajaraman et al. included AlexNet, VGG16, Xception, ResNet5, and DenseNet121. They claimed to have identified the layers of the best model through experimenting to extract features from the raw data. They claimed that trained CNNs successfully extracted features, and the results were statistically supported [9].

Cinar & Yildirim compared the classification performances of 6 different CNN models: ResNet50, AlexNet, GoogleNet, DenseNet201, VGG19, Inceptionv3, SqueezeNet, and InceptionResNetV2. Test results are obtained, and original data is used to train the networks. After preprocessing the data using the same procedures, the pictures are then subjected to median and Gaussian filters. They reported that they had achieved a maximum rate accuracy of 96.6% using the GoogleNet model while identifying malaria data using Gaussian-filtered data [10]. Dong et al. found that using transfer-based deep learning algorithms resulted in a success rate of over 95% in identifying malaria-infected cells [11]. Yang et al. generated a data set of 1819 thick smear images from 150 individuals. A malaria detection investigation employing deep learning approaches achieved a maximum accuracy of 94.33% Yang et al. (2019).

Reddy et al. achieved 95.91% accuracy in detecting malaria using the ResNet50 architecture. They worked with a data set of 27558 images [12]. In another study, researchers used AlexNet, Google Net, ResNet-50, MobileNet-v2, and VGG16 architectures. They claimed that the trained CNNs successfully extracted features. A maximum accuracy rate of 96.53% was obtained with the MobileNet-v2 model using the Adam optimizer [13].

In this paper, source fusion of original and filtered data is proposed to improve the classification performance of the state-of-the-art Deep learning methods. Two approaches have been proposed, where in the first approach the original data is supported by the Gaussian and median filtered data. In this approach, the training and test data go through Gaussian and median filtering to generate three distinct data sets. The three data sets are combined in a single data set to be used in CNN-based deep learning architectures such as AlexNet, ResNet50, DenseNet201, VGG16, InceptionV3, SqueezeNet, InceptionResNetV2, and GoogleNet. The CNN architectures have been trained and tested with 80% and 20% of the data respectively. The results obtained via the first method indicate that performance is generally increased in most of the classifiers where the increase in AlexNet reaches 7% using accuracy. The second proposed approach involved using original, Gaussian, and median-filtered data sets separately. Each of the eight CNN architectures is trained by original and filtered data sets separately. Fusion of the original and filtered data (3 data sets) improves the performance of the respective CNN models. Finally, the decisions of each of the 8 CNN models are combined using sum-rule decision fusion. The overall performance reaches an outstanding classification performance of 100% in accuracy, specificity, and sensitivity metrics.

The paper is organized as follows: Section 2 provides the related work, Section 3 explains the data set utilized in this study, Section 4 describes the method and the deep learning techniques, Section 5 describes the research outcomes, and Section 6 ends with conclusion and suggestions for further research.

2. Dataset & Software

Two types of data classes are selected from the Kaggle dataset [14]. The data of the first class is the healthy data, while the second is the parasitic data. These data consist of 3000 non-parasitic data and 3730 parasitic data. MATLAB R2021a environment used in our study. The data ratio has been distributed as 80% for training and 20% for testing, while 10% of the training is used as validation. The images are colored RGB with various sizes, thus they are resized to 224×224. The learning algorithm used is Adam optimizer, the learning rate is 0.0001, and the number of epochs is 5. Examples of uninfected/healthy (first row) and infected/parasitized (second row) images are shown in Figure 1. The figure shows that finding parasitized cells can be challenging, even with a close-up view.

Performance metrics that include sensitivity, specificity, precision (positive predictive value), negative predictive value, and accuracy rates are used to evaluate the investigated and developed models. Their formulae are provided below. The confusion matrices are used to calculate these performance metrics. These confusion matrices contain information about the correctly identified positive samples (true positive—TP), the correctly identified negative samples (true negative—TN), the incorrectly classified negative samples (false positive—FP), and the incorrectly classified positive samples negative (false negative—FN). AP and AN represent all positive and all negative samples, respectively.

$$Sensitivity \left(Recall\right)={\displaystyle \frac{TP}{TP+FN}}$$

(1)

$$Specificity={\displaystyle \frac{TN}{TN+FP}}$$

(2)

$$Positive Predictive Value \left(PPV-Precision\right)={\displaystyle \frac{ TP}{AP}}$$

(3)

$$Negative Predictive Value \left(NPV\right)={\displaystyle \frac{ TN }{AN}}$$

(4)

$$Accuracy={\displaystyle \frac{TP+TN }{AP+AN}}$$

(5)

3. Proposed Fusion Techniques

Deep learning architectures have been effectively used in the processing of medical data in different modalities [15,16,17,18,19]. Computers can process and learn from data with the help of deep learning. The primary feature that makes deep learning models above conventional neural networks is their multilayer structure. Deep learning has recently attracted the attention of researchers for several reasons, including the development of GPUs with faster processing speeds. The rise in interest toward deep learning also grew with the increase in the amount of collected data.

In this study, convolutional neural networks have been used as deep learning architecture. CNNs are one of the most effective deep learning networks for computer vision applications such as image classification. Network data is the main training source for CNN. The network goes through several layers to finish learning after receiving input images. These layers may be categorized as convolution, fully connected, pooling, dropout, normalization, and SoftMax layers [20].

In this study, 8 different architectures of pre-trained models will be studied and investigated, namely, AlexNet [21], GoogleNet [22], DenseNet201 [23], ResNet50 [24], Inceptionv3 [25], VGG16 [26], InceptionResNet [27], and SqueezeNet [28]. The Gaussian and median filtering techniques are used to augment the data set to boost the performance. The study proposes 3 data fusion methods: Source fusion, sum-rule decision fusion for each pre-trained model and sum-rule decision fusion for all eight models.

3.1. Source Fusion

In this method, the original and the filtered data are combined to form one dataset that contains a total of 20190 images. Two types of filters, Gaussian and median filters, are used for generation of the filtered data. The overall samples containing the original and the filtered data are classified by 8 different pre-trained models. Each CNN model consists of layers including input layer, convolutional layers, pooling layers, fully connected layers, and output layer. The structure of the proposed model is illustrated in Figure 2.

3.2. Sum-Rule Based Decision Fusion

3.2.1. Decision Fusion for Each Pre-Trained Model

This method utilizes the sum-rule decision fusion by employing it among the original, the Gaussian and the median filtered modalities using each CNN Model. The classification probabilities of each model are fused to generate the decision of the respective CNN model. In this way, 8 distinct classifiers have been formed with dedicated classification performances. Figure 3 illustrates the proposed sum-rule decision fusion method for each CNN model.

3.2.2. Decision Fusion Among All 8 Pre-Trained Models

Having 8 distinct classifiers using 8 different pre-trained CNN Models, we propose to employ another level of decision fusion where the probabilities generated by the 8 distinct classifiers are fused once more using sum-rule to generate the overall classification performance. The details of the developed framework are illustrated in Figure 4.

4. Results and Discussions

In this section extensive experiments are conducted on the dataset to evaluate the performance of the proposed 3 methods. The original data obtained from Kaggle had 6730 samples/images. After reshaping the images to 224×224 size and applying the Gaussian and the median filters the dataset increased to a total of 20190 samples/images. In the first method, the original data along with the filtered data are combined to obtain a single data set. This data set is used by 8 distinct classifiers with different pre-trained convolutional neural network (CNN) architectures. The median and Gaussian filters are adopted which can provide two distinct ways of filtering and complementing the original data set. Gaussian filter involves a linear process with a weighted average of nearby pixels to determine the filter response [29]. The median filter’s objective is to soften and reduce the sharp tone shifts such as impulses in the image [30].

Figure 2 illustrates the processing blocks of the proposed source fusion method which combines original and filtered data. The original and filtered images are individually processed with AlexNet, DenseNet201, ResNet50, VGG16, InceptionV3, GoogleNet, InceptionResNetv2, and SqueezeNet. The images are then classified as healthy/uninfected or infected/parasitized. Classification performance by means of accuracy and other metrics are provided in Table 1 and Table 2, respectively.

The performances of the proposed source fusion method compared with original data are given in Table 1. The results show that performance is generally increased after applying source fusion methodology. The highest performance increase was in AlexNet with a notable improvement of 7.2%. While marginal decrease was encountered in DenseNet201and InceptionResNetV2 (0.5% and 0.6%, respectively), the rest of the methods showed credible performance increase. Table 2 compares the performances of the 8 CNN models by the performance metrics listed in section 2 namely, sensitivity, PPV, NPV, and specificity. VGG16 stands out with the highest values in sensitivity, PPV and specificity. InceptionV3 also performs well, with high sensitivity and NPV, although its specificity and PPV are slightly lower. GoogleNet shows strong results with Sensitivity and PPV while AlexNet has solid performance across all metrics, especially sensitivity and NPV.

The second proposed method is the sum-rule decision fusion where each CNN model is employed in 3 different classification channels using original, median filtered, and Gaussian filtered data respectively. Figure 3 illustrates the proposed sum-rule decision fusion method for each CNN model. Classification performance by means of accuracy and other metrics of the sum-rule decision fusion are provided in Table 3 and Table 4 respectively.

The performance of the proposed sum-rule decision fusion method (method 2) compared with original data, median filtered, and Gaussian filtered for each model are represented in Table 3. The results show that performance is mostly increased after applying the proposed sum-rule decision fusion method for each CNN model. The highest performance was recorded by AlexNet with a notable improvement of 8.8% compared to the original data channel. GoogleNet achieves the highest accuracy performance (99.9%), while SqueezeNet reveals the lowest performance (97.4%). Table 4 compares the performance of the sum-rule decision fusion of the 8 different pre-trained CNN models.

The results provided in Figure 5 compare the performance of 8 CNN models using the three different data processing techniques: original data, source fusion, and sum-rule fusion. The highest performance on the original dataset without any fusion is achieved by InceptionV3 (96.4%), followed closely by GoogleNet (96.3%). With Source Fusion of the original data and Gaussian and median filtered data, InceptionV3 improves notably, reaching 99.2%, reflecting a strong increase from the original accuracy (96.4%). GoogleNet stands out here with the highest performance of 99.9% using sum-rule fusion, representing a major improvement from the original data and source fusion results. The sum-rule fusion method consistently yields higher accuracy compared to the original data and source fusion approaches across almost all models.

In the third method, we combined the sum-rule decisions generated from each of the 8 pre-trained CNN models to provide a higher level of decision fusion by using a second sum-rule to obtain the overall decision fusion. Figure 4 displays the overall sum-rule decision fusion for all 8 pre-trained CNN models.

Table 5 shows the performances of the proposed three fusion methods compared to other methods in the literature for binary classification of malaria as parasitic or uninfected using different deep learning techniques or models. The highest accuracy performance was in the study of Pan et al. [8], in which they achieved an accuracy of 99 % using LeNet-5 architecture with a small dataset (800 images). In this study, InceptionV3 architecture gave an accuracy of 99.2% when the source fusion approach (method 1) was applied. Furthermore, the sum-rule decision fusion method of 3 channels achieved an accuracy of 99.9% using GoogleNet (method 2). Finally, the sum-rule decision fusion method among all 8 pre-trained CNN models (method 3) outperformed all other models by achieving an accuracy of 100%.

5. Conclusions

In this paper, malaria images were binary classified using different pre-trained CNN architectures with and without data fusion at source and decision levels. Three different fusion methods of the original and filtered data to improve the classification performance. The first method combines the original data with Gaussian and median-filtered data at the source level before feeding the fused data into one of the pre-trained CNN architectures. The second method utilizes original, Gaussian-filtered filtered, and median-filtered data to go through separate CNN architectures where the resulting class probabilities are fused at the decision level using the sum-rule. Finally, the third method proposed using the resulting class probabilities generated from method 2 for each of the 8 pre-trained CNN models to go through a second sum-rule fusion stage before making the final decision. In method 1 accuracy reaches 99.2% using the InceptionV3 model. An accuracy value of 99.9% was obtained in the second method by the GoogleNet model. Finally, the third method generated the highest possible accuracy 100%. These results become more meaningful when compared with the CNN-based classification of the original data without any fusion approaches which only generates accuracy between 85.9% (SqueezeNet) and 96.4% (InceptionV3).