Enhancing Dermatological Diagnostics with EfficientNet: A Deep Learning Approach

I. Introduction

As the most common cancer in the world, skin cancer has increased dramatically over the last few decades. Skin cancer is divided by dermatologists into two major types: non-melanoma skin cancers (NMSC), and melanoma, recognized for its aggressive behavior and risk to metastasize. While occurring less frequently than NMSC, melanoma is responsible for the majority of skin cancer mortality, pointing out the urgent need for accurate diagnosis [1].

The most common type of skin cancer is NMSC, which includes mainly basal cell carcinoma (BCC) and squamous cell carcinoma (SCC). BCC rarely metastasizes but is associated with significant morbidity in those cases that are left untreated or treated with delay. SCC, although more aggressive than BCC in terms of metastasis, usually responds well to treatment if detected and treated on time. Increased incidence of NMSC has been associated with higher ambient UV, related to ozone depletion, and increased public awareness and screening activities [2].

Melanoma, although a relatively rare form of skin cancer, associates a highly metastatic behavior. Early detection is of extreme importance to support a high survival rate in effective melanoma management. The five-year survival rate for patients with early-stage melanoma is over 90%, but this figure decreases significantly with the higher stage at diagnosis [3].

Benign skin lesions, including actinic keratosis (AK), benign keratosis-like lesions, and melanocytic nevi, are widespread and typically non-malignant. Although with benign behavior, clinical differentiation from their malignant counterpart is sometimes challenging even for an experienced dermatologist. When distinguishing malignant from benign disease, this diagnostic challenge may result in either unnecessary invasive procedures or delayed interventions.

The differential diagnosis of skin lesions is a complex task that reflects a cumulative knowledge of their morphology, distribution, and evolution over time. For malignant and benign skin lesions, dermoscopy, a technique that permits improved visualization, has provided a great deal of diagnostic accuracy. However, dermoscopic analysis can be subjective and experience-dependent, which can result in diagnostic variability and outcomes [4].

Early and accurate diagnosis of skin lesions is of utmost importance for treatment outcomes, especially when considering skin cancer. According to the World Health Organization, 1.5 million cases of skin cancers were reported in 2022, of which 330.000 were melanoma cases, leading to almost 60.000 deaths [5]. Therefore, the precise and timely diagnosis of skin lesions is essential, as it directly impacts patient management and prognosis. Although established diagnostic methods are effective, they can be subjective when using clinical evaluation as the main diagnostic tool. Such heterogeneity highlights the importance of moving toward diagnostic tools that are less subjective and more repeatable and scalable, which can also enhance early detection and improve clinical outcomes.

Today, the diagnostic landscape in dermatology is changing through the great impact of emerging technologies with depth in science, such as deep learning (DL) and artificial intelligence (AI). DL, a subset of machine learning (ML) has emerged as a powerful tool that is predicted to be a game changer in multiple fields of medicine, including dermatology. Computers and improved ML models can now solve hard, complicated diagnostic tasks with high accuracy. AI models, particularly when trained on large-scale databases, have a potential ability similar or superior to dermatologists in skin lesion classification. Moreover, these AI systems have a major impact in providing standardization in diagnostic interpretations and in the reduction of the inter-observer variability and frequency of diagnostic errors [6].

This consistent, repeatable interpretation may revolutionize skin lesion diagnostics as AI becomes integrated into clinical practice. Further, such integration allows for enhanced distribution of dermatologic expertise to underserved regions, democratizing access to expert-level diagnosis and care. To accommodate the vast diversity of skin types and conditions we can expect to handle in clinical practice, a durable deployment of AI tools in dermatology may require extensive validation and continuous training on multiple datasets [7].

Up to the present, in dermatology, DL models have outperformed standard diagnostic approaches in several studies. For instance, research performed by Esteva et al. (2017) has demonstrated that convolutional neural networks (CNNs) can classify skin cancer with a level of competence comparable to dermatologists, using vast datasets of dermatoscopic images [8]. Similarly, Codella et al. (2017), reported that ensemble DL could remarkably improve melanoma detection in dermoscopy images [9]. In a more recent paper from Brinker et al. (2019), deep neural networks were found to outperform dermatologists after being trained on images of melanomas, indicating the potential of these models for clinical decision-making applications [10]. Moving forward, Liu et al. (2020) also reported an AI attempt to diagnose skin diseases with a differential diagnostic accuracy comparable to dermatologists [11]. These advances in AI are not only being transformed into plain research findings. Still, they are also being manifested into practical application results that could make use of saving lives through real clinical operations. These echo the belief within the medical field that AI, specifically DL algorithms can vastly improve both the accuracy and the consistency of diagnostic tests.

Nonetheless, as these advanced AI tools are being considered to be included in the clinic, significant challenges with deployment remain, driven by requirements in extensive training data and generalization across diverse patient populations. This includes also the validation process intended to demonstrate accuracy and reliability.

In this study, EfficientNetB3, an architecture that achieves state-of-the-art accuracy in image classification, was used. Developed by Tan and Le (2019), it is the ideal type of architecture for medical image analysis as it performs the best in benchmarks considering both size and computational efficiency [12]. The work presented in this article focuses on the usage of a custom model based on EfficientNetB3 that uses transfer learning to analyze skin lesions on an extended dataset. This study aims to bridge the gap between the rapid advancements in technology and the clinical utility, in a scalable and efficient fashion to improve diagnostic accuracy and speed in dermatology by applying advanced AI methods. The model was trained and validated across six classes of skin diseases, encompassing benign and malignant conditions such as melanoma, basal cell carcinoma, squamous cell carcinoma, actinic keratoses, benign keratosis-like lesions, and melanocytic nevi.

By integrating the latest AI methods, progress can be achieved, such as improving diagnosis or reducing clinicians’ workload, allowing more time for patient care and less for routine diagnosis. Combining AI strengths on the technical side with the nuanced understanding of skin pathology from expert dermatologists, we have created an interdependent mix that betters the accuracy and applicability of the diagnostic process.

There are several novel contributions introduced to the dermatological research and clinical practice:

• Ranks among the top-performing models in the European region, indicating its potential to effectively address regional medical challenges.
• Achieves competitive results with our custom model based on EfficientNetB3, demonstrating efficient utilization of limited training data.
• Enhances practical feasibility and cost-effectiveness of deployment due to its modest computational requirements.
• It shows robust performance with fewer images compared to models that achieve similar or better results with larger datasets.

The following part of the study is organized as follows: Section II details the training dataset, data preprocessing steps, the model architecture and hyper-parameters, and techniques to combat overfitting, along with the training and validation processes. Section III presents the model’s performance. Section IV interprets the results, highlighting their implications and limitations. Finally, the conclusion summarizes the findings and suggests directions for future research.

2.1. Description of the Training Dataset

The primary dataset used for training is a proprietary combination of files collected from the authors’ collection of images and the International Skin Imaging Collaboration 2019 (ISIC2019) archive [13]. The dataset used in the study comprises a total of 8,222 images, covering six categories of dermatological lesions. These categories include three malignant skin conditions- melanoma, BCC, and SCC—and three non-malignant conditions—AK, benign keratosis-like lesions, and melanocytic nevi (Figure 1).

The data was divided into training, validation, and testing sets, with the training set consisting of 80% of the data (6,578 images) and the validation and test sets containing 20% (1,644 images). Table 1 includes the distribution of all images across the disease categories.

Dermoscopic and close-up images were included in the dataset, ensuring that each image consistently belonged to one of the specified categories. The EfficientNetB3 model was initially trained on the ImageNet image database [14] using a resolution of 300x300 pixels, keeping the RGB color model. Consequently, all images in this project were resized to 300x300 pixels. This resizing process maintained the defining properties and shapes of the lesions, ensuring no distortion occurred. Additionally, this standardization reduced the dataset’s size, enabling faster model training.

2.2. Data Pre-Processing

Before feeding images into the neural network model, the following preprocessing steps were applied to optimize model performance. This preparation ensured efficient and effective learning:

• Resizing: Images were resized to 300x300, ensuring uniformity in input sizes.
• Normalization: Pixel values were scaled to the [0, 1] interval to reduce data discrepancies and aid training convergence.
• Mean Subtraction and Standardization: Each pixel’s value had the dataset’s mean subtracted and was then divided by the standard deviation to further normalize the data, enhancing model convergence.
• Data Augmentation: This technique creates new images by modifying existing ones. We employ mirroring, translation, rotation, scaling, brightness adjustment, and noise addition to augment the existing pictures [15]. These augmented images are then added to the categories with less data, thereby balancing the training dataset.

2.3. The Usage and the Architecture of the Model

The EfficientNet neural network class of models, first proposed by Tan and Le [12], is a family of image classification models known for its feature extraction capabilities, that achieved very good accuracy while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet neural networks [16]. The decision to use a model based on EfficientNetB3 was driven by its effective balance between classification performance and resource efficiency. Compared to larger models like EfficientNet-B4, B5, and B6, EfficientNet-B3 offers competitive classification accuracy while requiring fewer parameters. This efficiency translates to reduced demands on memory and CPU usage, making it a practical choice for achieving high performance in classification tasks without excessive computational costs.

In this study, we developed a customized model based on EfficientNetB3, using transfer learning [17,18], and leveraged Imagenet to reduce computational cost and carbon footprint. We used TensorFlow [19] and Keras [20] to integrate EfficientNet with new dense layers, allowing the model to handle complex image classification tasks. The dense layers learned high-level representations and made final predictions based on features extracted by the convolutional layers, enhancing the model’s ability to capture complex relationships and improve prediction accuracy. Fine-tuning our dataset further adapted the model to our project’s needs. The architecture of the custom model includes the added layers described below and is represented in Figure 2:

• On top of the EfficientNetB3 model, adding a Batch Normalization [21] layer improved accuracy by enhancing convergence and helped reduce overfitting. Batch Normalization contributed to smoother training and improved generalization on unseen data by stabilizing and normalizing activations throughout the network.
• Two additional dense layers significantly enhanced classification performance by introducing non-linear features, extracting higher-level features, reducing parameter count and dimensionality of input images, and serving as a regularization technique.
• Finally, one dropout layer [22] randomly deactivated the neurons during training, which helped prevent overfitting by encouraging the model to generalize better. This technique improves the robustness and performance of the neural network on unseen data.

2.4. Training and Validating the Model

During the study, we applied a sum of strategies and techniques to optimize our model’s training process and performance. These included leveraging pre-trained weights from the ‘Imagenet’ dataset [14] to provide a strong starting point for learning and utilizing mixed precision policy [23] to enhance computational efficiency. We applied patience, stop patience, [24] and learning rate reduction [25] to fine-tune the training process, preventing overfitting and ensuring efficient learning rate adjustments. Transfer learning and unfreezing [26] allowed us to adapt the pre-trained model to our specific task, ensuring improved convergence and flexibility. Additionally, we saved the best weights [27] to guarantee optimal model performance for future use. Batch training [28] improved computational efficiency and promoted stable convergence through batch normalization.

2.4.1. Hyperparameters

The model’s hyperparameters, listed in Table 2, fundamentally model the training process and overall performance, highlighting the importance of selecting the appropriate values.

Training a custom model for skin lesion classification using a relatively small dataset poses challenges due to noisy gradients.

We chose the Adamax optimizer [29] for its robustness against gradient fluctuations using the infinity norm, leading to stable parameter updates. This helped to improve the performance of our research on skin lesion classification, where image variability affects gradient consistency. Adamax handles maximum gradients better than SGD or Adam, supporting smoother convergence and enhanced generalization.

A learning rate of 0.001 was initially chosen and adjusted during the training as a function of the validation accuracy and loss to facilitate model convergence. The optimal batch size was set at 32, influencing convergence speed and memory demands throughout the training process. The Categorical Cross-Entropy loss and Relu activation functions were employed for multi-class classification. The model demonstrated the best performance by adopting the combination of the fine-tuned hyperparameters listed above.

2.4.2. Techniques Used to Combat Overfitting

Overfitting happens when a model is overtrained on its training data, leading it to perform poorly on new data. Essentially, the model strives to be as accurate as possible and it focuses too much on fine details and noise within its training dataset. These attributes are often not present in real-world data, so the model tends to not perform well. Overfitting can also occur when a model is too complex, relative to the amount of data. This can lead the model to hyper-focus on details present in the given data that may not be relevant to the general patterns the model must develop. Overfitting gives the illusion that a model is performing well, even though it has failed to make proper generalizations about the data provided [30].

To prevent overfitting, there were used several techniques, described below:

• Dropout: Dropout selectively deactivates neurons in neural network layers during training, simulating smaller networks within the model. This approach encourages the network to diversify its learning strategies, enhancing generalization and mitigating overfitting by preventing reliance on individual neurons. [31].
• Batch Normalization: Normalization adjusts data to a mean of zero and a standard deviation of one, aligning and scaling inputs. Batch Normalization speeds up training by preventing gradients from becoming too small, facilitating faster convergence with higher learning rates. It also acts as a regularizer, reducing overfitting and improving model generalization on new data. This stability reduces sensitivity to initial weight choices and simplifies experimenting with different architectures. [32].
• Regularization: We used the regularization techniques to reduce overfitting: L2 regularization with a strength of 0.016 for the kernel and L1 regularization at a strength of 0.006 for both activity and bias regularization. These methods were chosen to mitigate overfitting by penalizing large parameter values in the model, thereby promoting more straightforward and more generalized outcomes across varying datasets and scenarios.

3.1. Training and Validation Accuracy and Loss

Monitoring training and validation accuracy and loss provides insights into how well a machine learning model generalizes. Lower training loss and higher accuracy indicate effective learning on seen data, while validation metrics assess performance on unseen data, ensuring the model’s robustness and generalization. Figure 3 shows an accuracy of 95.4% when our custom model was trained on four classes, whereas Figure 4 shows an accuracy of 88.8% when our custom model was trained on six classes.

3.2. Classification Performance

The test results show the performance metrics of the classification model, side by side, across the four different classes: BCC, benign keratosis-like lesions, melanocytic nevi, and melanoma (Table 3), and across the six different classes, the initial four, plus AK and SCC (Table 4). Performance metrics reported are precision (proportion of true positive predictions out of all positive predictions made by the model), recall (proportion of true positive predictions out of all actual positive cases in the data) and F1-score (harmonic mean of precision and recall, providing a single metric that balances both precision and recall). The model’s overall accuracy, which measures the proportion of correctly classified instances out of the total instances and provides an overall assessment of the model’s performance across all classes, is 95.4% (4 classes)/88.8% (6 classes).

3.3. Receiver Operating Characteristic (ROC) Curve

Figure 5 and Figure 6 show the proposed model’s ROC curves. The curve shows excellent performance, with AUC values ranging from 0.98 to 1.0 across four classes and from 0.93 to 1 across six classes. These results are relevant to the model’s accuracy and promising clinical application potential.

3.4. Confusion Matrix and Errors by Class

Figure 7 and Figure 8 below show the confusion matrix of the custom model for the test with four and, respectively, six classes. The confusion matrix visually represents the classification results of the test dataset per each class. The majority of the examples per class, represented on the diagonal of the matrix, were accurately classified, i.e., for Melanoma, 205 images were correctly classified out of 207. The number of errors for each class is shown in Figure 9 and Figure 10.