Skin Cancer Disease Detection Using Two-Stream Hybrid Attention Based Deep Learning Model

1. Introduction

Skin cancer is one of the most common and significant types of cancer worldwide, representing approximately one-third of all cancer cases [1,2,3,4,5,6,7,8,9,10]. The rising incidence, especially in the United States, where over 9,500 [11] diagnoses are made daily, underscores the importance of early detection and intervention [12,13]. However, skin cancer diagnosis has historically been challenging, particularly due to variations in skin lesion characteristics such as color, shape, and the diverse types of diagnostic imaging systems used. Recent advances in imaging technology, including dermatoscopy, reflectance confocal microscopy (RCM), and 3D total body photography, have significantly improved the ability to analyze skin lesions, but traditional diagnostic approaches still rely heavily on subjective interpretations by healthcare providers, often resulting in variability and delays in diagnosis [14]. Although skin cancer is frequently curable, early detection and accurate diagnosis are crucial for achieving positive treatment outcomes and improving patient survival rates [15,16,17]. Traditional approaches for diagnosing skin cancer and other diseases [18,19,20], like visual inspections and histopathological exams, often suffer from being time-consuming, subjective, and prone to high variability among different observers. Approximately 53% of individuals diagnosed with melanomas personally conduct a self-examination before seeking medical evaluation [21]. The evolution of cancer therapeutics has seen a remarkable shift from traditional treatments like surgery, radiation, and chemotherapy to modern therapies such as immunotherapy and targeted treatments. These approaches aim to either directly target cancer cells or modulate their surrounding microenvironment, reflecting the growing emphasis on precision medicine in cancer care [2]. Initially, surgery was the first rational approach to treating cancer prior to 1970, evolving with advancements in anesthesia, surgical techniques, and early cancer detection. Radiation therapy, introduced after the discovery of X-rays, became a key treatment modality for targeting tumors while minimizing damage to healthy tissue. From 1970 to 2023, treatments like tamoxifen for breast cancer and targeted therapies such as trastuzumab and imatinib revolutionized cancer care by focusing on specific genetic alterations, significantly improving survival rates. Looking ahead to future treatment, emerging therapies, including PROTACs, molecular glue degraders, and gene-targeted treatments, promise to offer more precise treatments with fewer side effects. Additionally, advancements in immunotherapy, antibody drug conjugates (ADCs), and oncolytic virus therapy provide exciting new avenues for more effective, personalized cancer treatment strategies [22]. The clinical skin examination is a highly accurate method for affirmative skin cancer screening [14]. The discussion on skin cancer treatment paradigms focuses on the debate between directly targeting cancer cells and modulating the tumor microenvironment (TME) to control tumor growth and metastasis [23]. The TME, which includes tumor, stromal, and immune cells, plays a critical role in tumor progression and immune evasion. Targeting stromal cells, such as cancer-associated fibroblasts (CAFs), has shown promise, as they contribute to tumor growth, metastasis, and chemotherapy resistance. However, direct approaches like genetic therapies and metabolic reprogramming remain vital for reducing tumor cell viability and enhancing treatment sensitivity. Despite these advances, tumor heterogeneity and genetic instability often lead to resistance. Therefore, combining strategies that target both tumor cells and the TME—along with immune modulation through checkpoint blockade—may offer a more effective, comprehensive treatment approach to improve patient outcomes [23].

Emerging trends in oncology are moving towards integrating traditional therapies with innovative approaches like immunotherapy, personalized cancer vaccines, and machine learning-assisted imaging techniques. These advancements are expected to enhance early detection and treatment accuracy, offering a more comprehensive understanding of cancer dynamics and improved therapeutic strategies. Key developments in cancer therapies include nanomedicines, stem cell therapy, ablation therapy, and proton beam therapy, all of which promise safer, more efficient treatment options with minimal invasiveness. Additionally, gene therapy and natural antioxidants are being explored for their potential in cancer prevention and treatment. While these emerging technologies offer great promise, patient safety and cost-benefit considerations remain essential. A multimodal approach combining surgery, radiotherapy, chemotherapy, and newer therapies tailored to individual patient needs will likely become the future standard of care, improving overall outcomes [24]. Clinical examinations require significant time from physicians to analyze multiple dermoscopic images. Delays of weeks or months can postpone treatment and worsen skin conditions. Advances in imaging technology now enable more accurate skin lesion diagnosis through techniques such as reflectance confocal microscopy (RCM), 3D total body photography (3D-TBP), teledermatology, and dermoscopy. Dermoscopy, a non-invasive method, uses magnification to study lesions without relying on reflected light [14]. However, manual image processing is tedious, and results depend on subjective interpretation, making accurate skin lesion identification challenging. Previous methods, such as histogram thresholding and Principal Component Analysis (PCA) for colour space clustering, attempted to improve segmentation [25,26]. While these approaches reduce human involvement, they still struggle to distinguish all relevant features of skin lesions. In line with these advances, the role of deep learning in skin cancer detection has evolved significantly. Early approaches, such as those utilizing histogram thresholding or gradient vector flow, were limited by the need for significant human involvement and the inability to capture complex features. However, Convolutional Neural Networks (CNNs) [27,28], particularly with transfer learning, have proven highly effective for automating skin lesion analysis, achieving impressive accuracy levels [15,29,30]. While deep learning models like MobileNetV3 and EfficientNet have been used to facilitate skin cancer detection on mobile devices, they often fall short in terms of feature representation depth, limiting their accuracy [31,32,33,34]. To improve the performance accuracy, Hossain et al. proposed a Max Voting Ensemble Technique combining multiple pre-trained CNNs (e.g., ResNet50, InceptionV3, Xception) to enhance classification performance. This approach achieved 93.18% accuracy and an AUC of 0.9320 on the ISIC 2018 dataset [35]. Despite its effectiveness, the ensemble model is computationally expensive, making it unsuitable for real-time or resource-constrained applications. Further efforts have focused on optimizing Depthwise Separable Convolutions (DWSConv) to reduce model size while preserving performance [36]. Recent research has demonstrated that lightweight attention based deep learning for direct skin cancer detection, even on clinical PCs and mobile devices with limited processing resources [31,37,38,39,40]. Some hybrid models combine CNNs and Vision Transformers (ViTs) using either standard or DWSConv layers to compress feature dimensions before processing with transformer modules [39,41,42,43,44]. Most recently, Dai et al. proposed HierAttn, a lightweight network that employs a multi-stage, multi-branch attention mechanism guided by a single deep supervision loss [45]. HierAttn achieved 96.70% accuracy on ISIC2019 and 91.22% on PAD-UFES-20, outperforming other lightweight models. It addresses the insufficient feature depth of earlier lightweight networks and the computational burden of heavier ensemble methods. To further enhance performance, we propose a Two-Stream Hybrid Attention-Based Deep Learning Model. The first branch processes the original image through a convolutional layer integrated with three attention modules: Enhanced Separable Depthwise Convolution (SCAttn), stage attention, and branch attention. The second branch applies Contrast Limited Adaptive Histogram Equalization (CLAHE) to enhance local contrast and fine details, then follows a similar feature extraction pipeline. The main contributions of the proposed model are given below:

• We employed a novel hybrid attention dual-stream deep learning model specifically designed for skin lesion detection, which addresses the limitations of existing ensemble CNN models that are often too large and inefficient for contextual information processing.
• Our model effectively utilizes two distinct feature extraction branches: the first branch integrates a convolutional layer with three novel attention modules—Enhanced Separable Depthwise Convolution (SCAttn), branch attention, and stage attention—to capture both local and global hierarchical features from the original images.
• We enhance the input images in the second branch using Contrast Limited Adaptive Histogram Equalization (CLAHE) to improve local contrast and reveal finer details. This enhancement allows for a more nuanced feature extraction through the same attention-based pipeline as the first branch, thereby achieving better feature representation while maintaining computational efficiency.
• The model combines features from both branches, leveraging the strengths of enhanced local contrast and advanced attention mechanisms to produce a comprehensive and robust feature set, which is subsequently fed into a classification module.
• In the extensive experiment with the PAD2020 and ISIC 2019 datasets, we assessed the proposed model and obtained an accuracy rate of 98.59%, of the PAD2020 surpassing the state-of-the-art performance by 2% and stable performance accuracy for the ISIC 2019 dataset. In addition, our model achieved stable performance accuracy for the ISIC-2019 dataset. This shows that the model is better at combining different attention mechanisms and feature improvement methods for accurate and useful skin cancer detection.

We organized the rest of the paper as follows: the literature review is included in Section 2, and datasets are described in Section 3. Methodology of the study is described in the Section 4, results are shown in the Section 5 and lastly, the conclusion is included in the Section 6.

2. Literature Review

Early detection is critical for effective treatment, as delayed diagnosis can lead to metastasis, increased morbidity, and higher mortality rates; however, traditional methods such as visual examination and biopsy are often subjective, time-consuming, and observer-dependent, prompting growing interest in machine learning—particularly deep learning—to enhance diagnostic accuracy and efficiency [1,3,4,5,7,8]. Convolutional Neural Networks (CNNs) have shown significant promise in detecting and classifying skin cancer from medical images [15,29,30]. Esteva et al. [46] utilized CNNs trained on over 129,000 clinical images to detect melanoma, achieving 95% sensitivity, 85% specificity, and 91% overall accuracy, comparable to dermatologists. Similarly, Tschandl et al. [47] achieved an AUC score of 0.94 in distinguishing benign from malignant lesions, and Brinker et al. [48] demonstrated enhanced accuracy using transfer learning. Although deep learning approaches have shown significant promise in detecting and classifying skin cancer, comparative studies are essential to evaluate the effectiveness of different CNN models designed for this purpose [15,17,49,50,51]. Several researchers have conducted comparisons to assess the efficacy of various transfer learning. Transfer learning, leveraging pre-trained CNNs for feature extraction, has further improved classification performance. Han et al. [52] achieved 89.1% accuracy in melanoma detection. Haenssle et al. [52] optimized a VGG-19 model to achieve an AUC of 0.86, while Codella et al. [53] refined an Inception-v3 model to reach an AUC of 0.93. These advancements underscore the transformative potential of deep learning in skin cancer detection. Codella et al. [53,54] compared Inception-V3, ResNet-50, and DenseNet-121, finding that DenseNet-121 achieved the highest classification accuracy with an AUC-ROC of 0.91. Tschandl et al. [53] evaluated five CNN models, including Inception-V3, DenseNet-121, and Xception, and discovered that Xception attained the best performance with an AUC-ROC of 0.93. Similarly, Kawahara et al. [55] used a pre-trained CNN to extract features and trained an SVM classifier, achieving an accuracy of 83.6%. Brinker et al. [48] tested five CNN models, reporting Inception-V3 as the top performer with an AUC of 0.90. Patel et al. [56] highlighted the potential of transfer learning, noting that the InceptionV3 model demonstrated high efficiency. Zhang et al. [57] found that DenseNet-121 achieved the highest AUC value of 0.95 on dermoscopic images. Zaidan et al. [58] emphasized the need for further studies using larger and diverse datasets to improve the robustness of CNN models for skin cancer detection. Recent research has explored alternative deep learning methods, such as Generative Adversarial Networks (GANs), for skin cancer detection and classification [59,60,61]. For instance, Bi et al. [62] utilized a GAN to generate artificial skin lesion images and employed a Convolutional Neural Network (CNN) to classify these images as benign or malignant, achieving a classification accuracy of 83%. Attention mechanisms have also shown promise in computer vision tasks, including image classification and object segmentation [44,63,64,65,66]. For example, reverse attention directs polyp segmentation by using characteristic masks as guiding signals [67], while co-attention modules enhance correlations between branches in Siamese networks for video frame analysis [68]. SENet employs global average pooling to model channel-wise feature dependencies through a squeeze-and-excitation process [63]. CBAM extends SENet by incorporating spatial information using larger kernels [64]. Coordinated Attention (CoorAttn) improves upon SENet by encoding long-range constraints and channel relationships across multiple dimensions [66]. Although these attention-based models enhance feature extraction, they require more trainable parameters and computational resources compared to SENet, making them resource-intensive for real-world applications. To enhance computational efficiency and address scaling requirements, natural language processing techniques like self-attention, particularly transformers, have been introduced to computer vision. The Vision Transformer (ViT) replaces traditional convolutional methods with a transformer encoder that leverages self-attention [65]. ViT processes input images by dividing them into patches and embedding them, resulting in large models with approximately 100 million parameters. We utilized the standard transformer to analyze sequences of image patches and capture inter-patch representations. However, the original transformer approach lacks inductive biases such as translation equivariance and locality inherent in CNNs, leading to suboptimal performance with limited training data [65]. To address this, we incorporated a ViT branch into a CNN-based U-shaped architecture, enhancing long-range dependency modelling and contextual information extraction for skin lesion evaluation. While this approach improved feature representation, it also significantly increased the model’s computational complexity and weight. Despite its effectiveness in reducing the number of learnable parameters, knowledge distillation involves complex parameter updates between teacher and student networks, resulting in high computational costs during training [69,70,71]. On the other hand, Atrous Spatial Pyramid Pooling (ASPP), a lightweight module, enables spatial sampling at multiple scales for better localization of segment borders. However, ASPP does not fully capture feature interactions. Depthwise Separable Convolutions (DWSConv) offer an alternative to lightweight CNNs by replacing traditional convolutions with depthwise convolutions within channels and pointwise convolutions across channels, thereby significantly reducing parameter counts. Using DWSConv, networks like MobileNetV2, MobileNetV3, EfficientNet, MnasNet, and ShuffleNet demonstrated decent performance [72,73,74,75,76]. This study also explores lightweight Vision Transformer (ViT) networks using sparse attention [41], random feature attention [77], and low-rank approximation [78] to reduce computational overhead. To address inductive bias limitations in ViTs, MobileViT combines convolution and transformer blocks into a hybrid architecture, offering improved parameter efficiency and inference speed [44]. Despite the use of EfficientNet and MobileNetV2 for skin lesion diagnosis [79,80], further research is needed to optimize parameter count and computational costs for mobile applications in skin lesion identification.

3. Dataset

This research utilizes two publicly accessible skin disease datasets, ISIC2019 [47,81] and PAD2020 [82], to develop and assess the proposed methodologies. Dermoscopy and cellphones are two conventional techniques for obtaining photos of skin lesions. As a result, the two datasets accurately represent contemporary imaging data for identifying skin lesions.

The ISIC2019 dataset comprises 25,331 dermoscopy photos categorized into eight classifications: actinic keratosis (ACK), basal cell carcinoma (BCC), benign keratosis (BKL), dermatofibroma (DF), melanoma (MEL), melanocytic nevus (NV), squamous cell carcinoma (SCC), and vascular lesion (VASC). Three (SCC, BCC, and ACK) are non-melanoma skin cancers. The most aggressive form of skin cancer is melanoma, resulting from the uncontrolled proliferation of pigment-producing cells. Vascular lesions (VASC) may be benign or malignant and necessitate vigilant observation over time. The other kinds of skin tumors are benign disorders. Figure 1 (a) shows the sample examp lof the ISIC2019 dataset. The PAD2020 dataset comprises 2,298 skin lesion photos categorized into six classes: BCC, MEL, BKL, ACK, NV, and SCC, obtained using cellphones. Figure 1 (b) shows the sample examp lof the PAD2020 dataset and can be downloaded for below link https://www.kaggle.com/datasets/mahdavi1202/skin-cancer. PAD2020 contains two fewer categories, DF and VASC, than ISIC2019 due to the absence of skin lesion photographs.

4. Proposed Methodology

Figure 2 shows the proposed model architecture where we employed a hybrid attention-based two-stream deep neural network to develop a skin cancer recognition system which is the extension of the previous single stream model [45]. We began by preprocessing the input skin cancer dataset and employing augmentation techniques to effectively address overfitting issues. Our approach involves two distinct branches: the first extracts features from the original images, while the second branch extracts features from CLAHE-enhanced images. Each stream is designed with a convolutional layer integrated with a hybrid attention mechanism composed of three novel attention modules: Enhanced Separable Depthwise Convolution (SCAttn), stage attention, and branch attention-based deep neural networks. The primary objective of each stream is to capture both local and global hierarchical representations of the features, enabling a comprehensive understanding of the input data. The hybrid attention mechanism integrates three innovative attention modules—SCAttn [45], stage attention, and branch attention—to significantly enhance performance while maintaining a lightweight model architecture. The branch attention mechanism empowers the network to extract multi-level features from various layers, providing a deeper and more refined interpretation of the input data. The SCAttn module utilizes Depthwise Separable Convolutions (DWSConv) to effectively incorporate global information, which substantially improves the model’s diagnostic accuracy. These attention mechanisms are remarkably efficient, consuming minimal or no learnable parameters, thereby ensuring the hybrid model remains compact without compromising its effectiveness. Furthermore, the stage attention mechanism employs a powerful self-attention strategy that integrates convolutional operations to retain inductive biases and reduce latency, inspired by the MobileViT architecture.

4.1. Image Pre-Processing

The ISIC2019 dataset contains numerous images with large dark regions, which can hinder the accurate evaluation of deep learning models. To address this, we adopt an adaptive cropping technique to focus on the relevant skin lesion areas. First, each image is converted to grayscale and then binarized using OpenCV’s adaptive thresholding method with the mean adaptive approach (block size = 11, constant subtracted = 2) [83]. This method dynamically determines threshold values based on local neighborhood intensity. After binarization, we detect the curved surface to locate the central circular region. If the ratio of the circle’s area to the entire image lies between 0.01 and 0.9, the circular region is cropped and preserved for further processing.

4.2. Data Balance

In this study, an unbalanced ratio is created to measure the imbalance problem. The unbalanced ratio represents a minor portion of the discrepancy between the number of pictures belonging to the ruling class and the minority class. For ISIC2019, the mismatch ratio is 53.9, and for PAD2020, it is 16.3. Previous studies have demonstrated how imbalanced ratios of this magnitude can significantly lower training performance [84]. Because the data isn’t balanced, the small class might get a low validation result, but the overall validation result for all classes might be high. A useful way to deal with a very uneven collection is to balance the data by oversampling or undersampling each class. We simultaneously use both oversampling and undersampling to equalize the ISIC2019 and PAD2020 data. After sampling, we gathered 2500 pictures for the ISIC2019 dataset and 500 images for the PAD2020 dataset. This means that the total amount of data that has been adjusted is close to the total amount of unbalanced data, which is 20,000 for ISIC 2019 and 3,000 for PAD 2020. Oversampling to make new pictures from old ones uses random cuts, Gaussian blur, linear contrast, random translation, rotation, and shear on a small scale. Randomly selecting a set number of images can lead to a problem known as class overlapping. So, to fix this problem, we use instance hardness (IH), a flexible data analysis method [85].

4.3. Image Enhancement CLAHE

Contrast-Limited Adaptive Histogram Equalization (CLAHE) is an advanced image enhancement technique widely used in biomedical imaging to improve local contrast, particularly in regions with non-uniform illumination [86,87,88,89,90]. By enhancing local contrast, CLAHE helps reveal subtle structural details—such as lesion boundaries—critical for medical image analysis. Unlike global histogram equalization, CLAHE operates by dividing the image into small non-overlapping regions, or tiles, typically using an $8\times 8$ grid [91,92]. Within each tile, the local histogram is computed and clipped to a predefined clip limit before being redistributed. This clip limit, defined as $CL=N_{cl}\times N_{avg}$, controls contrast enhancement by preventing over-amplification of noise. Here, $N_{cl}$ is the normalized clip limit (commonly set to 2), and $N_{avg}={\displaystyle \frac{N_x\times N_y}{N_{gray}}}$ represents the average number of pixels per gray level, where $N_x$ and $N_y$ are the tile’s dimensions and $N_{gray}$ is the number of gray levels. If the frequency $n_j$ of a gray level j exceeds the clip limit $CL$, the excess pixels are clipped and redistributed uniformly across all gray levels to preserve the overall intensity distribution. This generates a contrast-limited histogram for each tile. To mitigate block artifacts that may occur due to independent tile processing, CLAHE uses bilinear interpolation to seamlessly blend the tiles, resulting in a more natural and continuous image enhancement [88,89]. By enhancing local contrast without amplifying noise excessively, CLAHE has proven especially effective when integrated into deep learning pipelines for medical imaging tasks, such as lesion detection and skin disease classification.

4.4. CTH Module

The Convolution-Transformer Hybrid (CTH) module, as shown in Figure 3 (a), integrates sequential components designed to process feature maps before and after encoding. The CTH block is composed of a multi-head self-attention (MHSA) component followed by a feed-forward neural network (FFN) layer, commonly referred to as a vision transformer (ViT) [45]. This ViT component effectively captures long-range spatial relationships within the feature maps. To address the need for capturing local features, the CTH block integrates standard convolution operations before and after the ViT module. This addition ensures that both local and global features are captured, as illustrated in Figure 3 (a). Furthermore, the CTH block includes a skip connection that connects the input and output, allowing for the preservation of low-level information from earlier layers. The Stage Attention module operates alongside the CTH block to extensively reorganize the feature maps through downsampling. The CTH block extracts the modified features and uses them for further processing. This integration of attention mechanisms and convolution allows the model to efficiently learn hierarchical features. The following steps outline the operations performed within the CTH block.

Step 1: Local Correlation (Convolutional Feature Processing) Before feeding the input into the ViT module, local correlations are applied to the input feature map. This is achieved using convolutional layers that preserve local feature information.

$$X_{\mathrm{local}}=\mathrm{LocalCorr}\left(X_{\mathrm{in}}\right)+X_{\mathrm{in}}$$

(1)

Where, $X_{\mathrm{in}}$ is the input feature map. $X_{\mathrm{local}}$ represents the output after local correlation is applied.

Step 2: Multi-Head Self-Attention (MHSA)

The next step applies the MHSA module to the input feature map to capture long-range dependencies between the patches:

$$Y=\mathrm{MHSA}\left(\mathrm{Encode}\left(X_{\mathrm{local}}\right)\right)$$

(2)

Where, $\mathrm{Encode}\left(X_{\mathrm{local}}\right)$ encodes the input feature map into patches for attention computation. Y is the feature map after applying the MHSA module.

Step 3: Feed-Forward Network (FFN)

The output from the MHSA is passed through a Feed-Forward Network (FFN), followed by a layer normalization to further refine the features:

$$Z=\mathrm{Decode}\left(\mathrm{LayerNorm}\right(\mathrm{FFN}\left(Y\right)\left)\right)$$

(3)

Where LayerNorm normalizes the features before applying the FFN. Z is the output after decoding and normalization.

Step 4: Local Correlation and Skip Connection

Finally, the output of the ViT module, combined with the original input, is processed through another convolutional layer for final feature extraction:

$$X_{\mathrm{out}}=\mathrm{LocalCorr}(Z+X_{\mathrm{in}})+X_{\mathrm{in}}$$

(4)

Where $X_{\mathrm{out}}$ is the final output feature map. This step ensures that local features are captured while retaining the global contextual information learned from the ViT. This hybrid approach allows the CTH module to effectively capture both local and global feature representations, leading to a more robust model that can handle a wide range of tasks.

4.5. SCADW

The block modifies the DWSConv with the SCAttn technique, resulting in a new SCADW block structure shown in Figure 3 (d). The SCADW block is structured to enhance the performance of the HierAttn architecture while maintaining efficiency [45]. The overall structure of the SCADW block can be summarized as follows: This structure allows the SCADW block to effectively learn and represent features that are crucial for skin lesion classification while ensuring that the model remains efficient and suitable for deployment in resource-constrained environments.

The benefits of the SCADW model are that it enhances feature representation by combining depthwise separable convolution with the SCAttn module, capturing local and global features effectively. It maintains a compact model size, making it suitable for mobile devices with limited computational resources. Here are the details regarding the structure of SCADW.

• Depthwise Separable Convolution(DWSConv): The SCADW block begins with a depthwise separable convolution which consists of two main operations that are- 1) Depthwise Convolution operation that applies a single filter to each input channel separately, which helps in reducing the number of parameters and computational cost compared to standard convolution, 2) Pointwise Convolution that follows the depthwise convolution. Pointwise convolution (1x1 convolution) is applied to combine the outputs from the depthwise convolution across all channels.DWSConv reduces computational complexity by separating convolution into depthwise and pointwise operations, maintaining performance in tasks like image classification and object detection, and allowing for more complex architecture design without increasing computational requirements.
• Integration of SCAttn Module: After the depthwise separable convolution, the SCADW block integrates the SCAttn (Same Channel Attention) module. The SCAttn module utilizes global average pooling to extract global features from the output of the depthwise convolution. This process allows the model to focus on the most relevant features without redundantly processing channel-wise information. SCAttn uses global average pooling to extract global features from the depthwise convolution output, improving feature representation. It reduces redundant operations, lowers computational costs, and is compatible with lightweight models, making it ideal for mobile and edge device applications.
• Output: The output of the SCADW block is a feature map that has been enhanced through the attention mechanism, allowing the model to capture important global context while preserving local feature information [45].

4.6. Stage Attention

Stage attention is key to the HierAttn architecture for skin lesion diagnosis which shows in the 3 (b). It combines a SCADW block with a convolution transformer hybrid (CTH) block to learn local and global features effectively [45]. This module downsamples feature maps, aggregates local features, and applies attention at different stages to enhance focus on relevant features. Stage attention, inspired by the MobileViT architecture, ensures efficient hierarchical learning with low latency, which is critical for mobile applications. Overall, stage attention improves the HierAttn network’s classification accuracy and AUC scores by enabling multi-level feature representation.

4.7. Branch Attention

To enhance feature interactions during training, Zhang et al. calculated three additional losses inside intermediate phases[93]. We used branch attention, a deep supervision method that uses hierarchical pooling to make tensors from later stages smaller [45]. It can be seen in the lower 3 (c). This approach uses tensor assembly to enhance the interactions of the shrunk features and learn the feature hierarchy from the built tensors. Furthermore, hierarchical pooling optimizes tensor representations by preserving the varying sizes of pooling results. This allows for the generation of a large tensor size ($C\times 5\times 5$) while capturing global representations, resulting in a lower tensor size ($C\times 1\times 1$). A medium output tensor of dimensions ($C\times 3\times 3$) is included as a buffer layer to maintain the integrity of both local and global characteristics. Following the channel-wise assembly of the pooled tensors from various branches, the ensembled tensors are next combined pixel-wise. Once the branch attention is complete, the Monte Carlo technique reorganizes the tensors pixel by pixel. Each stage attention block, also known as the key learning stage, implements the branch attention operation as a specific learning phase. Significantly, this pragmatic design avoids introducing any extra loss during the intermediate stage, thereby requiring relatively lower computational resources than the traditional deep supervision approach.

4.8. Stream-1: Original Image Based Feature

We implemented an initial convolutional layer to handle the preprocessed image in the first branch. Then, three SCAttn (Enhanced Separable Depthwise Convolution) modules with 1, 2, and 1 stride sizes were sequentially integrated [45]. This setup improves the model’s capacity to grasp both detailed and wide contextual information by facilitating efficient feature extraction at different receptive fields. The computational efficiency is maintained by each SCAttn module, which efficiently concentrates on key spatial features, thanks to its depthwise separable structure. The output from the SCAttn modules is then processed through the stage attention mechanism, which further refines the feature maps by dynamically adjusting the focus on important regions across different stages of the network. This stage attention ensures that both local and global feature representations are optimally balanced, reducing redundancy and enhancing feature diversity. After the features are improved, they are sent to the branch attention module, which groups and puts together features from different branches in a hierarchical way. This step greatly improves how features interact with each other and lets the model learn complicated hierarchical representations, bringing together specific local patterns with important global contexts. The hybrid attention modules process the combined features and then input them into an additional convolutional layer to capture and refine abstract concepts. Lastly, the final feature representation, $F_{GStream}$ is created by applying an average pooling layer. The model’s diagnostic accuracy and resilience are enhanced by the first branch’s ability to extract a rich set of features, which captures intricate local details while simultaneously preserving a knowledge of the overall spatial structure. The thorough integration of numerous attention mechanisms achieves this.

4.9. Stream-2: Enhanced Image-Based Feature

We improved the input image using the CLAHE method in the second branch. By altering the histogram of each small region, CLAHE successfully enhances local contrast and displays finer details in the skin lesion photos. To give the model a more complete input, this improvement is essential for bringing attention to delicate patterns and features that may go unnoticed during regular preprocessing. To make more accurate feature representations, CLAHE boosts the effectiveness of each layer of feature extraction by focusing on key details while keeping edge information. Extracting spatial characteristics from the CLAHE-enhanced image begins with its passage through a convolutional layer. Three SCAttn (Enhanced Separable Depthwise Convolution) modules, each having a stride size of 1, 2, and 1, are then fed this output in that order. Using the depthwise separable convolutions to efficiently zero in on important spatial details with minimal computational overhead, the integration of CLAHE enhancement with SCAttn modules takes advantage of the enhanced local contrast to extract more effective features. The stage attention mechanism processes the output from the SCAttn modules and dynamically shifts the emphasis to crucial regions throughout stages, ensuring a balance between refined local details and broader contextual information. Because it enables the model to focus on the most informative portions of the image, this step is especially beneficial when coupled with the CLAHE-enhanced input. The next step is to use the branch attention module to improve the features. This module hierarchically assembles and pools features from multiple levels. By studying a complicated hierarchy that captures both important global contexts and specific local patterns, this module improves the interactions between features even further. The output is refined and abstracted further by passing it through an additional convolutional layer after integrating these features using the hybrid attention techniques. The last step in creating the feature representation, $F_{CStream}$, is to apply an average pooling layer. With the help of CLAHE improvement and several attention modules, this branch can gather improved local textures as well as global structural patterns, leading to a more accurate and resilient model.

4.10. Feature Concatenation and Classification

To further enhance the model’s feature representation, we combined the two streams’ features by concatenating them, as shown in Equation 5:

$$F_{\mathrm{Concat}}=F_{\mathrm{GStream}}⨁F_{\mathrm{CStream}}$$

(5)

where $F_{\mathrm{Concat}}$ represents the concatenated feature vector fed into the classification module. This concatenation of features from the two branches, $F_{\mathrm{GStream}}$ and $F_{\mathrm{CStream}}$, effectively integrates complementary information from both the original and CLAHE-enhanced images. The first branch, $F_{\mathrm{GStream}}$, captures the fundamental features from the preprocessed images, providing a solid base of local and global hierarchical representations. Meanwhile, the second branch, $F_{\mathrm{CStream}}$, contributes enhanced features that emphasize critical local contrast and subtle patterns revealed by CLAHE. By merging these two streams, the model benefits from a richer, more comprehensive feature set that includes both the original spatial structures and the enhanced details. This fusion enables the model to leverage the strengths of both inputs, improving its ability to differentiate between skin lesions with subtle variations. The combined feature representation, $F_{\mathrm{Concat}}$, offers a more robust and discriminative input for the classification module, ultimately enhancing the overall diagnostic accuracy and effectiveness of the proposed model.

4.11. Classification and Training Procedure

After feature extraction through the dual-branch network, we employed a robust classification module to produce a probabilistic map for skin cancer detection. This module takes the concatenated feature vector, $F_{\mathrm{Concat}}$, and processes it through a series of fully connected layers. Each layer refines the features by learning complex, non-linear relationships, which are critical for distinguishing between different skin lesion types. The output layer of the classification module uses a softmax activation function to generate a probabilistic map, providing the likelihood of each input belonging to different skin cancer categories. This probabilistic map not only facilitates accurate disease detection but also offers insights into the model’s confidence in its predictions.

The training procedure is designed to optimize the classification performance by minimizing a cross-entropy loss function, which measures the divergence between the predicted probabilities and the true class labels. We employed a stochastic gradient descent (SGD) optimizer with momentum to accelerate convergence and avoid local minima. The training process includes data augmentation techniques and regularization methods, such as dropout, to prevent overfitting and improve the model’s generalization capability. The model is trained iteratively, continuously updating weights based on backpropagation until it reaches optimal performance, thus ensuring reliable and precise skin cancer detection.

5. Experimental Result

We evaluated the proposed model using various datasets with various modalities. Based on the old procedure, we divided the dataset into two parts: training and testing. Additionally, we display our environmental settings and the effectiveness of our information processing below.

5.0.1. Environmental Setup and Evaluation Metrics

The system was constructed using a GeForce RTX 4090 24GB graphics processing unit (GPU), CUDA 10.7, NVIDIA driver 515, and 64GB of random access memory (RAM). The training experiment had a learning rate of 0.005 and a batch size of 32. It was executed for 300 epochs utilizing the Adam optimizer on the RTX 4090 architecture. To achieve optimized graph convolution and attention with minimal computational expense, the Python environment used OpenCV, Pickle, Pandas, and PyTorch for model construction[94]. These packages, in addition to a few others[95,96], enabled the initial processing of data and the building of models. The assessment measure for assessing the accuracy of the proposed identification model, analogous to Equation 6, is calculated using TN for true negatives, TP for true positives, FN for false negatives, and FP for false positives.

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(6)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(7)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(8)

$$F1=2\ast {\displaystyle \frac{Precision\ast Recall}{Precision+Recall}}$$

(9)

We assessed the proposed model using two datasets and computed the accuracy of the performance for many models for each dataset.

5.1. Ablation Study

The ablation study evaluates the impact of applying augmentation before and after splitting the dataset on model performance shows in Table 1. In the augmentation before splitting approach, augmentation is applied to the entire dataset before the train-test split, ensuring that both the training and testing sets benefit from the enhanced features. This configuration showed the highest performance, achieving 98.59% accuracy on the PAD2020 dataset and 83.00% (average) accuracy on the ISIC-2019 dataset, with a peak of 88.62% in the best fold. This method leverages the full potential of data augmentation, improving the feature representation for both training and testing phases. In contrast, the augmentation after splitting, where augmentation is applied only to the training set post-split, resulted in lower performance. The accuracy dropped to 70.00% on PAD2020 and 62.65% on ISIC-2019, indicating that applying augmentation only to the training data after the split fails to fully exploit the benefits of enhanced feature extraction across the entire dataset. This highlights the importance of applying augmentation before the data split to achieve superior performance and feature representation.

5.2. Performance Accuracy for the PAD2020 Dataset

Table 2 shows the performance metrics of the proposed model across ten validation folds using the PAD2020 dataset. The evaluation includes Best Accuracy, Recall, Specificity, Precision, and F1 Score for each fold. The model demonstrates consistently high accuracy (average 98.59%) and specificity (average 99.71%), indicating a strong ability to correctly identify true negatives. The average recall (79.44%), precision (80.18%), and F1 score (79.78%) reflect a balanced performance in detecting positive cases, confirming the model’s effectiveness and robustness in Parkinson’s disease classification.

Table 3 provides a comparative analysis of recent studies focusing on various skin cancer datasets, specifically highlighting different models and their performance in terms of accuracy. The datasets considered include PAD2020, which shows that VGG16 models achieved an accuracy of 70.83%. Among the models applied exclusively to the PAD2020 dataset, the MobileNetV2 and MobileNetV3L models achieved high accuracy rates of 91.44% and 91.78%, respectively. ShuffleNetV2_1.× and EfficientNet_b0 also performed well, with accuracies around 91.89% and 91.13%. Meanwhile, more advanced methodologies, such as the HierAttn series models, reported the highest accuracy, with HierAttn_s reaching 91.22% [45], demonstrating attention mechanisms’ potential in enhancing model performance. This comprehensive comparison underscores the efficacy of various deep-learning models and techniques in enhancing skin cancer detection accuracy. Models like ResNet101, MnasNet1.0, and RegNetY6.4gf show an appropriate combination of computational efficiency and high accuracy, rendering them viable options for clinical applications and automated diagnostics in dermatology.

5.3. Performance Accuracy for the ISIC-2019 Dataset

Table 4 demonstrated the fold-wise performance accuracy for the ISIC-2019 dataset. To prove the stability and generalizability of the proposed model, we also evaluated the ISIC-2019 datasets. We achieved 82.00% accuracy for the single stream in our own experiment, which is a similar model to the existing article [45], and 83.00% for the proposed dual stream. This performance proves that our proposed model also produces stable performance accuracy with other datasets.

Table 5 compares several state-of-the-art models applied to various skin lesion datasets, with a focus on the ISIC 2019 dataset. Models such as CNN, ResNet50, Xception, DenseNet201, MobileNetV2, and others are evaluated based on different training configurations, including splitting first, followed by augmentation, or vice versa. Performance metrics include accuracy, F1-score, and specificity. The proposed dual-stream model outperforms others with 88.62% accuracy and 98.37% specificity, demonstrating superior classification efficiency and minimizing false positives. In comparison, our experiments with a single-stream model achieved 82.00% accuracy, similar to the architecture used by Dai et al. [45], which reported 96.70% accuracy. This discrepancy suggests potential differences in experimental conditions or dataset variations. However, the dual-stream model showed a slight improvement. This comparison highlights the proposed model’s stability cross different datasets, emphasising the importance of model architecture choices in achieving higher accuracy and specificity in skin lesion classification.

5.4. Dicussion

The proposed Hybrid Attention Dual-Stream Deep Learning Model introduces a lightweight and effective architecture for skin cancer detection. It leverages a dual-branch design where the first branch processes the raw image through multi-stage attention modules (SCAttn, stage attention, and branch attention), while the second branch uses CLAHE-enhanced images to extract fine-grained local details. Both branches’ features are fused and passed through a unified classification module under a single loss function, ensuring end-to-end learning and robust generalization. To evaluate the model’s effectiveness, we conducted experiments on two publicly available datasets: PAD2020 and ISIC-2019. The proposed model achieved a state-of-the-art performance of 98.59% accuracy on PAD2020, significantly surpassing prior models such as MobileNetV3L (91.78%), EfficientNet_b0 (91.13%), and even ensembled models (94.11%). The ablation study further revealed that applying augmentation before the dataset split considerably improves accuracy, supporting the model’s ability to extract rich, representative features across training and testing phases. On the ISIC-2019 dataset, the proposed model reached an average accuracy of 83.00% with a best fold performance of 88.62% and specificity of 98.37%, outperforming several existing deep CNN models including Xception, DenseNet201, and ResNet50V2, which reported lower accuracies ranging from 76.09% to 86.7% in previous studies. Although Dai et al. reported 96.70% with a different experimental setting, our dual-stream architecture demonstrated more stable cross-validation performance, confirming the model’s generalizability.

In addition to its medical imaging applications, the proposed hybrid attention dual-stream model has strong implications for Future Internet technologies, particularly in the domains of edge AI and smart healthcare systems. The lightweight and modular design of the model makes it suitable for deployment on resource-constrained devices such as mobile phones and embedded systems, enabling real-time analysis and feedback without relying on cloud-based computation. This supports intelligent human–machine interfaces, where patients or clinicians can interact with AI-powered tools directly through connected devices. Moreover, integrating this model within IoT-based medical networks could facilitate remote diagnostics and teledermatology, especially in underserved or rural areas. In such scenarios, the model enables efficient data processing at the edge, reducing latency and bandwidth usage—both of which are critical factors in the design of Future Internet architectures for healthcare. These capabilities align well with the vision of the Future Internet, where distributed intelligence, networked sensors, and user-centric interfaces drive personalized and accessible medical care.

6. Conclusions

This study tackles the significant difficulty of accurate and prompt skin cancer diagnosis by introducing an innovative hybrid attention dual-stream deep learning model for the identification of skin lesions. Our model overcomes the limitations of existing ensemble convolutional neural networks (CNNs), which are often too large and inefficient for contextual processing, as well as lightweight networks like MobileNetV3 and EfficientNet, which have inadequate feature representation depth. Our approach integrates two complementary branches: one that extracts local and global features using Enhanced Separable Depthwise Convolution (SCAttn), stage attention, and branch attention modules, and another that enhances images using CLAHE to capture more effective features. This dual-stream model, which combines multiple attention mechanisms and feature enhancement techniques, significantly improves the robustness and accuracy of skin lesion detection. Clinically, this advancement enhances early detection and diagnostic accuracy, which are critical factors in improving patient outcomes. The model’s ability to provide a comprehensive, hierarchical feature set makes it a promising tool for dermatologists, offering more reliable and efficient skin cancer diagnosis. Moving forward, future work will focus on refining the model’s architecture for handling more complex datasets, exploring additional attention mechanisms, and expanding its applicability to other medical imaging tasks, further improving its clinical utility.