Enhancing Skin Cancer Classification Using Efficient Net B0-B7 through Convolutional Neural Networks and Transfer Learning with Patient-Specific Data

Abstract

Background:

Skin cancer diagnosis challenges dermatologists due to its complex visual variations across diagnostic categories. Convolutional neural networks (CNNs), specifically the Efficient Net B0-B7 series, have shown superiority in multiclass skin cancer classification. This study addresses the limitations of visual examination by presenting a tailored preprocessing pipeline designed for Efficient Net models. Leveraging transfer learning with pre-trained ImageNet weights, the research aims to enhance diagnostic accuracy in an imbalanced multiclass classification context.

Methods:

The study develops a specialized image preprocessing pipeline involving image scaling, dataset augmentation, and artifact removal tailored to the nuances of Efficient Net models. Using the Efficient Net B0-B7 dataset, transfer learning fine-tunes CNNs with pre-trained ImageNet weights. Rigorous evaluation employs key metrics like Precision, Recall, Accuracy, F1 Score, and Confusion Matrices to assess the impact of transfer learning and fine-tuning on each Efficient Net variant’s performance in classifying diverse skin cancer categories.

Results:

The research showcases the effectiveness of the tailored preprocessing pipeline for Efficient Net models. Transfer learning and fine-tuning significantly enhance the models’ ability to discern diverse skin cancer categories. The evaluation of eight Efficient Net models (B0-B7) for skin cancer classification reveals distinct performance patterns across various cancer classes. While the majority class, Benign Kertosis, achieves high accuracy (>87%), challenges arise in accurately classifying Eczema classes. Melanoma, despite its minority representation (2.42% of images), attains an average accuracy of 80.51% across all models. However, suboptimal performance is observed in predicting warts molluscum (90.7%) and psoriasis (84.2%) instances, highlighting the need for targeted improvements in accurately identifying specific skin cancer types.

Conclusion:

The study on skin cancer classification utilizes EfficientNets B0-B7 with transfer learning from ImageNet weights. The pinnacle performance is observed with EfficientNet-B7, achieving a groundbreaking top-1 accuracy of 84.4% and top-5 accuracy of 97.1%. Remarkably efficient, it is 8.4 times smaller than the leading CNN. Detailed per-class classification exactitudes through Confusion Matrices affirm its proficiency, signaling the potential of EfficientNets for precise dermatological image analysis.

Introduction

Skin cancer is characterized by the abnormal growth of skin cells, and represents a pervasive and potentially life-threatening condition with a growing global issue. Its etiology is multifactorial with ultraviolet (UV) radiation exposure being a primary risk factor. The disease encompasses various types, including basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and melanoma, each with distinct clinical characteristics and prognoses. Despite advancements in prevention efforts, the incidence of skin cancer continues to rise, necessitating improved diagnostic and classification methodologies. In recent years, the integration of cutting-edge technologies, particularly machine learning (ML), deep learning (DL), and image processing techniques, has propelled significant progress in the field of skin cancer detection and classification. Notably, ML algorithms have demonstrated the potential to automate and enhance the accuracy of diagnostic processes, thereby mitigating the subjectivity associated with traditional methods reliant on visual inspection. The detection and classification of skin cancer, a pervasive and potentially life-threatening condition, have witnessed substantial progress due to the integration of cutting-edge technologies, primarily machine learning (ML), deep learning (DL), and image processing techniques. Recent years have seen a surge in studies dedicated to refining algorithms for accurate skin lesion classification, notably the contributions of Hameed et al., Nordmann et al. [2], Yang et al. [3], and Tan et al. [4], significantly advancing this field [1-7].

These studies have introduced diverse methodologies that have revolutionized skin cancer classification accuracy. Notably, the incorporation of convolutional neural networks (CNNs), transfer learning strategies, and innovative image preprocessing techniques proposed by Chatterjee et al., Lima et al., Gutman et al., and others have effectively addressed the challenges associated with feature extraction and classification of various skin lesion types [8-12]. These advancements have culminated in remarkably enhanced diagnostic precision, fostering confidence in automated diagnostic systems.

Moreover, pioneering research efforts led by Alom et al., Ali et al., Tahir et al., and Baig et al. have introduced novel architectural designs, lightweight CNN models, and modifications to existing network structures [13-18]. These innovations have showcased promising outcomes, significantly amplifying the efficiency and overall performance of skin cancer detection systems. Their efforts represent a vital stride in refining the algorithms capability to accurately identify and classify skin lesions, underscoring the continuous evolution in this field.

The integration of state-of-the-art technologies, such as hyperspectral imaging [19], quantum computing [20], and parallel CNN models [21], have further amplified the sophistication of skin cancer detection methodologies. [19-27]. These advancements hold significant promise in delivering heightened accuracy and clinical utility, envisioning a future where diagnostic tools harness the potential of advanced technologies for precise and efficient skin cancer identification.

Furthermore, studies by Abuared et al. [28], Esteva et al. [29], and Fu’adah et al. [30] have notably emphasized the potential for achieving dermatologist-level accuracy in skin cancer classification [28-30]. The Table 1 provides a comprehensive overview of recent skin cancer research, summarizing methodologies, technologies, key findings, and contributions from influential studies. The diverse approaches include multiclass multi-level classification algorithms, gamma knife radiosurgery, clinical skin lesion diagnosis using dermatologist criteria, intelligent skin cancer diagnosis, and segmentation/classification techniques using advanced CNN architectures. The findings range from improved classification accuracy to enhanced diagnostic precision, showcasing the continuous evolution and promising outcomes in the field of skin cancer detection and classification. Their findings highlight the remarkable strides made in bridging the gap between machine learning algorithms and expert human diagnosis, paving the way for more reliable and accessible diagnostic tools. By shedding light on the current landscape and contemplating future directions in this critical domain, this review aims to contribute to the continued advancements in skin cancer diagnosis, ultimately benefiting patients and healthcare practitioners.

Table 1.

Comparative Overview of Methodologies, Technologies, and Contributions in Recent Skin Cancer Research

Article	Methodology	Technology Used	Key Findings	Contributions
Hameed et al. (2020) [1]	Multiclass multi-level classification algorithm.	Machine learning techniques.	Improved skin lesion classification accuracy.	Enhanced classification algorithm.
Nordmann et al. (2017) [2]	Gamma knife radiosurgery & programmed cell death 1 receptor antagonists	Radiosurgery, Immunotherapy	Impact on metastatic melanoma	Treatment effects on melanoma
Yang et al. (2018) [3]	Clinical skin lesion diagnosis using dermatologist criteria	Representation techniques	Dermatologist-inspired diagnostic representations	Improved clinical diagnosis
Tan et al. (2019) [4]	Intelligent skin cancer diagnosis	Improved particle swarm optimization, Deep learning models	Enhanced skin cancer diagnosis using optimization methods	Improved diagnostic accuracy
Alom et al. (2019) [11]	Skin cancer segmentation and classification	NABLA-N, Inception recurrent residual CNNs	Improved segmentation and classification	Advanced CNN architectures
Abuared et al. (2020) [26]	Skin cancer classification based on VGG 19 and transfer learning	VGG 19, Transfer learning	Achieving dermatologist-level accuracy	Bridging ML with human diagnosis

The preceding studies have demonstrated notable efficacy in the binary classification of skin cancers. Nevertheless, the challenges posed by significant inter-class similarity and intra-class variability persist in the realm of multiclass skin cancer classification, also referred to as automated skin cancer classification. This paper addresses the classification of skin cancer using EfficientNets B0-B7 on the International Skin Imaging Collaboration (ISIC) 2020 challenge dataset. The approach involves leveraging pre-trained weights from ImageNet for transfer learning and fine-tuning the Convolutional Neural Networks (CNNs) on the dataset. Performance evaluation metrics, including Precision, Recall, Accuracy, F1 Score, Specificity, Roc Auc Score, and Confusion Matrices, are employed to assess the effectiveness of EfficientNets B0-B7. Additionally, the paper provides a detailed presentation of per-class classification exactitudes through Confusion Matrices for all eight models. Notably, our EfficientNet-B7 attains a groundbreaking state-of-the-art with 84.4% top-1 accuracy and 97.1% top-5 accuracy, while being 8.4 times smaller than the best existing CNN.

Dataset

The ISBI-2016 dataset, part of the ISIC Challenge, is pivotal for medical imaging in dermatology. Introduced in 2016, it facilitates the development of automated skin lesion analysis algorithms. This diverse dataset comprises high-resolution images of various skin lesions, annotated with crucial clinical details. Widely used in research, it serves as a benchmark for computer-aided diagnosis systems, fostering progress in lesion segmentation and classification. Leveraging ISBI-2016 advances automated dermatological analysis for early detection.

The proposed method was evaluated using the publicly available ISBI-2016 test dataset. Specifically designed for dermoscopic images, it includes around 379 de-identified images with both benign and malignant skin lesions. Each image is dermatologist-labeled for melanoma presence. The dataset’s standardized format and established ground truth make it invaluable for comparing skin cancer classification algorithms.

Materials and Methods

The methodology employed in this research is a systematic and thorough approach to skin disease classification using Convolutional Neural Networks (CNNs), with a particular emphasis on the EfficientNets B0-B7 architecture. Illustrated in Figure 1, the method encompasses the entire process from acquiring the image database to the final classification step. Each stage of the methodology is designed to address specific challenges associated with skin lesion images and optimize the performance of the CNN model.

Figure 1.

Architecture Diagram of Methodology

The initial step involves obtaining the image dataset from freely accessible pictures on the ISIC site. This dataset comprises RGB dermoscopic images alongside their corresponding ground truth segmentations. The inclusion of the ISBI-2016 test dataset enriches the variety of images available for analysis, providing a diverse set of skin lesion images for training and evaluation purposes. The images in the dataset primarily comprise pigmented skin lesions.

Given that the primary objective is to classify skin disease growth classes, the presence of hairs becomes irrelevant and contributes noise to the images. To address this, an image processing pipeline is employed. The pipeline prioritizes noise reduction, acknowledging that the presence of hair strands may cause the convolutional neural network (CNN) model to erroneously associate irrelevant connections between noise and the target skin disease class. By removing this noise, the CNN model can better discern relevant features for accurate classification.

For effective skin disease classification, CNNs need to be appropriately scaled. The Efficient Net model architecture, particularly the EfficientNets B0-B7, is selected for its ability to scale CNNs to achieve better precision and efficiency. This architecture allows for scaling in terms of depth, width, and image resolution. Experimentation involves evaluating the performance of all eight Efficient Net models on the dataset to identify the most suitable scaling strategy.

In the medical domain, acquiring labelled data for training neural networks is challenging and expensive. To overcome this limitation, the dataset is augmented through image expansion. This process involves creating variations of existing images by applying transformations such as rotations, flips, and zooms. Image augmentation not only increases the dataset size but also enhances the model ability to generalize to diverse skin lesion patterns.

The final classification step involves training the Efficient Nets B0-B7 on the augmented dataset. Transfer learning is employed, utilizing pre-trained weights from the ImageNet dataset. This approach leverages the knowledge gained by the model on a large-scale dataset (ImageNet) and adapts it to the specific features of skin lesion images. Fine-tuning of the Convolutional Neural Networks (CNNs) is performed to refine the model ability to classify skin disease growth classes accurately.

The classification results demonstrate the effectiveness of the proposed methodology. By incorporating techniques such as target scaling, data augmentation, and noise removal, the CNN model achieves high precision in identifying different skin disease classes. The experimentation with the EfficientNets B0-B7 reveals variations in performance across different groups of skin diseases. This suggests the potential for developing tailored models for specific types of skin cancer.

Figure 2 provides an activity diagram that visually represents the sequential steps involved in the methodology. Starting with the acquisition of the image database, the diagram illustrates the flow through the image processing pipeline, the evaluation of the EfficientNet models, image augmentation, and the final classification process. This visual representation offers a concise overview of the intricate methodology employed in this research. The methodology outlined in this section integrates various techniques and considerations to address the complexities of skin lesion images. From noise reduction to image augmentation and transfer learning, each step contributes to the robustness and accuracy of the CNN model in classifying skin diseases. The results and analysis provide insights into the effectiveness of the methodology and open avenues for further research in developing customized models for specific types of skin cancer.

Figure 2.

Activity Diagram

Results

The image preprocessing phase was pivotal in refining the raw image data to prepare it for accurate disease classification. Techniques like Gaussian blur and median blur were employed to eliminate undesired elements like background noise and body hair from pigmented skin sore images. These methods played a crucial role in accentuating the actual infected regions, thereby facilitating higher accuracy levels and minimizing errors within the dataset in Figure 3.

Figure 3.

Image Blurring

Following the blurring techniques, the system utilized multiple thresholding methods such as global thresholding, adaptive thresholding, and Otsu’s thresholding to effectively segregate the skin lesions from the surrounding tissues. This process significantly enhanced the visibility and clarity of the affected regions, making them more distinct and easier to analyze for accurate disease classification as shown in Figure 4. To further enhance image quality and ensure optimal feature extraction, histogram equalization and contrast stretching techniques were applied. These methods effectively improved the overall contrast and brightness within the images, consequently facilitating better feature extraction and higher accuracy rates in subsequent classification stages depicted in Figure 5.

Figure 4.

Image Thresholding

Figure 5.

Image Thresholding

The transformation of images into grayscale representations played a pivotal role in simplifying subsequent image analysis processes. This conversion streamlined computational complexities while retaining crucial texture and shape information, enabling more efficient feature extraction and enhancing the accuracy of classification algorithms and the image as shown in Figure 6. The study extensively focused on comprehending the intricate relationships between scaling dimensions and their impact on the model’s performance. This comprehensive evaluation aimed to optimize the trade-offs between computational efficiency and the model’s ability to capture intricate features and that is displayed in Figure 7.

Figure 6.

Grayscale Image

Figure 7.

Efficient Net Model Architecture

The deployment phase involved integrating the trained model into practical applications with a strong emphasis on user accessibility and functionality. The utilization of Flask, a Python-based web framework, facilitated the development of RESTful APIs, ensuring seamless interaction with the model. Additionally, the implementation of React JS for creating an intuitive user interface allowed users to effortlessly upload images for disease prediction.

A rigorous and comprehensive testing regimen was meticulously executed to validate the reliability and accuracy of the deployed system. The testing process encompassed unit testing and integration testing, meticulously scrutinizing both the front-end and back-end functionalities. These testing phases were essential in verifying the system’s robustness across diverse scenarios, ensuring its effectiveness and reliability in real-world applications refer to Figure 8.

Figure 8.

Block Diagram of Testing Process

Discussion

Iterative adjustments and exploration of scaling strategies were instrumental in refining the baseline EfficientNet architecture. Each variant, from EfficientNets B0 to B7, underwent rigorous evaluation to ascertain its ability to capture intricate features while managing computational demands effectively.

Experiments was conducted using EfficientNet models (B0-B7) on the ISBI-2016 test dataset, measuring accuracy at various tiers, including top-1%, top-2%, top-3%, top-4%, and top-5%. The ISBI-2016 dataset consists of medical images annotated for segmentation tasks, providing a challenging testbed for evaluating model performance. A standard evaluation metric is employed to assess the segmentation accuracy of each model and analyze their relative performance results revealing a consistent improvement in segmentation accuracy with increasing model complexity across all tiers. EfficientNet B7 demonstrates the highest accuracy among all models, showcasing its effectiveness in image segmentation tasks on the ISBI-2016 dataset. Moreover, the gap in accuracy between different tiers diminishes as model complexity increases, indicating the robust performance of higher-complexity models across a broader range of segmentation scenarios. The segmentation accuracy results obtained from the experiments are summarized in the following Table 2. EfficientNet B7 emerges as the top-performing model, offering superior accuracy across all tiers on the ISBI-2016 dataset. However, the computational cost associated with higher-complexity models must be carefully considered in practical applications. Experimental results illustrate how the suggested clinical descriptions not only accurately and consistently capture the signs of skin lesions with dermatological models but also enhance the predictive performance compared to state-of-the-art techniques based on uninterrupted features. Based on the illustrated chart in Figure 8, the efficient net algorithm had the highest percentage in a statistical analysis of techniques and dermatological diseases utilizing the image processing method. Acne, melanoma, psoriasis, eczema, keratosis, lichen planus, and rosacea are the most common diseases classified.

Table 2.

Segmentation Accuracy of Efficient Net Models on ISBI-2016 Test Dataset

Model	TOP 1 %	TOP2 %	TOP 3%	TOP 4%	TOP 5%
BO	81	82.7	84.2	85.7	87.5
B1	82.5	82.9	85.4	86.3	89.5
B2	82.7	83.3	86.2	87.5	91.3
B3	83.4	83.9	87.2	89.8	93.2
B4	83.7	84.5	88.1	90.3	94.5
B5	83.9	85.7	89.7	91.1	95.3
B6	84.1	86.5	90.2	92.2	96.7
B7	84.4	87.1	91.1	93.5	97.1

This study evaluates the performance of eight Efficient Net models (B0-B7) for the classification of skin cancer using confusion matrices shown in Figure 9. The analysis highlights distinctive patterns in model across various cancer classes. Notably, while the majority class, Benign Kertosis, achieved high accuracy (>87%), challenges emerged in classifying Eczema classes. Despite its minority representation (2.42% of images), the melanoma class attained an average accuracy of 80.51% across all models. Conversely, the models demonstrated suboptimal performance for excelling particularly in accurately predicting warts molluscum (90.7%) and psoriasis (84.2%) instances. These findings underscore substantial variations in model performance concerning different skin cancer classifications, emphasizing the need for targeted improvements in accurately identifying specific cancer types.

Figure 9.

Confusion Matrix

In conclusion, our research represents a significant stride in developing and evaluating a skin cancer diagnosis model. The research explored the distinction between a basic Convolutional Neural Network (CNN) and the more potent EfficientNet model, emphasizing the latter’s efficacy in skin disease classification. Through meticulous application of transfer learning and the dataset used for training with EfficientNets, creating a model that rapidly provides an overview of skin conditions. This tool not only serves as a quick diagnostic aid but also holds potential for healthcare professionals seeking a brief understanding of conditions and disease severity.Looking ahead, future work could involve integrating this model into mobile devices for global accessibility. Skin diseases are a prevalent health concern, and our research demonstrated that CNNs, particularly EfficientNets, outperform dermatologists in classifying various skin diseases.

This paper introduced a preprocessing image pipeline addressing challenges like hair removal, dataset enhancement, and image scaling tailored to each model’s requirements. Through transfer learning on pre-trained weights from ImageNet and fine-tuning the CNNs, Efficient Nets B0-B7 were trained on the dataset. Techniques like target scaling, data augmentation, and noise removal, along with successful transfer learning, contributed to high classification results. Analysis using Confusion Matrices revealed performance variations among different skin disease groups, suggesting potential refinement opportunities for specific disease types.

Results showcase the effectiveness of the preprocessing pipeline for EfficientNet models. Transfer learning and fine-tuning enhance models’ ability to discern diverse skin cancer categories. Evaluation of eight EfficientNet models (B0-B7) reveals distinct performance patterns across various cancer classes. While Benign Kertosis achieves high accuracy (>87%), challenges arise in classifying Eczema classes. Melanoma, despite its minority representation (2.42% of images), attains an average accuracy of 80.51% across all models. Suboptimal performance is observed in predicting warts molluscum (90.7%) and psoriasis (84.2%) instances, emphasizing the need for targeted improvements in identifying specific skin cancer types.

In summary, the study on skin cancer classification with EfficientNets B0-B7, utilizing transfer learning from ImageNet weights, achieved groundbreaking accuracy. EfficientNet-B7 stood out with a top-1 accuracy of 84.4% and top-5 accuracy of 97.1%, being 8.4 times smaller than the leading CNN. Detailed per-class classification through Confusion Matrices affirms its proficiency, highlighting EfficientNets’ potential for precise dermatological image analysis. The research underscores ongoing opportunities for improvement in skin disease diagnosis through advanced machine-learning techniques.

Author Contribution Statement

All authors contributed equally in this study.