Classification of Skin Lesions into Seven Classes Using Transfer Learning with AlexNet

Abstract

Melanoma is deadly skin cancer. There is a high similarity between different kinds of skin lesions, which lead to incorrect classification. Accurate classification of a skin lesion in its early stages saves human life. In this paper, a highly accurate method proposed for the skin lesion classification process. The proposed method utilized transfer learning with pre-trained AlexNet. The parameters of the original model used as initial values, where we randomly initialize the weights of the last three replaced layers. The proposed method was tested using the most recent public dataset, ISIC 2018. Based on the obtained results, we could say that the proposed method achieved a great success where it accurately classifies the skin lesions into seven classes. These classes are melanoma, melanocytic nevus, basal cell carcinoma, actinic keratosis, benign keratosis, dermatofibroma, and vascular lesion. The achieved percentages are 98.70%, 95.60%, 99.27%, and 95.06% for accuracy, sensitivity, specificity, and precision, respectively.

Introduction

Detection and accurate classification of skin lesions using computer-aided systems are steadily growing as a challenging scientific topic. Physicians can classify skin lesions into different kinds such as benign or non-malignant (nevus), cancerous (melanoma), pigmented benign keratosis (BKL), basal cell carcinoma (BCC), and squamous cell carcinoma (SCC). Dermatologists reported that melanoma is able to propagate to different places and organs of the human body [1]. Therefore, melanoma is the most aggressive type of skin cancer. For this reason, it is far responsible for the highest mortality rate despite its low occurrence [2]. The increased rate of skin cancer originates from direct and continuous exposure to the sun, which causes benign and malignant tumors. Both melanoma and nevus are regarded as melanocytic tumor types, which leads to the wrong distinction between these two types by the naked eyes of the dermatologist. Among the different types of skin cancer, despite its low incidence, melanoma is the most dangerous because it can spread to other places in the body, even if it was small [3]. There is a big chance of curing in the initial stages of the disease, so it is necessary to have a prior diagnosis. However, visual factors of humans, for example, eye fatigue, can hamper diagnosis, which regularly leads to wrong lesion detection [4]. In the other side, there are types of the lesion may be less deadly than melanocytic lesions like SCC and BCC; however, these lesions can spread through and behind the skin to other body places and parts that can cause disfigure, as well as threaten life [5].

Therefore, dermatologists have to use specific indications or signs for classifying and distinguishing melanoma aside from benign lesions—Digital Image Processing (DIP) techniques used to classify different types of skin cancer [6]. The DIP methods will increase the agility and reliability of lesion diagnosis than those dermatologists that have high skills. In the DIP methods, the contained information in the images was extracted and used in analysis and interpretation [7].

Dermoscopy techniques used to improve the performance of melanoma diagnosis [8]. According to its non-invasive imaging, dermoscopy can determine the skin lesions by magnifying and increase the illumination of the suspected spots [9]. It is a tedious and challenging task because of its variations in color, shape, location, texture, size, and intra-class. Besides, it has a high degree of similarity between malignant and non-malignant lesions, as well as other environmental conditions.

Many attempts made to overcome these challenges. Researchers used low-level handcrafted features to differentiate non-melanoma from melanoma lesions [10]. Other researchers tried to remove unwanted elements and backgrounds and only to keep the essential elements using various methods of segmentation. This approach led to unsatisfactory results because dermoscopy images suffer from high visual similarity, vast variations of intra-class, and artifacts [11].

According to the ability of the ABCD method to extract distinctive morphological features, this method used to detect benign lesions from malignant melanoma [10, 12]. Therefore, it has widely used for automated systems. The performance of this method is ranging from 85 to 91%. These low percentages motivate the researchers to find an alternative way either by modifying this method or by finding a new way to enhance the performance. A simple method to extract features is the gray-level difference method (GLDM) [13]. It works by indicating the differences that happened in the gray level by calculating angular second moment, contrast, entropy, and mean within the image in different directions (such as vertical, horizontal, and two diagonals) [14].

Also, a computer-aided system to detect skin cancer developed using machine learning methods [15–18]. A single pre-trained end to end CNN has been used by Esteva et al. [19] to classify skin lesions. They classify three different classes of lesions called melanoma, seborrheic keratosis, and benign or nevus. They made a binary classification in two stages. First, keratinocyte carcinomas versus benign seborrheic keratosis, and the second is between malignant melanomas versus benign nevi. They have been fine-tuned a pertained architecture from google called Inception v3 using their skin lesion dataset. The average accuracy of their proposed methods was about 71.2%.

A deep convolutional neural network (CNN) proposed by Yu et al. [20] for melanoma recognition. They used residual learning to deal with overfitting and degradation problems. They also built a fully convolutional residual network (F00CRN) for classification. The experimental results showed that they gained an average accuracy of 85.5%. In [21], they tried to classify only three lesion types by combining sparse coding, deep learning, and support vector machine (SVM) techniques, and the classification rate of their work was 93.1%. While in [22], they firstly enhanced the images by median filter and contrast limited adaptive histogram equalization technique (CLAHE), then images are segmented by normalized Otsu’s to extract the affected skin lesion from the background. To classify the segmented image, they combined the deep learning with hybrid Adaboost-(SVM). The classification rate accuracy of their work was 92.89%.

Wahba et al. [23] proposed a new discrimination technique for texture feature extraction. In this method, Wahba and his co-authors used the cumulative level-difference mean (CLDM) based on GLDM. The ABCD rule feature vector refers to symmetry, border irregularity, color variation, and diameter that have been used to classify the lesions into benign or melanoma. In the modified set of ABCD, each border feature considered as a separate feature, like the fractal dimension, edge abruptness, and compact index. These features are ranked using the eigenvector centrality (ECFS) method. Then a cubic support vector machine is used to classify these ranked features. The accuracy of this work was approximately 100%, which may appear as good results. It is biased, deceptive, and not accurate. They select only 300 images and discard the rest of the dataset. Also, they consider only four classes of skin lesion, melanoma, BCC, nevus, and BKL.

Titus et al. [24] proposed an automated skin lesion detection. They started by a preprocessing step using color normalization. They applied transfer learning to VGG16 and GoogLeNet architectures to classify skin lesions. The classification rate of their proposed model was 79.7% for VGG and 81.5% for GoogLeNet. Gessert et al. [25] attempted to classify the skin lesion by using different architectures like ResNet, Densenet, and SENet. They used different approaches such as balanced batch sampling and loss weighting to overcome the problem of imbalance numbers of images for each class. Finally, the ensemble different convolutional neural network architectures and fine-tuned them using their dataset. The classification rate of this work was 85.1%. In [26], they try to classify lesions using an SVM classifier with 200 handcrafted features after segmentation. The accuracy of this work was 93.01.

AlexNet proposed by Krizhevsky et al. [27], where this net possesses many essential characteristics, such as the following:

1. There are many more filters with every layer, which able to enhance features and reduce noise.

2. Each stacked convolution layer is followed by a pooling layer which able to reduce the number of features and extract essential features only.

3. The likelihood of gradient vanishing reduced because of using RELU as activation function instead of tanh, logistic, arctan, or Sigmoid, which is more biological inspired.

4. From the previous three points, the AlexNet is five times faster than other deep architectures.

5. Some deep architecture required specific hardware; AlexNet can work well with hardware and GPU limitations.

Even AlexNet was relatively old architecture; it successfully utilized in skin lesion classification. Hosny et al. [28–30] proposed modified models of AlexNet. In [28], they applied it with the public ph2 dataset [29]. In [30], the modified model was trained and tested using four datasets, Dermatology information system [31], DermQuest [32], MED-NODE [33], and ISIC 2017 [34].

In 2018, the ISIC 2018 challenge [35] released. In this challenge, the skin lesions divided into seven classes, melanoma (MEL), melanocytic nevus (N.V.), basal cell carcinoma (BCC), actinic keratosis (AKIEC), benign keratosis (BKL), dermatofibroma (D.F.), and vascular lesion (VASC).

Successful classification models must be able to classify the seven kinds of skin lesions with high accuracy. Besides, the ISIC 2018 challenge has two problems. First, the number of images in some classes is limited. Second, the imbalanced number of images in different classes makes the classifier biased toward the class with the dominant number.

To the best knowledge of the authors, there is no published work in which the seven skin lesions accurately classified. This challenge motivated the authors to modify the successful AlexNet-based model.

In this paper, a pre-trained deep convolutional neural network model proposed. This model utilized the transfer learning and AlexNet. The last three layers of the AlexNet, connected layer, softmax layer, and classification layer, have been dropped out and replaced with new layers to be appropriate for classifying seven classes of skin lesions. The weights of these layers have been randomly initialized and then updated during the training. The performance measures of the proposed method outperformed the existing methods.

We could summarize the main contributions in this paper in the following points:

1. We are achieving a very high classification rate, in addition to high sensitivity, specificity, and precision percentages.

2. The proposed model works smoothly with binary and multi-class detection due to using softmax. The probabilities of the softmax output ranging from 0: 1, so the summation of these probabilities must be one in binary classification. In multi-class detection, the possibility of each class will return, but only the one class with high probability will be the target class. For these characteristics, we use the softmax layer to compute the probability coming from the new, fully connected layer.

3. The utilization of different ways of augmentation has applied to overcome the two problems of class sizes.

The rest of this work organized as follows: a brief description of the utilized DCNN AlexNet and transfer learning presented in the “Method” section. The data set and the experiments discussed in the “Experiment results and discussion” section. The “Comparative study” section was for the comparison of the proposed method with state of the art—the conclusion presented in the “Conclusion” section.

Method

Usually, the depth of traditional neural networks consists of three layers. These layers are the input layer, an output layer, and a single hidden layer. These types of learning methods suffered from some problems, such as the gradient values may become zero or close to zero while updating the weights. This problem called gradient vanishing [36, 37], so the deep convolutional architecture introduced to solve problems that have appeared from older learning methods [38, 39].

Alex-Net

Deep architectures contain more than a hidden layer. These hidden layers help to enhance and extract features in a better way. So, the performance of image classification using the deep networks proved to gain a high rate of classification compared with other methods, which excites everyone to use deep networks. AlexNet is a big network consisting of several neurons equal to 650,000 and 60 million parameters. Krizhevsky et al. [27] carried out numerous enhancements to train these parameters. Overall Architecture of the network displayed in Fig. 1.

Fig. 1.

AlexNet architecture

The activation function was the first improvement. For nonlinearity in the classical neural network, the activation function limited to arctan, tanh, logistic function, etc. The gradient values using these activation functions will be significant only when the input is around small range 0, so these types of activations functions will fall into the problem of gradient vanishing. A new activation function called a rectified linear unit (ReLU) had been used to overcome this problem. If the input is not less than 0, the gradient of RELU is always 1. Also, it accelerates the training process. RELU defined through the next equation:

$$y=max\left(0,x\right)$$ 1

This network consists of several small sub-networks. Each sub-network can fall into overfitting, but they share the same loss function so that it may be a useful way to drop out some of these layers. The second improvement was to avoid overfitting by dropping out some of these layers. This improvement can be applied by dropping out the fully connected layers. During the dropout, a part of the neurons trained for each iteration. The joint adaptation will be reduced between neurons because neurons forced to cooperate by dropping out, which also enhances and improve the generalization. The average of the sub-networks is the output of the entire network. Now it can be clear that dropout also improves and increases the robustness.

Features are extracted automatically by the convolutional layers and reduced by the pooling layer. For an image I with height h and width w, where m is the convolutional kernel with height b and width c, the convolution can be defined as the following:

$$C\left(h,w\right)=\left(I\ast m\right)\left(h,w\right)=\sum _b\sum _{c}I\left(h-b,w-c\right)m\left(b,c\right)$$ 2

The model will able to learn from image features by convolution, and these parameters are shared to reduce the complexity of the model. The pooling layers used to reduce the extracted features. The pooling layers take a group of neighboring pixels from the feature map and generating values for representation. Max pooling is used in AlexNet to reduce the feature map. The max pooling takes a 4 × 4 block from the feature map to generate a 2 × 2 block contains the maximum values.

Feature generalization improved by the cross-channel normalization, which belongs to a local normalization method. Before feeding the feature maps to the next layers, these maps must normalize first. A sum from several adjacent maps with the same positions generated by cross-channel normalization. In real neurons, this mechanism also found. The classification was done in the fully connected layers. Softmax was used in the fully connected layers as an activation function, which can be computed by the next equation:

$$\text{softmax}{\left(x\right)}_i=\frac{exp\left(x_i\right)}{{\sum }_{j=1}^{n}exp\left(x_j\right)}\mathit{\text{for}}i=0,1,2\dots \dots ,k$$ 3

The output of softmax is constrained to be in range 0 to 1, which is the main advantage to ensure neurons activation; this is the reason behind using this activation function. Different techniques were used in [27] to train AlexNet, for example, multiple GPUs training and overlapping pooling. The classification rate of AlexNet has been tested against other methods using the same dataset. The classification rate of AlexNet exceeded these methods by 10% [40], and that is the main contribution. Here, we used this structure for skin lesion classification.

Transfer Learning

Although the outstanding classification ability of AlexNet, as discussed before, it requires a great time for training. Furthermore, the limitation of available skin lesions datasets is not sufficient to train a deep network from scratch because of having a few numbers of images. The best solution to overcome this problem is by applying transfer learning. The transferred AlexNet architecture is shown in Fig. 2.

Fig. 2.

Transfer learning to AlexNet

The last three layers replaced in AlexNet with the following:

1. A fully connected layer with seven nodes which used to classify seven different skin lesions

2. Softmax layer

3. The classification layer

The rest of the network parameters are the same as in the original model without any changes. In other words, the architecture of AlexNet is divided to a pre-trained network and transferred network. The pre-trained network parameters used as it has computed with the original model because it proved its ability to effectively extract features for classification during the training using millions of images in ImageNet [41]. The transferred network parameters expertly trained because it is a tiny part where be suitable to be prepared using small datasets. From the state-of-art, there are many researchers try to replace more than three layers, but they found that the performance measure of classification became less than replacing the last three layers only [40].

A practical and convenient way to train a deep neural network with a small number of labeled images is to apply transfer learning to a pre-trained deep architecture. Using all the pre-trained network parameters as initiation can take advantage of the features that have learned from large images. Features extracted using these layers, and the parameters that acquired from these layers can assist the training converging. Personal computers used to implement the transfer learning instead of using a high GPU and CPU performance to train deep networks from scratch. The stochastic gradient descent momentum (SGDM) used to train the transferred AlexNet. The max training epoch’s number set to 32, while the mini-batch size was 10, the initial learning rate was 0.0001, and the momentum was 0.9.

Experiment Results and Discussion

An IBM computer with a Core i7 processor with 16 GB DDRAM and a graphic card NVIDIA GeForce MX150 used to perform experiments and evaluations. MATLAB 2018 × 64-bit used to execute the codded program. This study tested and evaluated using a well-known dataset from the International Skin Imaging Collaboration (ISIC) 2018 challenge dataset [35]. The AlexNet and transfer learning trained and tested using the ISIC 2018 dataset, which consists of 10,015 images. ISIC dataset divided into seven classes. All of these classes have a different number of images MEL is 1113, NV is 6705, BCC is 514, AKIEC is 327, BKL is 1099, DF is 115, and VASC is 142. This dataset is one of the hardest challenges to classify different images into seven classes—the implemented work coded using CUDA run over the GPU. Using GPU allows using a massive number of training data with a low error rate of models. In the proposed model, the last three layers (fully connected 8, softmax layer, and the classification layer) dropped out and replaced with the new three layers. The pre-trained AlexNet previous three layers created to classify 1000 classes, but in this proposed work, there are only seven classes needed to be classified.

There are four performance measures have been computed to test the reliability of the proposed model. These measures named accuracy, sensitivity (TPR), specificity (TNR), and precision (PPV) [42]. These measures can be computed based on the following equations:

$$\text{Accuracy}=\frac{t_p+t_n}{t_p+f_p+f_n+t_n}$$ 4

$$\text{Sensitivity}\left(\mathrm{TPR}\right)=\frac{t_p}{t_p+f_n}$$ 5

$$\text{Specificity}\left(\mathrm{TNR}\right)=\frac{t_n}{f_p+t_n}$$ 6

$$\text{Precision}\left(\mathrm{PPV}\right)=\frac{t_p}{t_p+f_p}$$ 7

Where t_p, f_p,   f_n,   and t_n refer to a true positive, false positive, false negative, and true negative, respectively. The acronyms, TPR, TNR, and PPV, refer to true positive rate, true negative rate, and positive prediction rate. The rates of true negative and false positive should be large and small.

The proposed model tested by three different experiments. The first experiment carried out with the original ISIC 2018 images, which contains 100,015 images without any augmentation. The proposed model trained and tested using the original ISIC dataset. In this experiment, the 10-fold cross-validation used to divide ISIC 2018 dataset into groups for training and testing without any augmentation. Each group used at least once as training and once as testing but not in the same run. Then the modified AlexNet after applying transfer learning theory has been used. This process repeated ten times, and the average accuracy for the ten-run times computed to be the overall accuracy of the proposed model. The reason behind using this way with the first experiment is some classes contain a few numbers of images, which is not enough to train and test the proposed model. The accuracy, sensitivity, specificity, and precision for this experiment were 94.91%, 82.17%, 97.03%, and 82.17%, respectively. The confusion matrix of this experiment is shown in Fig. 3a.

Fig. 3.

Confusion matrix, a first, b second, and c third experiments

According to the deep learning networks, a vast number of images required to retrain a pertained network, so we have applied the augmentation in two ways. The first way of augmentation was in the second experiment. In the second experiment, we implemented the data augmentation where the images rotated using random rotation angles from 0°: 355° and flipping for all classes except “N.V.” to overcome the problem of images imbalance for each class. The other six classes augmented to have the same number of images equal to 6700 (± 5) to make the classifier not biased for the class that contains the largest number of images. The reason for not augment the N.V. class is that this class has enough images in 6705, so we do not need to expand this class. In the second experiment, as we discuss, six classes augmented to have the same number of images. The reason behind this way of augmentation is to make all classes contain approximately the same number of images to make balanced weights which make the measures more confidence and fair. In this experiment, the expanded datasets divided into 80% for training and 20% for testing. Then each set has been augmented separately to avoid sharing the same features. The accuracy, sensitivity, specificity, and precision were 98.2%, 93.84%, 98.96%, and 93.81% for the second experiment, respectively. The confusion matrix of this experiment is shown in Fig. 3b.

The second way of augmentation was in the third experiment. In the third experiment, images for all classes except “N.V.” augmented by random rotation from 0°: 355° and flipping. The number of images for augmented classes becomes 23,544, 37,008, 79,128, 8280, 79,992, and 10,224 for AKIEC, BCC, BKL, DF, MEL, and VASC, respectively. In this experiment, the dataset divided into 80% training and 20% testing, where each set augmented separately to avoid sharing the same features. The accuracy, sensitivity, specificity, and precision were 98.8%, 95.6%, 99.3%, and 95.6% for the third experiment, respectively. The confusion matrix of this experiment is shown in Fig. 3c.

Table 1 gives an overview of the obtained results for the performed experiments. From Table 1, the sensitivity and precision have low rates 82.17%, and 82.17% respectively in the first experiment because of the imbalance numbers of the images in each class in addition to that some classes contain not enough number of images like VASC, DF, AKIEC, and BCC. Figure 4 indicates the difference in image number. In the second experiment, the sensitivity becomes 93.84% while it was 82.17%, the precision becomes 93.81 while it was 82.17%, the specificity becomes 98.96% while it was 97.03%, and the accuracy becomes to 98.2% while it was 94.9%. These results show that data augmentation brings balance in all classes and improves the classification rate. In the third experiment, sensitivity and precision increased when the data augmentation applied differently. It is clear that the augmentation processes significantly improve the classification rates. The proposed method achieved a very high classification rate. Figure 5 visualizes the difference between classification rates for proposed model experiments.

Table 1.

Proposed model accuracy

Experiments	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)
First	94.90	82.17	97.03	82.17
Second (balanced)	98.20	93.84	98.96	93.81
Third	98.70	95.60	99.27	95.60

Fig. 4.

Visualization the difference of images number for each class

Fig. 5.

Visualization of the performance metrics for the proposed model experiments

Comparative Study

To prove the credibility of the proposed model here, AlexNet and transfer learning, we compare the acquired performance measure here with state-of-art. We choose a recent work that works on the same dataset, ISIC 2018. The state-of-art uses different architectures and methods to classify the same dataset ISIC that consists of seven classes. Table 2 lists the performance measures of the proposed model and recently published papers.

Table 2.

Proposed model accuracy

	Brinker et al. [24]	Gessert et al. [25]	Hardie et al. [26]	Proposed method
Preprocessing	Yes	Yes	Yes	No
No. of classes	7	7	7	7
Classification method	Ensemble (VGG and GoogLeNet)	Ensemble different deep architecture	SVM	AlexNet + transfer learning
Accuracy (%)	81.5	85.1	93.01	98.70
Sensitivity (%)	-	-	73.03	95.60
Specificity (%)	-	-	95.49	99.27
Precision (%)	-		61.97	95.60

From Table 2, it is evident that methods [24, 25] use the deep network but with different architectures while [26] use the SVM as classifier. In [24], they used VGG, GoogleNet separately, and ensembled the two architectures, but the accuracy found to be 81.5%, which may be very low to medical analysis. In [25], they try to use different architectures like PolyNet, SENet, and ResNeX; finally, they ensemble these different architectures. The accuracy of this was 85.1%, which is very low for medical purposes. In [26], they use the SVM classifier after a segmentation step for all images is carried out. The accuracy of this work was 93.01%. Here, the performance of the proposed model is 98.7%, which exceeds other methods. The obtained results are visualized and displayed in Fig. 6.

Fig. 6.

Classification rate comparative study visualization

Conclusion

A novel method for skin lesion classification developed here is based on deep neural network AlexNet and transfer learning. The proposed method was trained and tested using public dataset ISIC 2018 to compare with state of the art. The proposed method can classify seven different kinds of lesions accurately. The performance of the proposed classification model outperformed the existing classification methods. Explicitly, the accuracy, sensitivity, specificity, and precision for the proposed method were 98.7%, 95.6%, 99.27%, and 95.6%. The performance of the proposed method exceeds the performance of the state-of-art by at least 6%.