Automated cutaneous squamous cell carcinoma grading using deep learning with transfer learning

Abstract

Introduction: Histological grading of cutaneous squamous cell carcinoma (cSCC) is crucial for prognosis and treatment decisions, but manual grading is subjective and time-consuming. Aim: This study aimed to develop and validate a deep learning (DL)-based model for automated cSCC grading, potentially improving diagnostic accuracy (ACC) and efficiency. Materials and Methods: Three deep neural networks (DNNs) with different architectures (AlexNet, GoogLeNet, ResNet-18) were trained using transfer learning on a dataset of 300 histopathological images of cSCC. The models were evaluated on their ACC, sensitivity (SN), specificity (SP), and area under the curve (AUC). Clinical validation was performed on 60 images, comparing the DNNs’ predictions with those of a panel of pathologists. Results: The models achieved high performance metrics (ACC>85%, SN>85%, SP>92%, AUC>97%) demonstrating their potential for objective and efficient cSCC grading. The high agreement between the DNNs and pathologists, as well as among different network architectures, further supports the reliability and ACC of the DL models. The top-performing models are publicly available, facilitating further research and potential clinical implementation. Conclusions: This study highlights the promising role of DL in enhancing cSCC diagnosis, ultimately improving patient care.

Introduction

Cutaneous squamous cell carcinoma (cSCC), a prevalent skin cancer originating from the outer skin layer, arises from abnormal cell growth (dysplasia) that progresses to invasive cancer [1, 2]. It commonly affects sun-exposed areas, with a predilection for the head and neck [2, 3]. Incidence rates are increasing globally, particularly in Western countries, with higher rates observed in older individuals and men [4, 5, 6, 7, 8].

While generally less aggressive than melanoma, cSCC can be lethal, especially if diagnosed late or with certain high-risk subtypes [9, 10]. The global rise in cSCC cases underscores the need for comprehensive public health strategies, including prevention, early detection, and effective treatment [5, 8].

Histological grading, based on the degree of cellular differentiation, is crucial for prognosis and treatment decisions [2, 11]. It guides interventions, ranging from surgical excision for low-grade tumors to Mohs surgery, radiation therapy, or systemic treatments like Cemiplimab for high-grade or advanced cases [12, 13]. Furthermore, histological grading aids in identifying high-risk patients who require closer monitoring and facilitates research into the molecular underpinnings of cSCC progression [13, 14, 15].

Artificial intelligence (AI), as part of machine learning (ML) and deep learning (DL) in particular, is transforming histopathology by enhancing diagnostic accuracy (ACC), prognostication, and disease management [16, 17, 18]. ML algorithms, trained on digitized tissue slides, can identify patterns indicative of specific pathologies, augmenting pathologists’ capabilities and potentially leading to faster, more precise diagnoses [19, 20, 21].

DL, especially convolutional neural networks (CNNs), has emerged as a powerful tool for histopathology image analysis, facilitating tasks like tissue classification, tumor detection, and biomarker identification [22, 23, 24]. Moreover, AI models can predict disease progression and treatment responses, enabling personalized medicine approaches [18, 25].

Transfer learning (TL), a technique where a pre-trained model is adapted to a new task, is particularly valuable in histopathology due to the limited availability of annotated data [26]. By leveraging knowledge from models trained on large, general datasets like ImageNet, TL significantly reduces the need for extensive training data and accelerates model development while often enhancing performance on specific histopathology tasks [22, 27].

As shown above, histological grading of cSCC serves as a critical prognostic indicator, significantly influencing therapeutic decision-making and risk stratification, which are essential for optimal clinical management [10, 11, 12, 13, 28]. However, the current reliance on manual grading introduces inherent subjectivity, leading to possible interobserver variability due to differences in pathologists’ experience, training, and potential fatigue. Furthermore, manual grading is time-consuming, particularly with complex specimens, and suffers from limited reproducibility due to the potential for subjective interpretation of grading criteria. Additionally, the process necessitates trained pathologists, a resource that may be scarce in certain settings.

Aim

Given the inherent limitations of manual grading in cSCC, the aim of this study was to develop and validate a DL-based model for automated cSCC grading that can objectively and consistently classify cSCC in three grading classes (G1, G2, and G3), potentially reducing interobserver variability, improving diagnostic efficiency and time, and ultimately enhancing patient care.

Materials and Methods

Materials

Forty-seven consecutive cases of cSCC, obtained with approval from the Research Ethics Committees of the Emergency Clinical County Hospital, Cluj-Napoca, Romania (Approval No. 38789/13.09.2021) and the Iuliu Haţieganu University of Medicine and Pharmacy, Cluj-Napoca (AVZ2/8.11.2021), were included in this study. Following standard histological processing and Hematoxylin–Eosin (HE) staining, the excised tissue sections were scanned using a Pannoramic SCAN II slide scanner (3DHISTECH, Budapest, Hungary) with a ×40 objective. From each whole-slide image (WSI), 1000×1000 pixel crops in a 24-bit red, green, and blue (RGB) color space were extracted by a trained pathologist. A panel of three pathologists independently reviewed these crops, and only those with unanimous diagnostic agreement were retained for subsequent analysis.

For each of the three grades (G1, G2, and G3), 100 images were selected. Samples of the selected images for each grade are presented in Figure 1.

Figure 1.

Dataset sample. The first row resembles images labeled as G1, the second row as G2, and the third row as G3. Hematoxylin–Eosin (HE) staining, ×200

Methods

We trained three distinct classification deep neural networks (DNNs) with diverse architectures, originally designed for classifying real-world images into 1000 categories from the ImageNet dataset [29]. Utilizing the concept of TL [30], we adapted these networks to classify SCC images into three grading scores: G1, G2, and G3. The networks underwent modification, re-training, and thorough statistical evaluation. The top-performing network from each architecture were tested on 60 new images, with results compared to assessments by a panel of three pathologists. All computational tasks were conducted using MATLAB® (MathWorks, USA).

The three DNNs were selected based on previous research [31, 32] and their balance between ACC and prediction time [33]. The first network, AlexNet [34], is a CNN comprising five convolutional layers interspersed with max-pooling layers, followed by two fully connected layers, and concluding with a 1000-class Softmax layer. This model, introduced in 2012, can be freely downloaded [35] and is available in MATLAB. The second model, GoogLeNet [36], features a 22-layer architecture with the innovative “inception” modules, significantly different from AlexNet. Released in 2014, this model is also freely available [37] and accessible in MATLAB. The third model, ResNet-18 [38], incorporates a non-linear, 18-layer architecture known for its residual learning approach, introduced in 2016. This model is likewise available in MATLAB.

For each model, the final classification layers were replaced to adapt to the new task of classifying SCC images into three grading classes. Specifically, the last fully connected layer and the output layer were substituted to align with our dataset’s labels.

The modified networks were re-trained on the new dataset, with 10% of images reserved for validation and 90% used for training, employing a 10-fold cross-validation method. Training hyperparameters were standardized across all architectures, with a mini-batch size of 100, an initial learning rate of 0.0001, and a validation patience of four epochs. Stochastic gradient descent with momentum was utilized as the optimizer. Performance metrics included mean ACC, sensitivity (SN), specificity (SP), and area under the curve (AUC).

Clinical evaluation

A set of 60 images from six consecutive SCC cases were randomly selected from new WSIs, following the same procedure as the original dataset creation. These images were classified using the application and independently evaluated by three pathologists. Agreement between the networks and the pathologists was assessed.

Statistical assessment

Due to the stochastic nature of DNN algorithms, which are influenced by the specific training and validation datasets, we ensured robust results by running each model 100 times using 10-fold cross-validation. For each run, training and validation data were randomly selected but kept consistent across all three networks to enable direct performance comparisons. The normality of the data distribution was assessed using the Kolmogorov–Smirnov, Lilliefors, and Shapiro–Wilk tests.

The agreement between the networks and the panel of pathologists was evaluated using Cohen’s kappa (κ) coefficient [39]. The coefficient ranges between 0 and 1, with values close to 0 indicating no agreement and values close to 1 indicating perfect agreement. For interpretability and consistency, Landis & Koch [40] guidelines were used.

Results

A total of 300 trained networks resulted, 100 having AlexNet architecture, 100 having GoogLeNet architecture, and the last 100 having ResNet-18 architecture. Mean and standard deviation (SD) for ACC, SN, SP, and AUC of the networks, on the hole dataset are presented in Table 1.

Table 1.

Performance assessment, mean ± SD

	AlexNet	GoogLeNet	ResNet-18	ANOVA
ACC [%]	85.5±5.5	86.2±6.07	85.5±5.6	p >0.05
SN	0.854±0.055	0.862±0.060	0.855±0.056	p >0.05
SP	0.927±0.027	0.931±0.030	0.927±0.028	p >0.05
AUC	0.983±0.016*	0.978±0.021	0.973±0.023*	p <0.05

ACC: Accuracy; ANOVA: Analysis of variance; AUC: Area under the curve; SD: Standard deviation; SN: Sensitivity; SP: Specificity. *Student’s t-test, p<0.05

All normality tests (Kolmogorov–Smirnov, Lilliefors, and Shapiro–Wilk) indicated that the performance assessment variables did not follow a normal distribution. Nevertheless, in accordance with the central limit theorem [41], and given a sample size of 100, parametric tests were considered for comparing means.

One-way analysis of variance (ANOVA) showed that there are no significant differences between the performance parameters of the three networks, except for the AUC. Student’s t-test confirms there is a significant difference between the AUC of AlexNet and ResNet-18.

Confusion matrices on all dataset images from the best performing networks of each architecture are presented in Tables 2, 3, 4.

Table 2.

AlexNet confusion matrix

Predicted class	G1	G2	G3
G1	99	1	0
G2	2	97	1
G3	0	1	100
	Target class

Table 3.

GoogLeNet confusion matrix

Predicted class	G1	G2	G3
G1	99	1	0
G2	4	95	1
G3	1	1	98
	Target class

Table 4.

ResNet-18 confusion matrix

Predicted class	G1	G2	G3
G1	98	2	0
G2	0	99	1
G3	9	1	99
	Target class

The misclassification patterns are evident from the confusion matrices presented in Tables 2, 3, 4. These tables show that G1 was frequently misclassified as G2, and G3 was also often misclassified as G2. Conversely, G2 was misclassified as either G1 or G3. Misclassifications between G1 and G3 were very rare.

A sample of misclassified images is shown in Figure 2A, 2B, 2C, 2D, 2E. As anticipated, most G1 images were misclassified as G2. Similarly, G2 images were often incorrectly classified as G1 or G3, whereas G3 images were only misclassified as G2.

Figure 2.

Misclassified images: (A) A G1 pattern misclassified as G2; (B and C) A G2 pattern misclassified as G1; (D) A G2 pattern misclassified as G3; (E) A G3 pattern misclassified as G2. HE staining, ×200

The best performing networks from each architecture were packed as MATLAB files and are offered on a public repository (https://github.com/Mircea-Sebastian-Serbanescu/Automated-Cutaneous-Squamous-Cell-Carcinoma-Grading-Using-Deep-Learning-with-Transfer-Learning.git).

All Cohen’s kappa were above 0.8 (Table 5), showing almost perfect agreement and concluding that the observed agreement is not accidental. The average agreement between the pathologists (0.8889) exceeded the average agreement between the networks (0.8722). However, both are very high. Panel pathologist P3 had the best agreement with ResNet-18 close to 1 (0.9667).

Table 5.

Clinical evaluation assessment: Cohen’s kappa (κ) coefficient. P1–P3 represent the three panel pathologists. Sixty images from six different cases

	AlexNet	GoogLeNet	ResNet-18	P1	P2	P3
AlexNet		0.8833	0.9000	0.9000	0.8833	0.9000
GoogLeNet			0.8333	0.8667	0.9167	0.8667
ResNet-18				0.9333	0.8667	0.9667
P1					0.8667	0.9333
P2						0.8667

Discussions

AI has emerged as a valuable tool across various fields of medicine [42]. In radiology, AI algorithms can analyze medical images to detect and diagnose conditions like cancer and fractures with remarkable ACC [43, 44]. AI-powered tools in pathology can help identify abnormalities in tissue samples, aiding in disease diagnosis and prognosis [24]. In drug discovery, AI models have accelerated the identification of potential therapeutic targets and the development of new medications [45]. Furthermore, AI has shown promise in predicting patient outcomes and personalizing treatment plans based on individual characteristics and disease profiles [46].

The application of DL in pathology was delayed compared to other medical fields due to the necessity of digital slide scanners. These scanners were crucial for converting physical slides into digital data, which is essential for training DL models [26]. The availability of large datasets, coupled with advancements in computational power and algorithm development, finally allowed DL to emerge as a powerful tool in pathology, enabling significant progress in areas such as cancer diagnosis and prognosis [47].

Our research has demonstrated robust results in the grade scoring of cSCC through the application of TL techniques and general-purpose DNNs. The models achieved mean ACCs exceeding 0.85, SN of 0.85, SP of 0.92, and an AUC greater than 0.97. A statistical evaluation of the three networks employed in this study revealed comparable performance outcomes, except for a slight disparity in AUC between AlexNet and ResNet-18.

Furthermore, clinical evaluations showed a high degree of agreement between the network predictions and the assessments made by pathologists. There was also strong concordance among the pathologists themselves, as well as between different network architectures. These findings are consistent with the performance metrics obtained during the classification performance assessment phase, underscoring the possible reliability and ACC of the DL models in clinical applications and indicating that the grading features of cSCC are well-separated.

Analyzing the misclassified images shown in Figure 2A, 2B, 2C, 2D, 2E, we can intuitively understand the reasons behind the network’s decisions. The G1 pattern was misclassified as G2, likely due to the dense inflammatory infiltrate and potentially processing artifacts, such as the detachment of keratin pearls during sectioning. The G2 pattern was mistaken for G1, probably because no staining normalization or assessment was performed, leading to differences in staining. In hypereosinophilic slides, the network may misinterpret clusters of larger cells with pyknotic nuclei arranged in a whorl-like pattern as keratin pearls. Inflammation might also explain why the G2 pattern was misclassified as G3. Additionally, the small size of the keratin pearls could also contribute to this misinterpretation. Finally, the G3 pattern was misclassified as G2, likely because some necrosis resembled keratin formations. Furthermore, the presence of intervening stroma within the neoplastic cell sheets, particularly those containing blood vessels with brightly eosinophilic erythrocytes or sites of hemorrhage, may have contributed to the network’s misidentification of these areas as keratin pearls.

The DNNs have demonstrated strong classification capabilities across various medical imaging fields and pathologies. Our previous research has highlighted their effectiveness, including automatic segmentation of liver lesions in ultrasound (US) investigations [48] and the histological class prediction of breast tumors in mammography [31]. Achieving reliable results in the histopathological domain further underscores the potential utility of other imaging techniques, such as high-frequency US, which is an essential non-invasive tool for the pre-therapeutic assessment of basal cell carcinoma (BCC) [49].

These three network architectures – AlexNet, GoogLeNet, and ResNet-18 – have been previously utilized in the classification of BCC patterns [26]. While earlier studies reported good performance, the current approach has yielded superior results. Several factors may contribute to this improvement. Notably, the use of a balanced dataset likely enhanced the model’s performance. Additionally, the current task involved classifying images into only three categories, which may be inherently simpler. The grading task itself may also be more “clearly separable” with better inter-human agreement reinforcing this hypothesis. Comparable outcomes have been observed in automated Gleason grading of prostate cancer [32, 50], where TL from general-purpose DNNs achieved notable results. However, the performance in these studies was slightly lower compared to the current research.

Furthermore, while other malignancies often require costly investigations for reliable outcome prediction, such as immunohistochemistry predictive markers for primary colorectal cancer tumors [51], this is not the case for cSCC [2], making the current approach both efficient and safe.

The predominant application of DL in the context of cSCC research, as evidenced by existing literature, centers on the utilization of clinical and dermatoscopic imaging modalities. Studies conducted by Han et al. [52], Jain et al. [53], Bechelli & Delhommelle [54], Fraiwan & Faouri [55], Bassel et al. [56], Zia Ur Rehman et al. [57], and Hussain et al. [58] have employed different CNNs architectures and ML techniques to achieve automated diagnosis. The studies researched either the differentiation of benign from malignant skin lesions, including images of cSCC, or the classification of specific subtypes of skin malignancies, based on clinical or dermatoscopic image analysis. In addition to lesion classification, researchers such as Zhao et al. [59] have explored the use of DL models to assess the risk stratification of skin tumors, categorizing them into low-risk, high-risk, and dangerous categories.

While research utilizing DL on digital histopathology images of cSCC remains limited, Thomas et al. [60] developed an interpretable DL model for multi-class segmentation and classification of non-melanoma skin cancers from histopathology images. They used 290 slides of cSCC, and intraepidermal carcinoma, captured at 10× magnification with a resolution of 0.67 μm/pixel. The model, based on a U-Net architecture with a ResNet-50 encoder, achieved high ACC in both pixel-level segmentation (85%) and whole-image classification (93.6%). The study demonstrated the feasibility of using DL for automated skin cancer diagnosis and the potential for such models to assist pathologists in tasks like surgical margin assessment.

Wako et al. [61] developed a DL system to classify SCC margins in histopathology images. Using 3972 images for training, 492 for validation, and 84 for testing, they employed pre-trained models VGG16, ResNet-50, MobileNetV2, and EfficientNetB0, achieving 95.2% ACC [61]. This demonstrates the potential of DL to improve the ACC and efficiency of SCC margin assessment [61].

Davis et al. employed DL on 95 WSIs of frozen sections obtained from Mohs micrographic surgery for cSCC [62]. Utilizing 256×256-pixel image patches and a ResNet-101 model for tumor classification at the patch level, the algorithm achieved an AUC of 98.1%. Notably, performance was superior for poorly and moderately differentiated tumors (AUC 96.8%) compared to well-differentiated tumors (AUC 89.5%).

Ruini et al. developed a ML model using MobileNetV2 to automatically detect SCC in ex vivo confocal laser scanning microscopy (CLSM) images [63]. Trained and validated on 543 grayscale CLSM images, the model achieved a high AUC of 0.95, demonstrating the potential for faster and more accurate SCC diagnosis [633]. While promising, further validation with external datasets is needed to confirm its generalizability.

In a retrospective analysis involving 104 cSCC patients, Pérez-Baena et al. [64] developed a novel ML-derived cellular morphometric risk score (CMRS) and compared it to the Brigham and Women’s Hospital (BWH) staging system. The CMRS demonstrated independent prognostic value, but when combined with the BWH staging system, it significantly enhanced the ACC of risk stratification, especially regarding local recurrence and nodal metastasis. This approach objectively underscores the added value of ML in clinical prognostication.

Study limitations

A significant limitation of this study is the size of the dataset. Large datasets are essential for optimizing the performance of image analysis systems. Our dataset should be considered small for this specific task. Despite this, we have mitigated the issue of class imbalance, a common challenge in medical classification tasks, through the study’s design. This approach allowed us to achieve better than expected results.

Another limitation is that the data originates from a single medical center and was scanned using a single WSI scanner. More than that, the staining was not normalized [65].

Stroma and deep learning: a future direction

The tumor microenvironment (TME) plays a crucial role in cancer progression, and the stroma, a major component of the TME, has been implicated in cSCC development and metastasis [15]. The stroma consists of various cell types, including fibroblasts, immune cells, and endothelial cells, as well as extracellular matrix (ECM) components. Interactions between tumor cells and stromal cells can promote tumor growth, angiogenesis, and immune evasion [66]. The composition of the stroma, including the presence of specific immune cells and the organization of the ECM, can influence the aggressiveness of cSCC and its response to therapy [64].

DL models can be leveraged to analyze the stromal compartment of cSCC, potentially revealing novel insights into tumor–stroma interactions. For instance, DL algorithms could be trained to quantify stromal features such as fibroblast density, collagen deposition, vessel density and position and immune cell infiltration. These features could then be correlated with clinical outcomes, such as tumor aggressiveness and metastatic potential, to identify prognostic biomarkers and potential therapeutic targets.

Furthermore, DL models could be used to analyze spatial relationships between tumor cells and stromal components. This could help elucidate the mechanisms by which stromal cells promote tumor growth and metastasis, paving the way for the development of novel therapeutic strategies targeting the TME. For example, DL could be used to identify specific patterns of stromal cell infiltration that are associated with poor prognosis or resistance to therapy.

Conclusions

This paper presents an approach for cSCC grading using image classification based on TL in DL, trained on a dataset of only 300 images. DNN derived from AlexNet, GoogLeNet, and ResNet-18 were used, achieving average ACC, SN, SP, and AUC scores of approximately 0.85, 0.85, 0.93, and 0.97, respectively. For this task, all networks performed similarly, indicating that the grading features of cSCC are well-separated. In the clinical assessment, grading agreement measured through Cohen’s kappa showed that the neural networks had similar agreement levels with the pathologists and among themselves. There is also a very high agreement between pathologists on cSCC grading. The top-performing networks from each architecture have been made available in a public repository.

Conflict of interests

The authors declare that they have no conflict of interests.