Skin Cancer Diagnosis by Lesion, Physician, and Examination Type

Key Points

Question

What is the accuracy of skin cancer diagnosis, stratified by lesion type, physician specialty and experience, and examination method?

Findings

This systematic review and meta-analysis including 100 studies found that using dermoscopy compared with clinical examination substantially improved diagnostic accuracy for melanoma (relative odds ratio [ROR], 5.7) and for keratinocytic cancer (ROR, 2.5). Sensitivity and specificity using clinical examination and images of melanoma were 76.9% and 89.1% for experienced dermatologists compared with 78.3% and 66.2% for inexperienced dermatologists and 37.5% and 84.6% for primary care physicians; using in-person dermoscopy and dermoscopic images, they were 85.7% and 81.3% for experienced dermatologists, 78.0% and 69.5% for inexperienced dermatologists, and 49.5% and 91.3% for PCPs.

Meaning

These findings indicate that although accuracy varies by lesion type, physician specialty and experience, and examination method, dermatologists had the highest area under the receiver operating characteristic curve using a dermatoscope, and 13.3-fold increased accuracy over primary care physicians when diagnosing melanoma; that said, the data suggest that the greatest accuracy for dermatologists was achieved by using dermoscopy together with clinical examination.

Abstract

Importance

Skin cancer is the most common cancer in the US; accurate detection can minimize morbidity and mortality.

Objective

To assess the accuracy of skin cancer diagnosis by lesion type, physician specialty and experience, and physical examination method.

Data Sources

PubMed, Embase, and Web of Science.

Study Selection

Cross-sectional and case-control studies, randomized clinical trials, and nonrandomized controlled trials that used dermatologists or primary care physicians (PCPs) to examine keratinocytic and/or melanocytic skin lesions were included.

Data Extraction and Synthesis

Search terms, study objectives, and protocol methods were defined before study initiation. Data extraction was performed by a reviewer, with verification by a second reviewer. A mixed-effects model was used in the data analysis. Data analyses were performed from May 2022 to December 2023.

Main Outcomes and Measures

Meta-analysis of diagnostic accuracy comprised sensitivity and specificity by physician type (primary care physician or dermatologist; experienced or inexperienced) and examination method (in-person clinical examination and/or clinical images vs dermoscopy and/or dermoscopic images).

Results

In all, 100 studies were included in the analysis. With experienced dermatologists using clinical examination and clinical images, the sensitivity and specificity for diagnosing keratinocytic carcinomas were 79.0% and 89.1%, respectively; using dermoscopy and dermoscopic images, sensitivity and specificity were 83.7% and 87.4%, and for PCPs, 81.4% and 80.1%. Experienced dermatologists had 2.5-fold higher odds of accurate diagnosis of keratinocytic carcinomas using in-person dermoscopy and dermoscopic images compared with in-person clinical examination and images. When examining for melanoma using clinical examination and images, sensitivity and specificity were 76.9% and 89.1% for experienced dermatologists, 78.3% and 66.2% for inexperienced dermatologists, and 37.5% and 84.6% for PCPs, respectively; whereas when using dermoscopy and dermoscopic images, sensitivity and specificity were 85.7% and 81.3%, 78.0% and 69.5%, and 49.5% and 91.3%, respectively. Experienced dermatologists had 5.7-fold higher odds of accurate diagnosis of melanoma using dermoscopy compared with clinical examination. Compared with PCPs, experienced dermatologists had 13.3-fold higher odds of accurate diagnosis of melanoma using dermoscopic images.

Conclusions and Relevance

The findings of this systematic review and meta-analysis indicate that there are significant differences in diagnostic accuracy for skin cancer when comparing physician specialty and experience, and examination methods. These summary metrics of clinician diagnostic accuracy could be useful benchmarks for clinical trials, practitioner training, and the performance of emerging technologies.

This systematic review and meta-analysis assesses the accuracy of skin cancer diagnosis by skin lesion type, physician specialty and experience, and physical examination method.

Introduction

Skin cancer is the most frequently diagnosed cancer in the US. Approximately 100 640 new cases of invasive melanoma were diagnosed in 2024¹; the most recent study of nonmelanoma skin cancer reported 5.4 million cases diagnosed in 2012.² The 5-year survival among patients with localized melanoma is greater than 99%, whereas 5-year survival for those with distant or metastatic melanoma falls to 35%.³ Similarly, 5-year survival among patients with localized squamous cell carcinoma (SCC) is 95%, whereas 5-year survival for those with regional SCC decreases to 58%.⁴ Accurate screening for skin cancer coupled with earlier diagnosis can optimize outcomes, minimize the number of invasive diagnostic procedures for benign lesions (eg, skin biopsy), avoid associated morbidities, and reduce health care costs.

Using the naked eye is the current standard of care for skin cancer examination, and histopathologic testing remains the gold standard for skin cancer diagnosis.^5,6 Dermoscopy is a noninvasive in vivo magnification method that identifies features in skin lesions that may not be visible to the naked eye.^7,8,9 Adding dermoscopy to conventional naked-eye clinical examination in melanoma screening has been associated with more than 40% fewer biopsy procedures performed by dermatologists.⁸ Although dermoscopy has been adopted in dermatology training programs in the US, a lack of training has been reported to be a barrier to routine use by primary care physicians (PCPs).¹⁰

Accurate examination for skin cancer depends on the sensitivity and specificity of the physical examination method. Several large studies have shown an unclear benefit of screening the general adult population; however, the examination sensitivity and specificity of the physicians’ specialties were unclear because the accuracy of several practitioner types were reported in aggregate.^11,12,13 New technologies have been developed for skin cancer examination, including artificial intelligence analysis of images, tape sampling to collect skin surface messenger RNA, reflectance confocal microscopy, and impedance spectroscopy, among others.^14,15,16,17 However, there are no standard performance benchmarks with which to compare these technologies. Often, the validation studies include only a small number of dermatologists or PCPs for comparison.^18,19,20,21 The emergence of these novel diagnostic tools raises the question of how to best benchmark them against the current standard of care. In 2015, the US National Academies issued a consensus report, Improving Diagnosis in Health Care,²² which focused on supporting efforts toward implementing diagnostic excellence. Taken together, there is a need to assess the current state of diagnostic performance for skin lesion examination and for summary metrics of the diagnostic accuracy of skin cancer detection in both clinical and research settings.

Many previous studies have assessed the diagnostic accuracy of dermatologists and/or PCPs; however, there is considerable heterogeneity in study methodologies: varying ground truth, differing practitioner types, lack of or differing reported statistics. Previous systematic reviews have evaluated the diagnostic accuracy of either exclusively dermatologists, PCPs, or mixed practitioners (not by specialty) or have not stratified studies by practitioners’ experience level.^{23,24,25,26,27,28} In addition, most reviews have focused on melanoma or other specific classes of skin cancer.^{23,27,28,29,30,31,32,33} We conducted a systematic review and meta-analysis to assess the diagnostic accuracy of skin cancer diagnosis by lesion type (keratinocytic or melanocytic), practitioner type (dermatologist or PCP) and dermatologists’ experience level, and examination modality (magnified or nonmagnified).

Methods

The study protocol was prospectively registered with PROSPERO (CRD42022283603). The review and analysis were performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.³⁴

Data Sources and Search Strategy

PubMed, Embase, and Web of Science were searched for relevant articles, from database inception to December 31, 2021. The search used the following terms: diagnostic accuracy, dermoscopy, naked eye exam, skin cancer, dermatologist, PCPs, sensitivity, and specificity (Figure 1; eTable 1 in Supplement 1).

Figure 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses Flow Diagram.

^aFull search strategy for each database can be found in eTable 1 in Supplement 1.

Study Selection

Inclusion and Exclusion Criteria

Study types included were cross-sectional studies, case-control studies, randomized clinical trials, and nonrandomized controlled trials with examination performed by dermatologists or PCPs; those with midlevel practitioners, nurses, medical students, or mixed practitioners as skin examiners were excluded.

Target conditions were melanocytic and keratinocytic lesions, including benign lesions and lesions concerning for cutaneous melanoma, SCC, or basal cell carcinoma. Studies of rarer forms of skin cancer were excluded.

Studies that assessed diagnostic accuracy using nonmagnified (eg, unassisted in-person clinical examination, clinical digital images) or magnified methods (eg, in-person dermoscopy, dermoscopic digital images) and reported quantitative measures (eg, sensitivity, specificity, area under the receiver operating characteristic curve [AUROC]) were included. We also included all algorithms or checklists that were used to assist in diagnosis. Acceptable reference standards were histopathologic diagnoses or longitudinal follow-up when lesions were initially designated as benign.

Studies that reported the accuracy of specific morphologic features but not diagnosis, or reported the accuracy of clinical diagnoses but not whether inspection was by naked eye or dermoscopy were excluded. We excluded studies that obtained diagnoses by methods other than single observer (eg, by consensus or averages) because they did not reflect the usual clinical setting, did not report sensitivity and specificity, and/or were not published in English.

Selection Process

Each study was independently screened by 2 of 4 reviewers (J.C., K.F., R.F., R.R.). One of 4 reviewers extracted data, which was then verified by the second reviewer. Two of the 4 reviewers also independently assessed the risk of bias and applicability of included studies using the SIGN (Scottish Intercollegiate Guidelines Network) checklist³⁵ for diagnostic studies (eTable 3 in Supplement 1). Disagreements were resolved by consensus among 3 reviewers (J.C., K.F., R.R.); an attending dermatologist (M.W.) was available for consultation when disagreements were difficult to resolve. Complete details are available in eTables 2 and 3 in Supplement 1.

We initially identified 1430 unique references. After titles and abstracts were screened, 262 full-text articles were reviewed for eligibility. Of these, 104 studies were assessed for quality and bias using the Quality Assessment of Diagnostic Accuracy Studies, Version 2. We excluded 4 studies that did not pass the quality assessment (Figure 1).

Analytic Definitions

This review compared magnified with unmagnified methods of skin lesion examination, and the performance of experienced dermatologists (cutoff range, ≥3-6 years of training/dermoscopy experience), inexperienced dermatologists (cutoff range, <3-6 years of training/dermoscopy experience), and PCPs using those methods. Unmagnified methods were either in-person unassisted (naked eye) clinical examination or used clinical images. Magnified methods were either in-person dermoscopic evaluation or dermoscopic images. Dermoscopic evaluations used a dermoscope, a handheld device that provides magnification and enhanced visualization of skin lesions. Unless otherwise noted, combined unmagnified methods (in-person clinical examination and clinical images; hereafter, clinical examination/images) were compared with combined magnified methods (in-person dermoscopy and dermoscopic images (hereafter, demoscropy/images).

Statistical Analysis

Individual study results were combined for meta-analysis of sensitivity, specificity, AUROC, and relative odds ratio (ROR) of diagnostic accuracy results. For studies that reported head-to-head direct comparisons of examination methods (dermoscopy/images compared with clinical examination/images), the metrics of each examination method (sensitivity and specificity) was summarized as the odds ratio for accurate diagnosis. The relative odds ratios (RORs) for diagnostic accuracy were then computed for comparison within each study first and combined across the studies in a meta-analysis on a log-transformed scale.³⁶ The standard error of each ROR was defined as the square root of the sum of the odds of each examination method on the log-transformed scale. Two studies had zero false negatives, and we applied the standard adjustment method by adding 0.5 to the zero cell counts. I² statistic was computed to quantify the degree of between-study heterogeneity (as the percentage of variation across studies due to between-study heterogeneity).³⁷ When no head-to-head within study comparisons were available, we used other metrics (ie, sensitivity, specificity, AUROC) to summarize clinician diagnostic accuracy. These summary statistics, based on all eligible studies, gave a more accurate representation of overall sensitivity and specificity. Given that the sensitivity and specificity of a diagnostic test are typically (inversely) associated due to their dependence on each other through a cutoff value, a bivariate diagnostic mixed-effects model was used to account for the dependency between the sensitivity and specificity of a diagnostic test and variability in either statistic between studies.^38,39 Plots and tables show the bivariate diagnostic random-effects meta-analyses separately conducted for each examination method. Lastly, prevalence of melanoma or keratinocyte cancer in the sample may affect diagnostic accuracy. We used meta-regression to identify possible associations. Analysis was conducted in R, version 4.2 (The R Foundation for Statistical Computing) using the mada and metafor packages.⁴⁰ Statistical inference was performed using 2-tailed tests, with P < .05 considered statistically significant. Data analyses were performed from May 2022 to December 2023.

Results

The meta-analysis included 100 studies (eTable 3 in Supplement 1).^{41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140} These studies were conducted in Asia, Europe (the majority), North America, Oceania, and South America, and were published between 1988 and 2021 (eTable 4 in Supplement 1). Only 4 studies reported participants’ race and ethnicity in the datasets, and among these, most lesions involved White participants (eTable 5 in Supplement 1). Five studies reported Fitzpatrick skin type in their datasets, mostly types I to III (eTable 5 in Supplement 1).

Keratinocytic Carcinoma

When including all eligible studies that reported accuracy for diagnosis of keratinocytic carcinomas, experienced dermatologists had summary sensitivity of 79.0% (95% CI, 62.8%-89.3%) and specificity of 89.1% (95% CI, 70.5%-96.5%) when assessing lesions using clinical examination/images, and sensitivity of 83.7% (95% CI, 76.6%-89.0%) and specificity of 87.4% (95% CI, 78.9%-92.8%) when using dermoscopy/images (Table 1; Figure 2A).^{41,44,45,65,73,77,79,81,85,90,94,95,112,113,131,133,135,139} AUROC was 0.89 when assessed with clinical examination/images, and 0.91 with dermoscopy/images (Table 1; eFigure 1 in Supplement 1). Only 1 study⁸³ of inexperienced dermatologists diagnosing keratinocytic carcinomas met our eligibility criteria. It reported that dermatologists-in-training using clinical images achieved mean (SD) sensitivity and specificity of 94.2% (5.2%) and 68.5% (11.6%), respectively, with an AUROC of 0.88 (Table 1). PCPs’ accuracy for diagnosing keratinocytic carcinomas using dermoscopy had sensitivity and specificity of 81.4% (95% CI, 43.3%-96.1%) and 80.1% (95% CI, 54.2%-93.2%), respectively, with an AUROC of 0.87 (Table 1; Figure 2A).^108,137 No studies of PCPs using clinical examination to diagnose keratinocytic carcinomas met our inclusion criteria. We did not find prevalence of keratinocytic neoplasms in the study datasets to be significantly correlated with sensitivity (eFigure 4 in Supplement 1).

Table 1. Diagnostic Accuracy for Keratinocytic Carcinomas, Stratified by Physician Type and Experience and Examination Method.

Physician	Method	% (95% CI)		AUROC	Studies, No.a	Physicians, No.b	Lesions, No.
		Sensitivity	Specificity
Experienced dermatologist	Clinical examination/images	79.0 (62.8-89.3)	89.1 (70.5-96.5)	0.89	9	>16	8441
	Dermoscopy/images	83.7 (76.6-89.0)	87.4 (78.9-92.8)	0.91	18	>303	9564
Primary care physician	Dermoscopy	81.4(43.3-96.1)	80.1 (54.2-93.2)	0.87	2	>50	8515
Inexperienced dermatologist, % (SD)c	Clinical images	94.2 (5.2)	68.5 (11.6)	0.88	1	34	80

Abbreviation: AUROC, area under the receiver operating characteristic curve.

^a Some studies were included in multiple physician categories.

^b Not all studies specified how many physicians participated.

^c Based on data from only 1 study⁸³ that met inclusion criteria and reported clinical diagnostic accuracy for inexperienced dermatologists evaluating keratinocytic carcinoma. No studies that met inclusion criteria were found for inexperienced dermatologists using dermoscopy or for PCPs using clinical examination.

Figure 2. Diagnostic Accuracy of Experienced Dermatologists, Inexperienced Dermatologists, and Primary Care Physicians (PCPs) Using Dermoscopy/Images.

A, Analysis assessed 27 records from 20 studies that reported results for dermoscopy/images by experienced dermatologists^{41,44,45,65,73,77,79,81,85,90,94,95,112,113,131,133,135,139} or PCPs.^108,137 AUROC was 0.91 for experienced dermatologists and 0.87 for PCPs. B, Analysis assessed 92 records from 68 studies that reported results for dermoscopy/images by experienced dermatologists,^{41,42,43,46,47,48,49,50,51,52,53,54,55,57,58,60,64,66,67,68,71,72,74,76,79,80,82,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,105,106,107,110,111,114,116,117,119,121,123,124,126,127,129,131,132,134,138,140} inexperienced dermatologists,^{51,52,53,54,57,59,82,89,101,116,119,126,127,140} and PCPs.^{104,108,114,137,138} AUROC was 0.90 for experienced dermatologists, 0.81 for inexperienced dermatologists, and 0.65 for PCPs. AUROC indicates area under the receiver operating characteristic curve.

To minimize between-study heterogeneity, we restricted the analysis to studies that directly compared the accuracy for diagnosis of keratinocytic carcinomas by experienced dermatologists using dermoscopy/images compared with clinical examination/images. We found a 2.5-fold (95% CI, 1.3- to 5.0-fold) increase in the odds of accurate diagnosis of keratinocytic carcinomas using dermoscopy/images compared to using clinical examination/images (Figure 3A).^{44,77,81,90,131,133}

Figure 3. Comparative Diagnostic Accuracy, By Lesion Type, Examination Method (Dermoscopy/Images vs Clinical Examination/Images), and Physician Type (Dermatologist vs Primary Care Physician [PCP]) .

A-C, Random effects meta-analysis comparing examination by magnified (in-person dermoscopy or dermoscopic images) vs nonmagnified (in-person clinical examination or clinical images) methods. A, Each study directly compared magnified vs nonmagnified methods.^{44,77,81,90,131,133} for diagnosing keratinocytic carcinoma; I² = 10.0%. B, Each study directly compared magnified vs nonmagnified methods^{54,60,68,88,90,110,119,131} for diagnosing melanoma; I² = 66.5%. C, In person, no images; each study directly compared magnified vs nonmagnified methods^{60,68,88,110,131} for diagnosing melanoma; I² = 37.0%. D, Each study directly compared dermatologists with PCPs^114,138 in diagnosing melanoma using dermoscopic images. Random effects meta-analysis, with use of dermoscopic images; I² was not calculated due to low number of included studies. NA indicates not applicable, and NS, not stated.

Melanoma

When including all eligible studies, experienced dermatologists had sensitivity of 76.9% (95% CI, 69.3%-83.1%) and specificity of 89.1%, (95% CI, 76.9%-95.3%), with an AUROC of 0.84, when using clinical examination/images, and sensitivity of 85.7% (95% CI, 82.5%-88.3%) and specificity of 81.3% (95% CI, 76.3%-85.4%), respectively, with an AUROC of 0.90, when using dermoscopy/images (Table 2; Figure 2B).^{41,42,43,46,47,48,49,50,51,52,53,54,55,57,58,60,64,66,67,68,71,72,74,76,79,80,82,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,105,106,107,110,111,114,116,117,119,121,123,124,126,127,129,131,132,134,138,140} Inexperienced dermatologists diagnosing melanoma had sensitivity of 78.3% (95% CI, 54.9%-91.4%) and specificity of 66.2% (95% CI, 55.9%-75.1%), with an AUROC of 0.73, when assessing clinical images, and sensitivity of 78.0% (95% CI, 69.3%-84.7%) and specificity of 69.5% (95% CI, 52.9%-82.2%), with an AUROC of 0.81, when assessing dermoscopic images (Table 2, Figure 2B).^{51,52,53,54,57,59,82,89,101,116,119,126,127,140} We did not find eligible studies that reported the accuracy of inexperienced dermatologists diagnosing melanomas in person. PCPs’ had sensitivity of 49.5% (95% CI, 40.4%-58.6%) and specificity of 91.3% (95% CI, 78.0%-96.9%), with an AUROC of 0.65, for diagnosis of melanoma using dermoscopy/images (Table 2, Figure 2B).^{104,108,114,137,138} The single study¹⁰⁴ of PCPs using clinical examination to diagnose melanoma reported sensitivity of 37.5% (95% CI, 21.1%-56.3%) and specificity of 84.6% (95% CI, 80.0%-88.5%) (Table 2). We did not find prevalence of melanomas in the study datasets to be significantly correlated with sensitivity (eFigure 4 in Supplement 1).

Table 2. Diagnostic Accuracy for Melanoma, Stratified by Physician Type and Experience and Examination Method.

Physician	Method	% (95% CI)		AUROC	Studies, No.a	Physicians, No.b	Lesions, No.
		Sensitivity	Specificity
Experienced dermatologist	Clinical examination/images	76.9 (69.3-83.1)	89.1 (76.9-95.3)	0.84	11	>96	11 216
	Dermoscopy/images	85.7 (82.5-88.3)	81.3 (76.3-85.4)	0.90	64	>311	42 694
Inexperienced dermatologist	Clinical images	78.3 (54.9-91.4)	66.2 (55.9-75.1)	0.73	3	133	403
	Dermoscopic images	78.0 (69.3-84.7)	69.5 (52.9-82.2)	0.81	14	227	2270
Primary care physicianc	Dermoscopy/images	49.5 (40.4-58.6)	91.3 (78.0-96.9)	0.65	5	>116	9491
	Clinical examination	37.5 (21.1-56.3)	84.6 (80.0-88.5)	NA	1	63	374

Abbreviations: AUROC, area under the receiver operating characteristic curve; NA, not applicable.

^a Some studies were included in multiple physician categories.

^b Not all studies specified how many physicians participated.

^c Based on data from only 1 study¹⁰⁴ that met inclusion criteria and reported accuracy of primary care physicians diagnosing melanoma using clinical examination.

When restricting the analysis to studies that directly compared the use of dermoscopy/images with clinical examination/images by experienced dermatologists, we found that dermatologists had 2.6-fold (95% CI, 0.9- to 7.9-fold) higher odds of accurate diagnosis of melanoma with dermoscopy/images compared with clinical examination/images (Figure 3B).^{54,60,68,88,90,110,119,131} When analyses were further restricted to studies that directly compared dermoscopy to clinical examination, dermatologists had 5.7-fold (95% CI, 2.2- to 15.2-fold) higher odds of accurate diagnosis of melanoma using dermoscopy (Figure 3C).^{60,68,88,110,131}

Studies that directly compared the accuracy of experienced dermatologists and PCPs showed that experienced dermatologists had 13.3-fold (95% CI, 7.2- to 24.5-fold) higher odds of accurate diagnosis of melanoma compared to PCPs when using dermoscopic images (Figure 3D).^114,138 No studies directly comparing dermatologists with PCPs using clinical examination/images met eligibility criteria.

Discussion

We conducted a systematic review and meta-analysis of studies reporting the diagnostic accuracy of skin cancer diagnosis, stratified by skin lesion type (melanocytic or keratinocytic), physician specialty and experience, and method of examination (dermoscopy/images vs clinical examination/images). Experienced dermatologists had greater accuracy when diagnosing keratinocytic carcinomas using dermoscopy/images compared with clinical examination/images. For the diagnosis of melanoma, experienced dermatologists’ increased accuracy using dermoscopy/images compared with clinical examination/images improved even further and reached statistical significance when the analyses were based solely on comparing dermoscopy with clinical examination, consistent with previous reports that teledermoscopy is less accurate compared to in-person diagnosis of melanoma.¹⁴¹ Dermatologists had 13.3-fold higher odds of accurate diagnosis of melanoma compared to PCPs, a finding with implications for studies assessing the usefulness of skin cancer screening. Finally, for experienced dermatologists, sensitivity was greater with dermoscopy, but specificity was greater with clinical examination, suggesting that a skin examination should optimally include both methods.

Similar to prior studies,^23,33 we found that dermoscopy was superior to clinical skin examination when experienced dermatologists diagnose melanoma in person and keratinocytic carcinomas in person or using images. Diagnosis of melanoma is more difficult compared with diagnosis of keratinocytic carcinomas, likely producing greater accuracy when dermatologists screen for melanomas using dermatoscopes in person, given that in-person examination provides metadata not available with images alone. Our findings also concur with previous reports^30,33 that diagnostic performance for melanomas depended on the level of dermatologist experience. Also similar to previous reports, we found considerable heterogeneity among studies.^{23,24,25,27,29,30,31,142,143,144,145,146} Sources of heterogeneity included the small numbers of physicians included in the individual studies, differing training or experience levels of skin examiners, and the heterogeneous nature of datasets or lesions. To minimize heterogeneity among compared studies, we performed subanalyses that included only those studies that reported within-study direct comparisons of diagnostic accuracies of dermoscopic examination compared with clinical examination, and, separately, those that directly compared the diagnostic performance of dermatologists and PCPs.

Our meta-analyses yielded summary metrics for skin cancer diagnostic performance that could be useful in several settings. First, they serve as benchmarks for practitioners’ skin cancer diagnostic performance and help guide resident training with regards to target proficiency. Together with recommendations for dermoscopy foundational skills for dermatology residents, benchmarks for skin cancer examination could contribute to standardizing dermatology resident training across programs.¹⁴⁷ In addition, training PCPs in dermoscopy could increase diagnostic accuracy; an expert consensus statement has been developed detailing recommended dermoscopy proficiency skills for PCPs.¹⁴⁸

Second, the findings of this study suggest metrics for benchmarking emerging technologies. Clinical diagnostic tools that bring precision medicine to skin cancer examination and diagnosis are emerging; these tools range from reflectance confocal microscopy and RNA tape−sampling to computer vision models powered by artificial intelligence, among others.^15,16,17,18 To be useful, they need to be benchmarked against summary performance metrics, rather than a small group of practitioners examining a study-specific dataset. The stratification of metrics by specialty is useful because developers can compare their technologies to different practitioner benchmarks, depending on the clinical context for which the diagnostic tool is targeted.

Third, our findings highlight the differences in the diagnostic performance of skin cancer detection between specialties. Major studies assessing the effectiveness of skin cancer screening¹⁴⁹ included predominantly nondermatology physicians and few dermatologists, using a combination of predominantly visual examination and less often dermoscopy. Our findings suggest that large studies with PCPs examining skin among the general adult population should be reconsidered; PCPs receive minimal training on skin examination, and are not routinely trained in dermoscopy.^150,151,152 Sensitivity for melanoma detection by PCPs using either dermoscopy or clinical examination was less than 50%. However, PCPs conduct many of the skin examinations in the US, and accessibility to specialists can be limited.^{152,153,154,155} Developing technologies could assist in augmenting PCPs’ diagnostic performance, increase effectiveness of examination, and increase equitable access to high-quality skin examinations.¹⁵⁶ Furthermore, as suggested by the workplace screening study by Schneider et al¹⁵⁷ and expert consensus panels,^158,159 a risk-stratified targeted skin-screening approach by dermatologists may be an effective means of detecting skin cancer at a level sufficient to affect outcomes. Future studies are needed to assess both approaches to skin examination, individually and together.

We identified gaps in the literature and future research opportunities. Few studies assessed PCPs’ diagnostic accuracy by clinical examination, the standard of care method used by PCPs. Also, few studies reported the demographic characteristics of the patients whose lesions were examined; of those that did, their patients were mostly White and of Fitzpatrick skin types I to III. There is a need to assess the accuracy of skin examination among more diverse skin types. In addition to summary diagnostic performance statistics by our study, there is a need to develop standardized test datasets with clinical and dermoscopic images for benchmarking new technologies and physician diagnostic capabilities. Given the growing use of advanced practice practitioners for skin cancer diagnosis, future studies could assess their summary diagnostic metrics.

Strengths and Limitations

The strengths of this study include the breadth of the included studies (spanning 5 continents), restricting the inclusion criteria to studies that use longitudinal follow-up for benign lesions and/or histopathologic results as ground truth for diagnosis, and stratifying results by lesion type, practitioner type and experience, and examination method.

Limitations of our study include the small number of studies assessing PCPs and the small numbers of physicians in many of the included studies. Some studies provided morphology only without any clinical information, which may affect clinical applicability given that physicians generally have access to both morphology and clinical information. We were not able to conduct meta-analyses comparing in-person examination with using images, due to the low number of studies in each category. Furthermore, in geographic regions where PCPs specialize in skin cancer diagnosis and are proficient in dermoscopy, their performance may be better than reported in the studies; for example a study in Australia reported that 49% of PCPs have dermoscopy training and 61% of skin or pigmented lesion checks used dermoscopy.¹⁶⁰ In comparison, only 9% of PCPs in the US reported using dermoscopy regularly.¹⁶¹

Conclusions

This systematic review and meta-analysis of diagnostic accuracy for melanoma and keratinocytic carcinomas by dermatologists and PCPs compared examination modality. We found significant differences in diagnostic accuracy by different physician specialty; accuracy improved when dermoscopic methods were used, and improved further with in-person dermoscopy. These results suggest that clinical examination combined with dermoscopy likely yields better accuracy than either method alone.

Supplement 1.

eTable 1. Full search strategy

eTable 2. Excluded Studies

eTable 3. Bias Assessment of included studies using SIGN-2 Checklist

eTable 4. Geographic distribution of included studies

eTable 5. Studies that reported race/ethnicity or Fitzpatrick skin type of patients or dataset

eFigure 1. Accuracy of experienced dermatologists diagnosing keratinocytic carcinomas using magnified vs. non-magnified methods

eFigure 2. Accuracy of experienced dermatologists diagnosing melanomas using magnified vs. non-magnified methods

eFigure 3. Accuracy of inexperienced dermatologists diagnosing melanomas using magnified vs. non-magnified methods

eFigure 4. Correlation between sensitivity and prevalence in the included studies for keratinocytic carcinomas and melanoma

Supplement 2.

Data Sharing Statement