Abstract
Background:
Patients with lesions suspicious for skin cancer often present to primary care physicians (PCPs), who may have limited training in skin cancer diagnosis.
Objective:
To measure the impact of an adjunctive handheld device for PCPs that employs elastic scattering spectroscopy (ESS) on the diagnosis and management of skin cancer.
Methods:
Fifty-seven PCPs evaluated 50 clinical images of skin lesions (25 malignant and 25 benign), first without and then with knowledge of the handheld ESS device output, and in each case indicated if a lesion was likely to be benign or malignant.
Results:
The diagnostic sensitivity of the PCPs with and without the use of the ESS device was 88% (95% CI, 84%-92%) and 67% (95% CI, 62%-72%), respectively (P < .0001). In contrast, no significant difference was observed in the diagnostic specificity. The management sensitivity of the physicians with and without the use of the ESS device was 94% (95% CI, 91%-96%) and 81% (95% CI, 77%-85%), respectively (P = .0009). Similarly, no significant difference was observed in the management specificity.
Conclusion:
The use of the ESS device may have the potential to help improve skin cancer diagnosis and confidence in management decision-making in a primary care setting.
Keywords: skin cancer, dermatology, dermatologist, melanoma, basal cell carcinoma, squamous cell carcinoma, machine learning, artificial intelligence, spectroscopy, convolutional neural network
Introduction
Skin malignancies are the most common type of cancer diagnosed in the United States (US), 1 yet the clinical diagnosis of this cancer remains a challenge for most primary care physicians (PCPs), to whom initial presentations usually occur. Incidence rates of skin cancer, including malignant melanoma (MM) and keratinocyte carcinoma (KC), have increased by 44% and 77%, respectively, in recent decades.1,2
Skin cancer is highly curable if detected early, with a 99% 5-year survival rate for MM when diagnosed at a localized stage. However, this figure drops to 66% for regional stage, and 27% for distant stage. 1 There is a dire need to improve the early diagnosis of skin cancer, in order to maximize survival rate and decrease morbidity as well as associated healthcare costs. 3
Patients are most likely to visit PCPs for the initial examination of suspicious skin lesions before being referred to a specialist. 4 The standard method used by PCPs to diagnose skin cancer is visual inspection, with or without the use of diagnostic aids such as the ABCDE rule, the Chaos and Clues method, the 2 Step Algorithm, or the Glasgow 7-point checklist.5 -8 However, the accuracy of this naked-eye examination is highly dependent on the PCP’s experience, due to the high similarity between early-stage malignant lesions and benign lesions. 9 Consequently, reviews of the literature have reported that PCPs generally have lower sensitivity of both diagnostic accuracy and biopsy/referral decision accuracy as compared to dermatologists. 10
Recently, optical technology-based methods have been developed that significantly improve the accuracy of non-invasive skin cancer diagnosis. These approaches include educational interventions, dermoscopy, reflectance confocal microscopy, optical coherence tomography, multiphoton excited fluorescence imaging, and others.11 -14 However, significant barriers to their use in primary care include the need of specialized training, high cost of implementation, and generally a lack of comprehensive validating evidence.11 -13 A long-standing challenge is to develop new technologies for the early diagnosis of skin cancer, which are non-invasive and accurate, easy to use, and cost-effective for general practice.
A recent comparative study evaluating the diagnostic performance of non-invasive techniques for melanoma diagnosis found that optical spectroscopy achieved the best performance in terms of specificity and sensitivity. 15 Elastic-scattering spectroscopy (ESS) is a specialized form of optical spectroscopy that is sensitive to the micro- and nano-scale structure of a diseased tissue and is therefore suitable for skin cancer assessment.16,17
Here, we investigate the potential for utilization of a spectral classification algorithm that employs ESS measurements in conjunction with the machine learning approach of convolutional neural networks, implemented in a portable ESS Device to help detect skin cancer. 17 The ESS device was developed to be used as an adjunctive tool for the evaluation of suspicious skin lesions in a primary care setting, not as a diagnostic tool. We compared the diagnosis and management performance (sensitivity, specificity, and AUC) in primary care physicians (PCP) with and without the use of the ESS device for detecting skin cancer. Ultimately, this study aims to determine whether the use of an ESS device can help PCPs improve their diagnostic accuracy of skin malignancies.
Methods
ESS and Artificial Intelligence System
The investigative device is a handheld device that employs elastic scattering spectroscopy (ESS) and convolutional neural networks (CNNs) to detect cancerous skin lesions. 17 Developed as an adjunctive tool to help PCPs in their diagnostic and management decision, the device classifies lesions as either malignant (“Investigate Further”) or benign (“Monitor”). Additionally, for “Investigate Further” classified lesions, a score from 1 to 10 is provided which corresponds to the amount of spectral similarity a lesion has to malignant lesions in studies, with 10 representing the highest amount.
The algorithm of the ESS device was trained and validated with over 20 000 spectral recordings from over 4500 skin lesions, including histologically-confirmed melanoma and KC; as well as unbiopsied benign lesions, identified by dermatologists.18 -22 None of the ESS spectral recordings used in the training process were employed in the testing set used in this study.
Test Lesions
A total of 50 cases of skin lesions were selected from the clinical validation study of the ESS device. 23 These cases included 25 malignant lesions and 13 benign lesions, which had been biopsied and assessed histologically; and 12 unbiopsied benign lesions, diagnosed by dermatologists. Characteristics of the test lesions are presented in Table 1. Of note, actinic keratoses were not treated as malignant for the purposes of this classification, and severely atypical nevi were considered to need further evaluation by a dermatologist. For each case, high resolution digital clinical images were used. In addition, the patient’s clinical information, including prior skin cancer history, risk factors, and physical examination results were included.
Table 1.
Anatomic location, n (%) | |||
---|---|---|---|
Trunk | 15 (30) | ||
Upper extremity | 14 (28) | ||
Lower extremity | 11 (22) | ||
Head | 10 (20) | ||
Surface, n (%) | |||
Elevated | 27 (54) | ||
Flat | 23 (46) | ||
Texture, n (%) | |||
Smooth | 25 (50) | ||
Rough | 25 (50) | ||
Color, n (%) | |||
Light | 27 (54) | ||
Dark | 17 (34) | ||
Light and dark | 6 (12) | ||
Pigmentation a , n (%) | |||
Pigmented | 21 (42) | ||
Non-pigmented | 29 (58) | ||
Melanocytic b , n (%) | |||
Yes | 13 (26) | ||
No | 37 (74) | ||
Biopsied, n (%) | |||
Yes | 38 (76) | ||
No | 12 (24) | ||
Final histopathologic read c , n (%) | |||
Actinic keratosis | 3 (6) | ||
Basal cell carcinoma | 9 (18) | ||
Benign melanocytic nevus | 3 (6) | ||
Benign other | 4 (8) | ||
Blue nevus | 1 (2) | ||
Lentigo | 2 (4) | ||
Melanoma | 4 (8) | ||
Mildly atypical melanocytic nevus | 2 (4) | ||
Seborrheic keratosis | 10 (20) | ||
Severely atypical melanocytic nevus | 3 (6) | ||
Squamous cell carcinoma | 9 (18) | ||
Fitzpatrick skin type of patients d , N (%) | |||
I. Always burns, never tans | 5/44 (11.4) | ||
II. Always burns, tans minimally | 28/44 (63.6) | ||
III. Sometimes mild burn, tans uniformly | 8/44 (18.2) | ||
IV. Burns minimally, always tans well | 2/44 (4.5) | ||
V. Very rarely burns, tans very easily | 1/44 (2.3) | ||
Device classification | Total | Device + | Sensitivity (%) |
Overall Sensitivity | 25 | 24 | 96.0 |
Melanoma | 4 | 4 | 100.0 |
Severely atypical melanocytic nevus | 3 | 3 | 100.0 |
Basal cell carcinoma | 9 | 9 | 100.0 |
Squamous cell carcinoma | 9 | 8 | 88.9 |
Total | Device− | Specificity (%) | |
Overall Specificity | 25 | 9 | 36.0 |
Seborrheic keratosis | 10 | 4 | 40.0 |
Mildly atypical melanocytic nevus | 2 | 0 | 0.0 |
Lentigo | 2 | 2 | 100.0 |
Blue Nevus | 1 | 0 | 0.0 |
Benign Other | 4 | 0 | 0.0 |
Benign Melanocytic nevus | 3 | 1 | 33.3 |
Actinic keratosis | 3 | 2 | 66.7 |
Pigmentation of lesion was determined by an internal dermatologist reviewing the images of the lesion cases and assessing whether or not the lesion was or was not pigmented.
Determination of if a lesion was melanocytic was performed by an internal dermatologist classifying the different parent pathology classifications into buckets of melanocytic or non-melanocytic.
Basal cell carcinoma, squamous cell carcinoma, melanoma, and severely atypical melanocytic nevus lesions were considered malignant and all others benign. Severely dysplastic melanocytic nevi were considered malignant because they are often managed with re-excision by dermatologists.
50 lesions utilized in the study were procured from 44 patients.
Lesions were randomly selected to match the prevalence of lesions in the clinical study that was performed. The device performance for the lesions in this study was 96% sensitivity and 36% specificity, compared to the prior clinical study performance of 97% and 26% (37% for unbiopsied benign lesions and 22% for biopsied benign lesions). Thus, the randomly selected cases were similar in performance to the device performance. The malignant and benign subgroups were based on predefined proportions applicable to primary care and enrolled lesions matching these criteria were selected from the clinical study dataset to match the appropriate diagnoses. The order in which lesions were presented was also randomized.
Primary Care Physicians’ Evaluation
A total of 57 U.S. board-certified PCPs participated in this study. All PCPs provided written informed consent. Eligibility criteria and PCPs’ experience as well as demographic information are provided in Table 2, respectively. All PCPs watched a training video about how to use the ESS device and were required to pass a training quiz before moving forward with the study.
Table 2.
Variable | n (%) |
---|---|
Gender | |
Male | 44 (77.2) |
Female | 13 (22.8) |
Other | 0 (0.0) |
Area of board certification and active practice, n (%) | |
Internal medicine | 33 (57.9) |
Family medicine | 24 (42.1) |
Time practicing medicine (years) | |
1-5 | 5 (8.8) |
6-10 | 13 (22.8) |
11-15 | 9 (15.8) |
16-20 | 10 (17.5) |
>20 | 20 (35.1) |
Type of practice | |
Employed in group private practice (multi-specialty) | 10 (17.5) |
Hospital - owned practice | 9 (15.8) |
Independent PCP - solo private practice | 9 (15.8) |
Owner in group private practice (primary care only) | 9 (15.8) |
Owner in group private practice (multi-specialty) | 7 (12.3) |
Academic center | 6 (10.5) |
Employed in group private practice (primary care only) | 6 (10.5) |
Federally qualified health center | 1 (1.8) |
Locum tenens | 0 (0.0) |
Other | 0 (0.0) |
Type of area | |
Urban area (population > 50 000) | 37 (64.9) |
Urban cluster (2500 < population < 50 000) | 14 (24.6) |
Rural (population < 2500) | 6 (10.5) |
Percentage of patients receiving skin checks (%) | |
0 | 0 (0.0) |
1-20 | 8 (14.0) |
21-40 | 12 (21.1) |
41-60 | 11 (19.3) |
61-80 | 15 (26.3) |
>81 | 11 (19.3) |
Frequency of biopsies performed | |
Never | 5 (8.8) |
A few times per year | 16 (28.1) |
1-3 times per month | 6 (10.5) |
1-3 times per week | 19 (33.3) |
>3 times per week | 11 (19.3) |
Frequency of referral to dermatologist | |
Always | 0 (0.0) |
Most of the time | 22 (38.6) |
Sometimes | 28 (49.1) |
Rarely | 7 (12.3) |
Never | 0 (0.0) |
Self-rated skin lesion assessment competence | |
Expert | 6 (10.5) |
Advanced | 22 (38.6) |
Intermediate | 28 (49.1) |
Beginner | 1 (1.8) |
No competence | 0 (0.0) |
The study was conducted in 2 phases; in each phase, PCPs evaluated all 50 cases independently and in random order. In the first phase, each PCP evaluated the skin lesion cases using 1 or more digital images and the patient’s clinical information for each case alone. PCPs were blinded to the classifier output from the device. For each case, PCPs completed a questionnaire about their diagnosis of the lesion (malignant or benign), their recommended management decision (biopsy/referral to a dermatologist or not), and their confidence level on their management decision (scale 0-3, where 0 = no confidence, 1 = slight confidence, 2 = moderate confidence, and 3 = high confidence).
In the second phase of the study, PCPs were asked to evaluate the same lesions from the first phase in a different randomized order; but this time, with the additional knowledge of the ESS device output, as part of the clinical information provided for each case. As in the first phase, for each case, all PCPs completed the questionnaire about their diagnosis, management decision, and level of confidence. At no point in the study were the physicians informed of the study’s distribution of benign and malignant lesions (ie, 25 of each).
Statistical Analysis
Each performance parameter, including sensitivity and specificity, was calculated using the total number of lesions evaluated (n = 1425; 25 lesions × 57 PCPs). Corresponding 95% confidence intervals (CIs) were calculated for both sensitivity and specificity using the Wilson method, as outlined in Saha et al. (2016), 24 to account for potential within-cluster correlation. To compare diagnostic sensitivity and specificity of the physicians with and without knowledge of the device output, we used the method of moments for clustered matched paired data, where a cluster was the set of lesion cases within each PCP combination. A 2-sided P-value of less than .05 was considered to indicate a statistically significant difference.
The same procedure was followed to determine the sensitivity and specificity of management decision, where the sensitivity was defined as the probability of deciding to further evaluate a lesion through a biopsy or referral, given that the lesion was malignant, and specificity was defined as the probability of deciding not to further evaluate a lesion given that the lesion was benign.
Inter-physician variability was assessed using logistic regression with Generalized Estimating Equations to account for within PCP correlation (assuming a compound symmetry correlation structure).
Shifts in the levels of confidence of the physicians in their management decision, with and without (baseline) knowledge of the device output, were compared using an unweighted Kappa statistic. Statistical associations between the level of confidence of the PCPs and their sensitivity and specificity were assessed using multivariate logistic regressions where the dependent variable is a correct diagnosis (yes/no). The model includes a random effect of the lesion and physician to account for correlations. The P-values come from the logistic regression and measure the association between each covariate and the outcome while adjusting for other variables in the model.
All statistics analyses were performed using SAS software (Version 9.4, SAS Institute Inc).
Results
The 50 lesions evaluated by PCPs were dispersed across the body, had a mixture of surfaces, textures, colors, and pigmentations; 26% were melanocytic, 76% were biopsied, and there were a range of final lesion etiologies as determined by histopathology; inherent device classification is also provided (Table 1).
Among 57 PCPs included, participating physicians represented a diversity of areas of practice, time practicing medicine, and practice types (Table 2). Further demographics, as well as range of patients, frequency of biopsies performed, referrals to dermatologist, and baseline level of confidence in skin lesion assessment can be found in Table 2.
The diagnostic sensitivity of the PCPs was 67% (958/1425; 95% CI, 62%-72%) independently, i.e. without the use of the ESS device, and 88% (1261/1425; 95% CI, 84%-92%) with the device (P < .0001). In contrast, the diagnostic specificity independently and with the use of the ESS device was 53% (761/1425; 95% CI, 49%-57%) and 40% (577/1425; 95% CI, 37%-44%), respectively (P = .0516; Table 3). Balanced accuracy (TP + TN/all cases) of PCP diagnostic performance was 0.588 and 0.645 independently without and with the use of the ESS device, respectively. Notably, the proportion of melanomas correctly diagnosed by PCPs included in the test lesion set were 82% (186/228) without use of the ESS device and 97% (221/228) with use. The number of false negatives for melanoma without the use of the device was 42 and with the use of the device was 7; for severely atypical nevi, false negatives without and with the use of the device were 39 and 3, respectively.
Table 3.
Performance | 95% confidence interval | P-value | |||
---|---|---|---|---|---|
Without the use of ESS device | With the use of ESS device | Without the use of ESS device | With the use of ESS device | ||
Diagnosis | |||||
Sensitivity % | 67 (958/1425) | 88 (1261/1425) | 62-72 | 84-92 | <.0001 |
Specificity % | 53 (761/1425) | 40 (577/1425) | 49-57 | 37-44 | .0516 |
Management decision | |||||
Sensitivity % | 81 (1160/1425) | 94 (1342/1425) | 77-85 | 91-96 | .0009 |
Specificity % | 36 (516/1425) | 31 (437/1425) | 31-42 | 28-34 | .3558 |
Data in parentheses are the number of cases correctly identified as malignant (for sensitivity analysis) and benign (for specificity analysis) over the total number of lesions evaluated (25 lesions × 57 physicians = 1425). Bolded values represent statistical significance (p < .05).
The management sensitivity of the PCPs without and with the use of the ESS device was 81% (1160/1425; 95% CI, 77%-85%) and 94% (1352/1425; 95% CI, 91%-96%), respectively (P = .0009; Table 3). The management specificity of the PCPs was 36% (516/1425; 95% CI, 31%-42%) without the device and 31% (437/1425; 95% CI, 28%-34%) with the device (P = .3558; Table 3).
The interclass correlation, as an estimate of inter-physician variability in diagnostic decision, improved from .19 (slight agreement between PCPs) without the use of the ESS device to .50 (fair agreement between PCPs) with the use of the device. Similarly, the interclass correlation for management decision improved from .14 (slight agreement between PCPs) without the use of the device to .51 (fair agreement between PCPs) with the use of the device. The level of confidence of PCPs in their diagnosis and management decision also significantly increased by using the device (Table 4, P < .0001).
Table 4.
Level of confidence without the device | Level of confidence with the device | ||||
---|---|---|---|---|---|
None | Slight | Moderate | High | Total | |
None | 3 | 12 | 20 | 18 | 53 |
Slight | 9 | 91 | 297 | 252 | 649 |
Moderate | 10 | 138 | 619 | 688 | 1455 |
High | 7 | 36 | 118 | 532 | 693 |
Total | 29 | 277 | 1054 | 1490 | 2850 |
Kappa statistic: .1490; OR (95% CI): 4.210 (3.764-4.708); P < .0001.
The area under the curve for diagnosis of malignancy for PCPs for 4 confidence levels increased from 0.619 to 0.683 with addition of the device (P < .001; Figure 1).
Discussion
Our results support a beneficial role for the ESS device, which uses ESS and CNN, as an adjunctive tool for the evaluation of potentially malignant skin lesions in primary care settings. Use of the ESS device by PCPs significantly improved their diagnostic and management sensitivity and was associated with a decrease in specificity that approached significance for diagnosis and was not significant for management. There is a net improvement of PCP detection accuracy with the availability of device output. ESS use also significantly improved inherent validity as reported by area under the curve (AUC) for both diagnosis and referral.
While other commercial options are available, there is a great degree of variation among accuracy and little data to support their use by PCPs. A multispectral image analysis device (MELA Sciences, Irvington, NY) was an FDA-approved device that used a proprietary algorithm to aid with the decision to biopsy a suspicious skin lesion. Studies showed the device had high sensitivity, over 98%, but low specificity, around 10%. It was discontinued for sale or clinical use in 2017. Another adjuvant tool for melanoma detection, a device utilizing electrical impedance spectroscopy, reported over 96% sensitivity and 34% specificity. 25 This is the only currently FDA-approved device for melanoma detection but is restricted to use by dermatologists, given a lack of validation/approval in a primary care context and restrictions for usage on benign lesions such as seborrheic keratoses.26,27 Therefore, additional evidence for use of efficacious devices in clinical settings outside of dermatologic practice is both high yield for primary care, the front line of most suspicious lesion presentation, and in contribution to the high degree of trust needed for regulatory recognition and implementation of such devices. The present study contributes evidence for a diagnostically valid device that can improve detection in this context and meaningfully impact behavior.
This context is important when considering the increase in AUC that we report from 0.62 to 0.68. In this study, the majority of cases involved physician-biopsied lesions, which are notably more difficult for physicians to classify as benign or malignant compared to patient-concerned lesions often featured in image-based AUC studies. The receiver operating characteristic (ROC) curve in this scenario, which describes performance at different classification thresholds, was computed factoring in the variability in physician performance across 50 lesion cases. The rise in AUC implies that PCPs using the device would improve clinical utility and the quality of dermatologist referrals. This is particularly substantial for primary care, where skin cancer detection accuracy is typically lower than dermatology specialty care. Furthermore, given similar ROC sensitivity, PCP specificity increases (ie, when sensitivity is maintained at 94%, the specificity of the PCPs is 13.7% without device output, and 30.9% with device output).
Our results showed an increase in the levels of confidence of the PCPs in their management decision with the use of the ESS device; importantly, our results also indicated that higher confidence is associated with higher sensitivity and specificity. The inter-physician variability decreased with the use of the ESS device, which may indicate a decline in the subjectivity of the PCPs in their assessments. Of note, use of this device in prior clinical studies has shown minimal user-related variations. 16 Another strength of our study are the skin lesions used include a wide range of cases representative of those found in primary care practice. Additionally, due to an approach that is agnostic to lesion and skin type given its lack of surface-level feature analysis, ESS provides an inherent advantage as compared to other detection methods like Pigmented Lesion Assays (PLAs) and image-based CNN algorithms in its utility for a variety of lesions including both melanocytic and non-melanocytic lesions.28,29
There are a few limitations to this study. First, the study design did not allow for in-vivo lesion evaluations; therefore, though clinical history was provided, tactile evaluation of the lesions was not possible. Second, PCPs who participated in the study expressed interest in the ESS device and may have had a particular interest in skin cancer, resulting in a selection bias. Further, the ratio of benign:malignant lesions among the test pool was not reflective of primary clinical practice, where the vast majority of encountered lesions are benign. Future studies will include the analysis of the performance of the ESS device per lesion type.
In conclusion, the ESS device has the potential to help PCPs decide which skin lesions warrant further biopsy/referral for possible skin cancer. This technology improves sensitivity and inherent validity of skin cancer diagnosis and management. The use of the ESS device in a primary care setting is further supported by a reduction in the subjectivity of the PCPs regarding their evaluations and the limited training required for its use, as well as the increased diagnostic and management performance.
Footnotes
Abbreviations (Listed in Order of Appearance): Elastic Scattering Spectroscopy (ESS)
Primary Care Physicians (PCPs)
Malignant Melanoma (MM)
Keratinocyte Carcinoma (KC)
Convolutional Neural Networks (CNN)
Pigmented Lesion Assay (PLA)
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Todd Thames and Thomaz de Campos Silva are advisors for DermaSensor, Inc. Laura Korb Ferris is a consultant for DermTech and an investigator for Castle Biosciences, SkinAnalytics, and DermTech.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was sponsored by DermaSensor, Inc.
ORCID iD: Erik Jaklitsch https://orcid.org/0000-0002-9311-1660
References
- 1. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2021. CA Cancer J Clin. 2021;71(1):7-33. doi: 10.3322/CAAC.21654 [DOI] [PubMed] [Google Scholar]
- 2. Mohan SV, Chang ALS. Advanced basal cell carcinoma: epidemiology and therapeutic innovations. Curr Dermatol Rep. 2014;3(1):40. doi: 10.1007/S13671-014-0069-Y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Mariotto AB, Robin Yabroff K, Shao Y, Feuer EJ, Brown ML. Projections of the cost of cancer care in the United States: 2010-2020. J Natl Cancer Inst. 2011;103(2):117-128. doi: 10.1093/JNCI/DJQ495 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Roetzheim RG, Lee JH, Ferrante JM, et al. The influence of dermatologist and primary care physician visits on melanoma outcomes among medicare beneficiaries. J Am Board Fam Med. 2013;26(6):637. doi: 10.3122/JABFM.2013.06.130042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Friedman RJ, Rigel DS, Kopf AW. Early detection of malignant melanoma: the role of physician examination and self-examination of the skin. CA Cancer J Clin. 1985;35(3):130-151. doi: 10.3322/CANJCLIN.35.3.130 [DOI] [PubMed] [Google Scholar]
- 6. Rosendahl C, Cameron A, McColl I, Wilkinson D. Dermatoscopy in routine practice— “chaos and clues”. Aust Fam Physician. 2012;41(7):482-487. Accessed November 29, 2022. https://pubmed.ncbi.nlm.nih.gov/22762066/ [PubMed] [Google Scholar]
- 7. Walter FM, Prevost AT, Vasconcelos J, et al. Using the 7-point checklist as a diagnostic aid for pigmented skin lesions in general practice: a diagnostic validation study. Br J Gen Pract. 2013;63:610. doi: 10.3399/BJGP13X667213 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Braun RP, Rabinovitz HS, Oliviero M, Kopf AW, Saurat JH. Pattern analysis: a two-step procedure for the dermoscopic diagnosis of melanoma. Clin Dermatol. 2002;20(3):236-239. doi: 10.1016/S0738-081X(02)00216-X [DOI] [PubMed] [Google Scholar]
- 9. Murchie P, Campbell NC. Pigmented lesions, cutaneous melanoma, and future challenges for primary care. Eur J Gen Pract. 2007;13(3):151-154. doi: 10.1080/13814780701627354 [DOI] [PubMed] [Google Scholar]
- 10. Chen SC, Bravata DM, Weil E, Olkin I. A comparison of dermatologists’ and primary care physicians’ accuracy in diagnosing melanoma: a systematic review. Arch Dermatol. 2001;137(12):1627-1634. doi: 10.1001/ARCHDERM.137.12.1627 [DOI] [PubMed] [Google Scholar]
- 11. Fahradyan A, Howell AC, Wolfswinkel EM, Tsuha M, Sheth P, Wong AK. Updates on the Management of Non-Melanoma Skin Cancer (NMSC). Healthc. 2017;5(4):82. doi: 10.3390/HEALTHCARE5040082 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Jones OT, Ranmuthu CKI, Hall PN, Funston G, Walter FM. Recognising skin cancer in primary care. Adv Ther. 2020;37(1):603-616. doi: 10.1007/S12325-019-01130-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Meng X, Chen J, Zhang Z, et al. Non-invasive optical methods for melanoma diagnosis. Photodiagnosis Photodyn Ther. 2021;34:102266. doi: 10.1016/J.PDPDT.2021.102266 [DOI] [PubMed] [Google Scholar]
- 14. Gonna N, Tran T, Bassett RL, Farris DP, Nelson KC. Sensitivity and specificity for skin cancer diagnosis in primary care providers: a systematic literature review and meta-analysis of educational interventions and diagnostic algorithms. J Cancer Educ. 2022;37(5):1563-1572. doi: 10.1007/S13187-022-02194-4/FIGURES/2 [DOI] [PubMed] [Google Scholar]
- 15. Blundo A, Cignoni A, Banfi T, Ciuti G. Comparative analysis of diagnostic techniques for melanoma detection: a systematic review of diagnostic test accuracy studies and meta-analysis. Front Med. 2021;8:637069. doi: 10.3389/FMED.2021.637069 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Shirkavand A, Sarkar S, Ataie-Fashtami L, Mohammadreza H. Detection of melanoma skin cancer by elastic scattering spectra: a proposed classification method. Iran J Med Phys. 2017;14(3):162-166. doi: 10.22038/IJMP.2017.21367.1203 [DOI] [Google Scholar]
- 17. Rodriguez-Diaz E, Manolakos D, Christman H, et al. Optical spectroscopy as a method for skin cancer risk assessment. Photochem Photobiol. 2019;95(6):1441-1445. doi: 10.1111/PHP.13140 [DOI] [PubMed] [Google Scholar]
- 18. Rodriguez-Diaz E, Manolakos D, Christman H, et al. Optical spectroscopy as a method for skin cancer risk assessment. Photochem Photobiol. 2019;95(6):1441-1445. doi: 10.1111/PHP.13140 [DOI] [PubMed] [Google Scholar]
- 19. Tepedino M, Baltazar D, Hucks C, Chatha K ZN. Use of elastic scattering spectroscopy on patient selected lesions that are concerning for skin cancer. Cutis. 2022;110(6 Suppl):35-36. [Google Scholar]
- 20. Salmon P, Bonning M. Use of elastic-scattering spectroscopy and machine learning when assessing skin lesions suggestive of skin cancer. J Dermatol Physician Assist. 2021;15(4): 64-65. [Google Scholar]
- 21. Merry SP, Chatha K, Croghan I, Nguyen VL, McCormick B LD. Clinical performance of novel elastic scattering spectroscopy (ESS) in detection of skin cancer: a blinded, prospective, multi-center clinical trial. J Clin Aesthet Dermatol. 2023; 16(4 Suppl):s16. [Google Scholar]
- 22. Hartman R, Tepedino K, Fung MA, et al. Validation of a handheld elastic-scattering spectroscopy device on lesions suggestive of melanoma. J Dermatol Physician Assist. 2022;16(4):51. [Google Scholar]
- 23. Manolakos D, Rabinovitz H, Geisse J, Bonning M, Rodriquez-Diaz E, Bigio ICA. Clinical validation of a handheld elastic scattering spectroscopy artificial intelligence device. Pod Present American Academy of Dermatology Innovation and Academy, Vancouver, CA, July 20–24, 2022. [Google Scholar]
- 24. Saha KK, Miller D, Wang S. A comparison of some approximate confidence intervals for a single proportion for clustered binary outcome data. Int J Biostat. 2015;2015(2):1-18. doi:10.1515/IJB-2015-0024/MACHINEREADABLECITATION/RIS [DOI] [PubMed] [Google Scholar]
- 25. Malvehy J, Hauschild A, Curiel-Lewandrowski C, et al. Clinical performance of the Nevisense system in cutaneous melanoma detection: an international, multicentre, prospective and blinded clinical trial on efficacy and safety. Br J Dermatol. 2014;171(5):1099-1107. doi: 10.1111/BJD.13121 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Beltrami EJ, Brown AC, Salmon PJM, Leffell DJ, Ko JM, Grant-Kels JM. Artificial intelligence in the detection of skin cancer. J Am Acad Dermatol. 2022;87(6):1336-1342. doi: 10.1016/J.JAAD.2022.08.028 [DOI] [PubMed] [Google Scholar]
- 27. Nevisense Clinical Reference Guide. In: Scibase. 2014. Accessed April, 2023. https://scibase.com/wp-content/uploads/2017/11/Clinical-Reference-Guide-1.pdf
- 28. Ferris LK, Jansen B, Ho J, et al. Utility of a noninvasive 2-gene molecular assay for cutaneous melanoma and effect on the decision to biopsy. JAMA Dermatol. 2017;153(7):675-680. doi: 10.1001/JAMADERMATOL.2017.0473 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi: 10.1038/nature21056 [DOI] [PMC free article] [PubMed] [Google Scholar]