Abstract
Objective: Risk prediction models for primary malignant melanoma thus far have relied on qualitative patient information. The authors propose a quantitative diagnostic predictive probability model using Multispectral Digital Skin Lesion Analysis for melanoma and other high-risk pigmented lesions and evaluate its effectiveness optimizing biopsy decisions by dermatologists. Design: Data from 1,632 pigmented lesions analyzed by a Multispectral Digital Skin Lesion Analysis device were used to perform a logistic regression analysis. This new quantitative melanoma or melanoma/atypical melanocytic hyperplasia/high-grade dysplastic nevus probability model was then evaluated to determine its impact on dermatologist decisions to biopsy pigmented lesions clinically suggestive of melanoma. Participants were given an electronic keypad and answered “yes” or “no” if they would biopsy each of 12 pigmented lesions when presented first with patient history, clinical images, and dermoscopic images and again when subsequently shown Multispectral Digital Skin Lesion Analysis data. Setting/participants: Study of 191 dermatologists at a medical conference. Measurements: Sensitivity, specificity, biopsy accuracy, overall biopsy rate, and percentage dermatologists biopsying all five melanomas. Results: Dermatologists were significantly more sensitive, specific, and accurate while decreasing overall biopsy rates with Multispectral Digital Skin Lesion Analysis probability information. Conclusion: Integration of Multispectral Digital Skin Lesion Analysis probability information in the biopsy evaluation and selection process of pigmented lesions has the potential to improve melanoma sensitivity of dermatologists without the concomitant costs associated with additional biopsies being performed.
Risk prediction models are often used to help identify individuals at higher risk of cancer in the general population. Developing statistical models to evaluate the probability of developing cancer over a defined period of time allows for enhanced early detection, patient education, and intervention. Several studies have attempted to assess a binary outcome for melanoma (MM) diagnosis, but have not proposed device-based models for MM diagnostic assessment.1
Several diagnostic tools for MM have emerged in the past decade, including confocal scanning laser microscopy, electrical impedance spectroscopy, noninvasive genomic detection, and multispectral imaging2 suggesting there is interest in improving MM diagnosis with quantifiable data. The authors propose a quantitative diagnostic predictive probability model for MM and other high-risk pigmented lesions using a Multispectral Digital Skin Lesion Analysis (MSDSLA) device (MelaFind®, MelaSciences, Inc., Irvington, New York) and evaluate its effectiveness optimizing biopsy decisions by dermatologists.
METHODS
Data from 1,632 pigmented lesions analyzed by an MSDSLA device were used to generate a logistic regression analysis.3 Diagnoses of these lesions were assigned to the following four distinct categories: 1) high-grade dysplastic nevus (HGDN); 2) atypical melanocytic hyperplasia (AMH); 3) MM; or 4) other. The MSDSLA classifier score is a numerical value based on the level of morphological disorder within a pigmented lesion. By analyzing the range of these values, logistic regression models were derived to determine the probability distribution for both MM and other high-risk pigmented lesions (MM/AMH/HGDN). The logistic regression model used in this study was:
logit(p) =a + b1x1+ b2x2 + ….+ bixi
where p is the calculated probability of MM and x1x2xi are explanatory variables. The model logit(p)=a+bx is equivalent to p=probability of MM=e(a+bx) / 1+e(a+bx). Results for each model are outlined in Table 1.
TABLE 1.
Probability of MM and MM/AMH/HGDN prediction model
| CLASSIFIER SCORE | PROBABILITY OF MM | 95% CI | PROBABILITY OF MM/AMH/HGDN | 95% CI |
|---|---|---|---|---|
| -10.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.00 |
| -9.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.00 |
| -8.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.00 |
| -7.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.00 |
| -6.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.01 |
| -5.00 | 0.00 | 0.00–0.00 | 0.00 | 0.00–0.01 |
| -4.00 | 0.00 | 0.00–0.01 | 0.01 | 0.00–0.01 |
| -3.00 | 0.00 | 0.00–0.01 | 0.01 | 0.01–0.01 |
| -2.00 | 0.01 | 0.00–0.01 | 0.01 | 0.01–0.02 |
| -1.00 | 0.01 | 0.01–0.02 | 0.02 | 0.01–0.03 |
| 0.00 | 0.02 | 0.01–0.03 | 0.04 | 0.03–0.05 |
| 1.00 | 0.04 | 0.03–0.05 | 0.06 | 0.04–0.07 |
| 2.00 | 0.06 | 0.05–0.07 | 0.09 | 0.07–0.10 |
| 3.00 | 0.10 | 0.08–0.11 | 0.14 | 0.12–0.15 |
| 4.00 | 0.15 | 0.13–0.18 | 0.20 | 0.17–0.23 |
| 5.00 | 0.23 | 0.19–0.28 | 0.29 | 0.24–0.35 |
| 6.00 | 0.34 | 0.27–0.42 | 0.40 | 0.33–0.48 |
| 7.00 | 0.47 | 0.36–0.58 | 0.52 | 0.42–0.62 |
| 8.00 | 0.60 | 0.47–0.72 | 0.64 | 0.52–0.74 |
| 9.00 | 0.72 | 0.57–0.83 | 0.74 | 0.61–0.84 |
| 10.00 | 0.81 | 0.67–0.90 | 0.82 | 0.70–0.90 |
This new quantitative MM or MM/AMH/HGDN probability model was then evaluated to examine its impact on dermatologist decisions to biopsy lesions clinically suggestive of MM. Older MSDSLA systems reported classifier scores based on a system of “low disorganization,” for scores below zero and “high disorganization,” for scores of zero and above. In 2014, the MSDSLA system changed from a binary reading to include the probability of high-risk lesions based on the logistical regression model noted above. This additional information was given as the probability of a pigmented lesion being MM or MM/AMH/HGDN.
One hundred ninty-one dermatologists evaluated 12 pigmented lesions (5 MMs and 7 other pigmented lesions) in this study. Participants were given an electronic keypad and answered “yes” or “no” if they would biopsy each pigmented lesion when presented first with patient history, clinical images, and dermoscopic images and again when subsequently shown MSDSLA probability data. Individual responses before and after MSDSLA were compared to determine the effect of the MSDSLA probability regression generated classifier score model on dermatologist biopsy decisions.
RESULTS
Demographic characteristics of the participants are outlined in Table 2 and results of the study are summarized in Table 3. For the 191 dermatologists, additional MSDSLA information improved the MM biopsy sensitivity from 67.7 to 89.2 percent (p<0.001) while specificity improved from 38.6 to 54.1 percent with MSDSLA information (p<0.001). Diagnostic accuracy improved with MSDSLA data by 18.0 percent (50.7–68.7%, p<0.001). With MSDSLA probability information, the rate of biopsying non-MMs significantly decreased from 61.4 to 45.9 percent (p<0.001). The number of dermatologists choosing to biopsy all five MMs increased by 48.9 percent (p<0.001) following MSDSLA. Interestingly, the biopsy rate of all pigmented lesions remained relatively unchanged (64.1 vs. 64.0%; NS) when MSDSLA probability information was provided.
TABLE 2.
Demographic characteristics
| YEARS IN PRACTICE* | (N, %) |
|---|---|
| 0–5 | 36 (19.0%) |
| 6–10 | 18 (9.5%) |
| 11–15 | 27 (14.3%) |
| 16–20 | 41 (21.7%) |
| 21+ | 67 (35.4%) |
| NUMBER OF MELANOMA SEEN IN PREVIOUS YEAR** | (N, %) |
| 0–5 | 31 (16.7%) |
| 6–10 | 51 (27.4%) |
| 11–15 | 42 (22.6%) |
| 16–25 | 37 (19.9%) |
| 26+ | 25 (13.4%) |
2 participant entries missing
5 participant entries missing
TABLE 3.
Reader study sensitivities and specificities
| BEFORE MSDSLA | AFTER MSDSLA | P VALUE | |
|---|---|---|---|
| Sensitivity | 67.70% | 89.20% | P<0.001 |
| Specificity | 38.60% | 54.10% | P<0.001 |
| Diagnostic accuracy | 50.70% | 68.70% | P<0.001 |
| Percentage of non-melanomas biopsied | 61.40% | 45.90% | P<0.001 |
| Dermatologists choosing to biopsy all 5 melanomas | 16.50% | 65.40% | P<0.001 |
| Percentage of all lesions biopsied | 64.10% | 64.00% | NS |
CONCLUSION
The ability to quantify MM diagnostic risk in a pigmented lesion using MSDSLA has tremendous potential because MM is virtually curable if identified early. New technologies have emerged including noninvasive, in vivo techniques not limited to digital photography, dermoscopy, computerized image analysis systems, confocal scanning laser microscopy, and a high-definition laser Doppler imaging system.5–7 MSDSLA is the first to provide a quantifiable risk assessment with the potential to be widely utilized in clinical practice.
Compared to an earlier study4 using the original, binary MSDSLA output on the same set of lesions, the change in specificity improved to a greater extent (15.5 vs. 10.2%, p<0.05) with the logistical regression model. This suggests the additional probability information was more helpful in ruling out benign lesions that may have otherwise been chosen for biopsy. There was also a reduction in the overall biopsy rate with the new logistical regression derived MSDSLA model without change in the total number of biopsies. Most importantly, there does not appear to be any negative impact to the safety and effectiveness of the MSDSLA system by incorporating logistic regression derived probability information.
A quantifiable risk prediction model can improve diagnostic sensitivity, specificity, and accuracy without increasing the number of biopsies performed. This is the first study, to the authors’ knowledge, that has evaluated a dermatological diagnostic device for its quantitative predictive capacity for the presence of MM and other high-risk pigmented lesions. Integration of these data into the biopsy decision process may improve early MM detection while having the potential to decrease healthcare costs associated with unnecessary biopsies.
Footnotes
DISCLOSURE:Dr. Winkelmann is a clinical research fellow funded in part by MelaSciences Inc. Dr. Yoo has worked as a consultant for MelaSciences Inc. Ms. Tucker is employed by MelaSciences Inc. Mr. White reports no relevant conflicts of interest. Dr. Rigel is a consultant to MelaSciences Inc.
REFERENCES
- 1.Masood A, Al-Jumaily AA. Computer aided diagnostic support system for skin cancer: a review of techniques and algorithms. Int J Biomed Imaging. 2013;2013:323268. doi: 10.1155/2013/323268. Epub 2013 Dec 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ferris LK, Harris RJ. New diagnostic aids for melanoma. Dermatol Clin. 2012;30:535–545. doi: 10.1016/j.det.2012.04.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Monheit G, et al. The performance of MelaFind: a prospective multicenter study. Arch Dermatol. 2011;147:188–194. doi: 10.1001/archdermatol.2010.302. [DOI] [PubMed] [Google Scholar]
- 4.Rigel DS, Roy M, Yoo J, et al. Impact of guidance from a computer-aided multispectral digital skin lesion analysis device on decision to biopsy lesions clinically suggestive of melanoma. Arch Dermatol. 2012;148:541–543. doi: 10.1001/archdermatol.2011.3388. [DOI] [PubMed] [Google Scholar]
- 5.Hunger RE, Della Torre R, Serov A, et al. Assessment of melanocytic skin lesions with a high-definition laser Dopller imaging system. Skin Res Technol. 2012;18:207–211. doi: 10.1111/j.1600-0846.2011.00555.x. [DOI] [PubMed] [Google Scholar]
- 6.Oliveria SA, Sachs D, Belasco KT, et al. Adoption of new technologies for early detection of melanoma in dermatologic practice. J Am Acad Dermatol. 2003;49:955–959. doi: 10.1016/s0190-9622(03)02464-2. [DOI] [PubMed] [Google Scholar]
- 7.Dhawan AP, D’Alessandro B, Patwardhan M. Multispectral optical imaging of skin lesions for detecion of malignant melanomas. Conf Proc IEEE Eng Med Biol Soc. 2009;2009:5352–5355. doi: 10.1109/IEMBS.2009.5334045. [DOI] [PubMed] [Google Scholar]