Real-world application of a non-invasive 2-gene expression test for melanoma diagnosis

Letter:

A 2-gene non-invasive molecular test for the diagnosis of melanoma is commercially available in the United States [Pigmented Lesion Assay (PLA), DermTech]. The test measures gene expression for LINC00518 and preferentially expressed antigen in melanoma (PRAME) from stratum corneum samples.(Gerami et al., 2017) The 2017 validation study and 2018 registry study, however, provided different estimates of the test’s diagnostic accuracy (sensitivity 91%-95%; specificity 53%-91%), raising concerns about its validity.(Ferris et al., 2018, Gerami et al., 2017) The factors that contribute to different estimates of the PLA’s diagnostic accuracy are not well characterized, which has raised concerns about its validity and potential for harm.(Beatson and Weinstock, 2019) Here, we used 2020 registry data(Brouha et al., 2020) to model the real-world application and utility of the PLA.

In the community-based registry, 3,418 pigmented lesions suspicious for melanoma underwent PLA testing; 324 (9.5%) tested positive and 3,094 (90.5%) tested negative.(Brouha et al., 2020) Tests were ordered by 90 providers (dermatologists, primary care physicians, physician assistants, and nurse practitioners) from 53 US dermatology practices. A reference standard diagnosis was not provided. The median age was 48 years and 61% were female. Lesions were located on the trunk (55%), extremities (27%), and head/neck (18%). Our objective was to identify the melanoma prevalence that would lead to a PLA positivity rate of 9.5%.

First, we applied the most optimal diagnostic accuracy measures from the validation study (model 1) to a hypothetical cohort and found that no melanoma prevalence estimate could lead to a PLA positivity rate of 9.5% under these assumptions (Table). This was primarily due to low specificity. Second, we imputed the diagnostic accuracy measures from the 2018 registry (model 2) and found that a melanoma prevalence of 0.6% would lead to a PLA positivity rate of 9.5%. Third, as a sensitivity analysis, we used the melanoma prevalence and sensitivity estimates from the 2018 registry (model 3) and found that a specificity of 95% would be required to yield a PLA positivity rate of 9.5%.

Table.

Estimates of the diagnostic accuracy of the Pigmented Lesion Assay and melanoma prevalence in published studies and hypothetical models

	Data from Published Studies			Data from Hypothetical Models
	Validation Dataset (n=398)a	2018 Registry (n=381)b	2020 Registry (n=3418)c	Model 1d	Model 2e	Model 3f
Sensitivity	0.91	0.95	NR	0.91	0.95	0.95
Specificity	0.69	0.91	NR	0.69	0.91	0.95g
Melanoma Prevalence	0.219	0.052	NR	Noneg	0.006g	0.052
PLA+	0.440	0.134	0.095	0.095	0.095	0.095
Positive Predictive Value	0.45	0.37	NR	NA	0.06	0.52

^aThe reference standard diagnosis was consensus histopathological examination (unanimous agreement by three expert dermatopathologists) for all lesions

^bThe reference standard diagnosis was histopathological examination by local dermatopathologist(s) in 55 cases (51 PLA+ and 4 PLA-) and 3-6 month clinical follow-up in 326 cases (326 PLA-).

^cNo reference standard diagnosis

^dSensitivity, specificity, and PLA positivity rate are independent variables and melanoma prevalence is a dependent variable. There is no melanoma prevalence that can lead to a PLA positivity rate of 9.5% under these assumptions.

^eSensitivity, specificity, and PLA positivity rate are independent variables and melanoma prevalence is a dependent variable.

^fSensitivity, melanoma prevalence, and PLA positivity rate are independent variables and specificity is a dependent variable.

^gdependent variables

PLA = Pigmented Lesion Assay; NR = not reported; NA = not applicable

The clinical utility of a test can be difficult to interpret from statistical measures of classification accuracy. Decision curve analysis, however, is a simple method for visualizing whether a test would provide more benefit than harm across a range of risk thresholds for the treatment (e.g. skin biopsy) under consideration (Figure).(Vickers et al., 2019) Here, we used the diagnostic accuracy and melanoma prevalence estimates from model 2 and specified 2-10% as a clinically relevant range of risk thresholds for melanoma biopsy. Compared to the competing strategies of ‘biopsy all lesions’ and ‘biopsy no lesions’, PLA was associated with the highest net benefit for risk thresholds of 2%-5%. At risk thresholds of 6%-10%, PLA was associated with net harm and was inferior to ‘biopsy no lesions’.

Figure. Decision curve analysis of the Pigmented Lesion Assay.

Decision curve analysis assesses the clinical value of information provided by diagnostic or prognostic tests in medical decision making.(Vickers et al., 2019) A curve is drawn for each potential test along with a line to show what happens when no treatment is ever given and a curve to show what happens if all patients are treated, irrespective of any test results. For any risk or probability threshold (horizontal axis), the curve with the highest net benefit (vertical axis) is the best choice. Treatment can refer to a wide range of interventions, such as additional diagnostic work-up, delaying therapy, medical treatment, or lifestyle changes.

Threshold probability refers to the level of predictive certainty at which a treatment would be chosen by a patient and physician. It reflects the relative value of a treatment for a certain disease, if present, to the value of avoiding the treatment if the disease is not present. For example, if a clinician is willing to conduct 10 skin biopsies to find 1 case of melanoma, the probability threshold would be 0.1 (or 10%).

To draw a decision curve, net benefit is calculated for a range of relevant threshold probabilities: Net benefit = (true positives / N) – [(false positives / N) * (weighting factor)] Weighting factor = Threshold probability / (1 – threshold probability) N = total number of patients in population

In a hypothetical data set (model 2) assuming a melanoma prevalence of 0.6% and PLA sensitivity of 95% and specificity of 91%, the PLA was associated with the highest net benefit for risk thresholds from 2% to 5% when compared with competing strategies in which all lesions or no lesions are biopsied. However, the PLA was associated with net harm and was inferior to ‘biopsy no lesions’ at risk thresholds from 6% to 10%.

Herein, we used published data regarding the diagnostic accuracy and positivity rate of the PLA to estimate how the test is being ordered in the real-world setting. First, we showed that the diagnostic accuracy estimates from the 2017 validation study, particularly specificity, are not relevant to the community setting. Possible reasons for disparate estimates of the specificity of the PLA are likely due to the composition of benign lesions selected for testing. For example, a greater proportion of benign lesions were non-melanocytic in the original validation set compared to the training set (19% vs. 13%).(Gerami et al., 2017) Previous studies reveal that the test is being used on non-melanocytic lesions, including seborrheic keratoses, actinic keratoses, basal cell carcinoma, and squamous cell carcinoma, which may have different rates of PRAME and/or LINC00518 gene expression than melanocytic lesions.(Brouha et al., 2020, Ferris et al., 2018, Ferris et al., 2019) Indeed, a recent study found that diffuse PRAME staining varied among melanoma subtypes.(Lezcano et al., 2018) Most melanocytic nevi (86.4%) completely lacked PRAME staining, but among melanocytic nevus subtypes, Spitz nevi demonstrated the highest rates of having any PRAME immunoreactivity (20%).(Lezcano et al., 2018) Rare isolated melanocytes immunoreactive for PRAME were seen in 15% of solar lentigines and 50% of cases of benign non-lesional sun-damaged skin.(Lezcano et al., 2018) These data suggest that the PLA’s specificity could be lower in patients with the atypical mole syndrome or older patients with sun-damaged skin, compared to younger patients without these characteristics.

Our data also suggests that the PLA is primarily used on lesions with a low melanoma probability (i.e., 1 out of every 167 lesions) in the community setting, implying it is used to evaluate lesions that would have been monitored, not as a “rule-out” diagnostic test on lesions selected for biopsy. The net effect of such a strategy would be an increase in clinical sensitivity at a cost of lower specificity; whether this reduces mortality is unknown. Therefore, the PLA may not prevent up to 88% of surgical biopsies, as has been suggested by some authors.(Rivers et al., 2018) The conclusions from a previous economic analysis of the PLA may not be valid because it assumed a pretest melanoma probability of 6%.(Hornberger and Siegel, 2018) Nonetheless, the PLA may provide benefit, depending on the relative value the patient and physician places on melanoma detection compared to the value of avoiding unnecessary biopsies in shared decision making. However, it is unknown how PLA net benefit would compare to emerging diagnostic tools.(Fried et al., 2020) Descriptions of the potential utility of PLA in specialized melanoma screening clinics are promising but limited by the potential for reporting bias.(Childs, 2018, Shah et al., 2019)

Our study is limited by its use of published estimates of diagnostic accuracy, which may have led to spurious conclusions, particularly as the reference standard differed across studies. Indeed, we also show that if the PLA specificity is as high as 95%, the melanoma prevalence among tested lesions could be 5%. Registries may be limited by case selection bias and inclusion of nondermatologists, affecting generalizability of findings. Given the uncertainty and variability in estimates, further testing and validation of the PLA on diverse populations in different settings is necessary to best understand how to use this promising technology. Future studies would benefit from rigorously adhering to reporting guidelines for diagnostic accuracy studies.(Cohen et al., 2016)