Comparative performance of predictors of death from thin (≤1.0 mm) melanoma

Dear Editor,

Despite overall favourable prognosis,¹ thin (≤1.0 mm) cutaneous melanoma account for 23% of melanoma deaths in the high-risk general population of Queensland, Australia (2005–2009), because of the sheer volume of disease.² The prospect of adjuvant systemic therapy has increased the focus on identifying patients with thin melanoma who are at high risk of death, such as through gene expression profiling (GEP), sentinel lymph node biopsy (SLNB), or prognostic models.^3–5

In an earlier study⁶ on single invasive thin (≤1.0 mm) melanoma (n=27,660), we analysed data from the population-based Queensland Cancer Register from 1995 to 2014 and found that scalp location of the primary tumour and 0.8–1.0 mm thickness were strong predictors of melanoma-specific death, the latter in concordance with the current staging guidelines.¹

Extending these findings, we examined how these two simple predictors would perform in identifying death within five years from thin melanoma in a prognostic model. We analysed the same cohort, but this time only included people who had either died from their thin melanoma within five years (n=228), or survived at least five years post-diagnosis (n=17,148). Using a flexible parametric survival model with cubic splines (3 df), we constructed survival models including combinations of the covariates of scalp location (Yes/No), 0.8–1.0 mm thickness (Yes/No) and ulceration (Yes/No). We used Liu’s method to find the optimal probability cut-point (melanoma death) for our scalp+thickness model: 0.009, corresponding to an AUC of 0.69 (95% CI 0.66–0.72) (Table 1). This cut-point meant that patients with a melanoma located on the scalp, of 0.8–1.0 mm thickness, or both, were categorised as “predicted melanoma death”. At the cut-point, the model correctly identified 118 of 228 deaths within five years of diagnosis among the whole cohort (sensitivity 52%), but incorrectly predicted 2,408 “melanoma deaths” (positive predictive value [PPV] 5%) among the 2,526 people with a melanoma on the scalp and/or of 0.8–1.0 mm thickness (Table 1). Also, the model correctly classified 14,740 of the 17,148 survivors among the whole cohort (specificity 86%) but failed to predict 110 deaths (negative predictive value [NPV] 99%) among the 14,850 people with neither scalp location nor 0.8–1.0 mm thickness. The 5-year melanoma-specific case fatality of patients predicted as melanoma deaths was 4.7% and the case fatality of those predicted as being alive was 0.7%. Further, we generated additional models including each covariate singly. To emulate the present staging guidelines, we also generated two separate models including ulceration; nevertheless, ulceration status did not improve the prediction. Then, we tested the predictive ability of our model after 10-years of follow-up, finding that the AUC was similar (0.66) to that of the 5-year model, irrespective of the inclusion of ulceration (data not shown). Similarly, restricting the study period to before 2010, prior to the use of advanced therapies for metastatic melanoma, did not change the predictive ability of the highlighted clinical parameters.

Table 1:

Predicted¹ and observed melanoma deaths after 5-years of follow-up among people diagnosed with thin (≤1.0 mm) melanoma in Queensland, Australia, 1995 to 2014

			Covariates in prognostic model
Predicted (n)	Observed (n)	Classification (n)	Scalp location	Thickness 0.8–1.0 mm	Scalp location and/or thickness 0.8–1.0 mm	Ulceration and/or thickness 0.8–1.0 mm	Ulceration and/or scalp location and/or thickness 0.8–1.0 mm
Melanoma death	Melanoma death	Correct	16	109	118	115	123
Alive	Melanoma death	Incorrect	212	119	110	113	105
Melanoma death	Alive	Incorrect	173	2,258	2,408	2,468	2,616
Alive	Alive	Correct	16,975	14,890	14,740	14,680	14,532
Quality of model (%)
Sensitivity			7.0 (4.1–11.1)	47.8 (41.2–54.5)	51.8 (45.1–58.4)	50.4 (43.8–57.1)	53.9 (47.2–60.5)
Specificity			99.0 (98.8–99.1)	86.8 (86.3–87.3)	86.0 (85.4–86.5)	85.6 (85.1–86.1)	84.7 (84.2–85.3)
Positive predictive value			8.5 (4.9–13.4)	4.6 (3.8–5.5)	4.7 (3.9–5.6)	4.5 (3.7–5.3)	4.5 (3.8–5.3)
Negative predictive value			98.8 (98.6–98.9)	99.2 (99.1–99.3)	99.3 (99.1–99.4)	99.2 (99.1–99.4)	99.3 (99.1– 99.4)
AUC			0.53 (0.51–0.55)	0.67 (0.64–0.71)	0.69 (0.66–0.72)	0.68 (0.65– 0.71)	0.69 (0.66–0.73)

¹Predicted from a flexible parametric survival model including combinations of the covariates scalp location (Yes/No), 0.8–1.0 mm thickness (Yes/No), and ulceration (Yes/No). The analysis was also performed with a 10-year follow-up, with similar results. Full details available from corresponding author.

Model performance should be interpreted in the context of other prognostic tests. SLNB is a robust prognostic factor for melanoma >1mm thickness but only considered for T1 melanoma with high-risk features due to cost, morbidity, and low positivity rates.¹ Among T1 patients selected for SLNB, sensitivity and specificity for melanoma death after five years is approximately 18% and 94%, respectively⁴. However, these data only inform on the predictive ability of SLNB for the patients who received the procedure. Further, it has been argued that SLNB does not offer better prognostic information for T1 patients than thickness alone.⁷ More recently, GEP has been applied to the prediction of melanoma outcomes.³ Among 281 T1 patients with disease recurrence or 5-years of follow-up, however, only 1 melanoma death was reported, limiting appraisal of its performance. The sensitivity and specificity of 31-GEP for distant metastasis after 5-years of follow-up were estimated to be 21% and 90%, respectively.^3,8

Given that patients with thin melanoma experience 96% 10-year survival rates¹, a prognostic test or model would need to have excellent performance to be used for identification of candidates for adjuvant systemic therapy. Here we show that two readily available clinicopathologic factors, namely scalp location and 0.8–1.0 mm thickness, identify a group of patients at higher than average risk of death with a sensitivity that exceeds SLNB and 31-GEP testing. These variables were also found to be robust prognostic factors in a US population-based SEER-based classification tree of ≤1.0 mm melanoma after 10-years of follow-up, supporting their generalizability.⁵ Ultimately, a prognostic model that provides an individualized absolute risk of melanoma outcomes is likely to be most informative to clinicians and patients for medical decision-making. Additional research is needed to understand how to best integrate costly and/or potentially hazardous tests to the prediction of thin melanoma outcomes. At a minimum, however, they should clearly be shown to provide a net benefit beyond clinicopathologic factors.