Development and Validation of Time-to-Event Models to Predict Metastatic Recurrence of Localized Cutaneous Melanoma

Abstract

Background:

The recent expansion of immunotherapy for stage IIB/IIC melanoma highlights a growing clinical need to identify patients at high risk of metastatic recurrence and, therefore, most likely to benefit from this therapeutic modality.

Objective:

To develop time-to-event risk prediction models for melanoma metastatic recurrence.

Methods:

Patients diagnosed with stage I/II primary cutaneous melanoma between 2000 and 2020 at Mass General Brigham and Dana-Farber Cancer Institute were included. Melanoma recurrence date and type were determined by chart review. Thirty clinicopathologic factors were extracted from electronic health records. Three types of time-to-event machine-learning models were evaluated internally and externally in the distant versus locoregional/non-recurrence prediction.

Results:

This study included 954 melanomas (155 distant, 163 locoregional, and 636 1:2 matched non-recurrences). Distant recurrences were associated with worse survival compared to locoregional/non-recurrences (HR: 6.21, p<0.001) and to locoregional recurrences only (HR: 5.79, p<0.001). The Gradient Boosting Survival model achieved the best performance (concordance index: 0.816; time-dependent AUC: 0.842; Brier score: 0.103) in the external validation.

Limitations:

Retrospective nature and cohort from one geography.

Conclusions:

These results suggest that time-to-event machine-learning models can reliably predict the metastatic recurrence from localized melanoma and help identify high-risk patients who are most likely to benefit from immunotherapy.

Capsule Summary

• Clinicopathologic factors can be used to identify localized melanomas at high risk of metastatic recurrence and select patients who are most likely to benefit from adjuvant immunotherapy.

• Time-to-event models achieved a concordance index 0.816 and a time-dependent area under the receiver operating characteristic curve 0.842 in the metastatic recurrence prediction.

Introduction

Dramatic improvements in cancer cell biology and immunology are constantly pushing forward therapeutic options, and treatment strategies for managing melanoma have continued to evolve over the past decade. Notably, pembrolizumab was approved in December 2021 as adjuvant therapy for patients with high-risk node-negative melanoma (stage IIB/IIC) following complete resection of the primary tumor.¹ However, though there are established surveillance guidelines for advanced (stage III/IV) melanoma, the management of patients with stage IIB/IIC disease is largely up to the physician’s discretion, as there is limited evidence supporting specific recommendations. The decision to proceed with adjuvant immunotherapy is based on the recurrence risk, treatment-related toxicities, and patient preferences. This highlights a significant opportunity for developing decision-making tools to stratify patients with localized melanoma (stage I/II) according to their risk for recurrence and mortality.

Previous studies have demonstrated that typically up to 30% of patients with localized melanoma experience a disease recurrence.^2,3 Adjuvant immunotherapy, if utilized in the appropriate population, may prevent recurrence and improve patient outcomes. However, this treatment class is associated with significant and potentially permanent toxicities and should therefore be targeted toward patients with the highest risk of recurrence.^4,5,6,7,8 Furthermore, methods to identify patients at the highest risk of recurrence are lacking, underscoring the importance of re-evaluating current management and surveillance patterns to identify those whose risk profile favors systemic therapy. Specifically, prior studies have demonstrated that routine imaging detects only 21% of recurrent cases in patients with stage II disease.² Besides, distant metastases following surgical resection of melanoma often portend one of the worst prognoses compared to locoregional recurrences.⁹ Also, distant recurrence and locoregional recurrence require different treatment strategies. For example, patients with distant recurrence may need systemic therapies, while patients with locoregional recurrence may benefit from local treatments, such as re-excision and intralesional therapy.¹⁰ Thus, a greater understanding of distant recurrence from localized melanoma is increasingly important.

This study is built on a recent publication from our research group, which successfully developed risk prediction models for any type of recurrence from early-stage melanoma.¹¹ The goal of the current study is to specifically identify patients at the highest risk of distant recurrence, who are therefore most likely to benefit from adjuvant therapy. Since the timing of the recurrence is an important criterion in evaluating the most appropriate time for systemic intervention, we specifically focus on the methodological approach to time-to-event prediction. Compared to prior studies of melanoma recurrence that relied on conventional statistical models or concentrated on the binary classification of recurrent versus non-recurrent melanomas^12,13, our time-to-event machine-learning approach will enable improved representation of the complexity of disease progression and the reality of clinical practice. In this study, we apply time-to-event algorithms from three categories including linear, ensemble, and deep learning-based models, and compare their performance in the distant recurrence risk prediction.

Methods

We developed time-to-event machine-learning models to predict distant recurrence versus locoregional/non-recurrence of localized melanomas. We also examined model performances in the prediction of distant versus locoregional recurrence and the prediction of distant recurrence versus non-recurrence. We additionally predicted distant, locoregional, and non-recurrence together with competing risk modeling.

Data Collection

This study included stage I/II melanomas at the Massachusetts General Hospital (MGH) and Brigham and Women’s Hospital/Dana-Farber Cancer Institute (DFCI) between January 2000 and February 2020. Figure 1 shows the flow diagram of how the study population was obtained. All recurrent melanomas were first identified, then non-recurrent melanomas were 2:1 best matched with recurrent melanomas on the primary melanoma diagnosis year using the “matchControls” function in R 4.1.0.¹⁴ eMethod 1 describes the inclusion criteria.

Figure 1. Study Design Flowchart.

Stage I and stage II cutaneous melanomas with no evidence of nodal or distant metastasis at the time of primary diagnosis were included. Thus, melanomas with positive sentinel lymph node biopsy and melanomas with microscopic satellites were excluded. Acral, mucosal, uveal, and desmoplastic melanomas were excluded. Melanomas without a pathology report available in electronic health records were excluded. All recurrent melanomas were first identified, then non-recurrent melanomas were 2:1 best matched with recurrent melanomas on diagnosis year of primary melanoma.

A manual review of electronic health records (EHRs) was conducted to ascertain the recurrence status (locoregional, distant, non-recurrence) and date. Melanomas that were stage IV at the time of recurrence, based on the American Joint Committee on Cancer 8^th edition (AJCC-8) staging guidelines, were labeled as “distant recurrence.”¹⁵ All recurrent melanomas without distant metastases were labeled as “locoregional recurrence.” All melanomas without any recurrence were labeled as “non-recurrence.” The time-to-event was defined as the duration from primary melanoma diagnosis to the first recurrence if the melanoma recurred; otherwise, the duration to the date of death or last follow-up.

We extracted variables of interest (eTable 1) from two institutional clinical databases: Research Patient Data Registry¹⁶ and Enterprise Data Warehouse¹⁷. eMethod 2 details information extraction. Median household income was extracted from the U.S. Census data using the patient’s zip code.¹⁸ ICD codes were used to calculate the Charlson Comorbidity score (CCS)¹⁹. Based on clinical guidelines, sentinel lymph node biopsy (SLNB) is recommended for melanomas with Breslow thickness >1.00 mm and not recommended for primary melanomas that are <0.8 mm thick and non-ulcerated lesions.²⁰ Primary melanomas that are 0.8 to 1.0 mm thick or are <0.8 mm thick and ulcerated may be offered SLNB after a clinical discussion of the procedure’s advantages and disadvantages.²⁰ We incorporated the SLNB complexity to reflect real-world clinical settings.

Time-to-Event Machine Learning Methods

We compared prediction performances of three types of supervised time-to-event machine-learning algorithms (eMethod 3): ensemble models, including GradientBoostingSurvivalAnalysis (GBS)²¹ and RandomSurvivalForest (RSF)²², linear models, including CoxnetSurvivalAnalysis (Coxnet)²³ and CoxPHSurvivalAnalysis (CoxPH)²⁴, and deep learning models, including DeepSurv²⁵ and CoxTime.²⁶ The DeepHit²⁷ was applied for competing risk modeling. Models were evaluated internally and externally by concordance index,²⁸ time-dependent area under the receiver operating characteristic curve (AUC),²⁹ and Brier score³⁰ using: 1. five-fold cross-validation of the entire cohort (internal validation); 2. the MGH cohort for model development and the DFCI cohort for validating independently (external validation). Each experiment was repeated 50 times. Mean and 95% confidence interval (CI) were reported. Experiments were implemented by using scikit-survival 0.18.0³¹ and pycox 0.2.3.³²

eFigure 1 presents the model development and validation pipeline. In Phase 0, we examined model performances using all extracted features. In Phase 1, we conducted feature selection using the MGH cohort. We first investigated the predictive features by conducting permutation importance.³³ Given the multicollinearity of features, we performed hierarchical clustering on features’ Spearman rank-order correlations, and a single feature in each highly correlated cluster was retained. In Phase 2, models were evaluated using the selected features. We also ranked the selected features by conducting permutation importance.

Statistical Analyses

We conducted Pearson’s Chi-squared test for categorical variables and t-test for continuous variables to compare groups. Kaplan-Meier curves for overall survival and distant recurrence free probabilities were utilized. Guarantee-time bias occurs when the outcome of interest competes with mortality (e.g., patients must be alive long enough to develop a recurrence). To account for this potential bias, time-varying Cox proportional hazards regression models were used to compute Hazard Ratios (HRs).³⁴ Statistical analyses were conducted using R 4.1.0.¹⁴

Results

Participant Characteristics

Among the 954 stage I/II primary cutaneous melanomas, 155 melanomas recurred distantly, and 163 melanomas recurred locoregionally. The characteristics of the study population are described in Table 1 and eTable 3 (all variables). The comparison between distant and locoregional recurrences is presented in eTable 4.

Table 1.

Characteristics of the study population.

	Locoregional/Non-Recurrence (N=799)	Distant Recurrence (N=155)	P-value
Institution
DFCI	235 (29.4%)	53 (34.2%)	0.275
MGH	564 (70.6%)	102 (65.8%)
Duration of follow-up (year)
Median [IQR]	6.9 [2.8, 11.2]	5.7 [3.3, 8.9]	<0.001
Mortality status
Alive	599 (75.0%)	60 (38.7%)	<0.001
Dead	200 (25.0%)	95 (61.3%)
Age at diagnosis (year)
Median [IQR]	62 [51, 72]	63 [54, 72]	0.105
Sex
Female	362 (45.3%)	47 (30.3%)	<0.001
Male	437 (54.7%)	108 (69.7%)
Race
White	789 (98.7%)	154 (99.4%)	0.813
Unavailable/Other	10 (1.3%)	1 (0.6%)
Ethnicity
Non-Hispanic	786 (98.4%)	154 (99.4%)	0.572
Unavailable/Other	13 (1.6%)	1 (0.6%)
Histology type
Lentigo maligna melanoma	55 (6.9%)	14 (9.0%)	<0.001
Nodular melanoma	103 (12.9%)	36 (23.2%)
Superficial spreading melanoma	497 (62.2%)	62 (40.0%)
Melanoma, not otherwise specified	144 (18.0%)	43 (27.7%)
Tumor site
Skin of face	89 (11.1%)	24 (15.5%)	<0.001
Skin of lower limb and hip	167 (20.9%)	18 (11.6%)
Skin of scalp and neck	49 (6.1%)	35 (22.6%)
Skin of trunk	294 (36.8%)	54 (34.8%)
Skin of upper limb and shoulder	200 (25.0%)	24 (15.5%)
AJCC stage
1A	429 (53.7%)	19 (12.3%)	<0.001
1B	213 (26.7%)	45 (29.0%)
2A	64 (8.0%)	29 (18.7%)
2B	55 (6.9%)	30 (19.4%)
2C	38 (4.8%)	32 (20.6%)
Breslow thickness (mm)
Median [IQR]	0.7 [0.4, 1.4]	2.1 [1.1, 4.2]	<0.001
Anatomic (Clark’s) level
Median [IQR]	4 [2, 4]	4 [4, 4]	<0.001
Laterality
Left	363 (45.4%)	73 (47.1%)	0.113
Midline	79 (9.9%)	23 (14.8%)
Right	357 (44.7%)	59 (38.1%)
Sentinel lymph node biopsy
Not indicated	419 (52.4%)	19 (12.3%)	<0.001
All nodes negative	312 (39.0%)	102 (65.8%)
Not performed: unknown reason	17 (2.1%)	5 (3.2%)
Not performed: due to age/comorbidity	41 (5.1%)	27 (17.4%)
Not performed: deferred by patient	10 (1.3%)	2 (1.3%)
Ulceration
Absent	701 (87.7%)	101 (65.2%)	<0.001
Present	94 (11.8%)	50 (32.3%)
Unavailable	4 (0.5%)	4 (2.6%)
Mitotic rate (mitoses/mm2)
Median [IQR]	1 [0, 3]	4 [1, 12]	<0.001
Total surgical margins (cm)
Median [IQR]	1 [1, 2]	2 [1, 2]	<0.001
Tumor infiltrating lymphocytes (TIL)
Present: brisk	56 (7.0%)	7 (4.5%)	0.044
Present: non-brisk	400 (50.0%)	98 (63.2%)
Absent	160 (20.0%)	25 (16.1%)
Unavailable	183 (22.9%)	25 (16.1%)

When comparing the distant recurrence group to the combined locoregional/non-recurrence group, there were no significant differences in age at diagnosis, race, ethnicity, marital status, CCS, history of previous melanoma (HOM), laterality, presence of regression, and perineural invasion. The percentage of males in the distant recurrence group was higher (70% versus 55%, p<0.001). The median income in the distant recurrence group was lower (95,000 versus 101,000 dollars, p=0.04). More patients with stage II melanoma were in the distant recurrence group (59% versus 20%, p<0.001). Figure 2.C presents the Kaplan-Meier curves of distant recurrence free probability stratified by AJCC-8 stage among the study population.

Figure 2. Kaplan-Meier curves and the distribution of recurrent melanomas.

A. A Kaplan-Meier curve of overall survival comparing the distant recurrence group and the locoregional/non-recurrence group. The Hazard Ratio for overall survival was computed using a univariate time-varying Cox Proportional Hazards model.

B. A Kaplan-Meier curve of overall survival comparing the distant recurrence group and the locoregional recurrence group. The Hazard Ratio for overall survival was computed using a univariate time-varying Cox Proportional Hazards model.

C. A Kaplan-Meier curve for distant recurrence stratified by AJCC stage.

D. The number of distant recurrences within a period from diagnosis compared to the number of locoregional recurrences.

When comparing the distant recurrence group to the locoregional recurrence group (eTable 4), there were no significant differences in all tumor characteristics except tumor site, laterality, and sentinel lymph node biopsy status. The time from primary diagnosis to distant recurrence was longer than the time to locoregional recurrence (2.6 versus 1.6 years, p=0.003). The histogram of recurrent melanomas is presented in Figure 2.D. Within two years, there were fewer distant recurrences than locoregional recurrences (40% versus 64%, p<0.001).

The median follow-up was 5.7 years (IQR: 3.3–8.9) for distant recurrences, 6.1 years (IQR: 3.2–10.8) for locoregional recurrences, and 7.0 years (IQR: 2.7–11.2) for non-recurrences (eTable 4). The distant recurrence group had the highest mortality rate compared to the locoregional recurrence and the non-recurrence groups (61.3% versus 38.7% versus 21.5%, p<0.001). The Kaplan-Meier curves for overall survival are presented in Figure 2.A and Figure 2.B. The Hazard Ratio of overall mortality for the distant recurrence group was 6.2 (95% CI: 4.8–8.0, p<0.001) when compared to the combined locoregional/non-recurrence group and was 5.8 (95% CI: 4.2–8.1) when compared to the locoregional recurrence group.

Performance of Prediction Models

We built time-to-event models for distant recurrence risk prediction using ensemble, linear, and deep-learning algorithms. We evaluated the model performance first by experimenting on all extracted variables and then on the 27 selected variables (see below for details). The results are presented in Table 2.

Table 2.

Time-to-event distant recurrence versus locoregional/non-recurrence prediction.

		Ensemble models		Linear models		Deep learning models
		RSF	GBS	CoxPH	CoxNet	DeepSurv	CoxTime
All extracted variables	Concordance index a
	Internal (Mean and 95% CI)	0.836 0.830–0.841	0.844 0.839–0.850	0.811 0.801–0.822	0.805 0.793–0.816	0.811 0.799–0.822	0.820 0.811–0.829
	External (Mean and 95% CI)	0.798 0.797–0.799	0.811 0.811–0.812	0.711 0.710–0.712	0.710 0.709–0.711	0.738 0.728–0.748	0.749 0.743–0.755
	P-value b	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
	Time-dependent AUC a
	Internal (Mean and 95% CI)	0.867 0.861–0.873	0.877 0.869–0.885	0.845 0.833–0.857	0.846 0.837–0.856	0.852 0.839–0.865	0.860 0.850–0.869
	External (Mean and 95% CI)	0.811 0.810–0.812	0.837 0.836–0.838	0.738 0.737–0.739	0.737 0.736–0.738	0.770 0.760–0.780	0.779 0.773–0.785
	P-value b	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
	Integrated time-dependent Brier score a
	Internal (Mean and 95% CI)	0.097 0.094–0.101	0.109 0.102–0.116	0.108 0.101–0.115	0.109 0.102–0.116	0.109 0.103–0.115	0.108 0.103–0.113
	External (Mean and 95% CI)	0.107 0.106–0.107	0.103 0.103–0.104	0.126 0.126–0.127	0.127 0.126–0.127	0.127 0.125–0.129	0.126 0.123–0.129
	P-value b	<0.001	0.087	<0.001	<0.001	<0.001	<0.001
27 selected variables	Concordance index a
	External (Mean and 95% CI)	0.810 0.809–0.811	0.816 0.815–0.816	0.774 0.773–0.774	0.777 0.776–0.777	0.767 0.756–0.778	0.772 0.764–0.78
	P-value c	<0.001	<0.001	<0.001	<0.001	<0.001	<0.001
	Time-dependent AUC a
	External (Mean and 95% CI)	0.822 0.821–0.823	0.842 0.842–0.843	0.792 0.791–0.793	0.796 0.796–0.797	0.784 0.772–0.795	0.786 0.777–0.795
	P-value c	<0.001	<0.001	<0.001	<0.001	0.079	0.197
	Integrated time-dependent Brier score a
	External (Mean and 95% CI)	0.105 0.105–0.106	0.103 0.103–0.104	0.112 0.111–0.112	0.110 0.108–0.112	0.126 0.123–0.128	0.129 0.125–0.133
	P-value c	<0.001	0.823	<0.001	<0.001	0.446	0.087

^aWhen the concordance index and time-dependent AUC tend to 1, the model is more accurate. When the Brier score tends to 0, the model is more accurate.

^bP-value: t-test between the internal validation and the external validation.

^cP-value: t-test of the external validations when all extracted variables were used and when 27 selected variables were used.

The best performances when all extracted variables were used are highlighted in blue, and the best performances when 27 selected variables were used are highlighted in green.

For experiments on all extracted variables, the ensemble models (GBS and RSF) achieved better performance than the linear and deep-learning models (p<0.001). In the internal validation, GBS and RSF models achieved similar performance (concordance index: 0.844 versus 0.836, p=0.026; time-dependent AUC: 0.877 versus 0.867, p=0.057). In the external validation, GBS outperformed RSF (concordance index: 0.811 versus 0.798, p<0.001; time-dependent AUC: 0.837 versus 0.811, p<0.001).

We selected features by ranking all extracted features based on permutation importance (eFigures 2-4) and conducting correlation analysis (eFigure 5). Breslow thickness and mitotic rate were the two most important features in the GBS and RSF models (eFigure 2). AJCC stage, mitotic rate, and vertical growth phase type (VGT) appeared to be important in the linear models (eFigure 3). Breslow thickness, health insurance type, and total surgical margins were the most important features in the deep-learning models (eFigure 4). Other important features included SLNB, ulceration, melanoma histology type, age at diagnosis, tumor site, first visit due to melanoma (FVM), HOM, patient sex, and laterality. We initially selected 30 one-hot encoded features and conducted a correlation analysis. “Stage:1A”, “SLNB: Not indicated”, and “Laterality: Left” were removed since they were highly correlated with the other variables (eFigure 5). The final 27 selected features are specified in eTable 2.

When experimenting on the 27 selected features, all models achieved better performance in terms of concordance index in the external validation (p<0.001). Ensemble and linear models achieved better time-dependent AUC (p<0.001) compared to the results when all available features were included. The GBS models outperformed other models in the external validation (concordance index: 0.816; time-dependent AUC: 0.842, integrated time-dependent Brier score: 0.103). The results of the internal validation are presented in eTable 5.

Figure 3 displays the importance ranking of the selected features in each model. Breslow thickness, mitotic rate, SLNB:Age.comorbidity (inability to perform SLNB due to age or comorbidity of the patient), and Site:Scalp.neck were the common top five features in the ensemble models. Stage:2C, Stage:2B, Stage:2A, SLNB:Age.comorbidity, and Insurance:Self.Pay were the top five features in the linear models. Mitotic rate, Breslow thickness, Age at diagnosis, SLNB:Age.comorbidity, Site:Face, and Gender:Male played important roles in the deep-learning models.

Figure 3. Mean feature importance in the time-to-event distant recurrence versus locoregional/non-recurrence prediction on the MGH cohort.

First column (A and B): Ranking of the 27 selected features in the MGH cohort by the ensemble models (RSF and GBS, respectively).

Second column (C and D): Ranking of the 27 selected features in the MGH cohort by the linear models (CoxPH and Coxnet, respectively).

Third column (E and F): Ranking of the 27 selected features in the MGH cohort by the deep learning models (DeepSurv and CoxTime, respectively).

Secondary Analyses

We additionally evaluated model performances in the distant versus locoregional recurrence prediction (eTable 6) and the distant versus non-recurrence prediction (eTable 7). All models did not achieve satisfactory performance in the distant versus locoregional recurrence prediction. The best concordance index and time-dependent AUC in the external validation were less than 0.7 (eTable 6). The models achieved better concordance index and time-dependent AUC in the prediction of distant versus non-recurrence (eTable 7) compared to the results in the prediction of distant versus locoregional/non-recurrence (eTable 4), especially in the internal validation (p<0.01 for all models). For competing risk modeling, the model achieved concordance index of 0.767 (95% CI: 0.760–0.773) and Brier score of 1.49 (95% CI: 0.147–0.151) in the external validation.

Discussion

Overall, we found that models trained to predict distant recurrence versus locoregional/non-recurrence had better discriminative prediction than those trained to predict distant versus locoregional recurrence. While all models showed satisfactory performance, ensemble models had greater discriminating power (external: time-dependent AUC 0.822–0.842) than linear and deep-learning models (external: time-dependent AUC 0.784–0.796). Since patients with distant recurrence have significantly worse survival, these models can help identify patients who would benefit most from adjuvant immunotherapy. The enhanced performance of ensemble models is likely due to the combination of multiple diverse models used in the methodology and their ability to excel with a relatively modest volume of data.

In our models, Breslow thickness, mitotic rate, tumor site, and the reason for SLNB deferral were ranked as the most important features, which is largely consistent with the literature.^35,36 Similarly, our prior study found Breslow thickness and mitotic rate to be the most predictive features for overall primary melanoma recurrence.¹¹ Furthermore, another recent study demonstrated that the binary classification power of stage-related features alone has limitations and may lead to missed recurrent cases.¹³ Our findings, in conjunction with prior studies, highlight that there are features beyond those included in AJCC staging criteria that can help predict recurrence with improved accuracy to inform clinical recommendations and adjuvant therapy consideration in patients with early-stage melanoma.

Compared with our prior work¹¹, this study focuses on distant metastases and delineates between distant and locoregional recurrences, given the increased risk of mortality and potential benefits from adjuvant immunotherapy for those at risk of distant recurrence. In addition, we included a comprehensive set of time-to-event algorithms. This study underscores opportunities for more sophisticated risk-stratifying tools to assess a patient’s risk for recurrence to inform patient selection for adjuvant therapy. By identifying those at the highest risk of distant recurrence, prediction models can prevent overtreatment and optimize adjuvant therapy benefits. A recent trial showed that adjuvant treatment with pembrolizumab for up to approximately one year reduced the risk of distant metastases by two-fold in patients with stage IIB/IIC melanoma.³⁷ Patients often describe the possibility of recurrence as a psychological and emotional burden, as rapid disease progression can occur after a melanoma recurrence.³⁸ Management considerations, including prognosis, potential adverse events, treatment burden, and costs to the healthcare system and the individual, are complex.^39,40 Although the occurrence of grade 3 or worse immune-related adverse events is relatively uncommon, some patients experience life-long toxicities or toxicities that require long-term therapy, such as systemic corticosteroid or hormone replacement.³⁷ As our study illustrates, time-to-event machine-learning algorithms demonstrate the improved ability to capture the complexities of melanoma recurrence that are underrepresented in the current literature and likely hold the promise of future advancements in precision medicine.³⁹

Limitations of this study include the retrospective nature of clinical data extraction, where some variables were unavailable. Nonetheless, our machine-learning models performed satisfactorily in the external validation. Also, our models did not include immunosuppression information and were developed using a relatively small cohort of patients seen in an overall similar geography. Future studies should leverage time-to-event algorithms with immunosuppression information and larger generalized cohorts to guide clinically deployable prediction models. Furthermore, the incorporation of prognostic molecular markers will strengthen the discriminatory power and advance personalized treatment.

Code Availability

The code is available at https://github.com/SemenovLab/Time2Event-MelanomaRecurrence.

Data Availability

The data generated for this study can only be shared per specific institutional review board (IRB) requirements. Upon request to the corresponding author, a data sharing agreement can be initiated following institution-specific guidelines.