Machine learning models for outcome prediction of Chinese uveal melanoma patients: A 15‐year follow‐up study

Abbreviations

AUC

Area Under Curve

LogMAR

Logarithm of the Minimum Angle of Resolution

UM

Uveal Melanoma

Dear Editor

Uveal melanoma (UM) is the most frequent primary malignant intraocular tumor in adults with an estimated incidence of 4‐5 per million per year in western countries [1]. About 50% of UM patients eventually develop metastasis. In a previous study, the prognosis of Chinese UM was mainly correlated with visual clinical features and gene sequencing results. Models designed to predict UM prognosis have been previously described [2], but these studies were based on Caucasians, not Chinese. Further, our previous study [3] determined that the clinical characteristics were different between Chinese and Caucasian UM patients. Therefore, in this study, we analyzed the clinical characteristics and survival status of 1553 patients diagnosed with UM over a period of 15 years in China, and also included factors that were not included in our previous study, such as the largest basal diameter, thickness, tumor size after updating the criteria, pigmentation, whether complicated with intraocular hemorrhage, ciliary body involvement, extraocular extension and TNM stage. We also constructed comprehensive prognostic models to predict the risk of metastasis within 2 years and death of UM after 2 years following treatment according to clinical characteristics using machine learning. The methods of this study are described in the Supplementary File.

The clinical data of the UM patients are shown in Supplementary Table S1. The mean age of subjects was 47.2 ± 12.5 years (median, 48.0 years; range, 5‐85 years), indicating that the onset age is much younger than Caucasians [4] for whom it usually occurs in the fifth to sixth decades. This demonstrates one of the racial variations of the disease. Nearly 70% of patients’ visual acuity was worse than the 0.30 logarithm of the minimum angle of resolution (LogMAR). Intraocular pressure was within the normal range of 10‐21 mmHg in 1279 (85.6%) patients, while it was lower than 10 mmHg in 148 (9.9%) patients and higher than 21 mmHg in 67 (4.5%) patients. The mean largest basal diameter was 12.0 ± 3.6 mm (median, 11.7 mm; range, 1.1‐23.0 mm), and the mean tumor thickness was 7.1 ± 3.3 mm (median, 6.9 mm; range, 0.4‐23.0 mm). According to the criteria of the American Joint Committee on Cancer classification (7^th edition), there were 198 (12.9%) patients at Stage I, 1089 (71.1%) at Stage II, and 243 (15.8%) at Stage III. It was reported that the mixed cell‐type tumor is the most common UM [5], but in the present study, the spindle cell‐type UM was the most common pathological type (44.6%).

The mean follow‐up time was 49.5 ± 32.5 months (median, 43 months; range, 0‐197 months). We followed 78.4% of patients for more than 2 years, 32.5% for more than 5 years and 3.2% for more than 10 years. The estimated 5‐, 10‐, and 15‐year overall survival rates after treatment initiation were 84.0%, 72.0%, and 66.7% (Figure 1A). Shields et al [4] published a cohort study of 8100 UM patients, demonstrating UM‐related survival rates of 91.4%, 86.5% and 83.8% and metastasis rates of 15%, 25% and 31%. Our 5‐, 10‐ and 15‐year UM‐related survival rates were 85.6%, 75.5% and 69.5%, and were thus lower than Shields et al results [4]. However, the metastasis rates in our study were higher at 18.9%, 27.2% and 31.4% (Figure 1B). Similarly, our UM‐related survival rates were lower than those reported in Scotland (92.3%, 87.4% and 83.8%) [6]. A total of 183 patients in our study died of UM metastasis, and the median duration from metastasis to death was 7 months. 75.5% of these patients died within 1 year of metastasis. A previous study showed that 80% of UM patients died within 1 year of metastasis and 92% within 2 years [7]. The most common site of metastasis observed was the liver (Supplementary Table S2). Multiple‐site metastasis occurred in 70 patients.

FIGURE 1.

Survival analysis and predicting models of 1553 UM patients. A. Kaplan‐Meier curve of overall survival. B. Kaplan‐Meier curve of metastasis. C. ROC curve of model for predicting death after 2 years of treatment (UMDeath model). D. ROC curve of model for predicting metastasis within 2 years of treatment (UMMetastasis model). Abbreviations: UM, Uveal Melanoma; AUC, Area Under Curve; ROC, Receiver Operating Characteristic

We assessed the variables with significant differences in univariate analysis for Cox multivariate regression analysis (Supplementary Table S3). There were few patients with known pathological types (n = 386), including spindle cell‐type (n = 172), mixed cell‐type (n = 131), and epithelioid cell‐type (n = 83). Thus, pathological information was not included in the multivariate analysis. Using the log‐rank test, we found that the prognosis of patients with epithelioid cell‐type UM was significantly worse than spindle cell‐type UM (Bonferroni corrected P < 0.001); which was consistent with another study [8]. Cox multivariate regression analysis of some other possible prognostic factors showed that older age, larger basal diameter and presence of subretinal fluid were associated with high mortality and metastasis, while hemisphere tumor was associated with better prognosis (Supplementary Figures S1 and S2). In addition, tumor pigmentation and position were only related to death (Supplementary Figure S1).

We used random forest [9, 10] to construct two models (UMDeath and UMMetastasis): whether a patient will survive for more than 2 years after treatment and whether the tumor will metastasize within 2 years of treatment using demographic attributes, general ocular features and tumor‐specific features, which are listed in Supplementary Table S3. The imbalanced ratio (the ratio between the numbers of majority samples and minority samples) was different when we chose a different censored time, and the ratio was abnormal when the censored time was 3 years. Thus, we chose to predict the prognosis of patients within 2 years. Moreover, four‐fold cross‐validation was used to fairly assess the performance of random forest, where the training dataset and the testing dataset are subject‐independent (each sample was collected from one patient and all samples were split into training and testing datasets in cross‐validation, therefore, one sample were used only once in the training or testing dataset). Also, 0.5 was selected as the threshold for both classification problems, on which all statistical results were based. Finally, we applied genetic feature selection to investigate which features were more related to the two classification problems (whether a patient would die after 2 years or have metastasis within 2 years after treatment) (Supplementary Material). When the model for the prediction of death was established, we found that the largest basal diameter, thickness, size, intraocular pressure and initial treatment were related to death. The boxplots show the results of four‐fold cross‐validation before and after feature selection (Supplementary Figure S3A and S3B). The sensitivity, specificity, accuracy, area under the curve (AUC) and their 95% confidence intervals are shown in Supplementary Table S4. PRiMeUM [2], a model for predicting the risk of metastatic UM, used the clinical characteristics and chromosomal information to determine the risk of metastasis within 2 years after the primary treatment; and showed that the accuracy of risk prediction could reach 83% using only clinical characteristics.

Moreover, we established a model for predicting metastasis within 2 years after treatment. It was found that the largest basal diameter, size, stage, age, intraocular pressure and gender were associated with metastasis. The boxplots show four‐fold cross‐validation results before and after feature selection (Supplementary Figure S4A and S4B). The sensitivity, specificity, accuracy, AUC and their 95% confidence intervals are shown in Supplementary Table S5. Figures 1C and 1D, Supplementary Figures S3C and S4C depict the receiver operating characteristic and precision‐recall curves of one‐fold out of four‐fold cross‐validation for predicting death and metastasis, respectively. Two predictive models each found that intraocular pressure was an influencing factor for UM metastasis and death, and intraocular pressure and prognosis were related to tumor volume. The larger the tumor, the greater the impact on the eyeball structure and physiological conditions, and thus is more likely to cause changes in intraocular pressure.

This present study is currently the largest retrospective study on UM in China. The age of onset for Chinese UM patients was earlier and the prognosis was worse than Western patients. The largest basal diameter and age were found to be related to prognosis in both statistical results and predictive models. Our predictive models found that intraocular pressure was related to both death and metastasis. These models achieved a high sensitivity and specificity for predicting survival and can predict metastasis to a certain extent.

FUNDING

Supported by The Capital Health Research and Development of Special (2020‐1‐2052), Science & Technology Project of Beijing Municipal Science & Technology Commission (Z201100005520045, Z181100001818003), The Beijing Municipal Administration of Hospitals’ Ascent Plan (DFL20150201), The National Natural Science Foundation of China (82101180), Beijing Natural Science Foundation (7204245), Scientific Research Common Program of Beijing Municipal Commission of Education (KM202010025018), Beijing Municipal Administration of Hospitals’ Youth Programme (QMS20190203), Beijing Dongcheng District Outstanding Talents Cultivating Plan(2018).

DECLARATIONS

CONFLICT OF INTEREST

No conflicting relationship exists for any author.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

This research adhered to the tenets of the Declaration of Helsinki and was approved by the Institutional Review Board of Beijing Tongren Hospital. We obtained informed consent from all of these patients.

AUTHORS’ CONTRIBUTION

YML, YL, and WBW contributed to the study conception. YNC, YNW and MXC designed the current study. HHZ, YNH, YF,YJL and JTL collected the data and implemented the follow‐up. KZ, YNC and YNW performed the analyses and verified the underlying data. RTC, RF and HW arranged and checked the consistency of results. YNC and YNW wrote the initial draft of the manuscript. WBW and KZ commented on the manuscript and gave final approval of the version to be published.

CONSENT FOR PUBLICATION

Written informed consent was obtained from the patients for participation in the study and publication of this article. A copy of the written consent is available for review by the Editor‐in‐Chief of this journal.

DATA AVAILABILITY STATEMENT

Correspondence and requests for materials and code should be addressed to WBW (weiwenbin@mail.ccmu.edu.cn), YL (liyang_8151@126.com) or YML (liuyueming2005@163.com).

Supporting information

Supporting information