Malignant Transformation of Choroidal Indeterminate Melanocytic Tumors

This cohort study investigates if the predicted risk of malignant transformation of choroidal nevus is accurate.

Key Points

Question

Is the predicted risk of malignant transformation of choroidal nevus accurate?

Findings

In this cohort study including 123 participants, predicted values from the Collaborative Ocular Melanoma Study (COMS) model and the model from Wills Eye Hospital overlapped between melanoma and nevus. Both models overdiagnosed melanoma in 19% to 31% of cases; in addition, the diagnosis of choroidal nevus might have been incorrect in 21% to 36% of cases.

Meaning

Results suggest that these analyses raise questions regarding the accuracy of risk prediction of patients with indeterminate melanocytic tumors.

Abstract

Importance

The accuracy of the predicted risk of malignant transformation of a large choroidal nevus or indeterminate melanocytic tumor (IMT) is not known.

Objective

To estimate the risk of malignant transformation (predicted risk) in a cohort of patients with IMT of known outcomes (observed status; benign [large nevus] or malignant [small melanoma]).

Design, Setting, and Participants

This was a cohort study of patients from a single center. Patients diagnosed with IMTs that were benign (large nevus) or malignant (small melanoma) were included in the analysis. Those lesions classified as large nevus (benign; 0% risk) had documented stability over 24 or more months. IMTs classified as small melanoma (malignant; 100% risk) had quantified growth or confirmatory pathology. Data were analyzed from October to December 2024.

Exposures

Prediction of malignant transformation of a large choroidal nevus or IMT.

Main Outcomes and Measures

The primary outcome included the predicted 5-year Kaplan-Meier probability of malignant transformation using combinations of risk factors of predictive models, the Collaborative Ocular Melanoma Study (COMS) and Wills Eye Hospital (WEH) model.

Results

A total of 123 patients (median [IQR] age, 63 [56-67] years; 89 male [72%]), 62 with large nevus and 61 with small malignant melanoma, were included in this study. The mean predicted 5-year Kaplan-Meier probability of melanoma for observed melanoma was 0.39 (95% CI, 0.32-0.46) by the COMS model and 0.44 (95% CI, 0.39-0.49) by the WEH model. The difference of −0.05 (95% CI, −0.14 to 0.04) was not statistically significant. However, the mean predicted 5-year Kaplan-Meier probability of melanoma for observed nevus was 0.18 (95% CI, 0.12-0.23) by the COMS model and 0.31 (95% CI, 0.24-0.38) by the WEH model. The difference of −0.13 (95% CI, −0.22 to −0.05) was statistically significant. There was a significant difference in mean 5-year Kaplan-Meier probability of melanoma between observed melanoma and nevus of 0.21 (95% CI, 0.12-0.31) by the COMS model and 0.13 (95% CI, 0.05-0.21) by the WEH model. Optimal cut points of 0.18 and 0.34 for the COMS model and the WEH model, respectively, were identified using the Youden index. The sensitivity was lower for the COMS model than the WEH model (−15.2% difference; 95% CI, −25.6% to −4.8%), and the specificity was higher for the COMS model than the WEH model (11.7% difference; 95% CI, 2.0%-21.4%).

Conclusions and Relevance

Findings of this cohort study suggest that predicted risk for malignant transformation estimated by 2 different models based on combinations of risk factors was suboptimal and may lead to overtreatment in approximately 30% of patients. These findings support pursuing other methods for prediction that should be validated before use in clinical practice.

Introduction

The majority of tumors labeled as small choroidal melanoma within the Collaborative Ocular Melanoma Study (COMS) remained stable during observation with clinical behavior compatible with a diagnosis of a large choroidal nevus rather than a small melanoma.¹ As the tumors within the COMS size criteria (1.0-2.5 mm in height and 5.0-16.0 mm in their largest basal diameter [LBD])² include both nevi and melanoma, these tumors are best referred to as indeterminate choroidal melanocytic tumors (IMTs) with the need to differentiate large nevus from a small melanoma.^2,3 Over the years, extrinsic and intrinsic features have been identified as risk factors that are predictive of growth.^3,4,5,6,7,8 In general, presence of orange pigment and subretinal fluid (SRF) favor a diagnosis of a small choroidal melanoma,⁹ whereas drusen and retinal pigment epithelium (RPE) changes imply chronicity and are likely to indicate a benign lesion (such as nevus).²

It is worth clarifying certain attributes of these predictive risk factors. First, these risk factors are applicable to IMTs as previously defined and not to typical nevi (<5.0 mm in LBD and <1.0 mm in height), which have a low risk of malignant transformation (estimated to be 1 in 8845).¹⁰ Second, growth (increase in thickness or BD) observed over a period of 6 to 12 months,^1,11,12,13 particularly an increase of BD by 1 to 2 mm per year, is equivalent to the observed growth rate of biopsy-proven small choroidal melanoma.¹⁴ Such observable growth over months is distinct from minimal growth of choroidal nevi (0.015-0.03 mm per year) detected over decades and documented only by comparison of high-quality photographs.¹⁵ Hence, the presence of risk factors can also be considered as markers of melanoma.¹⁶ Third, time-to-event analysis indicates that the tumors growing in the first 2 years (early growth) may be melanoma in evolution, and in these tumors, risk factors for growth may be interpreted as the factors predictive of small choroidal melanoma.¹⁷

Two clinically used methods grade IMT by combinations of risk factors, one based on prospective COMS data of 188 patients (COMS model)⁶ and the other based on retrospective analysis of a large number of patients that in recent iterations have incorporated imaging studies (model from Wills Eye Hospital [WEH]).^18,19 These models assign a predicted risk of growth (malignant transformation) based on 5-year Kaplan-Meier (KM) probability of growth. To our knowledge, these 2 predictive models have not been externally validated or tested for accuracy.² We compared predicted risk with observed status of having a melanoma in a well-defined large cohort of patients with IMT with known outcomes as large nevus (benign, stable over a follow up of >24 months, 0% risk) or small melanoma (malignant, quantified growth or pathology, 100% risk).

Methods

The study protocol (17-397) was approved by Cleveland Clinic institutional review board (IRB). The Cleveland Clinic IRB determined that patient consent was not required as the study was based on review of medical records. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.

Test Dataset

All patients were evaluated using a standard slitlamp and fundus examination. Detailed fundus drawings depicting the entire extent of the lesion along with color fundus photography were performed for all the patients. The clinical records were reviewed for the following variables at the initial examination: patient age and sex, laterality, visual symptoms, presenting best-corrected visual acuity (BCVA), quadratic distribution (superotemporal, superonasal, inferotemporal, inferonasal, juxtapapillary, or macular), posterior tumor margin in relation to optic disc and foveola (<3 mm or ≥3 mm), and tumor dimensions. The LBD was estimated in millimeters by ophthalmoscopy, and the greatest tumor height in millimeters was measured by ultrasonography. Specific tumor features, such as the presence of SRF, surface orange pigment, drusen, and RPE atrophy, were also assessed by 90-diopter ophthalmoscopic examination and supplemented by ancillary studies such as optical coherence tomography and autofluorescence. The record of each patient was reviewed to establish if there was documented evidence of growth at any time during follow-up. Growth was judged by an increase in BD of at least 0.5 mm by meticulous comparison of serial fundus photographs or by an increase in thickness of 0.3 mm by serial ultrasonograms. The time interval between the initial examination and the documentation of tumor growth was recorded (in months).¹⁶ Data regarding race and ethnicity was not recorded as the vast majority of patients with choroidal nevus or melanoma tend to be White race.

Observed Risk

Because the definition of small choroidal melanoma in the COMS model was purely size based, IMTs considered to be benign were labeled as large nevi rather than suspicious nevi or suspicious melanomas.² The data included 123 patients with IMTs seen at a tertiary ophthalmology clinic between 2010 and 2018. IMTs were classified as small choroidal melanoma (n = 61) either by growth (growth-confirmed group, n = 30) or pathology (pathology-confirmed group, n = 19) or both (combined group, n = 12). Comparison of disease characteristics between growth-confirmed, pathology-confirmed, and combined groups showed similar distribution of age, sex, laterality, BD, tumor height, presence of orange pigment, and SRF among the 3 groups.¹⁴ The only difference noted was in the pathology group, wherein patients more frequently presented with symptoms (68% pathology vs 20% growth vs 42% combined; P = .004), had juxtapapillary location (<3 mm from optic disc; P = .03), and lacked features suggestive of chronicity such as drusen (11% pathology vs 60% growth vs 50% combined; P = .003) and RPE atrophy (11% pathology vs 23% growth vs 67% combined; P = .003). Hence, tumors in the pathology-confirmed group underwent prompt therapeutic intervention rather than observation for growth confirming that these tumors were indeed melanomas and not nevi.¹⁴

A total of 62 patients were considered to be free of melanoma (large choroidal nevus) after 24 months of stability under observation.¹⁶ The mean follow-up duration was 48 months (range, 26-84 months; median, 47 months).

In this study cohort, benign IMTs (large choroidal nevus) were assigned 0% observed risk of malignant transformation, and malignant IMTs (small melanoma) were assigned 100% observed risk of malignant transformation.

Predictive Models (Predicted Risk)

COMS Model

Risk factors predictive of choroidal nevus transformation into melanoma were identified for each tumor (tumor thickness >2.0 mm, LBD >12.1-16.0. mm, drusen, and orange pigment).⁶ Published predicted risk expressed as KM 5-year probability of malignant transformation was attributed to each tumor. Risk greater than 50% was observed with the combination of 2 risk factors: greater thickness and LBD, greater thickness and presence of orange pigment, and greater LBD and presence of orange pigment.⁶

WEH Model

Risk factors predictive of choroidal nevus transformation into melanoma were identified for each tumor (tumor thickness >2.0 mm, subretinal fluid, symptoms of visual acuity loss to 20/50 or worse, orange pigment, hollow acoustic density, and tumor LBD >5.0 mm).¹⁹ Published predicted risk expressed as KM 5-year probability of 1% of with no risk factors, 11% with 1 factor, 22% with 2 factors, 34% with 3 factors, 51% with 4 factors, and 55% with 5 factors was attributed to each tumor.¹⁹

Statistical Analysis

Patient and disease characteristics were summarized using the number and percentage for categorical variables. Differences between observed status as melanoma and nevus were calculated as the difference in percentage along with a 95% CI for the difference and tested using either the χ² test or Fisher exact test, as appropriate based on the expected cell counts.

Each patient was assigned a predicted 5-year KM probability of melanoma from both the COMS and WEH predictive models, based on their combination of risk factors included in each model. The distribution of predicted risks according to observed status by each model was plotted using box plots for each model. The distribution of predicted risk for each model was also displayed in a waterfall plot, with patients ordered from lowest to highest predicted risk for the respective model and with colors indicating the observed status. Finally, the predicted risks according to each model were plotted on a scatterplot to directly compare predictions for each patient according to the 2 models. Predicted risks were summarized by the mean and 95% CI. Differences according to observed status as melanoma or nevus, and according to COMS vs WEH model, were summarized using the mean difference and 95% CI for the difference and tested using a 2-sample t test. CIs were calculated using the Wald method, unless otherwise noted.

Then, we explored different cutoff values of predicted risk for assigning predicted status as either melanoma or nevus and calculated the sensitivity and specificity at each cutoff. Sensitivity was calculated as the total number of patients correctly identified as having melanoma (true positives by the predictive model) divided by the total number of patients who did have melanoma (true positives and false negatives by the predictive model). A model with high sensitivity is useful for ruling patients out of treatment because for patients predicted to have a nevus, the sensitivity represents the probability that they truly do not have melanoma. Specificity was calculated as the total number of patients correctly identified as having a nevus (true negatives by the predictive model) divided by the total number of patients who did have a nevus (true negatives and false positives by the predictive model). A model with high specificity is useful for ruling patients in for treatment because for patients predicted to have melanoma, the specificity represents the probability that they truly do have melanoma.

Cut points of 0/100, 10/90, 25/75, and 50/50 were investigated, where patients with predicted risk less than or equal to the first number were classified as having a nevus, and patients with predicted risk greater than or equal to the second number were classified as having melanoma. In addition, the optimal cut point, as defined by the Youden index, was identified for each model. The Youden index finds the cut point that simultaneously maximizes sensitivity while minimizing 1 − specificity and represents the point that achieves maximal area under the receiver operating characteristics curve.

All statistical analyses were conducted in R, version 4.2 (R Core Team). All P values were 2-sided, and no adjustments were made for multiple comparisons. Data were analyzed from October to December 2024.

Results

A total of 123 patients (mean [SD] age, 63 [56-67] years; 34 female [28%]; 89 male [72%]) were included in this study. Using the COMS predictive model, there were significant differences between observed melanoma and nevus according to greater than 2-mm tumor thickness (44% difference; 95% CI, 29%-60%), presence of drusen (−40% difference; 95% CI, −57% to −22%), and orange pigment (46% difference; 95% CI, 29%-64%). Presence of RPE changes (−16% difference; 95% CI, −34% to 2.6%) and LBD (>12 mm, 4.9% difference; 95% CI, −3.6% to 14%) were not statistically significantly associated with observed status (eTable 1 in Supplement 1). Using the WEH predictive model, there were significant differences between observed melanoma and nevus according to tumor thickness greater than 2 mm (44% difference; 95% CI, 29%-60%), presence of SRF (56% difference; 95% CI, 40%-72%), orange pigment (46% difference; 95% CI, 29%-64%), and hollow melanoma acoustic density (34% difference; 95% CI, 15%-54%). Visual acuity loss (<20/60, 0.16% difference; 95% CI, −10% to 11%) and LBD (>5mm, 3.1% difference; 95% CI, −8.1% to 14%) were not statistically significantly associated with observed status (eTable 2 in Supplement 1).

The distribution of predicted 5-year KM probabilities of melanoma, according to combinations of risk factors, ranged from 9% to 100% by the COMS model (eTable 3 in Supplement 1) and from 0% to 100% by the WEH model (eTable 4 in Supplement 1). Note that 5 patients did not have any of the features included in the COMS model and, therefore, do not have a predicted risk. In addition, 29 patients had a combination of 2 features (drusen + RPE or drusen + orange pigment) and did not have a predicted risk value assigned by the COMS model due to no patients with those combinations of features being observed in the COMS development data. Therefore, all results for the COMS model included 89 of 123 patients (72.4%) in the sample. In the COMS model, the median predicted risk was much higher for patients with observed melanoma vs nevus, but there was overlap in the 2 distributions, especially on the low end of the range. In the WEH model, the median predicted risk was again higher for patients with observed melanoma, but there was more overlap in the distributions of predicted risk according to observed status, with the 75th quartile in observed nevus exceeding the median in observed melanoma (Figure 1). The mean predicted risk of melanoma by the COMS model was 0.39 (95% CI, 0.32-0.46) for observed melanoma and 0.18 (95% CI, 0.12-0.23) for observed nevus, a difference of 0.21 (95% CI, 0.12-0.31). The mean predicted risk by the WEH model was 0.44 (95% CI, 0.39-0.49) for observed melanoma and 0.31 (95% CI, 0.24-0.38) for observed nevus, a difference of 0.13 (95% CI, 0.05-0.21). The mean difference of −0.13 (95% CI, −0.22 to −0.05) between the COMS and WEH models for the predicted 5-year KM probability of melanoma for observed nevus was statistically significant. The mean predicted 5-year KM probability of melanoma for observed melanoma was 0.39 (95% CI, 0.32-0.46) by the COMS model and 0.44 (95% CI, 0.39-0.49) by the WEH model. The difference of −0.05 (95% CI, −0.14 to 0.04) was not statistically significant.

Figure 1. Box Plots of 5-Year Kaplan-Meier (KM) Predicted Risk According to Observed Status and Model.

The middle line of the box displays the median, the upper and lower lines of the box display the 75th and 25th quartiles, respectively, and the whiskers extend to the largest point that is no more than 1.5 times the IQR from the top or bottom of the box. Points beyond that range are considered outliers and are indicated by a point. COMS indicates Collaborative Ocular Melanoma Study; WEH, Wills Eye Hospital.

Prediction of individual patient risk revealed that the COMS model had several patients with melanoma with very low predicted risks, whereas the WEH model had a number of patients with observed nevus with very high predicted risk (Figure 2). Clinical examples of a tumor correctly classified as melanoma (WEH model, 10/90 cutoff) (Figure 3) and a tumor not correctly classified as melanoma by both models (COMS and WEH model, 10/90 cutoff) (Figure 4) are included. Predictions from the 2 models were not concordant, as demonstrated by the spread around the diagonal line. In particular, the tumors in the lower range of predicted risks (<0.25) by the COMS model had much higher predicted risks by the WEH model (Figure 5).

Figure 2. Waterfall Plot of 5-Year Kaplan-Meier (KM) Predicted Risk for Each Model, According to Observed Status.

On each panel, patients are separately ordered from lowest to highest predicted risk for the respective model. COMS indicates Collaborative Ocular Melanoma Study; WEH, Wills Eye Hospital.

Figure 3. Clinical Example.

Tumor correctly classified as melanoma (Wills Eye Hospital model, 10/90 cutoff). A, Symptomatic juxtapapillary and macular amelanotic choroidal lesion (6.0 × 6.0 × 2.3 mm) and associated subretinal fluid in a 51-year-old female patient. Fundus autofluorescence-confirmed (B) and optical coherence tomography (OCT)–confirmed (C) subfoveal fluid. D, Ultrasonography revealed a dome shape and medium to high reflectivity. Diagnostic fine-needle aspiration biopsy was done (melanoma) before enucleation (mixed epithelioid and spindle cell type melanoma).

Figure 4. Clinical Example.

Tumor not correctly classified as melanoma by both models (10/90 cutoff). A, Symptomatic juxtapapillary melanotic choroidal lesion (8.5 × 8.0 × 2.3 mm) with orange pigment, subretinal fluid, and subretinal hemorrhage in a 68-year-old female patient. Fundus autofluorescence-confirmed (B) and optical coherence tomography (OCT)–confirmed (C) subfoveal fluid. D, Ultrasonography revealed a dome shape and medium to high reflectivity. Due to concern for optic nerve invasion, enucleation was done (mixed epithelioid and spindle cell type melanoma).

Figure 5. Scatterplot of Predicted Risks According to Collaborative Ocular Melanoma Study Model (COMS) and Wills Eye Hospital Model (WEH) According to Observed Status.

The diagonal line represents perfect concordance between the 2 models. KM indicates Kaplan-Meier.

Sensitivity and specificity according to various cutoffs and by model are presented in eTable 5 in Supplement 1. The COMS model could classify only 1 patient with a correct prediction of melanoma using a criterion of 100% predicted risk, and no patient was correctly classified as having a nevus using a criterion of 0% predicted risk. By the WEH model, only 2 patients were correctly classified as having a melanoma with 100% predicted risk, and 1 patient was correctly classified as having a nevus with predicted risk of 0%. For all the other predefined cutoffs, the COMS model had higher specificity than the WEH model, with differences of 55.6% (95% CI, 34.5%-76.7%) for the 10/90 cutoff, 13.2% (95% CI, 5.1%-21.3%) for the 25/75 cutoff, and 12.3% (95% CI, 3.5%-21.1%) for the 50/50 cutoff. The optimal cut points according to the Youden index were both low, 0.18 and 0.34 for the COMS and the WEH models, respectively (eFigure in Supplement 1). Patients with predicted risk less than the optimal cut point were classified as having a nevus whereas patients with predicted risk greater than or equal to the optimal cut point were classified as having a melanoma. The sensitivity was lower for the COMS model than the WEH model (−15.2% difference; 95% CI, −25.6% to −4.8%), and the specificity was higher for the COMS model than the WEH model (11.7% difference; 95% CI, 2.0%-21.4%) (eTable 6 in Supplement 1).

Discussion

Controversy regarding management of small choroidal melanoma stems from the diagnostic uncertainty.²⁰ Because a benign tumor such as choroidal nevus can be safely observed, therapeutic intervention for small choroidal melanoma would be advisable to minimize risk of metastasis, albeit with some risk of vision loss. Therefore, predictive models that can discriminate tumors with risk of malignant transformation (malignant or likely to become malignant) from those that are benign (or likely to remain benign) become relevant. Commonly used predictive models (COMS and WEH model) use combinations of risk factors to assign risk expressed as 5-year KM probability of growth, although the majority (range, 52%-83%; mean, 67%) of tumors are observed to grow in the initial 2 years.^1,17,18

To our knowledge, these models have not been adopted as standard tools for clinical practice per guidelines from systematic reviews or from the American Academy of Ophthalmology. We compared predicted risk with observed status as melanoma or nevus in a well-defined cohort that included only IMT. Observed status was classified as having a large nevus (benign, stable over a follow up of >24 months, 0% risk) or small melanoma (malignant, quantified growth or pathology, 100% risk). The need for statistical assessment of the predictive models becomes evident when critical analysis of the underlying data used to determine predictions is undertaken.⁹ The risk estimate of some single factors such as presence of orange pigment (37% at 5 years) may be more than that for combinations of 2 risk factors (visual acuity ≤20/50 and LBD >5 mm, 12% at 5 years).¹⁹ The number of cases is less than 10 in some combination groups,¹ and for some combinations, no patients with growth were observed. Therefore, a predicted risk of 0 is provided, or alternatively, no patients without growth were observed; therefore, a predicted risk of 100 is provided. For some combination groups, the 95% CIs for the predicted risks are either not reported¹⁴ or are as wide as 0% to 100% (4 factors 51% [0%-100%] and 5 factors 55% [0%-100%]), implying lack of any predictive value.¹⁹ In addition, the use of distinctive combinations of features can exclude certain patients from getting a predicted risk, if their combination of features was not included in the model, as seen by our inability to obtain predicted risk by the COMS model for 27.6% of the patients analyzed here.

When assessed for distribution of individual risk factors, in both the COMS predictive model and the WEH predictive model, high risk factors such as tumor thickness greater than 2 mm, presence of orange pigment and SRF, absence of drusen, and low melanoma acoustic density were more significantly associated with melanoma than with nevus, indicating applicability of these predictive models to our dataset. Overall, the range of predicted risk for various combinations of risk factors was similar between the COMS model (9% to 100%) and the WEH model (0%-100%). The mean difference between predicted risk for observed melanoma and observed nevus of 0.21 (95% CI, 0.12-0.31) for the COMS model and of 0.13 (95% CI, 0.05-0.21) for the WEH model indicates applicability of these predictive models to our dataset. However, in the COMS model, the IQR was wide for melanoma and narrow for nevus, with minimal overlap between the melanoma and nevus, whereas in the WEH model, the IQRs were wide for both melanoma and nevus, with overlapping values indicating limitation of model in discriminating between nevus and melanoma (Figure 1 and Figure 3).

We noted discordance of predictions between 2 models particularly for tumors with lower range of predictions for melanoma (<0.25) by the COMS model with much higher predictions by the WEH model (Figure 4). The discordance between the models can be explained on the source data (COMS: IMT only; WEH: nevus + IMT) and methods (WEH included imaging features such as ultrasound based acoustic density) and the prospective data collection in COMS model and retrospective data in the WEH model.

We investigated model performance across a variety of predefined cut points. The cut point of 50/50 is common, where patients with less than 50% predicted probability are classified as not having disease (here, the disease being a nevus) and patients with 50% or greater predicted probability are classified as having the disease (here, the disease being melanoma). However, the choice of a cut point depends on the various trade-offs between overtreating and undertreating for a specific disease. We included the cut point of 0/100 as this represents performance of a perfect prediction model—if everyone without disease was predicted to have 0% risk and everyone with the disease was predicted to have 100% risk. It is not expected that most models would perform well at this high standard. We additionally identified a so-called optimal cut point, identified by the Youden index, which yielded higher specificity for the COMS model than the WEH model (11.7% difference (95% CI, 2.0%-21.4%). Even so, relying on these models implies approximately 19% to 31% of the tumors predicted to be melanoma are not. If we were to rely on these predictive models to select cases for immediate treatment, overtreatment (of nevi) in the range of 19% to 31% can be expected. On the contrary, a predictive model with high sensitivity is good at ruling patients out of treatment. The Youden index yielded higher sensitivity for the WEH model than for the COMS model (79% and 64%, respectively) implying that approximately 21% to 36% of tumors diagnosed to be nevus were confirmed to be melanoma.

Machine learning–based models are being developed with higher level of discrimination (0.861) for IMT¹⁶ and for wider spectrum of choroidal melanocytic tumors that include nevus and IMT (0.864).²¹ Proof-of-concept studies using artificial intelligence are also under way.²²

Limitations

There are several limitations of this study. Our analysis is based on small number of selected tumors with observed outcomes, growth rate estimates of which may not be representative of the entire population of small choroidal melanomas. The melanoma was defined using multiple criteria (growth, pathology, or both) with growth observed predominantly in BD. Although tumors within 3 groups were similar,¹⁴ subtle unrecognized differences within them may influence interpretation of the results. It was not possible to obtain predicted risks for all patients in this validation dataset using the COMS model. This is more a limitation of the original COMS model, which failed to use data representing the full range of patient risk profiles, than of this validation. The WEH model includes ultrasonographic acoustic density, but not all patients in the validation dataset could be assessed for it due to limited thickness of the tumor. Prediction models that include easily assessed clinical variables are more practically applicable than models that rely on subjective or difficult-to-assess features. Finally, given the rarity of malignant transformation, the predictive values for some of the feature combinations are likely to be inaccurate making the estimates less reliable. Because original models only reported the 5-year KM probabilities for each combination of features, probabilities for other time periods could not be assessed. Our results should be considered as preliminary that need to be validated in a larger dataset or in a prospective study using standardized definitions, imaging techniques, and follow-up intervals. Despite limitations, our study was unique as it tested outcomes of predictive models.

Conclusions

In conclusion, in this cohort study, testing of accuracy of commonly used predictive models revealed pitfalls when comparing predicted risk for malignant transformation estimated by combination of risk factors models. Two tested models were suboptimal and may lead to overtreatment in approximately 30% of patients. Newer methods for prediction need to be developed and validated prior to clinical usage.

Supplement 1.

eTable 1. Distribution of Risk Factors Included in COMS Predictive Model

eTable 2. Distribution of Risk Factors Included in WEH Predictive Model

eTable 3. Combinations of Risk Factors Included in COMS Predictive Model

eTable 4. Combinations of Risk Factors Included in WEH Predictive Model

eTable 5. Sensitivity and Specificity According to Prediction Cutoff and Model

eTable 6. Sensitivity and Specificity According to Optimal Cut Point

eFigure. Receiver Operating Characteristic Curve

Supplement 2.

Data Sharing Statement