Development and Validation of a Nomogram for Predicting Survival in Patients with Thyroid Cancer

Abstract

Background

The AJCC staging system is inadequate for use in patients with thyroid carcinomas. Here, we aimed to establish a nomogram for thyroid cancer, and we compare its prognostic value with the AJCC staging system in adults diagnosed with thyroid carcinoma.

Material/Methods

Patient records were obtained from the Surveillance, Epidemiology, and End Result database. The 8491 included patients were divided into a modeling cohort (n=5943) and a validation cohort (n=2548). The variables included in the modeling cohort were selected using a backward stepwise selection method with Cox regression, and the prognosis nomogram was constructed. In the validation cohort, we compared our survival model with the AJCC prognosis model using the concordance index, the area under the time-dependent receiver operating characteristic curve, the net reclassification improvement, the integrated discrimination improvement, calibration plotting, and decision curve analysis.

Results

Twelve independent prognostic factors were identified and used to establish the nomogram. In particular, marital status was included in a survival prediction model of thyroid cancer for the first time. The concordance index, area under the time-dependent receiver operating characteristic curve, net reclassification improvement, integrated discrimination improvement, calibration plotting, and decision curve analysis for the nomogram showed better performance compared to the AJCC staging system.

Conclusions

We have developed and validated a highly accurate thyroid cancer prognosis nomogram. The prognostic value of the nomogram is better than that of the AJCC staging system alone.

Background

There were approximately 567 000 new cases of thyroid cancer worldwide in 2018, and the global incidence rate is 3 times higher in women (10.2 per 100 000) than in men. The mortality rate of thyroid cancer ranges from 0.4 to 0.5 per 100 000 in both men and women, with an estimated 41 000 deaths annually. In the USA, thyroid cancer incidence rates were about 6.9 per 100 000 in men and 23.1 per 100 000 in women in 2018. Incidence rates are 4 and 5 times higher in men and women in the United States than in countries with lower Human Development Index. The incidence is increasing faster than that for any other solid tumor, ranking in ninth place for incidence in 2018 [1],and it will replace colorectal cancer as the fourth leading cancer diagnosis by 2030 in the USA [2].

Papillary thyroid carcinoma (PTC), follicular thyroid carcinoma (FTC), anaplastic thyroid carcinoma (ATC), and medullary thyroid carcinoma (MTC) arising from thyroid parafollicular cells are the main pathological types of thyroid carcinoma [3]. Differentiated thyroid cancer (DTC) includes papillary and follicular cancer. PTC constitutes 90% of thyroid cancers, and they are a major differentiated adenocarcinoma that shows papillary proliferation pathologically. Most cases have an excellent prognosis, but approximately 10% of PTC patients exhibit recurrences such as lymph node recurrence and lung metastasis [4], or even death. Clinicopathologically, age >45 years, extra-thyroidal invasion, distant metastasis, large tumor, vascular invasion, and poorly-differentiated histology are well known factors predicting poor prognois [5]. Most FTCs are minimally invasive, with only slight tumor capsular invasion, and rarely cause distant metastasis [6]. Although much less common, FTC is widely invasive, with 80% of these tumors causing distant metastasis, causing a high mortality rate of around 20% [5]. The poor prognostic factors are large tumor, distant metastasis, extra-thyroidal extension, extensive vascular invasion, age >45 years, and wide invasion [7]. ATC is an extremely aggressive undifferentiated tumor, with almost 100% disease-specific mortality [8], and while representing about only <2% of thyroid cancers, they are responsible for 40% of thyroid cancer deaths. The median survival time from diagnosis is around 6 months [9]. ATC extensively invades into surrounding structures, and distant metastases are observed at the diagnosis in one-third of ATC patients. Most (>70%) of the patients are women, and the peak age of patients is older than that of DTC [10]. MTC represents <5% of thyroid carcinomas; that is, neuroendocrine tumors originating from parafollicular cells of the ultimobranchial body of the neural crest, and it secrets calcitonin. Most (70–80%) MTCs are sporadic, with 20–30% being familial [11]. The peak age is younger for familial MTC (approximately 35 years) than for sporadic MTC (40–60 years). The overall 5-year survival rate of patients with MTC is 86%. Poor prognostic factors include older age at diagnosis, the presence of lymph node metastasis at diagnosis, advanced stage, and somatic RET mutation [12]. Adequate risk stratification is crucial in malignant neoplastic disease to avoid both the undertreatment of high-risk subjects and the overtreatment of low-risk patients [13]. For thyroid cancer, surgery and radioiodine therapy remain the established therapeutic procedures. However, the effect of chemotherapy is not clear [14]. This means that personalized treatment according to the potential prognosis for individuals with thyroid cancer is critically important.

Suh et al. demonstrated that the AJCC staging system predicted patient outcome more accurately than other staging systems, which suggests that it is a reliable and cost-effective predictor of outcome in patients with PTC [15]. However, the current AJCC staging system for MTC appears to be less than optimal in distinguishing the risk of mortality among different stage groups [16]. The AJCC staging system can predict the outcome in patients with PTC, but it is inadequate in patients with FTC, MTC, and ATC. This is because factors other than the AJCC stage also influence the prognosis and metastasis of thyroid cancer, including the involvement of lymph nodes [17], age at diagnosis, completion of surgical resection, and pathological subtype [18]. We therefore established a comprehensive prognostic nomogram including all of the factors mentioned above and determined whether its performance is better than that of the AJCC staging system in patients diagnosed with PTC, FTC, ATC, and MTC.

Material and Methods

Patients

We reviewed patient data from the latest version of the SEER (Surveillance, Epidemiology, and End Result) database using SEER*Stat version 8.3.5 released on March 6, 2018 (https://seer.cancer.gov/). We searched for patients whose ICD-O-3/WHO 2008 histological type codes were 8330/3 (FTC), 8050/3 (PTC), 8510/3 (MTC), or 8021/3 (ATC). We also searched for patients with a positive diagnostic confirmation in histology, categorized as either alive or with thyroid carcinoma as the cause of death, and as active follow-up. We excluded patients under the age of 18 years, those initially confirmed by a death certificate or only an autopsy, and cases with unknown or incomplete important variables.

There were 8491 patients identified from the SEER database between 2004 and 2015. These patients were randomly divided into a modeling cohort (n=5943, 70%) for constructing the prognosis nomogram and a validation cohort (n=2548, 30%) for evaluating the constructed nomogram. This retrospective study was exempted from consent requirements by the SEER database administrators.

Variables from the SEER database were selected by a backward stepwise selection method in the Cox regression model. The following variables were examined: age at diagnosis, sex, race, insurance recode, marital status, tumor size, AJCC stage, derived AJCC stage T, derived AJCC stage N, derived AJCC stage M, extent of disease, ICD-O-3 histology, surgery, radiation, and chemotherapy. All variables except sex, race, and insurance recode were entered into the nomogram. Death was attributed to thyroid cancer if this was listed as the cause of death on the death certificate.

Statistical analyses

All statistical analyses were performed using SPSS (version 21.0) and R software. A 2-sided probability value of P≤0.05 was considered to be statistically significant. Mean ±SD values were used to express continuous variables conforming to a normal distribution, and all other variables were expressed as median (25^th–75^th percentile) values. Variables were included in the multivariable Cox regression analyses at P≤0.1.

Statistical analyses to identify risk factors were performed by applying the backward stepwise selection method of Cox regression to the modeling cohort. All related statistical analyses and the establishment of the nomogram were performed using R software. The discrimination performance of our nomogram was compared with that of the traditional AJCC staging system by measuring the concordance index (C-index) and the area under the time-dependent receiver operating characteristic curve (AUC). In addition to these indices, we calculated the integrated discrimination improvement (IDI) and the net reclassification improvement (NRI) to evaluate the predictive accuracy of our model. Calibration plots were generated to evaluate the predictive performance by comparing the nomogram-predicted and actual observed 3-, 5-, and 10-year survival rates. Decision curve analysis (DCA) was used to evaluate the clinical usefulness of the nomogram by quantifying the net benefits at different threshold probabilities.

Results

Clinicopathological characteristics of the patients

The median age at the time of diagnosis was 50 years in both the modeling and validation cohorts. Most of the patients in both cohorts were female, white, and married, had insurance and any medical and a tumor size of ≤5 mm, and were in AJCC stage I, derived AJCCT1, derived AJCC stage N0, and derived AJCC stage M0. Most of the patients had a localized tumor and a PTC or FTC in both cohorts, and received surgery but not radiation or chemotherapy. The median follow-up time was 66 months in both cohorts. The demographics and tumor characteristics of the patients are summarized in Table 1.

Table 1.

Clinical and pathological characteristics of patients in the modeling and validation cohorts.

	Modeling cohort	Validation cohort	P-value
Number of Patients n (%)	5943 (70)	2548 (30)
Median age at diagnosis, year (interquartile range)	50 (39–62)	50 (39–60)	0.16
Sex n (%)			0.84
Male	1680 (28.3)	714 (28.0)
Female	4263 (71.7)	1834 (72)
Race n (%)			0.89
White	4883 (82.2)	2094 (82.2)
Black	556 (9.4)	232 (9.1)
Other	504 (8.5)	222 (8.7)
Marital status n (%)			0.93
Married	3691 (62.1)	1573 (61.7)
Unmarried	1983 (33.4)	861 (33.8)
Unknown	269 (4.5)	114 (4.5)
Insurance recode n (%)			0.89
Uninsured	142 (2.4)	65 (2.6)
Insured and any medical	4234 (71.2)	1807 (70.9)
Unknown	1567 (26.4)	676 (26.5)
Tumor size n(%)			0.06
≤50 mm	5016 (84.4)	2190 (85.9)
50–100 mm	748 (12.6)	285 (11.2)
>100 mm	63 (1.1)	16 (0.6)
No/unknown	116 (2.0)	57 (2.2)
AJCC n (%)			0.28
I	3296 (55.5)	1470 (57.7)
II	856 (14.4)	344 (13.5)
III	821 (13.8)	343 (13.5)
IV	970 (16.3)	391 (15.3)
Derived AJCC T n (%)			0.06
T0	15 (0.3)	8 (0.3)
T1	2434 (41.0)	1116 (43.8)
T2	1492 (25.1)	623 (24.5)
T3	1393 (23.4)	533 (20.9)
T4	609 (10.2)	268 (10.5)
Derived AJCC N n (%)			0.83
No	4739 (79.7)	2026 (79.5)
N1	1204 (20.3)	522 (20.5)
Derived AJCC M n (%)			0.11
M0	5540 (93.2)	2400 (94.2)
M1	403 (6.8)	148 (5.8)
Extent of disease n (%)			0.07
Localized	3268 (55.0)	1443 (56.6)
Regional	2129 (35.8)	909 (35.7)
Distant	546 (9.2)	196 (7.7)
ICD-O-3 histology n (%)			0.44
Papillary carcinoma	2349 (39.5)	1050 (41.2)
Follicular carcinoma	2284 (38.4)	942 (37)
Medullary carcinoma	959 (16.1)	415 (16.3)
Anaplastic carcinoma	351 (5.9)	141 (5.5)
Surgery n (%)			0.54
Yes	5703 (96.0)	2453 (96.3)
No	240 (4.0)	95 (3.7)
Radiation n (%)			0.85
Yes	2660 (44.8)	1134 (44.5)
None/unknown	3283 (55.2)	1414 (55.5)
Chemotherapy n (%)			0.77
Yes	236 (4.0)	97 (3.8)
No	5707 (96)	2451 (96.2)
Median follow-up, months (interquartile range)	66 (26–104)	66 (26–104)	0.78
Thyroid cancer-specific mortality (%)	495 (8.3)	191 (7.5)	0.21

Race-Other – American Indian & AK Native & Asian & Pacific Islander. Marital status – Unmarried: Single & Separated & Divorced & Widowed & Unmarried or Domestic Partner.

Independent prognostic factors in the modeling cohort

The variables of age at diagnosis, marital status, tumor size, AJCC stage, derived AJCC stage T, derived AJCC stage N, derived AJCC stage M, extent of disease, histology, surgery, radiation, and chemotherapy were entered into the multivariable Cox regression analyses. The multivariate analyses revealed the following significant risk factors for survival: age at diagnosis (hazard ratio [HR]=1.034, P<0.001), being unmarried (HR=1.468, P<0.001 vs. married), AJCC stage II (HR=2.863, P=0.027 vs. AJCC stage I), AJCC stage IV (HR=11.317, P<0.001 vs. AJCC stage I), derived AJCC stage T3 (HR=3.535 P=0.037 vs. derived AJCC stage T0), derived AJCC stage T4 (HR=7.944, P<0.001 vs. derived AJCC stage T0), derived AJCC stage M1 (HR=2.175, P<0.001 vs. derived AJCC stage M0), distant extent of disease (HR=1.863 P=0.02 vs. localized extent of disease), follicular histology (HR=1.744, P=0.007 vs. papillary histology), medullary histology (HR=1.593, p=0.026 vs. papillary histology), anaplastic histology (HR=6.316, P<0.001 vs. papillary histology), no surgery (HR=2.261, P<0.001 vs surgery), and no/unknown radiation (HR=1.388, P<0.001 vs. radiation). In particular, we found that tumor size was also a risk factor affecting survival: HR=1.316 (P=0.017) for 50–100 mm vs. ≤50 mm, and HR=1.925 (P=0.001) for >100 mm vs. ≤50 mm (Table 2).

Table 2.

Selected variables by multivariate Cox regression analysis (modeling cohort).

Variables	Multivariate analysis
	HR	95% CI	P-value
Age at diagnosis	1.03	1.03–1.04	<0.01
Marital status n (%)
Married	Reference
Unmarried	1.47	1.22–1.77	<0.01
Unknown	0.32	0.15–0.67	<0.01
Tumor size n (%)
≤50 mm	Reference
50–100 mm	1.32	1.05–1.65	0.02
>100 mm	1.93	1.30–2.85	<0.01
NO/unknown	1.69	1.24–2.31	<0.01
AJCC n (%)
I	Reference
II	2.86	1.13–7.26	0.03
III	2.04	0.88–4.71	0.10
IV	11.32	5.32–24.07	<0.01
Derived AJCC T n (%)
T0	Reference
T1	2.15	0.60–7.69	0.24
T2	1.87	0.52–6.67	0.34
T3	3.54	1.08–11.60	0.04
T4	7.9	2.46–25.70	<0.01
Derived AJCC N n (%)
N0	Reference
N1	1.21	0.98–1.49	0.08
Derived AJCC M n (%)
M0	Reference
M1	2.18	1.60–2.95	<0.01
Extent of disease n (%)
Localized	Reference
Regional	1.502	0.9603–2.3479	0.074
Distant	1.863	1.1019–3.1512	0.02
ICD-O-3 histology n (%)
Papillary carcinoma	Reference
Follicular carcinoma	1.74	1.17–2.60	0.01
Medullary carcinoma	1.59	1.06–2.40	0.03
Anaplastic carcinoma	6.32	4.22–9.45	<0.01
Surgery n (%)
Yes	Reference
No	2.26	1.79–2.85	<0.01
Radiation n (%)
Yes	Reference
None/unknown	1.39	1.14–1.68	<0.01
Chemotherapy n (%)
Yes	Reference
No	1.25	0.99–1.58	0.06

Marital status – Unmarried: Single & Separated & Divorced & Widowed & Unmarried or Domestic Partner.

Prognostic nomogram for 3-, 5-, and 10-year survival rates

Based on the significant independent factors that were selected variables with HRs, we constructed a nomogram for predicting the 3-, 5-, and 10-year survival rates in the modeling cohort. The nomogram showed that the age at diagnosis was the strongest contributor to the prognosis, followed by the AJCC stage, derived AJCC stage T, histology, marital status, surgery, derived AJCC stage M, tumor size, extent of disease, radiation, chemotherapy, and derived AJCC stage N. Each variable was given a score on a scale. These scores were then added to obtain the total score, and a vertical line was dropped down from the total points row to estimate the 3-, 5-, and 10-year survival rates (Figure 1).

Figure 1.

Nomogram predicting 3-year, 5-year, and 10-year survival. Unmarried: Single & Separated & Divorced & Widowed & Unmarried or Domestic Partner. EOD – extent of disease. SUR – surgery; RAD – radiation; CHE – chemotherapy. Hist – ICD-O-3 histology. PC – papillary carcinoma; FA – follicular carcinoma; MC – medullary carcinoma; AC – anaplastic carcinoma.

Validation of the prognostic nomogram

The prognostic nomogram and the AJCC staging system were compared using the verification cohort. The C-index was higher for the nomogram than for the AJCC staging system (0.975 vs. 0.929), as were the AUCs for the nomogram (0.999, 0.997, and 1.000 for 3-, 5-, and 10-year survival rates, respectively, vs. 0.958, 0.962, and 0.965, respectively), which indicated the good discriminative ability of the nomogram (Figure 2). Compared with the AJCC stage, the NRIs for 3, 5, and 10 years of follow-up were 0.643 (95% confidence interval [CI]=0.515–0.739), 0.565 (95% CI=0.452–0.674), and 0.549 (95% CI=0.423–0.767), respectively; the corresponding IDIs were 0.264, 0.271, and 0.241, respectively (all P<0.001). These indicators demonstrate that the nomogram showed better discrimination performance than the AJCC staging system.

Figure 2.

ROC curves. The ability of the model to be measured by the C-index. In the validation cohort, predicted probabilities for 3-,5- and 10-years survival (A–C) based on the nomogram and AJCC in the validation sets.

Calibration plots of the nomogram showed that the predicted 3-, 5-, and 10-year survival rates for the modeling and validation groups were almost identical to the actual observations (Figure 3).

Figure 3.

Calibration plots. These show the relationship between the predicted probabilities for 3-, 5- and 10-years survival (A–C) based on the nomogram and actual values in the validation sets.

These results show that although both models yield net benefits, the 3-, 5-, and 10-year DCA curves for the nomogram yielded net benefits greater than when using the traditional AJCC staging system in the validation set (Figure 4).

Figure 4.

Decision curve analysis. In the figure, the abscissa is the threshold probability and the ordinate is the net benefit rate. The horizontal one indicates that all samples are negative and all are not treated, with a net benefit of zero. The oblique one indicates that all samples are positive. The net benefit is a backslash with a negative slope. A–C show prediction for 3-, 5- and 10-year survival in the validation sets. Survival probability new: the nomogram. Survival probability: AJCC.

Discussion

The incidence of thyroid cancer is reportedly 3- to 4-fold higher among females than males worldwide, ranking as the sixth most common malignancy diagnosed in women [3]. Thyroid cancer can occur at any age, but it is rare in childhood, with most tumors being diagnosed during the third to sixth decades of life [19]. The main pathological types of carcinomas are PTC, FTC, and ATC. MTC arises from thyroid parafollicular cells. PTC constitutes 85–90% of all thyroid cancer cases, followed by FTC (5–10%) and MTC (about 2%). ATC accounts for less than 2% of thyroid cancers, but it is still an important lethal disease [19]. The 10-year overall relative survival rates for US patients with PTC, FTC, MTC, and undifferentiated carcinoma/ATC are 93%, 85%, 75%, and 14%, respectively [20].

A nomogram is a useful predictive tool that is tailored to the profile of an individual patient and creates a more precise prediction compared to the traditional AJCC staging system [16]. In recent years, nomograms have been applied in most types of cancer [3,18]. However, nomograms have been developed for PTC rather than other types of pathological thyroid carcinoma to generate individualized predictions [21–25], and there has also been an overall lack of evaluations of the developed nomograms. As is well known, characterization according to 2 related properties of discrimination is the most basic assessment for predictive models [26], Moreover, comparisons of calibration methods and the clinical usefulness of predictive models are also very important [27]. We have established a comprehensive prognostic nomogram and compared its prognostic value with the AJCC staging system from the 3 aspects above in patients diagnosed with PTC, FTC, ATC, and MTC.

The multivariate Cox regression performed in this study revealed that age at diagnosis, being unmarried, AJCC stage II, AJCC stage IV, derived AJCC stage T3, derived AJCC stage T4, derived AJCC stage M1, distant extent of disease, follicular, medullary, and anaplastic histology, no surgery, no/unknown radiation, and large tumor are risk factors for survival. It is worth noting that we discovered the marital status in a survival prediction model of thyroid cancer for the first time. To the best of our knowledge, there has been no previous description of the effect of marital status on survival of patients with thyroid cancer. This new information can therefore further help clinicians to make more effective clinical decisions.

We have established and validated a nomogram for predicting the survival rates of thyroid cancer patients at 3, 5, and 10 years. Our nomogram model contains risk factors that can be easily collected from historical medical records. The clinical applicability and ease of use are important advantages of the nomogram we constructed. To further determine whether our prognostic model performed better than the traditional AJCC staging system, we evaluated the performance of our survival model using several parameters that are commonly assessed in model validations: C-index, AUC, NRI, IDI, calibration plots, and DCA. The receiver operating characteristic curve and C-statistic are typically used to assess discrimination [28]. The IDI and NRI were also used in the present study to assess the discrimination performance based on the additional diagnostic value of our model compared to the AJCC staging system. Although the C-index and AUC of the nomogram were only a little higher than those of the AJCC staging system (Figure 2), the addition of the nomogram to the AJCC staging system significantly improved the reclassification performance in the validation cohort: NRI=0.643, 0.565, and 0.549, and IDI=0.264, 0.271, and 0.241 for 3, 5, and 10 years, respectively (all P<0.001).

The calibration plots approximating a 45-degree line indicated that the nomogram predictions were well calibrated (Figure 3). DCA is used for evaluating clinical usefulness, and it can show a minimal net benefit of modified scores in an incorporated index. Some studies have demonstrated the benefits of DCA and recommend its use [29,30]. The present results indicate that the 3-, 5-, and 10-year DCA curves of our model yielded net benefits greater than those of the traditional AJCC staging system (Figure 4).

Our newly developed nomogram can be used to improve the prediction performance when using the AJCC staging system alone. This supports the use of our nomogram as a tool for helping to redirect and optimize treatment in this clinical setting.

Limitations

The patients included in this study were mainly white, and so it might not be valid to extrapolate the results to other racial groups. The analyzed data set was extracted from the SEER database, making this a retrospective study with the inevitable inherent bias. Selection bias was present in the selection and exclusion of patients, because we only included those patients with complete information. In addition, many factors were not included, such as RET mutation. The other limitations of this study include the relatively small sample, so more data need to be analyzed to improve the accuracy of model performance assessments. Finally, the values predicted using the nomogram should only be used by clinicians for reference purposes.

Conclusions

We have developed and validated a highly accurate thyroid cancer prognosis nomogram. The prognostic value of the nomogram is better than that of the AJCC staging system alone. In particular, marital status has been included in a survival prediction model of thyroid cancer for the first time. The nomogram developed in this study may be a valuable tool when explaining 3-, 5-, and 10-year survival rates to patients in clinical practice.

Abbreviations

SEER

Surveillance, Epidemiology, and End Result

C-index

concordance index

AUC

area under the time-dependent receiver operating characteristic curve ()

NRI

net reclassification improvement

IDI

integrated discrimination improvement

DCA

decision curve analysis

PTC

papillary thyroid carcinoma

FTC

follicular thyroid carcinoma

ATC

anaplastic thyroid carcinoma

MTC

medullary thyroid carcinoma

DTC

differentiated thyroid cancer