Establishment and validation of a nomogram to predict structural incomplete response in papillary thyroid carcinoma patients: a retrospective study

Abstract

Objective

To identify risk factors related to structural incomplete response (SIR) in papillary thyroid carcinoma (PTC) and develop a nomogram for PTC patients.

Methods

In this respective study, clinical, ultrasonic, and pathological data of PTC patients treated at our institute between 2016 and 2020 were analyzed. Patients were randomly split into training and validation sets at a ratio of 7:3. Multivariate Cox regression analysis was conducted to determine independent prognostic factors. On the basis of these factors, a nomogram was built to predict SIR. P value, concordance index, calibration plots and decision curve analysis were used to evaluate the model.

Results

Multivariate Cox regression analysis showed that BRAF V600E status, lymph node metastasis, sex, tumor size, margin, and surgical procedure were independent prognostic factors. In the validation set, the concordance index of the nomogram was 0.774 (95% confidence interval: 0.703–0.845). Calibration plots at 3 and 5 years showed no apparent difference between predicted SIR probability and the actual SIR proportion. Additionally, the nomogram had good net clinical benefit according to the decision curve analysis compared with cases that were treat-all or treat-none.

Conclusion

We build a nomogram to predict individualized outcomes and help postoperative surveillance in PTC patients.

Introduction

Thyroid cancers have become the most common malignant endocrine tumors in adults,^1,2 and are currently the fifth most common cancer diagnosis in women. By 2030, thyroid cancers are estimated to become the second leading cancer diagnosis in women and the ninth leading cancer diagnosis in men.³ With improvements in people’s health awareness, there has been increasing detection of thyroid cancer, especially small ones, an increase of 3- to 6-fold in recent years in some Asian countries, including China and South Korea.^4–6

Papillary thyroid carcinoma (PTC) is the most common thyroid cancer in clinical practice, accounting for more than 80% of all thyroid malignancies. PTC is an indolent disease and has an excellent 10-year overall survival rate of more than 90%.⁷

Given the favorable prognosis, predicting the risk of recurrence is a new challenge and more meaningful. However, some staging systems, such as metastases, age at diagnosis, completeness of resection, invasion, size of the tumor (MACIS) scoring system, and American Joint Committee on Cancer (AJCC)/Union for International Cancer Control tumor, node, and metastasis (TNM) staging systems, have been commonly used to predict the risk of death in recent years.⁸ Since 2009, the American Thyroid Association (ATA) guidelines established a risk stratification scheme, which classified patients as having high, intermediate, or low risk of recurrence, and the guidelines were updated in 2015.^9,10 The system was more available for the population with similar nodules features but could not provide a personalized prediction of PTC, especially for the low- patients.

The dynamic risk stratification (DRS) system was recommended to evaluate responses to initial therapy among PTC patients who have undergone total thyroidectomy in 2015 by the American Thyroid Association guidelines. The European Society for Medical Oncology guidelines in 2019¹¹ updated the DRS system for PTC patients who have undergone total thyroidectomy or lobectomy. Structural incomplete response (SIR) refers to imaging evidence of disease regardless of thyroglobulin (Tg) or anti-thyroglobulin antibody (TgAb) levels.

Nomograms are graphical representation tools based on a statistical predictive model and are commonly used to predict specific outcomes for individual patients by combining significant factors. Nomograms have been applied to various diseases, including cancer, to indicate treatment outcomes.¹²

To study the application of nomograms in PTC, we searched the PubMed database using the search term “nomogram,” “thyroid cancer,” and “predictive.” In addition to articles on the survival of medullary thyroid carcinoma and papillary thyroid carcinoma and some data mining articles on molecular markers, related markers, and methylation group genes, there were more than 10 predictive articles for PTC. However, most of these were focused on predicting lymph node metastasis or recurrence. There were only two articles related to this study’s goal. In one, the authors predicted the recurrence of PTC.¹³ The primary related factors were clinical data, age, nodule size, sex, pathological characteristics, extranodal diffusion, lymph node metastasis, but relevant molecular markers and sonographic features were not included. The other article only predicted incomplete biochemical response in patients but without structural recurrence after total thyroidectomy.¹⁴ Among these models, only three pieces included sonographic features, which predicted PTC diagnosis, but no recurrence.

In accordance with these publications, we included sonographic features, clinical data, and molecular markers, such as the BRAF V600E mutation, to model and individually predict structural recurrence of PTC in our hospital.

Materials and methods

Patients

This retrospective study was performed using our institutional database of patients who underwent operations for thyroid nodules between June 2016 and December 2020. The Ethics Committee of Qilu Hospital deemed that formal approval was not required owing to the retrospective and anonymous nature of this study. Written informed consent forms were signed by all patients. All patient data were de-identified prior to analysis. All specimens were submitted for routine pathological evaluation and diagnostic confirmation. For this analysis, the inclusion criteria were patients postoperatively diagnosed with PTC and regularly followed up after discharge. For total thyroidectomy, surgical procedures included at least ipsilateral central lymph node dissection and therapeutic lateral neck dissection for clinically proven nodal metastasis or unilateral lobectomy accompanied by routine ipsilateral central lymph node dissection.¹⁵ Decisions for radioactive iodine RAI (ablation) were made in accordance with the French Societies of Nuclear Medicine and Endocrinology guidelines.¹⁶

Patients were excluded if they had one of the following features: (1) more than two malignant lesions; (2) previous invasive procedures such as thyroid surgery; (3) previous radiation exposure; (4) a family history of PTC.

Instruments and methods

Ultrasound imaging was performed using Philips iU Elite platforms with a 5- to 12-MHz linear (L12-5) transducer (Philips Healthcare, Bothell, WA, USA). Morphological features of lesions on B-mode ultrasound, such as maximum diameter, shape, margin, orientation, internal echotexture, nodule contents, and calcification, were recorded. Two radiologists with eight years of experience in thyroid ultrasound examination analyzed ultrasound images. If there were disagreements, another radiologist with 15 years of expertise in thyroid ultrasound examination reviewed the image until a consensus was reached. Shape was categorized as regular or irregular. Margins were defined as the morphological features of the boundary between lesions and surrounding tissues and were classified as well-defined or ill-defined. A taller-than-wide ratio was defined as a nodule with a greater anteroposterior diameter than transverse diameter. The internal echotexture was described as hypoechoic, isoechoic, hyperechoic, or mixed in comparison with the thyroid parenchyma. Nodule contents were categorized as solid (<25% cystic), cystic (≥75% cystic), or mixed (25%–74% cystic). Calcification was defined as no calcification or calcification.^8,17

Tumor tissues were fixed with formaldehyde and embedded in paraffin. All pathological specimens used in this study were reviewed by a single pathologist in accordance with the Word Health Organization’s Classification, 4th edition.¹⁸ In cases where there was doubt regarding the diagnosis, two other pathologists reviewed the specimens to obtain a consensus diagnosis. Each patient was staged in accordance with the AJCC/UICC 8th edition staging system.¹⁹ BRAF V600E, Galectin-3, and HBME-1 expression were determined by standard immunohistochemical protocols.

Follow-up

Patient follow-up started upon discharge from the hospital. Patients were usually followed-up every 3 months during the first year and at 6-months intervals later. Routine checkups including neck ultrasound and thyroid function tests were performed at each visit. Follow-up data from qualified local hospitals were also accepted. All neck ultrasounds of patients during final follow-up were also obtained. The definition of SIR was in accordance with the European ESMO guidelines (2019).¹¹ Regardless of Tg or TgAb levels, patients with evidence of suspicious loco-regional disease on imaging, including the presence of suspicious lymph nodes in the central or lateral neck, abnormal tissues of the thyroid bed, or distant metastases (ultrasound, computed tomography, or magnetic resonance imaging) or with cytologically- or histologically-proven disease were considered to have structural recurrences. Disease-specific mortality was defined as death related to the tumor and also an endpoint.^8,9

Cox regression analysis

Cox regression analysis was used for prognostic analysis. Significant variables in univariate Cox regression analyses were included in the multivariate Cox regression analysis to determine independent prognostic factors.

Establishment and validation of the nomogram

Patients were randomly split into training and validation cohorts at a ratio of 7:3. The independent prognostic factors determined by multivariate Cox regression analysis in the training set were used to establish the nomogram. The performance of the nomogram was assessed using the concordance index (C-index) and 95% confidence intervals (CIs). A C-index of 0.5 (0.5 ≤ C-index ≤ 1) suggests no predictive discrimination power, while an index of 1.0 indicates perfect discriminatory power.²⁰ The validation set was used to assess the performance of the predictive nomogram model.

Statistical analysis

Statistical analyses and plots were performed and generated, respectively, with R software (version 4.0.4; www.r-project.org) using the lattice, survival, Formula, foreign, plyr, and ggplot2 packages. Descriptive statistics are presented as mean ± standard deviation. Age and size were analyzed using Student's t-tests. Comparative analyses for categorical variables (shape, margin, internal echotexture, nodule contents, calcification, and orientation) were performed using the Chi-square test or Fisher exact test. A P value <0.05 represented statistical significance.

Results

Pathologic diagnoses

During the study period, 1154 PTC patients with pathologically confirmed disease were treated at our hospital. Among these cases, the final cohort with complete follow-up included 906 patients. The detailed workflow of this study is shown in Figure 1. PTCs of ≤10 mm in diameter were called papillary microcarcinomas (PTMCs) and accounted for 58.7% (n = 532) of all cases.²¹ The median follow-up was 29.5 ± 15.8 months (range: 3–60 months). There were 239 male (26.4%) and 667 female (73.6%) patients. Total thyroidectomy was performed in 43.4% (n = 393) and lobectomy in 56.6% (n = 513) of cases. The majority of our patients had stage I disease (n = 838, 92.5%), while 68 (7.5%) had stage II lesions.¹⁹ At the end of the follow-up period, the overall prognosis was excellent, and SIR was only identified in 7.9% (n = 72) of patients. There were no cases of death from disease. We randomly split the dataset into at a 7:3 ratio into a training set to establish the nomogram and a validation set to validate the nomogram, respectively. For reproducibility, the random seed was set to 1000. Clinicopathologic characteristics of the training and validation sets are shown in Table 1.

Figure 1.

Study flow chart. DCA, decision curve analysis.

Table 1.

Characteristics of patients in the training and validation sets.

Baseline variables	Training set (%)(n = 635)	Validation set (%)(n = 271)	P
Follow-up time (month, mean ± SD)	29.4 ± 15.8	29.8 ± 15.9	0.72
Age (year, mean ± SD)	44.9 ± 12.1	44.8 ± 12.2	0.87
Sex			0.12
Female	458(72.1)	209(77.1)
Male	177(27.9)	62(22.9)
Size (mm, mean ± SD)	12.0 ± 9.4	11.3 ± 7.5	0.55
Shape			0.16
Regular	56(8.9)	31(11.4)
Irregular	579(91.1)	240(88.6)
Orientation			0.82
Wider than tall	277(43.6)	116(42.8)
Taller than wide	358(56.4)	155(57.2)
Margin			0.13
Well-defined	115(18.1)	38(14.0)
Ill-defined	520(81.9)	233(86.0)
Capsule			0.43
Invasion	233(36.7)	92(32.7)
Not invasion	402(63.3)	179(67.3)
Internal echotexture			0.35
Hypoechoic	613(96.5)	258(95.2)
Isoechoic	5(0.8)	4(1.5)
Mixed	17(2.7)	9(3.3)
Nodule contents			0.68
Solid	623(98.1)	265(97.8)
Mixed	12(1.9)	5(1.8)
Cystic	0	1(0.4)
Calcification			0.79
Negative	167(26.3)	69(25.5)
Positive	468(73.7)	202(74.5)
Lymph node metastasis			0.3
Yes	161(25.4)	60(22.1)
No	474(74.6)	211(77.9)
Surgery			0.72
Total thyroidectomy	273(43.0)	120(44.3)
Lobectomy	362(57.0)	151(55.7)
BRAF V600E			0.62
Positive	632(99.5)	269(99.3)
Negative	3(0.58)	2(0.7)
Galectin_3			0.073
Positive	630(99.2)	265(97.8)
Negative	5(0.8)	6(2.2)
HBME_1			0.78
Positive	619(97.5)	265(97.8)
Negative	16(2.58)	6(2.2)
Recurrence			0.14
Yes	56(8.8)	16(5.9)
No	579(91.2)	255(94.1)

Variables are reported as number (percentage). P-values were calculated by Student's t-test or Chi-square test.

Risk factors for recurrence in the training set

Univariate analysis showed that sex (P < 0.001), tumor size (P = 0.003), margin (P = 0.001), lymph node involvement (P = 0.003), surgery (P = 0.015), and BRAF V600E status (P < 0.001) were associated with SIR. Coincidentally, all of these risk factors were also significant in the multivariate analysis, including male sex (hazard ratio [HR]: 1.94, 95% confidence interval [CI]: 1.11–3.38; P = 0.02), lymph node metastasis (HR: 2.54, 95%CI: 1.44–4.48; P = 0.001), and positive BRAF V600E status (HR: 16.48, 95%CI: 3.66–74.27; P < 0.001). Additionally PTC size (HR: 0.48, 95%CI: 0.27–0.86; P = 0.013), thyroidectomy (HR: 0.34, 95%CI: 0.18–0.66; P = 0.001), and unclear margin (HR: 0.48, 95%CI: 0.27–0.85; P = 0.012) had strong prognostic benefits. Among these, we found that BRAF V600E status provided the largest contribution to the model and lymph node metastasis provided the next largest contribution. Detailed information is shown in Table 2.

Table 2.

Univariate and multivariate analysis in the training cohort.

Independent variable	Univariate analysis			Multivariate analysis
	Hazard Ratio	95%CI	P Value	Hazard Ratio	95%CI	P Value
Age	1.04	0.62–1.76	0.885	NA	NA	NA
Sex	2.89	1.71–4.88	<0.001	1.94	1.11–3.38	0.02
Size	0.43	0.25–0.75	0.003	0.48	0.27–0.86	0.013
Shape	0.74	0.34–1.6	0.44	NA	NA	NA
Orientation	1.66	0.98–2.82	0.06	NA	NA	NA
Margin	0.39	0.23–0.67	0.001	0.48	0.27–0.85	0.012
Capsule	0.56	0.3–1.05	0.071	NA	NA	NA
Internal echotexture	0	0–Inf	0.996	NA	NA	NA
Nodule contents	0	0–Inf	0.996	NA	NA	NA
Calcification	1.04	0.58-1.88	0.89	NA	NA	NA
Lymph node involvement	2.24	1.32–3.79	0.003	2.54	1.44–4.48	0.001
Surgery	0.47	0.26–0.87	0.015	0.34	0.18–0.66	0.001
BRAF V600E	22.03	5.22–92.93	<0.001	16.48	3.66–74.27	<0.001
Galectin_3	5.55	0.76–40.72	0.092	NA	NA	NA
HBME_1	1.98	0.48–8.15	0.343	NA	NA	NA

CI, confidence interval.

Prognostic nomogram for recurrence

The significant factors associated with SIR were integrated to form the nomogram (Figure 2). The C-index of the nomogram was 0.774 (95%CI: 0.703–0.845) in the validation set. Calibration plots at 3 and 5 years showed no apparent departure between the predicted recurrence probability and actual recurrence proportion in the validation set (Figure 3). Additionally, decision curve analysis was used to estimate the potential clinical utility of the nomogram at 3 and 5 years. Compared with cases of treat-all and treat-none, this nomogram had good net clinical benefit (Figure 4).

Figure 2.

Nomogram to predict the probability of 1-, 3-, and 5-year structural recurrence. By drawing a straight line perpendicular to the score axis, each predictor corresponds to a specific point. The total points on the risk axis represents the possibility of recurrence in 1, 3, and 5 years.

Figure 3.

Calibration plots of the nomogram in the validation set. (a) Calibration plot of predicted structural recurrence probability and actually structural recurrence proportion in the validation set at 3 years and (b) Calibration plot of predicted structural recurrence probability and actually structural recurrence proportion in the validation set at 5 years.

Figure 4.

Decision curve analysis (DCA) for the model. The x-axis represents probability, and the y-axis represents net benefit. The red line represents the nomogram. (a) The DCA for 3-year structural recurrence by the nomogram in the validation set and (b) The DCA for 5-year structural recurrence by the nomogram in the validation set.

Discussion

To examine the risk of SIR in low- and median-risk patients, we included patients with stage 1 and 2 (primarily stage 1) single PTC lesions in this study. The inclusion factors included complete clinical data, ultrasound findings, and molecular markers. Compared with another related model,¹³ our study had a higher C-index and increased ability to evaluate prognosis.

In our nomogram, BRAF V600E status was important for predicting SIR. At the same time, the prognostic significance of BRAF V600E status for recurrence is currently controversial. Mahmoud et al.²² found that BRAF V600E status did not affect loco-regional recurrence, distant metastasis, overall survival, or disease-free survival in 128 patients. Emunah et al.²³ found that BRAF mutation was an independent predictor of long-term recurrence through the follow-up of 599 patients after total thyroidectomy. They believed that understanding molecular markers would be more helpful for long-term monitoring of patients. Using a 20-year follow-up study of 53 patients with advanced PTC, Taciana et al.²⁴ concluded that BRAF V600E status was related to extra-thyroid invasion and local invasiveness and should be considered a prognostic marker. This model also found that its BRAF V600E status was related to recurrence and provided some basis for the impact of this mutation on prognosis.

Factors that had a large impact on this model included lymph node metastasis, sex, and surgical method. Lymph node metastasis was recently recognized to be a prognostic factor by previous studies.^25–27 As described in the ATA risk stratification system (2015), more attention should be paid to the number and maximum diameter of metastatic lymph nodes. The degree of invasion in metastatic lymph nodes may increase the risk of recurrence for PTC.¹⁵ Some have believed that early lymph node metastasis was a more critical factor.²⁸ Although we did not analyze the lymph nodes in detail, our study also reflected this result. Similarly, consistent with the results of other studies, male sex was associated with a higher risk of recurrence in this study.^29–31

The choice of surgical methods has always been controversial because thyroidectomy may have higher rates of damage to the neck, such as recurrent laryngeal nerve injury or damage to the brachial plexus nerve, lymphatic fistula, trachea, or large blood vessels. The 2015 ATA guidelines also recommend lobectomy for low-risk patients. Yossi et al.³² found that lobar resection did not affect recurrence for low-risk patients after a follow-up of 109 patients with stage I disease after lobectomy and 50 patients with thyroidectomy. In another study, Min et al.³³ examined 147 patients with PTC of 1 cm to 4 cm who were followed-up for up to 7 years, and they concluded that lobectomy did not affect recurrence compared with thyroidectomy. Zhu et al.³⁴ concluded that surgical method was involved with recurrence, especially total thyroidectomy combined with lymph node dissection, which may reduce the risk of recurrence. The results of this study were that surgical method, especially lobectomy, was an independent risk factor for recurrence, similar to Zhu et al.³⁴ Further studies are required to resolve this issue.

One innovation of this model was that this study included ultrasonic features, including margin, shape, echotexture, nodule contents, and calcification. However, only margin was included in the model. Generally, the strongest independent predictor for PTC has been considered to be unclear margin. In contrast, we found that the recurrence of nodules with clear boundaries may be higher than previously thought. This difference may be due to subjectivity in evaluating margins. If a nodule was located in the lobe without invasion through the thyroid capsule, ill- or well-defined margins have a subjective nature, especially for novices. There may be more PTMC in the cohort, and these margins are not easily distinguished. In such cases, the personal judgment of operators will influence the result to a great degree. Therefore, specific findings need to be studied.

Regarding the relationship between size and recurrence, some studies have concluded that size (>4 cm) affects recurrence.³⁵ Our findings also showed that nodule size had an impact on recurrence, although we only divided nodules into PTMC and PTC. Interestingly, we found that the recurrence rate of PTMC patients was higher. Ultrasound is widely used in the clinic, and the micropapillary subtype has become the dominant form of PTC.³⁶ In addition, there was a 58.7% proportion of PTMC in our cohort. This phenomenon of a high recurrence rate of PTMC does not mean that PTMC is more invasive but may be due to the fact that these cases were a relatively high proportion in this cohort. In particular, most PTMC cases were treated by lobectomy. Nevertheless, it cannot be ruled out that some PTMCs are highly invasive and that cases with simple lobectomy are more likely to relapse.

In this study, we constructed a nomogram model on the basis of clinical information that is easy to obtain and can quickly predict SIR. In particular, we added molecular markers that were not included in the 8th edition AJCC staging system. At the same time, the nomogram is different from other models and has specific innovations.

There are some limitations to this study. First, it was a retrospective study with all the inherent drawbacks, and the nomogram was not validated in newly-diagnosed patients. Second, the analysis was performed at a single center, PTMC accounted for most cases, and there were some limitations in extracting features such as margin and shape. Third, tumor multifocality and biochemical indexes that are currently considered to affect prognosis (such as Tg level) were not included.^37–40 Fourth, the study was conducted in Qingdao, which has sufficient iodine and therefore may not necessarily be applicable to patients in iodine-deficient areas. Fifth, according to the 4th World Health Organization Classification of Thyroid Tumors,¹⁸ there are more than 10 subtypes of PTC. Among them, tall cell PTC, column cell PTC, and hobnail PTC are aggressive PTCs. In this study, PTC subtypes were not included in the analysis; therefore, the aggressive PTCs may affect prognosis in ways not detailed in this study.

Conclusion

This study focused on the combined use of clinical data, ultrasonic findings, and molecular markers to predict structural recurrence in PTC patients. Our nomogram had a high C-index, which showed that it was feasible in these cases. Nevertheless, further studies are required to confirm these findings. These patients will be followed-up for 10 years, at which point we will also verify the nomogram in newly-diagnosed patients.

Data availability statement

The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

ORCID iD

Chenchen Geng https://orcid.org/0000-0002-7495-9068