Comparison of Multimodal Ultrasound Imaging with Conventional Ultrasound Risk Stratification Systems in Presurgical Risk Stratification of Thyroid Nodules

Abstract

Background:

Ultrasonography (US) is an indispensable tool in the management of thyroid nodules, not only for assessing tumor characteristics but also to assign risk of malignancy and guide in management. Various guidelines and US-based risk stratification systems have been proposed for this purpose. This study aims to compare the diagnostic performances of multimodal US-based risk scores (French TIRADS, TMC-RSS) with conventional US-based scoring systems (Korean TIRADS, ACR-TIRADS, ATA risk stratification).

Material and Methods:

A total of 168 nodules from 139 patients were studied and categorized in each of the risk stratification systems. Sensitivity, specificity, positive and negative predictive values, and accuracy of each system were computed. ROC curves were plotted and area under curve (AUC) for each scoring system noted.

Results:

Thirty five (21%) of the 168 nodules were malignant on final histopathological examination. TMC-RSS fared the best in predicting malignant nodules with a sensitivity of 96.2% and specificity of 88.6%, while the PPV and NPV were 97% and 86.1%, respectively. The AUC for TMC-RSS was 0.924 (95% CI, 0.860–0.988; P < 0.001).

Conclusion:

Multimodal US-based risk stratification incorporating non-grayscale characteristics in addition to conventional systems like the TMC-RSS improves the diagnostic performance of ultrasound imaging of thyroid nodules.

INTRODUCTION

The prevalence of thyroid nodules has increased over the past few decades, mostly due to advances in imaging techniques. The reported prevalence ranges from 2% in iodine sufficient areas to up to 45% in iodine-deficient areas.[1] High-resolution ultrasound (US) can detect thyroid nodules in 19–68% of random individuals with increased incidence in females and elderly.[2] It is important to exclude malignancy in these nodules, seen in 7–15%.[2] In India, thyroid malignancies account for 1.8% of all cancers, with about 18,600 cases diagnosed every year.[3] Mortality rates, however, are very low, responsible for 0.4–0.5% of all cancer-related deaths. This mortality rate has remained rather stable in spite of increase in the incidence of thyroid cancers, attributable to improvements in diagnostics and possible change of risk factors.[4]

US has become an indispensable tool in the management of thyroid nodules, not only for assessing the tumor characteristics but also to assign risk of malignancy and formulate management strategies. Various guidelines and US-based risk stratification systems have been proposed to guide surgeons toward optimal management in thyroid nodules. The first among such classification systems was proposed in 2009 by Horvath et al., based on an already established risk classification system for breast lumps, called the Thyroid Imaging Reporting and Data System (TIRADS).[5] Many other versions of the TIRADS have been proposed since then, like the KWAK-TIRADS, Korean TIRADS, ACR-TIRADS, and so on. Professional academic bodies like the American Thyroid Association (ATA) and the British Thyroid Association also devised a risk stratification system based on the US findings. These systems utilize different parameters in US parameters and hence differ in their diagnostic performances. There are many US characters (like vascularity, tissue elasticity, etc.) not featured in the TIRADS but are known to improve the diagnostic capabilities of US when incorporated. Multimodal scoring systems including these additional features were proposed to improve the diagnostic accuracy, like the French TIRADS, TMC-RSS, etc. Many studies comparing conventional US-based risk scores have been performed but there is a dearth of studies comparing multimodal risk scores with the conventional scoring systems. The purpose of this study was to compare the diagnostic performances of conventional versions of TIRADS, ATA risk stratification system, and multimodality scoring systems in identifying malignant thyroid nodules.

MATERIAL AND METHODS

This cross-sectional, observational study was performed in the Departments of Endocrine Surgery and Radiodiagnosis at the King George's Medical University, Lucknow, India. Patients with thyroid nodules who satisfied the inclusion criteria were recruited in the study after obtaining written consent. Clearance from the institutional ethical committee was obtained on 20/ 02/ 2019. A total of 161 patients were approached and after exclusion, 168 nodules from 139 patients were studied from March 2018 to October 2019. A dedicated radiologist performed the US evaluation and findings were recorded on predesigned proforma. Before starting the study, training sessions were held to establish a baseline consensus on the performance, evaluation, and interpretation of US. The optimal US was defined as image that was acquired while the patient held his or her breath, without any motion artifacts.

Ultrasonography was performed by using LA533 apple probe linear array transducer (Esaote) of 12 MHz frequency. Adequacy of external compression was assessed via the quality indicator and a compression of more than 50% on the scale was considered optimum. Elastograms were obtained from the transverse plane by manually setting the region of interest within the lesion. Both colorimetric elastograms (Asteria classification) and strain ratio were obtained.[6] Color Doppler was used to assess vascularity and flow pattern was noted. The histopathological examination (HPE) reports were obtained postoperatively.

Ultrasound findings were analyzed for baseline parameters. Nodule characteristics were studied and categorized in each of the following scoring systems:

• Korean TIRADS: proposed by the Korean Society of Thyroid Radiology.[7]

• ACR-TIRADS: proposed by the American College of Radiology.[8]

• ATA risk stratification.[2]

• French TIRADS: proposed by the French Society of Thyroidology.[9]

• Thyroid Multimodal Imaging Comprehensive Risk Stratification System (TMC-RSS): proposed by Tata Memorial Hospital, Mumbai.[10]

The US features considered in each scoring system are shown in Table 1.

Table 1.

US features

	K-TIRADS	A-TIRADS	ATA	F-TIRADS	TMC-RSS
Composition	+	+	+	+	+
Echogenicity	+	+	+	+	+
Orientation	+	+	+	+	+
Margins	+	+	+	+	+
Calcification	+	+	+	+	+
Lymph nodes				+	+
Elastography				+	+
Vascularity					+
Halo					+
Comet tail artifact					+
Negative score (for benign features)					+
Comments/Criticism	Emphasis on US patterns, therefore difficulty in classification of nodules into risk groups	Technically complex, as numerical values are used	Importance to all suspicious features, but little emphasis on independent risk factors like composition of nodule	Difficulty in reproducibility, requires experience to perform and interpret elastography	A quantitative scoring system which has conventional features with Color Doppler, elastography, and negative scores for benign features

Inclusion and exclusion criteria

All nodules measuring 4 cm or less were included in the study. Patients with diffuse thyroid enlargement, autoimmune and inflammatory disorders, and those patients not willing to participate in the study were excluded.

Analysis of data

Data were analyzed and reported as the mean ± SD for continuous variables and frequency (percentage) for categorical variables. The P values were calculated by the t-test or Mann–Whitney U test for continuous variables and the Chi-square test or Fisher exact test for categorical variables. Multivariate logistic regression was performed to test the association of different parameters. Significance was set at P value equal to or less than 0.05. All statistical tests were performed using SPSS software (version 23).

RESULTS

Among the 139 patients, 115 (82.8%) were females and 24 (17.2%) males. Mean age of the patients was 35.3 + 13.2 years (range 9–70 years). Thirty five (21%) of the 168 nodules were malignant on final HPE. Mean tumor size was 2.93 + 0.67 cm for benign nodules, while malignant nodules it was 3.1 + 0.78 cm. Cytological and histological characteristics of nodules are shown in Table 2.

Table 2.

Cytological and histological characteristics of nodules

Bethesda category	n (%)	Histopathology (postoperative)	n (%)
I	0 (0)	Colloid nodule	129 (76.8%)
II	140 (83.3%)	NIFTP	4 (2.4%)
III	2 (1.2%)	PTC	15 (8.9%)
IV	14 (8.3%)	FVPTC	7 (4.2%)
V	2 (1.2%)	FTC	11 (6.5%)
VI	10 (6%)	HCC	1 (0.6%)
		MTC	1 (0.6%)

For analyses of data and risk assignment, three groups were formed. The low risk group comprised TIRADS 1–3 of Korean and ACR, 4A of the French systems, benign through low suspicion subcategories of ATA, and category 1 of TMC-RSS. TIRADS 4 of Korean and ACR, TIRADS 4B of the French system, intermediate-suspicion category of ATA, and category 2 of TMC-RSS were grouped as intermediate risk for malignancy, while the remainder were classified as high risk for malignancy. The risk stratifications of the nodules, according to different scoring systems, are presented in Table 3.

Table 3.

US Risk stratification

	K-TIRADS		ACR-TIRADS		ATA risk		F-TIRADS		TMC-RSS
	Total	Malignant	Total	Malignant	Total	Malignant	Total	Malignant	Total	Malignant
Low risk	124	9 (7.2%)	127	9 (7%)	126	10 (7.93%)	132	6 (4.5%)	132	3 (2.3%)
Intermediate risk	28	12 (42.8)	27	13 (48.1%)	11	1 (9%)	18	13 (72.2%)	9	5 (55.5%)
High risk	16	14 (87.5%)	14	13 (92.8%)	31	24 (77.4%)	18	16 (89%)	27	27 (100%)
Comments-from our study	On comparison with F-TIRADS or TMC-RSS, there were both risk upgrades and downgrades, details in Table 4 TMC-RSS was used for comparison as it comprises grayscale features along with additional non-grayscale parameters

Risk reassignment from conventional TIRADS and ATA to multimodal systems is shown in Table 4. Notable risk reassignment was observed from conventional TIRADS to TMC-RSS. Most significant reassignment was seen from the intermediate-risk category, where 12 nodes were downgraded to low risk, while another 12 nodules were upgraded to high risk, reducing the number of nodules in the intermediate category.

Table 4.

Risk reassignment

Risk upgrade
	F-TIRADS		TMC-RSS
	A-TIRADS → F-TIRADS	ATA → F-TIRADS	A-TIRADS → TMC-RSS	ATA → TMC-RSS	F-TIRADS → TMC-RSS
Low to intermediate	4 (3)	6 (3)	7 (6)	7 (6)	2 (1)
Intermediate to high	5 (5)	1 (0)	12 (12)	2 (1)	9 (9)
Comments-from our study	8 of the 9 nodules upgraded to intermediate risk are malignant on final HPE	3 of the 7 nodules upgraded to intermediate risk are malignant on final HPE	18 of the 19 nodules upgraded to intermediate risk are malignant on final HPE	7 of the 9 nodules upgraded to intermediate risk are malignant on final HPE	10 of the 11 nodules upgraded to intermediate risk are malignant on final HPE
*Total malignant nodules on final HPE in brackets
Risk downgrade
	F-TIRADS		TMC-RSS
	A-TIRADS → F-TIRADS	ATA → F-TIRADS	A-TIRADS → TMC-RSS	ATA → TMC-RSS	F-TIRADS → TMC-RSS
Intermediate to low	8 (8)	7 (6)	12 (12)	6 (6)	4 (4)
High to intermediate	-	8 (4)	-	2 (1)	-
High to low	1 (1)	2 (2)	1 (1)	4 (4)	-
Comments-from our study	All 9 nodules downgraded to low risk are benign on final HPE	8 of the 9 nodules downgraded to low risk are benign on final HPE	All 13 nodules downgraded to low risk are benign on final HPE	All 10 nodules downgraded to low risk are benign on final HPE	All 4 nodules downgraded to low risk are benign on final HPE
*Total benign nodules on final HPE in brackets

HPE=Histopathological examination

The diagnostic performances of all the US-based scoring systems in differentiating benign and malignant nodules were analyzed. Sensitivity, specificity, positive and negative predictive values, and accuracy of each system were computed. For this purpose, low-risk score was considered as a predictor of benignity, while high-risk scores were considered malignant. As the intermediate-risk score is an area of uncertainty, two separate analyses were performed, one considering the intermediate score as a benign and the other as an indicator of malignancy. Table 5 shows the overall performance for both considerations. Specificity increased but sensitivity reduced when intermediate risk was considered as an indicator of malignancy. For further analyses, intermediate-risk group was considered as an indicator of malignancy. Figure 1 shows the Positive predictive value (PPV), Negative predictive value (NPV), and accuracy of all scoring systems. TMC-RSS performed better than the other scoring systems.

Table 5.

Sensitivity and specificity

	Intermediate risk considered benign		Intermediate risk considered malignant
	Sensitivity	Specificity	Sensitivity	Specificity
K-TIRADS	97	48.6	86.5	71.4
ACR-TIRADS	99.2	37.1	88.7	71.4
ATA	96.2	68.6	91	74.3
F-TIRADS	99.2	48.6	94.7	80
TMC-RSS	99.2	68.6	96.2	88.6

Figure 1.

PPV, NPV, and accuracy

Receiver operating characteristics curves were plotted for all the systems. The area under curves (AUC) improved with addition of auxiliary parameters [Figure 2]. All scoring systems showed AUC of more than 0.8 indicating an excellent performance, except the K-TIRADS which had an AUC of 0.78. TMC-RSS showed maximum AUC of 0.92, reiterating its superior performance in identifying malignant nodules.

Figure 2.

ROC curves and AUC

Test Result Variable(s)	Area	Sth. Error	Asymptotic Sig.	Asymptotic 95% Confidence Interval
				Lower Bound	Upper Bound
K-TIRADS	0.789	0.048	0.000	0.695	0.884
A-TIRADS	0.801	0.048	0.000	0.707	0.895
ATA	0.826	0.046	0.000	0.736	0.917
F-TIRADS	0.874	0.041	0.000	0.792	0.955
TMC-RSS	0.924	0.033	0.000	0.860	0.988

DISCUSSION

US is considered as an extension of clinical examination in the context of evaluation of a thyroid nodule. Due to the high prevalence of thyroid nodules, it is very important to identify those at high risk for malignancy. Although many US features are proven to be robust indicators of malignancy, no single feature is reliably predictive.[2] Hence, many risk-stratification models have been developed that combine several suspicious US features in order to improve the diagnostic ability of US. Each system ascribes differential degree of risks to the individual US features in order to determine a nodule's risk of malignancy, and this risk assigned to a particular feature varies substantially in each system. As a result, no system is universally accepted.[11]

The 4-tier K-TIRADS is simple to use and analyze but has been criticized for laying emphasis on US patterns rather than the high-risk findings themselves. This makes it difficult to classify nodules which lack a typical pattern but carry high-risk findings.[12] The ACR-TIRADS integrates all US features, which are assigned a numerical score based on their malignant potential. It is technically more complex than other systems. The reported drawback of this system is that nodules with mixed echogenic patterns may be placed in a lower grade, resulting in false-negative diagnoses.[11] In our study, we observed similar difficulties with categorization and risk assignment in the conventional US-based scoring systems due to overlapping findings. False negativity was 7.2% in K-TIRADS and 7% in ACR-TIRADS, while false positivity was 12.5% and 7.1%, respectively. Nodules classified into the intermediate-risk category were high (16.5% in K-TIRADS and 16% in ACR-TIRADS). The ATA guidelines, first proposed in 2009 and revised in 2015, give a 5-tier risk stratification system based on US features.[2] The limitation of this system is that it gives equal importance to all suspicious features, while laying little emphasis on independent risk factors like composition of a nodule.[11] In our study, the ATA risk stratification showed false negativity of 7.93% and highest false positivity (22.5%) among all scoring systems. About 6.5% of all nodules were of intermediate risk.

Attempts to standardize the US terminologies for diagnosis of thyroid nodules are still ongoing, and this has led to further advancements in US techniques. Addition of multiple non-grayscale parameters to the conventional US findings has shown a lot of promise in this regard.[13] The F-TIRADS is a 5-tier system which along with the conventional US high-risk features has stiffness of the nodule on elastography (ES) and suspicious lymph nodes as indicators of malignancy.[14] Initially criticized for its difficulty in reproducibility, it was subsequently shown to have a better interobserver agreement.[15] Requirement of fair amount of experience to perform and interpret elastographic findings may limit the extensive use of this system.[16] The false negativity in F-TIRADS was 4.5% while false positivity was 11.1%, and 11% nodules were categorized as intermediate risk. F-TIRADS showed better sensitivity and specificity compared to conventional US-based scoring systems (94.7% and 80%, respectively). This improvement in diagnostic performance has been shown in other studies evaluating utility of ES with conventional TIRADS.[17]

Along with elasticity, the role of nodule vascularity in diagnosis of malignancy in imaging studies has always been debated. Previously, studies have shown that vascularity alone is not a reliable indicator of malignancy.[18] But recent reports using advanced techniques have reemphasized the role of vascularity in diagnosis of malignancy on imaging.[19,20] The TMC-RSS is a quantitative algorithm for characterizing nodules and consists of conventional US features in combination with Color Doppler, ES, and cervical nodal status. It assigns a positive score for suspicious features and negative score for benign features. As it is completely a quantitative scoring system, it reduces interobserver reporting variability.[21] TMC-RSS had a false-negative rate of 2.27% while the false positivity was zero. The number of nodules classified as intermediate risk was the least among all scoring systems (5.35%). The initial study on TMC-RSS showed a sensitivity of 90%, specificity of 89%, and accuracy of 91%.[10] Our study showed 96.2% sensitivity, 91.4% specificity, and an accuracy of 94.7%. The most significant aspect of TMC-RSS was recategorization of intermediate nodules and reduction in number of both false positives and false negatives. Few nodules were also downgraded to intermediate risk from high risk and this reassignment may not have had any impact on the final results, as intermediate-risk category is also considered as an indicator of malignancy for final analysis. Many other studies have demonstrated improvement in diagnostic performance using additional multimodal parameters.[22,23] The ROC curves for each system were plotted. There was an incremental trend in the AUC from conventional TIRADS systems through multimodality systems. The TMC-RSS showed maximum AUC, confirming its better performance and improved diagnostic accuracy.

Similar attempts at utilizing multimodal US features in evaluation of thyroid nodules have shown promise. Another study incorporating minor features and negative score for benign characters showed comparable outcomes, with an AUC of 0.921, overall sensitivity of 82%, and specificity of 87.6%.[24] This study also concluded that it is possible to categorize all the nodules into one of the risk categories, unlike conventional US-based scoring systems where overlapping suspicious features may prevent appropriate categorization. Our study reflects a similar pattern, where TMC-RSS was successful in assigning a risk category to all the nodules. There are some limitations to our study. First, we have considered only nodules which are of 4 cm or less. Although initially restricted to small nodules, recent studies have shown that with advances in techniques, ES can be useful in large nodules as well.[25] Inclusion of nodules of all sizes will yield a better picture of applicability of the multimodality scoring systems. Second, an increase in the sample size will render the data more robust. Third, the study is performed in patients of a single specialty center by a single trained ultrasonologist. Hence, the results may not be replicable when applied in the community. In conclusion, multimodal ultrasound imaging risk stratification systems like the TMC-RSS improve the diagnostic performance of ultrasound imaging of thyroid nodules compared to scoring systems incorporating only conventional grayscale features.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.