Diagnostic accuracy of thyroid nodules with indeterminate cytology when combined with ultrasonographic risk

ABSTRACT

Objectives:

To evaluate the added diagnostic value of ultrasonographic risk levels to cytological diagnoses in nodules with Bethesda III and IV cytology.

Methods:

This retrospective study was conducted in Jeddah, Saudi Arabia, and involved analyzing thyroid nodules from patients who underwent thyroidectomy between 2016 and 2023. We focused on pathology, US risk stratification based on the 2015 American Thyroid Association (ATA) guidelines, and fine-needle aspiration cytology (FNAC) using the Bethesda system. We calculated malignancy rates for each ultrasound (US) and FNAC category, with the indeterminate cytology groups defined as follicular lesions of undetermined significance (FLUS) and follicular neoplasms (FN).

Results:

In 290 patients, malignancy rates in the high-risk US group were 72%. In patients with FLUS, the malignancy rate was 36.9%, while in those with FN, it was 50%. For FLUS and FN, high-risk US features showed a sensitivity of 47%, a specificity of 81%, and a kappa of 29%. Combining FLUS with high-risk US features improved sensitivity to 50%, specificity to 80%, and kappa to 32%.

Conclusion:

For indeterminate thyroid cytology, US features can guide decision-making, supporting surgery in patients with high-risk US findings rather than follow-up or repeat FNAC.

Thyroid nodules are a common health problem, with a 40-50% prevalence in some populations. This high prevalence is likely related to the advances and frequent use of imaging modalities in modern days.^1,2 In Gulf Cooperation Council States generally, and in Saudi Arabia specifically, the incidence of thyroid nodules and thyroid malignancies are increasing as well.³

Fine-needle aspiration cytology (FNAC) is one of the most accurate diagnostic tests for thyroid nodules, with sensitivity of 83% and specificity of 92% when excluding the indeterminate cytology.⁴ Nevertheless, the cytology result of about 25% of all thyroid nodules remains indeterminate, and those are the challenging ones.⁵ Indeterminate thyroid cytology is grouped into three categories based on the Bethesda system for reporting thyroid cytopathology. The first is atypia of undetermined significance/follicular lesion of undetermined significance (AUS/FLUS), Bethesda III, which is now called AUS in the updated Bethesda system. The other 2 groups are follicular neoplasm/suspicious for a follicular neoplasm (FN/SFN), Bethesda IV, and suspicious for malignancy (SUSP), Bethesda V. The risk of malignancy ranges between 5 and 50% in FLUS and FN cases.^6,7

Ultrasonography (US) is a noninvasive, accurate, and widely available method for evaluating thyroid nodules. Several guidelines have been employed to stratify the risk of malignancy of thyroid nodules based on ultrasonographic features, such as the American Thyroid Association (ATA) guidelines and the American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS) guidelines. Based on the presence or absence of suspicious sonographic features, the nodules are stratified to the respective malignancy risk. Ultrasonographic findings of solid, irregular nodule margins; marked hypoechogenicity; a taller than wider nodule dimension; and microcalcifications suggest malignancy.⁸

Overall, diagnosing thyroid malignancy in indeterminate cytology is still challenging despite advances in imaging and cytopathology techniques. However, the combination of US risk stratification with the Bethesda cytological classification has been shown to increase the accuracy of diagnoses. Nonetheless, studies on this issue are limited.⁹

This study aims to assess the added diagnostic value of ultrasonographic risk level to the cytological diagnosis of nodules with Bethesda III and IV cytology.

Methods

This retrospective study was conducted at King Abdul-Aziz University Hospital, in Jeddah, Saudi Arabia. Patients enrolled in the study were adults who underwent total or hemithyroidectomy secondary to thyroid nodules during the period 2016-2023. Patients without thyroid nodules, or FNAC, or US, or pediatric patients (<18 years of age) were excluded from the study. Ethical approval was obtained from the Biomedical Research Ethics Committee, King Abdul Aziz University Faculty of Medicine, Jeddah, Saudi Arabia (Grant numbers 114-23).

The following data were collected from patients’ electronic medical records: age, gender, preoperative thyroid-stimulating hormone (TSH), and US risk stratification of the nodule from which the FNAC was obtained based on the 2015 ATA thyroid nodule guidelines (benign, very low, low, intermediate, and high). The US procedures were conducted by a radiographer using a high-frequency linear transducer (10 MHz) and subsequently reviewed by a consultant radiologist. The ATA risk was determined based on the nodule description in the US report. In cases where the ATA ultrasonographic score was unclear from the US description, the images were reviewed by a head and neck radiologist to assign the risk score.

Fine-needle aspiration cytology results were reported based on the Bethesda categories (Bethesda I = nondiagnostic, Bethesda II = benign, Bethesda III = FLUS/AUS, Bethesda IV = FN/SFN, Bethesda V = SUSP, Bethesda VI = malignant).

Statistical analysis

Means (SD) and frequencies with percentages were used to describe continuous and categorical variables, respectively. The overall study sample was categorized based on the thyroid histopathology into malignant or benign tumors. To compare descriptive statistics between malignant and benign tumors, we used the t-test and Chi-square test for means of normally distributed variables and frequency distributions of categorical variables, respectively. We conducted further analyses on patients with indeterminate cytology (FLUS combined with FN, and FLUS alone).

To determine whether US features may improve diagnostic accuracy in patients with indeterminant cytology, we considered high-risk US features as the radiological cutoff for malignancy. Since the usual challenge is in Bethesda III and IV, we assessed the added diagnostic value of the ultrasonographic risk features to these FNAC categories in diagnosing malignancy. Then we calculated the respective sensitivity, specificity, and kappa statistics. All analyses were performed with SAS 9.4 (SAS Institute, Cary, NC) with a significance level set at 0.05.

Results

There were 290 patients with valid US risk assessment and FNAC results, and the majority were female patients (85%). The mean age of the patients was 43.7±14.3 years, and the median size of the detected nodules was 3.0 cm (Q1, Q3 are 2.0, 4.5). Table 1 further describes the baseline characteristics of the study cohort.

Table 1.

- Baseline characteristics of the study sample based on histopathology.

Variables	Total N=290	Histopathology		P-value
		Benign n=149	Malignant n=141
Age, years, mean ± SD	43.7 ± 14.3	44.2 ± 13.4	43.2 ± 15.2	0.54
Preoperative TSH median (Q1, Q3)	1.5 (0.8, 2.6)	1.4 (0.8, 2.4)	1.7 (0.9, 2.9)	0.04
Size of the nodule, cm, median (Q1, Q3)	3.0 (2.0, 4.5)	3.3 (2.0, 4.7)	2.8 (1.8, 4.4)	0.24
Age categories, n (%)				0.59
≤40 years	131 (45)	65 (44)	66 (47)
>40 years	159 (55)	84 (56)	75 (53)
Gender, n (%)				0.13
Female	246 (85)	131 (88)	115 (82)
Male	44 (15)	18 (12)	26 (18)
Type of surgery, n (%)				0.0002
Hemithyroidectomy, n (%)	71 (25)	51 (34)	20 (14)
Total thyroidectomy, n (%)	217 (75)	98 (66)	119 (86)

TSH: thyroid-stimulating hormone

US findings were benign in 14 nodules, very low risk in 9 nodules, low risk in 114 nodules, intermediate risk in 59 nodules, and high risk in 94 nodules. The final histopathological diagnosis is shown in Table 2. Of the 290 FNAC, 32 were inadequate, 78 were benign, 74 were AUS/FLUS, 44 were FN/SFN, 32 were SUSP, and 30 were malignant. The final histopathology results in different Bethesda classes are shown in Table 3.

Table 2.

- Proportion of malignancy in each ultrasound (US) category.

US Features	Histopathology		Total
	Benign	Malignant
Benign	11 (79)	3 (21)	14 (5)
Very low risk	5 (56)	4 (44)	9 (3)
Low risk	72 (63)	42 (37)	114 (39)
Intermediate risk	35 (59)	24 (41)	59 (20)
High risk	26 (28)	68 (72)	94 (33)
Total	149	141	290

Values are presented as numbers and percentages (%)

Table 3.

- Proportion of malignancy in each Bethesda category.

Bethesda category	Histopathology		Total
	Benign	Malignant
Nondiagnostic	17 (53)	15 (47)	32 (11)
Benign	60 (77)	18 (23)	78 (27)
FLUS or AUS	46 (62)	28 (38)	74 (26)
FN/suspicious of FN	21 (48)	23 (52)	44 (15)
Suspicious of malignancy	4 (13)	28 (87)	32 (11)
Malignant	1 (3)	29 (97)	30 (10)
Total	149	141	290

Values are presented as numbers and percentages (%). FLUS: follicular lesion of undetermined significance, AUS: atypia of undetermined significance, FN: follicular neoplasm

In patients with FLUS/FN cytology, we found the presence of high-risk US features was associated a 65% risk of malignancy, while the absence of high-risk features was associated with only 33.3% malignancy rate. Sensitivity (47%), specificity (81%), and kappa (29%, fair agreement) as measures for the accuracy of high-risk US features for determining malignancy. Table 4 has the full description of the results. A similar trend was also observed when FLUS was examined separately with high-risk US features individually. When restricting the analysis to those with only FLUS, the risk of malignancy in high-risk US features is 60.9%. The sensitivity (50%), specificity (80%), and kappa (32%, fair agreement) remained almost similar (Table 5).

Table 4.

- Accuracy of high-risk ultrasound (US).

US Features	Histopathology		Total
	Malignant	Benign
High risk	24 (64.9%)	13 (35.1%)	37 (100%)
Non-high risk	27 (33.3%)	54 (66.7%)	81 (100%)
Total	51	67	118
	Sensitivity (95% CI)	Specificity (95% CI)	Kappa (95% CI)
	0.47 (0.33, 0.62)	0.81 (0.69, 0.89)	0.29 (0.12, 0.46)

Values are presented as numbers and percentages (%)

Table 5.

- Accuracy of high-risk ultrasound (US).

US Features	Histopathology		Total
	Malignant, n (%)	Benign, n (%)
High risk	14 (60.9%)	9 (39.1%)	23 (100%)
Non-high risk	14 (27.5%)	37 (72.5%)	51 (100%)
Total	28	46	74
	Sensitivity (95% CI)	Specificity (95% CI)	Kappa (95% CI)
	0.50 (0.31, 0.69)	0.80 (0.66, 0.91)	0.32 (0.09, 0.54)

Discussion

In this study, we found that combining thyroid nodule US risk score with the challenging Bethesda III and IV cytology results improves the prediction of malignancy, with the rate of malignancy reaching approximately 70% in nodules with high-risk features and indeterminate cytology compared with only 38-52% when relying on regional cytology alone.

In a study from Brazil among 142 postoperative thyroid nodules, it was found that ATA high-risk US features yielded an 80% malignancy rate, while the intermediate-risk US yield was 43%, and low-risk yield was 13% malignancy rate.¹⁰ This is comparable to our results for the high and intermediate risks, but the low-risk US features in our cohort still had a higher rate of malignancy (37.5%). These rates are different from those reported in the ATA guidelines, where thyroid cancer rates in ultrasonographic images labeled as high-risk nodules are 70-90%, 10-20% in intermediate-risk, 5-10% in low-risk, and 3% in very low-risk nodules.⁸ Depending on regional thyroid cancer epidemiology and the availability of thyroid cytology expertise, cancer rates in thyroid cytology samples differ globally and might vary from center to center in the same region. In a local study from Saudi Arabia performed solely on AUS/FLUS cytology, the overall malignancy rate was 46.5%.¹¹ This is marginally higher than the one reported by our center, which is 36.9%.

In this research, we found that integrating the US risk with the Bethesda category enhances cancer prediction in indeterminate thyroid cytology. Similarly, another study on Bethesda AUS/FLUS and FN thyroid nodules concluded that the malignancy rate for patients with the high-risk US was 70.7%; in intermediate-risk, it was 45.9%. It also reported that nuclear atypia played a key role in influencing the risk of malignancy, which aligns with the most updated Bethesda classification in 2023.^12,13 Some authors found that the US stratification system exhibited higher sensitivity and better negative predictive value (NPV) than the Bethesda scoring system. When retrospectively analyzing 192 post-total thyroidectomy patients, Boudina et al¹⁴ concluded that the sensitivity of the US score was 64.2% versus 33.3% for the Bethesda score, and the NPV was 34.5% versus 24.4%. Both scoring systems demonstrated equivalent specificities (90.9% versus 100%) and positive predictive values (97.1% versus 100%).¹⁴

Additionally, Barbosa et al¹⁵ investigated thyroid malignancy rates in indeterminate cytology and linked it to the ATA US risk stratification. They divided the cohort into patients with FLUS/AUS and patients with FN or SUSP. They concluded that in patients with AUS/FLUS cytology and non-high-risk US, the probability of malignant final pathology is 5.7%. In comparison, it was 55.5% if the patient had high-risk US features. Furthermore, if the patient had FN or SUSP cytology, the rate of malignancy is 36.7% if the patient had a non-high-risk US, while it is 87.2% if the patient had a high-risk US for the same Bethesda category.¹⁵

In a similar study by Kim et al,¹⁶ conducted exclusively on cases with indeterminate cytology, the malignancy rate in patients with high-risk US features was 98% and 79.6% with intermediate-risk US. They also categorized their sample into nodules with positive BRAF mutation and negative BRAF mutation, and expectedly BRAF positive mutation increased the probability of cancer within the same Bethesda and US risks.¹⁶ With the introduction of molecular genetic testing for thyroid nodules, there was a paradigm shift in the management of thyroid nodules with indeterminate cytology. Molecular diagnostic markers can identify benign outcomes with excellent sensitivity and NPV, thus obviating the need for surgery in some cases.¹⁷ In addition, multiple studies have proven the cost-effectiveness of molecular testing for indeterminate thyroid nodules versus diagnostic lobectomy surgeries.^18,19

Nevertheless, these techniques are still costly and not widely available worldwide. In that case, it is essential to interpret thyroid cytology in the light of the radiological and cytologic assessment of the targeted nodules. This notion was reflected by the most recent international thyroid nodules guidelines by the European Thyroid Association and Korean Thyroid Association, both published in 2023, where the management of each Bethesda category was tailored based on the nodule EU-TIRADS/K-TIRADS risk and the nodule size.^20,21

Based on the aforementioned findings, it is reasonable to consider the surgical option as the primary management in patients with indeterminate cytology and high-risk US features rather than repeating the FNAC or follow-up. However, our study has several limitations. Firstly, it is a single-center and retrospective experience. Secondly, US images were assessed by different radiologists, so interobserver variability is possible.

In conclusion, in the era of individualized patient management, it is essential to integrate radiological features with cytology results to decide the best course of action for each thyroid nodule.