Malignancy Risk of Thyroid Nodules That Are Not Classifiable by the American Thyroid Association Ultrasound Risk Stratification System: A Systematic Review and Meta-Analysis

Abstract

Background:

Sonographic evaluation is fundamental to thyroid nodule assessment. The American Thyroid Association (ATA) ultrasound risk stratification system (USRSS) is widely used, but the appearance of some nodules has been considered nonclassifiable (NC-ATA). The risk of malignancy (RoM) of NC-ATA nodules varies widely between studies, leading to uncertainty in clinical management. The aim of this study was to comprehensively evaluate the prevalence and malignancy risk of NC-ATA nodules.

Methods:

A systematic review was performed searching PubMed/MEDLINE and EMBASE to identify original studies of thyroid nodules classified using the ATA USRSS from 2016 to 2022 and reporting the outcome of NC-ATA nodules. Meta-analysis was conducted to obtain pooled RoM estimates and meta-regression sensitivity analyses were used to explore sources of between-study heterogeneity.

Results:

Of 6377 screened studies, 135 underwent full-text review, and 16 studies reporting 21,271 nodules were included. Within these, the pooled prevalence of NC-ATA nodules was 7.8% (1872 nodules; [confidence interval; CI 5.1–11.1]). The pooled RoM estimate for NC-ATA nodules was 20.3% [CI 13.0–28.7] and there was significant heterogeneity between studies (I² = 92.8%, p < 0.001). NC-ATA nodule RoM estimates were significantly different by study type: single-center versus multicenter studies (24.8% vs. 12.3%, respectively, p = 0.031) and study design: retrospective versus prospective studies (25.1% vs. 8.5%, respectively, p = 0.003). No significant difference was observed in RoM based on inclusion of <1 cm nodules or geographic region. Meta-regression analysis showed study design and use of surgical histology for diagnostic criteria contributed significantly to differences in the reported RoM estimates.

Conclusion:

In this first meta-analysis comprehensively assessing the RoM of NC-ATA nodules, the malignancy risk was found to be comparable with the current ATA USRSS intermediate suspicion category. Significant heterogeneity was observed between studies and limits the interpretation of these results. In future iterations of the ATA USRSS that seek into incorporate categorization of NC-ATA nodules, these meta-analysis data may help to inform proper malignancy risk stratification. The study protocol was registered on PROSPERO, the international prospective register of systematic reviews (CRD42020182498), on July 14, 2020.

Introduction

Thyroid nodules are highly prevalent and there has been a significant increase in their detection and evaluation for the past several decades.¹ Thyroid ultrasound evaluation has emerged as an integral part of thyroid nodule assessment to stratify the risk of malignancy based on sonographic features. To improve implementation, standardized ultrasound risk stratification systems (USRSS) have been devised, including in the 2015 American Thyroid Association (ATA) Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer.² This system utilizes sonographic patterns to categorize nodules (with associated malignancy risk) as benign (<1%), very low suspicion (<3%), low suspicion (5–10%), intermediate suspicion (10–20%), or high suspicion (>70–90%). Based on their respective risk of malignancy (RoM), each category has an associated size cutoff at which fine needle aspiration (FNA) is recommended.

While the ATA USRSS has performed well in validation studies, a subset of nodules does not fit any of the described patterns. These “nonclassifiable” (NC-ATA) nodules generally consist of a combination of lower suspicion and higher suspicion features, such as iso- or hyperechoic or mixed cystic/solid nodules that have one or more high-risk features (e.g., punctate echogenic foci, taller than wide shape, and irregular margins). Some prior studies have suggested that NC-ATA nodules may be encountered relatively frequently^3–7 and have high RoM,^5,8–11 while other data indicate that NC-ATA nodules are uncommon^8,12–15 and RoM is low.^3,4,6,16,17

This wide variability in the reported findings among individual studies, which is perhaps due to differences in study populations or methods, results in uncertainty as to the true prevalence and RoM of NC-ATA nodules. Therefore, this systematic review and meta-analysis aims to comprehensively determine the frequency and RoM for NC-ATA nodules described in the literature and assess variables influencing malignancy risk estimates.

Methods

Search strategy

We performed a literature search according to the Preferred Reporting Items for a Systematic Review and Meta-analysis (PRISMA) guidelines. A systematic search was done from January 2015 until August 2022 of PubMed/MEDLINE and EMBASE databases to identify English language publications that recognized and reported on NC-ATA nodules (Supplementary Data). Identified references with abstracts were exported to Covidence (Covidence Systematic Review Software, Veritas Health Innovation, Melbourne, Australia; www.covidence.org) for initial screening. This study was considered exempt by the governing institutional review board.

Study selection

After removal of duplicates, at least two authors (T.E.A., D.K., M.K., E.J., and/or R.R.M.) independently performed a title and abstract screen followed by a full-text review by two independent reviewers (T.E.A., D.K., M.K., and/or R.R.M.) using previously agreed upon inclusion and exclusion criteria. The reference lists for all studies that underwent full-text evaluation were screened for additional relevant articles for review. Any disagreements were resolved by discussion and review by two authors (T.E.A. and D.K.). Studies met initial inclusion criteria if they were original publications of thyroid nodules in adult subjects; provided categorization using the ATA USRSS; and reported pathologic outcomes of NC-ATA nodules, either by histopathology and/or definitive biopsy result (i.e., Benign [Bethesda II] or Malignancy [Bethesda VI] cytology).

Studies were excluded if only a specific cytopathologic subgroup (e.g., indeterminate cytology) or histopathology subgroup (e.g., follicular thyroid carcinoma) was evaluated. In addition, studies were excluded if they did not report the existence of NC-ATA nodules or explicitly recategorized them into another ATA USRSS category. Studies that recognized the presence of NC-ATA nodules but did not report their pathologic outcomes were excluded. In studies that included both <1 and >1 cm nodules, only data for >1 cm were included if these data were separately reported. Studies of <1 cm nodules were generally excluded, but to allow for the greatest number of relevant data after initial review, studies that had <15% of nodules <1 cm were considered to have an acceptably low proportion and were included.

When inclusion or exclusion criteria were unanswered in the article, personal correspondence to the authors was attempted. If additional data were collected from study authors, the study was assessed for inclusion/exclusion as aforementioned. Studies were not included if additional data could not be collected or if criteria could not be confirmed. The following data were extracted from the selected articles: study design and location, diagnostic criteria (histopathology vs. histopathology and definitive FNA), inclusion of subcentimeter nodules, number of patients and nodules investigated, proportion of nodules with indeterminate cytology and no surgical pathology that were excluded, proportion of NC-ATA nodules, proportional of nodules with histopathologic diagnosis, and RoM of NC-ATA nodules.

Risk of bias assessment

Risk of bias and assessment of applicability was conducted by two authors (D.K., R.R.M., and/or M.K.) with disagreements resolved by discussion between authors (T.E.A. and D.K.) using the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.¹⁸

Statistical analyses

Meta-analysis to obtain pooled estimates for the proportion of NC-ATA nodules that were malignant used the metaprop procedure in Stata (V17.0, College Station, TX, USA).¹⁹ Each study summarized the number of NC-ATA nodules and the number of malignancies. The proportion of NC-ATA malignancies was computed for each study, with a 95% score-based confidence interval [CI] on binomial proportions. Random effects meta-analysis of proportions used the DerSimonian and Laird approach²⁰ on Freeman-Tukey double arcsine normalizing transformations of the study proportions; the study weights used the inverse variance of the transformed values. Between-study heterogeneity in malignancy proportions was assessed by I² statistics.

Prevalence of malignancy was also summarized by subgroups: study type (single- vs. multicentered); study design (prospective vs. retrospective); inclusion of <1 cm nodules; and geographic location (Asia, Europe, and America). To investigate sources of between-study heterogeneity in estimates of malignancy proportions, mixed effects logistic regression with a random study effect was conducted, testing for possible heterogeneity by single- versus multicentered studies, prospective versus retrospective study designs, diagnostic criteria used, total number of patients evaluated, total number of NC-ATA nodules evaluated, percentage of nodules histologically verified, and percentage of nodules that were excluded for because of indeterminate cytology without verified diagnosis. Continuous data were evaluated as dichotomous variables with cutoff at the median value.

Results

Study selection

As depicted in Figure 1, a total of 6377 titles/abstracts were screened after duplicate removal. Of those, 126 articles underwent full-text review. Reasons for study exclusion are shown in Supplementary Table S1. Ultimately, 16 articles fulfilled the inclusion and exclusion criteria and were analyzed for this study.^3–17,21 The authors of two articles were contacted for additional information, of which one author provided additional data that led to inclusion of the study.

FIG. 1.

PRISMA diagram. PRISMA, Preferred Reporting Items for a Systematic Review and Meta-Analysis.

Study characteristics

Study characteristics are summarized in Table 1. Of the included studies, 12 were retrospective analyses, 3 were prospective,^4,14,17 and 1 study reported on both retrospective and prospective cohorts.⁶ Single-center analyses represented 11/16 (68.8%) of included studies^{3,5–8,10–13,15,16,21} and most of the studies (11/16, 68.8%) reported data on populations from centers in Asia.^{3,5,6,8,10–13,15,16,21} Six studies (37.5%) included a subset of <15% subcentimeter nodules,^4,7,14–16 and four studies (25%) reported only thyroid nodules that underwent surgical resection.^8,10,11,15

Table 1.

Characteristics of Included Studies

Study	Country	Design	Diagnostic criteria	Includes nodules <1 cm	Total nodules	NC-ATA nodules, n (%)	NC-ATA RoM, n (%)
Yoon et al.12	South Korea	Multicenter, retrospective	Surgery + FNA	No	1293	44 (3.4)	8 (18.2)
Middleton et al.3	United States	Multicenter, retrospective	Surgery + FNA	Yes	3422	477 (13.9)	45 (9.4)
Chnget al.8	Singapore	Single-center, retrospective	Surgery only	No	167	3 (1.8)	1 (33.3)
Ha et al.16	South Korea	Multicenter, retrospective	Surgery + FNA	Yes	586	53 (9.0)	4 (7.5)
Ha et al.21	South Korea	Multicenter, retrospective	Surgery + FNA	No	2000	100 (5.0)	19 (19.0)
Persichetti et al.14	Italy	Single-center, prospective	Surgery + FNA	Yes	987	31 (3.1)	6 (19.4)
Gao et al.15	China	Single-center, retrospective	Surgery only	Yes	2614	70 (2.7)	13 (18.6)
Grani et al.4	Italy	Single-center, prospective	Surgery + FNA	No	502	90 (17.9)	9 (10.0)
Ha et al.13	South Korea	Single-center, retrospective	Surgery + FNA	No	1938	34 (1.8)	7 (20.6)
Peng et al.5	China	Single-center, retrospective	Surgery + FNA	Yes	230	45 (19.6)	31 (68.9)
Slowinska-Klencka et al.9	Poland	Single-center, retrospective	Surgery + FNA	No	1000	51 (5.1)	16 (31.4)
Merhav et al.17	Israel	Single-center, prospective	Surgery + FNA	No	255	15 (5.9)	0 (0.0)
Paker et al.10	Israel	Single-center, retrospective	Surgery only	No	238	16 (6.7)	9 (56.3)
Kuru et al.11	Turkey	Single-center, retrospective	Surgery only	No	1143	87 (7.6)	55 (63.2)
Seifert et al.7	Germany	Multicenter, retrospective	Surgery + FNA	Yes	1211	135 (11.1)	16 (11.9)
Yang and Na6	South Korea	Single-center, retro- and prospective	Surgery + FNA	No	3685	621 (16.8)	62 (10.0)

ATA, American Thyroid Association; FNA, fine needle aspiration; NC-ATA, nonclassifiable; RoM, risk of malignancy.

Sample size varied from 167 to 3685 thyroid nodules, with a total pooled sample of 21,271 nodules. A total of 1872 nodules were classified as NC-ATA, representing 1.75–19.6% of all nodules among studies. The random effects pooled prevalence was 7.8% [CI 5.1–11.1]. The criteria used to categorize nodules as NC-ATA were described in all publications. NC-ATA nodules were described as isoechoic with at least one high-risk feature, hyperechoic with at least one high-risk feature, mixed solid/cystic with at least one high-risk feature, or when composition, echogenicity or margins could not be determined. The RoM for NC-ATA nodules ranged from 0% to 68.9%.

Meta-analysis of malignancy rate

As shown in Figure 2, the overall pooled RoM estimate for NC-ATA nodules in the 16 included studies was 20.3% [CI 13.0–28.7]. There was significant heterogeneity among study estimates (I² = 92.8%, p < 0.001). In subgroup analyses, single-center studies showed higher pooled RoM estimate of 24.8% [CI 13.0–38.7] compared with multicenter studies (12.3% [CI 8.3–16.8], p = 0.031; Fig. 3A). There was significant interstudy heterogeneity found across single-center studies (I² = 94.4%, p < 0.001), but not multicenter studies (I² = 57.1%, p = 0.053). Furthermore, the pooled RoM was higher in retrospective versus prospective studies (25.1% [CI 15.2–36.3] vs. 8.5% [CI 4.2–14.1], p = 0.003; Fig. 3B). Significant interstudy heterogeneity was seen among retrospective studies (I² = 93.8%, p < 0.001), but not prospective studies (I² = 47.6%, p = 0.13).

FIG. 2.

Meta-analysis of NC-ATA nodule RoM. The RoM for thyroid nodules evaluated using the ATA USRSS with sonographic patterns deemed nonclassifiable (NC-ATA) from 16 included studies. The pooled RoM for NC-ATA nodules was 20.3% [CI 13.0–28.7], with significant heterogeneity identified among study estimates (p < 0.001). Ha 2018a¹⁶ and Ha 2018b.²¹ ATA, American Thyroid Association; CI, confidence interval; RoM, risk of malignancy; USRSS, ultrasound risk stratification system.

FIG. 3.

Meta-analysis of RoM for NC-ATA nodules. Subanalyses of RoM for ATA USRSS nonclassifiable (NC-ATA) nodules for 16 included studies, stratified by (A) study centers: multicenter versus single-center studies; (B) study design: retrospective versus prospective; (C) diagnostic criteria: Histopathology + Cytology versus Histopathology alone; (D) nodule size inclusion: only ≥1 cm nodules versus inclusion of <1 cm nodules, and; (E) geographic location: Asian versus Europe/American sites.

The pooled RoM for studies that only included nodules with surgical pathology was 42.9% [CI 13.6–74.9] compared with 16.1% [CI 10.6–22.3] for studies including both surgical and other nonsurgical outcomes (e.g., definitive biopsy result; trend at p = 0.057, Fig. 3C). Of note, studies that only included nodules >1 cm had similar pooled RoM (20.7% [CI 10.7–32.7]) to studies that included a subset of nodules <1 cm (20.0% [8.6–34.4]) (Fig. 3D). Finally, with respect to different geographic regions (Fig. 3E), we found no difference in pooled RoM by geographic region between studies from Asia (23.3% [CI 12.1–36.5]) versus Europe/America (14.4% [CI 8.5–21.1]; p = 0.142).

Because of significant heterogeneity observed in the primary analysis, sensitivity analyses were performed to identify sources of between-study heterogeneity that contributed to differences in the RoM estimates. After removal of one study representing a statistical outlier,⁵ the overall pooled RoM estimate for NC-ATA nodules in these studies was 19% [CI 13.0–25.0], with heterogeneity remaining significant (I² = 83.0%, p < 0.001). Mixed effects logistic regression performed to evaluate study variables included single- versus multicentered studies, prospective versus retrospective design, study location (Europe/America vs. Asia), use of histology for diagnostic outcome, total and NC-ATA nodule sample size, nodule size threshold, and rate of exclusion due to unconfirmed nodule diagnosis.

This analysis showed that RoM was lower in multicenter versus single-center studies (p = 0.049) and in prospective versus retrospective studies (p = 0.015), while higher RoM was observed in studies with only surgically resected nodules (p = 0.013) and studies with >35% of nodules with histopathologic diagnosis (p = 0.01). Other variables did not show significant contribution to differences in RoM.

Quality assessment

The methodological quality and risk of bias in the included studies were assessed using the revised QUADAS-2 and are reported in Table 2. All studies selected patients with thyroid nodules who underwent evaluation and treatment and, therefore, inherently did not evaluate patients with nodules who did not undergo additional testing. Within the parameters of patients receiving treatment for thyroid nodules, most studies were considered low risk of impactful bias in patient selection, testing, and final diagnosis. We did note higher risk of bias for patient selection and patient selection applicability in Slowinska-Klencka et al.,⁹ due to unclear recruitment of their patient cohort, as well as an unclear risk of bias in the index test for two studies in which blinding was not clearly stated.^8,11 The four studies that reported only nodules with surgical pathology were also graded to have higher risk of bias in patient selection.^8,10,11,15

Table 2.

Quality Assessment of Included Studies (QUADAS-2)

Discussion

Ultrasonography is a primary modality for assessment of thyroid nodule risk and guiding clinical management. Despite our clinical reliance upon ultrasound evaluation, there remain limitations to sonographic systems for nodule evaluation. Here we address the subset of nonclassifiable nodules within the widely used ATA USRSS system to attempt to better delineate their prevalence and malignancy risk through a comprehensive systematic review and meta-analysis comprising 16 studies and >20,000 nodules.

Importantly, we found that NC-ATA nodules are common with a pooled prevalence of 7.8% [CI 5.1–11.1] and portend a malignancy risk of 20.3% [CI 13.0–28.7]. This malignancy risk approximates the upper range assigned to ATA USRSS intermediate suspicion nodules (10–20%) and may overlap with the risk implied for the American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS)²² category 4, as well as some in category 5 (those with 7 points).^7,16,23 In subgroup analyses, risk estimates varied significantly by aspects of study design.

We found significantly lower RoM reported in multicentered (12.3%) and prospective studies (8.5%) compared with single-center and retrospective studies, respectively. No identified studies were both prospective and multicentered. Even if the true malignancy risk of NC-ATA nodules is closer to the lower rate seen in these subgroups, it is comparable with that assigned to intermediate suspicion nodules and represents a non-negligible risk to patients. These findings broadly indicate that FNA should be considered in patients with NC-ATA nodules in a manner comparable with those with an intermediate suspicion ATA sonographic pattern.

Significant heterogeneity observed among the studies should be considered when interpreting the pooled RoM estimates in this meta-analysis. The analysis of study characteristics to assess sources of heterogeneity revealed that lower NC-ATA nodule RoM was observed in multicenter studies and prospective studies, whereas higher RoM was found for studies that only used histopathology diagnosis and in those that had a percentage of diagnosis from histopathology that was above the median (>35%). In addition, there may be other aspects of thyroid nodule assessment that were not reported in the studies that affected the individual RoMs.

Differences in thyroid ultrasound expertise could lead to variability in the criteria for classifying nodules as NC-ATA. If nodules with ATA low or ATA high suspicion sonographic phenotypes were included as NC-ATA nodules, this would lead to lower or higher RoM, respectively. Similarly, differences in cytological reporting and use of indeterminate (Bethesda III–V) categories, as well as how patients with indeterminate cytology were referred for surgical resection could lead to heterogeneity in the observed RoMs. These were not data available in the published studies. Although whether and how these variables could alter the pooled RoM estimates is not known, the majority of studies indicate standard of care criteria and clinical decision-making that likely would be applicable to general populations.

The strengths of this study include the large population of nodules derived from systematic review, well-defined criteria to include a broad and high-quality sample, and subanalyses to evaluate variables that could impact RoM in NC-ATA nodules. We acknowledge some limitations as well. The included studies only reported on nodules with a confirmed diagnosis, either by surgical histopathology or definitive biopsy (Benign [Bethesda II or Malignancy [Bethesda VI] cytology), which excluded indeterminate cytology nodules without histopathologic standard and, therefore, our results may include some selection bias.

Cytologically indeterminate nodules that were determined to be more suspicious may have been resected while less concerning indeterminate nodules were managed conservatively and not reported, which could result in a higher observed RoM than if all indeterminate nodules were included. No studies reported use of molecular testing. Thyroid malignancies other than PTC (e.g., follicular thyroid carcinoma and medullary thyroid carcinoma) may be under-reported in studies evaluating USRSSs that rely on confirmed diagnosis, perhaps due to less specific sonographic features and higher chances of indeterminate cytology in these less common thyroid cancers.²⁴

In addition, in older studies, tumors now defined as noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP), which is an indolent nonmalignant tumor, may have been reported as follicular variant of PTC and been counted as malignant. Since NIFTP may present as a thyroid nodule with NC-ATA appearance,²⁵ the reported malignancy risk for NC-ATA nodules may be overestimated. Because the diagnosis of NIFTP can be made only upon histopathologic assessment, surgical resection is still accepted management for nodules that are ultimately determined to be NIFTP.

Moving forward, these data further support the development of more robust criteria for thyroid nodule evaluation. Previous studies have attributed much of the interobserver variability seen during use of the ATA USRSS system to the presence of nonclassifiable isoechoic or hyperechoic nodules with high-risk features.²⁶ USRSS that evaluate nodule features individually, such as ACR-TIRADS, are able to assess risk for any combination of nodule characteristics and, therefore, can categorize all nodules. In contrast, more holistic nodule stratification systems such as the ATA USRSS can help to avoid pitfalls of feature misclassification and subsequent inaccurate risk assessment. In future iterations of the ATA USRSS that seek into incorporate categorization of NC-ATA nodules, these meta-analysis data can inform proper malignancy risk.

Conclusion

Data from this meta-analysis provide a pooled estimate for the prevalence and RoM of NC-ATA nodules. Although significant heterogeneity observed between studies indicates the need for caution when interpreting these results, they indicate that the RoM is comparable with that of the current ATA USRSS intermediate suspicion category. This suggests that for NC-ATA nodules, use of similar criteria for performing FNA is likely warranted.

Authors' Contributions

Conceptualization (equal), methodology (equal), investigation (equal), data curation (lead), supervision, writing—original draft (lead), and writing—review and editing (equal) by D.K. Investigation (equal) and writing—original draft (supporting) by M.K. Formal analysis (lead) and writing—review and editing (supporting) by W.J.M. Investigation (equal) and writing—review and editing (supporting) by R.M.M. and E.J. Conceptualization (equal), methodology (equal), investigation (equal), supervision (lead), visualization (lead), writing—original draft (supporting), and writing—review and editing (equal) by T.E.A.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Funding Information

This study was supported by grants UL1TR001855 from the National Center for Advancing Translational Science (NCATS) of the U.S. National Institutes of Health.

Supplementary Material

Supplementary Data

Supplementary Table S1