Effects of Rapid On-Site Evaluation on Diagnostic Accuracy of Thyroid Fine-Needle Aspiration

Abstract

Introduction

Fine-needle aspiration (FNA) is well-established for the evaluation of suspicious thyroid nodules. However, a significant proportion is nondiagnostic. Rapid on-site evaluation (ROSE) has been proposed to improve the overall adequacy of FNA.

Methods

Retrospective cohort study comparing adequacy of thyroid FNA findings pre- and postimplementation of ROSE at a tertiary center in Switzerland. Patients undergoing thyroid FNA from January 2016 to December 2019 were included. The primary outcome was the rate of nondiagnostic findings (Bethesda System for Reporting Thyroid Cytopathology category I).

Results

In total, 410 thyroid nodule FNAs were performed. Of those, 309 with standard FNA and 101 with ROSE. The majority of patients were female (71%), with a median age of 56 years (IQR 46–68) and a nodule diameter of 1.9 cm (IQR 1.2–2.9). Implementation of ROSE led to a decrease in nondiagnostic findings from 41.1% to 23.8%, with an odds ratio of 0.42 (95% CI: 0.24–0.72; p = 0.002). Implementation of ROSE was associated with significantly higher rates of Bethesda category III (27.7% vs. 19.1%), category IV (15.8% vs. 5.5%), and Bethesda category VI (6.9% vs. 2.3%). Repeated FNA was performed in 29.1% before and 20.8% after implementation of ROSE (p = 0.18). The mean number of FNA per nodule was reduced from 1.4 (0.6) to 1.2 (0.4) with ROSE (p = 0.04).

Conclusions

Implementation of ROSE of thyroid nodule specimen improved diagnostic adequacy of FNA, reducing nondiagnostic findings. However, due to increased equivocal findings (Bethesda category III), there was no significant reduction of repeat FNA.

Introduction

Thyroid nodules are common and their lifetime occurrence rate is estimated to be around 60%, with a female preponderance [1]. Wider availability as well as improved resolution and quality of ultrasound and other imaging modalities has resulted in an increased detection rate of thyroid incidentalomas over the past decades [2]. Although the majority of nodules are known to be benign, in case of characteristic clinical and sonomorphologic features, further investigations are warranted since around 7–15% will prove to be malignant, mostly corresponding well-differentiated papillary thyroid cancer [3].

Ultrasound-guided fine-needle aspiration (FNA) has become the standard diagnostic tool for the investigation of thyroid nodules since it is rapidly performed, has low complication rates [4], and has proven to be cost-effective [4, 5, 6]. As a consequence, millions of thyroid nodule FNAs are performed every year worldwide, with a strong upward trend [7]. Cytological findings are reported using systematic cytological reporting systems such as the Bethesda System for Reporting Thyroid Cytopathology (BSRTC) [8]. However, 2–20% of thyroid nodule samples are inadequate for evaluation, commonly due to insufficient numbers of evaluable thyrocytes in the specimen and thus are classified as nondiagnostic or unsatisfactory (BSRTC category I) [9]. In such cases, repeat thyroid nodule FNA is warranted, leading to a longer time to definitive diagnosis and being associated with higher healthcare costs as well as avoidable uncertainty, anxiety, and psychological stress for the patients.

It has been suggested that rapid on-site evaluation (ROSE) of FNA specimens by trained cytologists may decrease the rate of nondiagnostic findings as well as the number of repeat FNAs. Hence, this study aimed at investigating whether the prospective implementation of ROSE in a teaching hospital would result in fewer nondiagnostic findings, shorter time to a definitive diagnosis, fewer numbers of repeated FNAs, and higher rates of definitive diagnosis.

Materials and Methods

Study Design and Participants

This is a retrospective cohort study including patients undergoing thyroid nodule FNA from January 2016 to December 2019 at the University Hospital Basel, a tertiary care center in Switzerland. ROSE was implemented in June 2017 at the interdisciplinary endocrine and nuclear medicine outpatient clinics. Using a mixed-method pre- and postimplementation design, the performance of ROSE was assessed, comparing the period before and after implementation of this quality measure.

All thyroid nodules were assessed by the TIRADS ultrasound features before the decision for FNA [10]. Ultrasound-guided FNA was performed using a 22- to 27-gauge needle with a 5–10 mL syringe.

ROSE was performed by two experienced cytotechnicians. In brief, conventional smears were prepared for each pass, fixed in Delaunay's solution, and one to two smears were stained with rapid Papanicolaou stain and immediately assessed for adequacy under the microscope [11]. For a thyroid sample to be classified as adequate for cytological evaluation, a minimum of six cell groups, each containing at least ten well-visualized follicular cells, was required [9]. FNAs were carried out by a physician in training accompanied by a senior endocrine or nuclear medicine physician. At least, two passes were performed. Additional passes were performed until adequate material was present or the senior physician decided to end the procedure (e.g., due to patient discomfort). The needle was rinsed in physiological saline solution and, along with the remaining fixed smears, processed and Papanicolaou stained according to routine procedures in the cytopathology laboratory. Final interpretation of the specimens was performed by a board certified cytopathologist according to the BSRTC. A total of three cytopathologists with an average of 10 years of experience provided diagnostic interpretation of the thyroid FNAs.

In case multiple thyroid nodules were examined by FNA in a patient, each thyroid nodule was included separately in the study. The institutional review board of Northwestern and Central Switzerland (EKNZ) approved this study and waived patients' informed consent owing to the use of de-identified data. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology reporting guideline [12].

Outcomes

The primary outcome was the rate of nondiagnostic FNA findings (BSRTC category I) pre- versus postimplementation of ROSE. Secondary outcomes included the following: (i) number of total FNAs per nodule; (ii) time to diagnosis, defined as time of definitive cytological or histological diagnosis after first FNA; (iii) number of repeat FNAs; and (iv) rate of definitive diagnosis as defined as the relative number of patients achieving a definitive diagnosis by the end of the study period. Definitive diagnosis was defined as benign if the cytological diagnosis was classified as BSRTC category II. In all other cases, the definitive diagnosis was defined by the histological result, if thyroid surgery was performed. Since all patients with BSRTC category VI underwent surgery, histological diagnosis was available.

Statistical Analysis

Unless stated otherwise, categorical variables are expressed as number (percentage) and continuous variables as median (interquartile range). We used univariable and multivariable logistic regression models to calculate odds ratio and 95% confidence interval (CI) for the risk of BSRTC category I classification. For all secondary endpoints, we calculated unadjusted and adjusted estimates of the effect size and corresponding 95% confidence intervals (CIs) using linear, logistic, or Cox proportional hazards regression as appropriate. To account for the fact that comparison groups were treated at different time periods, multivariate adjustment was performed for patient age, gender, family history, number of thyroid nodules, maximum diameter of nodules, TSH level, and comorbidities to minimize bias.

The significance level was set as α = 5%, and all p values are 2-sided. Statistical analyses were performed using Stata 15.1 (Stata Corp 2015, College Station, TX, USA).

Results

In total, 410 FNAs between January 2016 and December 2019 were included in the study. From January 2016 to May 2017, standard FNA was performed in 309 thyroid nodules and from June 2017 FNA was performed with ROSE in 101 thyroid nodules.

Baseline characteristics of included patients are presented in Table 1. Patients undergoing thyroid nodule FNA had a median age of 56 years (IQR 46–68), and the majority (70.5%) were female. Most of the nodules had been detected incidentally, with only 36% being symptomatic prior to FNA. The median nodule diameter measured 1.9 cm (IQR 1.2–2.9), and in around 90% of cases, the thyroid function was euthyroid. According to ultrasound morphometry, the majority of nodules were either classified as TIRADS class III (31%), or class IV (42%).

Table 1.

Baseline characteristics and clinical variables of enrolled patients

Characteristic/variable	Total (N = 410)	Standard procedure (n = 309)	ROSE (n = 101)	p value
General characteristics
Age, years	56 (46, 68)	56 (47, 69)	54 (45, 67)	0.28
Sex
Male	121 (29.5)	94 (30.4)	27 (26.7)	0.48
Female	289 (70.5)	215 (69.6)	74 (73.3)
Thyroid function
Euthyroid	367 (89.5)	279 (90.3)	88 (87.1)
Hypothyroid	19 (4.6)	14 (4.5)	5 (5.0)
Hyperthyroid	6 (1.5)	3 (1.0)	3 (3.0)	0.45
Latent hypothyroid	7 (1.7)	4 (1.3)	3 (3.0)
Latent hyperthyroid	11 (2.7)	9 (2.9)	2 (2.0)
TSH, lU/mL	1.62 (0.99, 1.93)	1.63 (0.97, 1.93)	1.62 (1.15, 1.93)	0.38
Thyroid nodule characteristics Symptomatic nodule	148 (36.1)	118 (38.2%)	30 (29.7)	0.12
Nodules, n
1 nodule	345 (84.1)	274 (88.7)	71 (70.3)
2 nodules	26 (6.3)	18 (5.8)	8 (7.9)
3 nodules	8 (2.0)	2 (0.6)	6 (5.9)	<0.001
4 nodules	3 (0.7)	2 (0.6)	1 (1.0)
≥5 nodules	28 (6.8)	13 (4.2)	15 (14.9)
TIRADS classification
I	1 (1)	0 (0)	1 (1)
II	6 (7)	2 (11)	4 (6)
III	27 (31)	5 (28)	22 (31)	0.92
IV	37 (42)	7 (39)	30 (44)
V	17 (19)	4 (22)	13 (18)
Maximum diameter of nodule, cm	1.9 (1.2, 2.9)	1.8 (1.1, 2.7)	2.2 (1.5, 3.2)	0.003
Cold nodule in scintigraphy	30 (7.3)	15 (4.9)	15 (14.9)	<0.001
Thyroid-specific comorbidities
Graves' disease	6 (1.5)	1 (0.3)	5 (5.0)	<0.001
Hashimoto's disease	22 (5.4)	14 (4.5)	8 (7.9)	0.19
Autonomous nodule	15 (3.7)	11 (3.6)	4 (4.0)	0.85
Radioiodine therapy	9 (2.2)	6 (1.9)	3 (3.0)	0.54
Unilateral thyroidectomy	21 (5.1)	18 (5.8)	3 (3.0)	0.26
Family history for thyroid cancer	47 (11.5)	25 (8.1)	22 (21.8)	<0.001
Medications
L-thyroxine	26 (6.3)	19 (6.1)	7 (6.9)	0.78
Lithium	3 (0.7)	3 (1.0)	0 (0.0)	0.32
Carbimazole	8 (2.0)	4 (1.3)	4 (4.0)	0.093
Amiodarone	2 (0.5)	1 (0.3)	1 (1.0)	0.40

Data are presented as median (IQR) or n (%). ROSE, rapid on-site evaluation; TIRADS, thyroid imaging reporting and database system; TSH, thyroid stimulating hormone.

Specimen Adequacy

Implementation of ROSE led to a decrease of nondiagnostic BSRTC category I findings from 127/309 (41.1%) with standard FNA to 24/101 (23.8%), resulting in an odds ratio of 0.42 (95% CI: 0.24–0.72; p = 0.002) (Table 2). The distribution of the cytopathological BSRTC categories between the two groups is illustrated in Figure 1. Implementation of ROSE was associated with significantly higher rates of BSRTC category III (27.7% vs. 19.1%), BSRTC category IV (15.8% vs. 5.5%), and BSRTC category VI (6.9% vs. 2.3%) when compared to standard procedure, respectively. Representative images of cytological findings by ROSE are shown in Figure 2.

Table 2.

Primary and secondary endpoints of thyroid nodule FNA with and without ROSE

	Standard FNA (n = 309)	FNA with ROSE (n = 101)	Univariate regression analysis OR or regression coefficient (95% CI)	p value	Adjusted regression analysis OR or regression coefficient (95% CI)	p value
Primary endpoint
BSRTC category I	127 (41.1)	24 (23.8)	0.45 (0.27, 0.74)a	0.002	0.42 (0.24, 0.72)a	0.002
Secondary endpoints
FNAs, n	1.4 (0.6)	1.2 (0.4)	−0.15 (−0.27, −0.02)b	0.02	−0.14 (−0.27, −0.01)b	0.04
Rate of definitive diagnosis	177 (57.3)	58 (57.4)	1.01 (0.64, 1.58)a	0.98	0.95 (0.59, 1.54)a	0.85
Repeated FNA	90 (29.1)	21 (20.8)	0.64 (0.37, 1.10)a	0.10	0.68 (0.39, 1.19)a	0.18
Time to definitive diagnosis, * days	73.1 (141.3)	60.7 (84.4)	−12.44 (−52.69, 27.81)	0.54	−14.99 (−56.82, 26.84)b	0.48

Data are mean (SD) or n (%) unless otherwise stated. Multivariate analyses adjusted for patient age, gender, family history, number of thyroid nodules, maximum diameter of nodules and TSH, level; BSRTC, Bethesda System for Reporting Thyroid Cytopathology; FNA, fine-needle aspiration; ROSE, rapid on-site evaluation.

^aOdds ratio.

^bRegression coefficient (95% CI).

^*Definitive diagnosis defined as BSRTC category II, or histology.

Fig. 1.

Distribution of the cytopathological BSRTC categories between the two groups. BSRTC category I is much more prevalent with standard procedure compared to ROSE.

Fig. 2.

Representative images of cytological findings by ROSE (rapid Papanicolaou stain, original magnification, ×200 and ×400, respectively). a, b Papillary thyroid carcinoma (BSRTC category VI): cellular smear with papillary tissue fragment (a) and crowded, pale nuclei with nuclear grooves (arrow) and intranuclear cytoplasmic pseudoinclusions (arrow head) (b). c, d Atypia of undetermined significance with architectural atypia (BSRTC category III): sparsely cellular smear with clusters of microfollicles, cell crowding, and absence of colloid. e, f Nondiagnostic smear (BSRTC category I): only one cluster of follicular cells and abundant blood. ROSE, rapid on-site evaluation; BSRTC, Bethesda System for Reporting Thyroid Cytopathology.

Secondary Endpoints

After implementation of ROSE, there was a trend towards reduced necessity of repeat-FNA from 29.1% with standard FNA to 20.8% with ROSE (p = 0.18). With ROSE the mean number of FNAs per nodule was reduced from 1.4 (SD 0.6) to 1.2 (SD 0.4), yielding a relative reduction of 14% (p = 0.04).

The mean time to definitive diagnosis (BSRTC category II or histological diagnosis) for patients undergoing standard FNA was 73 days (95% CI: 52–94 days) compared to 61 days (95% CI: 37–84 days) with ROSE (Table 2). After implementation of ROSE, there was a slight reduction of the time to definitive diagnosis by 15 days, however, not meeting statistical significance (95% CI: −56.8–26.8; p = 0.48) (Table 2).

Discussion

The key findings of this study can be summarized as follows: introduction of ROSE led to a significant reduction of nondiagnostic FNAs with concomitant shift of results towards BSRTC category III, IV, and VI, as well as a reduction of the number of FNA procedures per nodule. After implementation of ROSE, there was a tendency for a reduced time to definite diagnosis and reduced risk for repeat FNA, however this was not statistically significant.

It is well-established that there is a negative correlation between the investigator's experience and the rate of nondiagnostic FNA findings [13, 14]. Nonetheless, especially in teaching hospitals, FNA procedures are mainly performed by less experienced physicians in training under the supervision of senior staff members. Therefore, especially in such institutions the current results suggest that ROSE appears to be an appropriate method to obtain adequate diagnostic accuracy of FNA.

While the use of ultrasound for guided FNA has already increased the diagnostic accuracy in general [15], implementation of ROSE has the capacity to additionally improve the diagnostic yield with less necessity for repeat FNA. In 1992, Eisele et al. [16] observed that an immediate cytopathological procurement and evaluation of specimen led to a significant reduction of unsatisfactory results in head and neck masses. Results from studies investigating the diagnostic performance of ROSE in thyroid FNA likewise showed higher adequacy of FNA findings [13, 17, 18] or at least reduction in the number of needle passes and in the procedural time [19]. In a recent retrospective analysis of 1975 thyroid nodule FNAs, ROSE led to a reduction in the rate of nondiagnostic findings, even among physicians with high procedural rates [20]. A previous meta-analysis summarized the data showing that implementation of ROSE led to a reduction of the rate of nondiagnostic thyroid FNA results by 44% [17]. Due to the fact that in our study FNAs were predominantly performed by physicians in training, the rates of nondiagnostic findings before and after implementation of ROSE were in general higher when compared to the literature, though improvements achieved with ROSE seemed to be greater [17]. This observation is in line with Witt and Schmidt, who reported that the impact of ROSE seemed to be lower for sites with high initial adequacy rates in comparison to sites with initially lower adequacy rates [17]. In a more recent study, the benefit of ROSE strongly depended on the investigator's experience, with less experienced operators benefiting most from ROSE [13] which would highlight the need of such a procedure in teaching hospitals.

Other than expected, in our study, the time interval until definitive diagnosis was not shortened significantly with the introduction of ROSE. This is possibly due to the fact that repeat FNAs were generally performed shortly after nondiagnostic results were communicated. Hence, the time to definitive diagnosis might be underestimated and probably much longer in hospitals with less capacities. It is also possible that in some nondiagnostic cases with BSRTC category I, a watch-and-wait strategy was chosen and thus no definitive diagnosis was made until the end of our study. This might have contributed to an underestimation of the time saved after the implementation of ROSE. These factors may also have resulted in nonstatistical differences for the need of repeat FNAs, as not every nondiagnostic BSRTC category I result was followed up by repeat FNA.

However, it has to be noted that implementation of ROSE led to higher rates of BSRTC categories III and IV which in turn either require repeat FNA or molecular analyses in order to guide clinical decision-making since these findings are of unknown significance [21, 22]. ROSE by a cytotechnician might be suitable to assure a specimen to be satisfactory for evaluation, however it does not assure that a definitive diagnosis can be made. For this reason, on-site evaluation with a cytopathologist would further improve quality of care, since not only adequacy for evaluation would be assured, but also definitive diagnosis could be achieved at the time of FNA procedure. This would also allow to trigger additional passes for ancillary testing if needed. For example, to prepare a cell block for immunochemistry, for flow cytometry if lymphoma is suspected or for diagnostic molecular testing. However, ROSE performed by a cytopathologist would significantly increase the costs incurred by the additional fees and require additional resources, which in turn might be problematic for coverage by health insurances [11].

Our study has the following limitations. First, the pre- and postimplementation design poses the risk that results of the implemented ROSE procedure may be confounded by other structural changes over time. Second, the current study did not distinguish between experienced and inexperienced investigators − previous studies suggested less benefit of ROSE with higher experience. Third, due to the fact that this is a single-center study in a teaching hospital with FNA procedures performed by staff in training, the results cannot be generalized to clinical settings with higher procedural volume and experience per investigator. Fourth, due to the retrospective design of the study a power analysis was not performed beforehand and the relatively small number of cases may limit strong conclusions. However, our study has several strengths. During the conduct of the study, there were no further alterations, neither in the diagnostic procedure, nor in the equipment used for FNAs, thus helping to exclude other confounding factors over time. Similarly, experience levels of investigators, in particular senior staff members, did not vary during the corresponding time periods, and performance of FNA was highly standardized. Finally, due to that large between-group difference in adequacy rates, post hoc analysis of the achieved power indicates that with the included sample size a strong statistical power was achieved. In conclusion, in our study a standardized cytotechnician-assisted ROSE of thyroid nodule cytology was associated with a higher diagnostic adequacy of FNA procedures and a decreased necessity of repeat FNA.

Statement of Ethics

This study was conducted ethically in accordance with the World Medical Association Declaration of Helsinki. This study protocol was reviewed and approved by the Institutional Review Board of Northwestern and Central Switzerland (EKNZ). Written informed consent was not required.

Conflict of Interest Statement

The authors have no conflicts of interest to declare.

Funding Sources

The funding sources had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Author Contributions

C.F., E.C., and F.E. designed the study and wrote the manuscript. S.E.C. and S.S.P. assisted in data collection. M.H. and J.H. performed the ROSE. C.F. and F.E. had access to all the data. E.C. and F.E. were responsible for the decision to submit the manuscript. All the authors provided comments on drafts and approved the final report.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Emanuel Remigius Christ and Fahim Ebrahimi shared last authorship.