A Prospective Study Evaluating the Accuracy of Using Combined Clinical Factors and Candidate Diagnostic Markers to Refine the Accuracy of Thyroid Fine Needle Aspiration Biopsy

Abstract

Background

Approximately 30% of thyroid fine needle aspiration (FNA) biopsies have inconclusive results. We conducted a prospective trial to determine if clinical and molecular markers could be used in combination to improve the accuracy of thyroid FNA biopsy.

Methods

Clinical, tumor genotyping for common somatic mutations (BRAF V600E, NRAS, KRAS, RET/PTC1, RET/PTC3, NTRK1), and the gene expression levels of 6 candidate diagnostic markers were analyzed by univariate and multivariate methods in 341 patients to determine if they could reliably distinguish benign from malignant thyroid tumors and a scoring model was derived.

Results

By multivariable analysis FNA cytology classification, presence of a NRAS mutation, and TIMP1 expression level were jointly associated with malignancy. The overall accuracy of the scoring model, including these 3 variables, to distinguish benign from malignant thyroid tumors was 91%; 67% for the indeterminate and 77% for the suspicious FNA subgroups.

Conclusions

FNA cytology classification, presence of NRAS mutation, and TIMP1 mRNA expression levels in combination provide a higher diagnostic accuracy than FNA cytology alone to allow selection of more definitive initial surgical treatment. The sensitivity of the scoring model, however, was too low to avoid the need for diagnostic thyroidectomies for indeterminate FNA findings.

INTRODUCTION

The incidence of thyroid cancer has doubled over the past three decades, and an estimated 37,200 people were diagnosed with thyroid cancer in 2009¹. Fine needle aspiration biopsy (FNA) was adopted into widespread use in the 1970s to evaluate patients with thyroid nodules, and it has not only dramatically reduced the number of thyroidectomies for benign thyroid neoplasms, but has also allowed for more definitive initial surgical treatment of patients with malignant thyroid neoplasms^2–4. However, it may also be nondiagnostic or demonstrate indeterminate or suspicious cytologic features in 20 to 30% of all biopsies^3–7. The reported risk of malignancy in these FNA cytologic groups ranges from 5% to 75%^5,6. This limitation of FNA biopsy and cytologic examination is a result of indistinct cytologic features between benign and malignant tumors which commonly encompass follicular and Hürthle cell neoplasms, hyperplastic nodules, and follicular variant of papillary thyroid cancer. At least a diagnostic thyroidectomy is usually recommended in instances in which the FNA result is indeterminate or suspicious. Unfortunately, no preoperative clinical, imaging, or cytologic factors studied thus far can reliably distinguish which of those patients should undergo thyroidectomy.

A recent National Cancer Institute (NCI) Thyroid Fine-Needle Aspiration State of the Science conference proposed a more expanded classification for FNA cytology that substratifies the indeterminate and suspicious FNA results into atypical follicular lesion of undetermined significance (risk of malignancy 5% to 10%), follicular or hürthle cell neoplasm (risk of malignancy 15% to 25%), and suspicious for malignancy (risk of malignancy 50% to 75%)⁷. However, use of this classification varies among pathologists across various institutions and diagnostic thyroidectomies are still necessary because the risk of malignancy is not negligible⁸.

The development of adjunct diagnostic approaches to thyroid FNA biopsy has been an active area of thyroid cancer research. Activating somatic genetic alterations have been described in the signal transduction pathways involving tyrosine kinase receptors (RET/PTC, NTRK) and signaling proteins (RAS, BRAF), and in nuclear proteins (PAX8-PPARγ) in thyroid cancer of follicular cell origin^9–16. Molecular testing for common somatic mutations in thyroid FNA biopsy has emerged as a promising approach because about two-thirds of thyroid cancers of follicular cell origin have at least one of the common genetic alterations, which are absent in benign thyroid neoplasms (BRAF V600E point mutation, and RET/PTC and NTRK1 rearrangements) ^11–14.

Our group has previously used pathway specific cDNA array analysis to identify candidate diagnostic and extent of disease markers in thyroid neoplasms that would be indeterminate or suspicious on FNA cytology¹⁷. From this analysis six novel candidate diagnostic and extent of disease markers were identified to have a high diagnostic accuracy. These genes included extracellular matrix protein 1 (ECM1); transmembrane protease, serine 4 (TMPRSS4); angiopoietin 2 (ANGPT2); tissue inhibitor of metalloproteinase 1 (TIMP1); ephrin-B2 (EFNB2), and epidermal growth factor receptor (EGFR). The goal of our current study was to determine if demographics, clinical factors, and FNA cytology classification in conjunction with somatic mutation analysis and expression level of 6 candidate markers could further improve the accuracy of thyroid FNA biopsy.

METHODS

Thyroid Tissue and Fine Needle Biopsy Samples

Thyroid FNA samples, thyroid tissue, clinical and histopathology data were prospectively collected for 341 patients with 423 dominant thyroid nodules on a clinical protocol approved by the Committee on Human Research at the University of California, San Francisco. Demographic data collected included age, gender, ethnicity, family history of thyroid disease, and history of radiation exposure. FNA samples were classified according to the NCI State of the Science recommendation⁷. The FNA biopsy samples were dispensed directly into TRIzol® and total RNA was extracted immediately. 174 samples were obtained at the time the patient underwent FNA biopsy of the dominant thyroid nodule in the outpatient clinic. One additional FNA pass was performed to obtain the sample for the study. The remaining 249 biopsies were performed with a 25-gauge needle at the time of thyroidectomy. All of the intraoperative FNA biopsy samples were of the dominant nodule that were biopsied before the operation and had a cytologic diagnosis. The FNA biopsies performed at thyroidectomy were done percutaneously with a single pass. The RNA amount obtained was similar between the intraoperative and clinic FNA samples as well as the RNA integrity number using the Agilent 2100 Bioanalyzer (Foster City, CA). 65 of these samples were classified as “unknown” FNA cytology diagnosis because the initial FNA biopsy was performed elsewhere and the cytology was not reviewed at our institution to confirm the diagnosis. Tissue diagnoses were confirmed by permanent histology, which was used as the gold standard to determine accuracy.

Molecular Analysis

Total RNA was extracted using the TRIzol reagent (Invitrogen). Total RNA (125 ng/μL) was reverse-transcribed using the RT script cDNA synthesis kit (USB Corp., Cleveland, OH). Real-time quantitative PCR was used to measure mRNA expression levels relative to human β-glucuronidase (Gus) mRNA expression. Gene expression level = 2^{−(Ct of gene of interest − Ct of Gus)} × 100%, where C_t is the PCR cycle threshold. The PCR primers and probes for ANGPT2, ECM1, EGFR, EFNB2, TIMP1, and TMPRSS4 were purchased from Applied Biosystems (Foster City, CA). All PCR reactions were performed as previously described¹⁷. All quantitative PCR reactions were done in triplicate and repeated at least twice.

FNA genotyping was performed for the following somatic mutations BRAF V600E, NRAS (codons 12, 13 and 61), KRAS (codons 12,13, and 61), RET/PTC1, RET/PTC3, and NTRK1 as previously reported¹⁸.

Statistical Analysis

Chi squared and Wilcoxon rank sum tests were initially used to screen for any possible association of categorical and continuous variables according to the histologic diagnosis (benign versus malignant). A Cochran-Armitage test was used to determine the significance of the association between the FNA cytology classification and histologic diagnosis. Parameters significant in this initial univariate analysis (p< 0.05) were then analyzed using univariate and multiple logistic regression analyses to determine the degree to which disease status could be predicted on the basis of factors selected. The data was divided at random into two approximately equal sets: half for training the logistic scoring model and the other half for testing the model. Exploratory models were constructed from the training data set and then applied to the testing set to determine the accuracy of the model. This scoring model was then applied to the entire dataset and the indeterminate and suspicious subgroups to obtain sensitivity, specificity, and overall accuracy (true positive and true negative results). All p-values reported are two-tailed and have not been adjusted for multiple comparisons. Statistical analysis was performed using commercially available software (SAS, Inc., Cary, NC).

RESULTS

Patients

The demographic and clinical data are summarized in Table 1. FNA biopsies were reviewed by our cytopathologist for 358 of 423 samples. Of the 181 biopsies with benign FNA cytology result, 135 did not undergo thyroidectomy (Figure 1). On final histology, 165 nodules were benign and 123 were malignant. Cytologic examination of thyroid FNA biopsy samples showed that 51% were benign, 21% were malignant, 11% were atypical lesions, 12% were follicular or hürthle cell neoplasms, and 4% were suspicious for malignancy. Average tumor size (greatest tumor diameter) on ultrasound was 2.8 cm (range 0.4 to 8.1 cm). By univariate logistic regression analysis age and FNA cytology classification were significantly associated with classification into benign and malignant tumors (p ≤ 0.03) but gender, family history, prior history of radiation, and tumor diameter were not.

Table 1.

Study cohort demographic and clinical characteristics. Listed p-values are from univariate analysis.

		p-Value (Univariate Analysis)
Average Age (years) ± SD	51.1 ± 14.7	0.0043
Sex		0.0075
Male	282
Female	59
Ethnicity		0.0002
White	211
Asian/Pacific Islander	69
African American	13
Other/Unknown	48
Family History of Thyroid Malignancy	47	0.069
Prior History of Radiation*	36	0.84
NCI Classification on FNA Cytology (n = 423)		<0.0001
Benign	181
Atypia of Undetermined Significance	41
Neoplasm	43
Suspicious for Malignancy	13
Malignant	78
Nondiagnostic	2
Average Tumor Diameter (cm) ± SD	2.8 ± 1.4	0.0015
Number of Thyroid nodules in patients		0.56
1 nodule	268
2 nodules	66
3 nodules	6
4 nodules	1

* Prior History of Radiation	36
Treatment for Acne in childhood	4
Treatment for prior malignancy	12
Chernobyl Accident	9
Occupational Exposure	7
Radioiodide Treatment	1
Treatment for tuberculosis	1
Thymic irradiation as infant	1
Neck irradiation for persistent cough	1

^*Specific type of prior history of radiation exposure

Figure 1.

Flow Diagram demonstrating FNA biopsy diagnosis and management.

Molecular Analysis of Thyroid FNA Samples

Out of 423 FNA samples, 24 BRAF V600E mutations, 7 KRAS mutations, 21 NRAS mutations, 4 PAX8-PPARγ rearrangements, 3 RET/PTC1, and 2 RET/PTC3 rearrangements were detected. 17 of 165 (10.3%) benign thyroid tumors had a somatic mutation as compared to 32 of 123 (26%) malignant tumors (p < 0.05). NTRK1 rearrangement was not detected in any of the FNA samples. On univariate analysis, ANGPT2, EGFR, EFNB2, TIMP1, and TMPRSS4 were significantly differentially expressed between benign and malignant tumors (p ≤ 0.0001).

Scoring Model

Variables significantly associated with malignancy on univariate analysis were included in a multiple logistic regression analysis. FNA cytology classification (p<0.0001), presence of NRAS mutation (p = 0.016), and TIMP1 expression level (p = 0.067) were jointly associated with malignancy in the multiple logistic model analysis. A scoring model was created using these variables in a training set and then applied to a testing set. In the training set, the model led to the following rule for distinguishing between benign and malignant tumors: calculate 1.532*FNA cytology classification + 0.00137*TIMP1 normalized expression + 1.8816 (if NRAS mutation present). A total score greater than 4.81 would predict a tumor to be malignant and a score less than 4.81 was considered benign. In the testing set, this rule had 85% accuracy, 89% specificity, and 77% sensitivity. When this model was reapplied to the entire dataset, the overall accuracy to distinguish benign from malignant thyroid tumors was 91% with a specificity of 97% and sensitivity of 76%. In the atypical follicular or hürthle cell lesion FNA biopsies, 15 out of 41 nodules were malignant (36.6%) (Table 2). The scoring model, when applied to this subgroup was 61% accurate with a specificity of 81% and sensitivity of 27% (p < 0.001). In the indeterminate neoplasm subgroup 31.8% were malignant. The scoring model was 61% accurate with 100% specificity and 14% sensitivity for this FNA subgroup (p < 0.001). In the suspicious FNA group, 77% were malignant, of which 100% were detected by the scoring model (p < 0.001). The overall accuracy of the scoring model for this group was 77%.

Table 2.

Comparison of Scoring Model Results with Preoperative Fine Needle Aspiration (FNA) Biopsy Result and the Permanent Histologic Diagnosis

NCI Category of FNA biopsy	Risk of Malignancy (per NCI Guidelines)	Number of patients in Study Cohort	Scoring Model Result	Final Histology	% Malignancy (in Study Cohort)
Benign	< 1%	186	186 Benign	41 Benign	11%
			0 Malignant	5 Malignant: 5 Papillary *
Atypia of Undetermined Significance	5 – 10%	41	34 Benign (12 False negative)	26 Benign	37%
			7 Malignant (4 False positive)	15 Malignant  4 Papillary, 5 follicular variant of Papillary  5 Follicular  1 Medullary
Neoplasm	20–30%	43	41 Benign (12 False negative)	29 benign	33%
			2 Malignant (Follicular)	14 Malignant  7 Papillary  5 Follicular  2 Hurthle cell
Suspicious for Malignancy	50–75%	13	0 Benign	3 benign	77%
			13 Malignant (3 False positive)	10 Malignant  8 Papillary, 1 follicular variant of papillary  1 Anaplastic
Malignant	100%	78	0 Benign	0 benign	100%
			78 Malignant	78 Malignant  72 Papillary, 3 follicular variant of papillary  2 Medullary  1 Anaplastic

^*All the tumors were conventional papillary thyroid cancer and 1.5 cm or less in greatest diameter but in the dominant nodule which was biopsied. Thus, the cytopathologic diagnosis was a false negative interpretation. These patients had a thyroidectomy either due to increased tumor size on follow up ultrasound or suspicious ultrasound features.

DISCUSSION

In this study, we analyzed demographic, clinical, and molecular markers in patients with thyroid neoplasms to determine if any of these variables could be used to improve the diagnostic accuracy of thyroid FNA biopsy and cytologic examination alone. We used univariate and multivariate analyses to develop a scoring model that could classify benign and malignant thyroid neoplasm based on a combination of variables. A scoring model utilizing normalized TIMP1 mRNA expression levels, FNA cytology classification group, and presence of NRAS mutation was the most accurate, correctly classifying 91% of all biopsy samples as benign or malignant neoplasms. Application of this model to a subgroup of patients with either an atypical lesion or indeterminate neoplasm FNA cytology demonstrated that it was 61% accurate, but with a low sensitivity. In the subgroup of FNA biopsies that were suspicious for malignancy the model correctly identified 100% of patients with a malignancy and was 77% accurate with a false positive rate of 23%.

Since its introduction, FNA has reduced the number of diagnostic thyroidectomies performed for benign nodules and increased the number of patients receiving complete initial surgical treatment for malignant thyroid neoplasms. Currently, FNA is the most reliable, widely used, and most cost effective initial test to distinguish between a benign and malignant thyroid nodule. However, it is inconclusive for certain histological subtypes of tumor which commonly result in biopsies that are interpreted as indeterminate or suspicious for malignancy thus requiring the need for a thyroidectomy to obtain a definitive histologic diagnosis^3–6.

Numerous studies have evaluated potential diagnostic markers to improve the accuracy of FNA, but few have had little clinical utility in tumors that would be indeterminate or suspicious on FNA cytology^9–15. Nikiforov and colleagues prospectively tested 470 consecutive FNA samples from 371 patients for a panel of mutations including BRAF, RAS, RET/PTC, and PAX8-PPARγ and found that the presence of any mutation was a strong predictor of cancer¹⁹. Additionally, they found molecular testing to be 100% accurate in the lowest risk subgroup, follicular lesion of indeterminate significance. Sapio and associates confirmed a diagnosis in 25% of 16 suspicious for papillary thyroid cancer FNA biopsy results by mutation analysis for RET/PTC1, RET/PTC3, BRAF, and NTRK²⁰. Our results also show a significantly higher rate of somatic mutations in malignant thyroid nodules but with low sensitivity. Thus, somatic mutation analysis is likely to be most useful when positive to allow for more definitive initial operations rather than to avoid the need for a diagnostic thyroidectomy when FNA biopsy shows indeterminate or suspicious for malignancy results²¹.

We had previously identified six novel diagnostic markers for thyroid neoplasms that would be indeterminate or suspicious on FNA biopsy in 31 thyroid FNA biopsy samples¹⁷. In this current study demographic, clinical, tumor FNA genotype for common somatic mutations (BRAF V600E, NRAS, KRAS, RET/PTC1, RET/PTC3, NTRK1) and the previously identified 6 candidate gene expression levels were analyzed in 423 FNA biopsy samples to determine if they could further distinguish benign from malignant thyroid tumors. Five of the six candidate genes (ANGPT2, EGFR, EFNB2, TIMP1, and TMPRSS4) were able to classify (p<0.05) between benign and malignant thyroid tumors when evaluated in 423 FNA biopsy samples.

In contrast to our prior studies, TIMP1 expression level was the only independent diagnostic marker in addition to presence of a NRAS mutation that added to the accuracy of FNA cytology. This is likely a result of having a much larger sample size in the current study which allowed for separation of candidate gene expression patterns in benign and malignant samples encompassing a much broader histology of thyroid tumors. An acceptable scoring model derived from a training set and applied to an independent testing cohort had an excellent diagnostic accuracy of 91% with a specificity of 97% when applied to the entire cohort. However, the sensitivity of this model for the entire cohort was only 76%. The accuracy dropped to 61% for atypical lesions or indeterminate FNA samples. In the atypical follicular or hürthle cell lesions FNA subgroup, the specificity was 81% with a low sensitivity of 27%. This suggests that this model would not be helpful to determine the need and extent of thyroidectomy for an atypical lesion or indeterminate neoplasm FNA result. In the subgroup of patients with indeterminate FNA results, the scoring model had a specificity of 100% and a sensitivity of 14% In the subgroup of suspicious for malignancy FNA results the model was 77% accurate with a 100% sensitivity, but 0% specificity. This suggests that this model may be useful to determine the extent of initial thyroidectomy for an indeterminate neoplasm or suspicious for malignancy FNA result however, the sample size in each group is relatively small. We recognize that management decisions in patients with inconclusive thyroid FNA biopsy may be variable as there are no clearly established thresholds when a patient and surgeon would elect to not undergo or perform a diagnostic thyroidectomy or to determine the extent of thyroidectomy needed.

To our knowledge, this is one of the largest comprehensive studies that incorporated multiple clinical factors with FNA genotyping for known somatic mutation in addition to six candidate markers previously identified by cDNA array analysis to create a scoring model to improve the accuracy of FNA. Although this model had a high diagnostic accuracy, we do not believe the sensitivity was high enough to avoid the need for a diagnostic thyroidectomy in patients with indeterminate FNA results. However, in patients with suspicious FNA results the model correctly identifies 100% of patients with cancer and may be used to guide initial surgical treatment. The main limitations of this study were the relatively smaller sample size of patients in the atypical, indeterminate, and suspicious FNA subgroups for us to make strong clinical management recommendations based on our result. Additionally, some variability may exist with FNA cytologic classification according to the NCI State of the Science conference, which may not be uniformly interpreted as such at other center or by other cytopathologist.

In conclusion, this study introduces a novel scoring model incorporating TIMP1 expression and presence of NRAS mutation to improve the diagnostic accuracy of FNA or at least to better quantitate the preoperative risk of malignancy. Classification of FNA cytology according to the NCI guidelines, presence of NRAS mutation, and TIMP1 mRNA expression levels in combination may better predict the risk of malignancy and thus, allow the selection of more definitive initial surgical treatment. We believe prospective clinical trials to validate many of the proposed markers identified for thyroid cancers are needed to determine their ultimate clinical utility.