Differentiating Noninvasive Follicular Thyroid Neoplasm with Papillary-Like Nuclear Features from Classic Papillary Thyroid Carcinoma: Analysis of Cytomorphologic Descriptions Using a Novel Machine-Learning Approach

Abstract

Background:

Recent studies show various cytomorphologic features that can assist in the differentiation of classic papillary thyroid carcinoma (cPTC) from noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP). Differentiating these two entities changes the clinical management significantly. We evaluated the performance of support vector machine (SVM), a machine learning algorithm, in differentiating cases of NIFTP and encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion (EFVPTC) from cases of cPTC with the use of microscopic descriptions. SVM is a supervised learning algorithm used in classification problems. It assigns the input data to one of two categories by building a model based on a set of training examples (learning) and then using that learned model to classify new examples.

Methods:

Surgical pathology cases with the diagnosis of cPTC, NIFTP, and EFVPTC, were obtained from the laboratory information system. Only cases with existing fine-needle aspiration matching the tumor and available microscopic description were included. NIFTP cases with ipsilateral micro-PTC were excluded. The final cohort consisted of 59 cases (29 cPTCs and 30 NIFTP/EFVPTCs).

Results:

SVM successfully differentiated cPTC from NIFTP/EFVPTC 76.05 ± 0.96% of times (above chance, P < 0.05) with the sensitivity of 72.6% and specificity of 81.6% in detecting cPTC.

Conclusions:

This machine learning algorithm was successful in distinguishing NIFTP/EFVPTC from cPTC. Our results are compatible with the prior studies, which show cytologic features are helpful in differentiating these two entities. Furthermore, this study shows the power and potential of this approach for clinical use and in developing data-driven scoring systems, which can guide cytopathology and surgical pathology diagnosis.

INTRODUCTION

Papillary thyroid carcinoma (PTC) constitutes approximately 84% of all thyroid malignancies.[1] There has been a substantial increase in the incidence of PTC, in part due to early detection of mainly microcarcinomas and also more relaxed histologic criteria for detection of follicular variant of papillary thyroid carcinoma (FVPTC). Histologic diagnosis of FVPTC is based on detection of nuclear features of PTC, which are frequently less evident than classic PTC (cPTC). Papillary structures and psammomatous calcifications are absent, and nuclear pseudoinclusions are rare.[2] Until recently, the encapsulated FVPTC without invasion (EFVPTC) represented up to 20% of all PTCs.[3] Encapsulated or partially encapsulated/well-circumscribed FVPTCs with no associated capsular penetration or lymphovascular invasion have virtually no metastatic potential or risk of recurrence. Moreover, although there is some variability in results, in most cases, FVPTC harbors RAS mutations and PAX8/PPARG rearrangements and lacks BRAF mutations.[4,5] This cancer has recently been reclassified as a neoplasm of low malignant potential and is now termed “noninvasive follicular thyroid neoplasm with papillary-like nuclear features” (NIFTP).[6] This change will result in a corresponding reduction in the number of thyroid malignancies. Before the introduction of NIFTP, many of these patients were treated with total thyroidectomy followed by radioactive iodine therapy. Now, these patients are spared the anxiety of cancer diagnosis, and they will be managed more conservatively, requiring only lobectomy without the need for radioactive iodine therapy.[7]

Fine-needle aspiration (FNA) is the most essential and reliable procedure for preoperative evaluation of thyroid nodules. The cytopathologists follow the Bethesda system for reporting thyroid cytopathology (TBS). One important objective of TBS is to link each diagnostic category with a risk of malignancy (ROM) and an evidence-based clinical management guideline.[8] NIFTP and tumors historically classified as FVPTC do not typically exhibit the full range of cytologic features associated with cPTC and as a result have tended to be classified in the indeterminate categories of TBS (i.e., atypia of undetermined significance/follicular lesion of undetermined significance [category III, AUS/FLUS], suspicious for a follicular neoplasm/follicular neoplasm [category IV, SFN/FN], and suspicious for malignancy [category V]).[9] The two studies done by Strickland et al. and Faquin et al.[10,11,12] showed a significant decrease in the ROM in the indeterminate categories of TBS after this new classification with very little change in non-diagnostic, benign, and positive for malignancy categories (categories I, II, and VI). Therefore, this reclassification will change the ROM associated with some of TBS diagnostic categories and to triage patients with NIFTP to more conservative clinical management, cytopathologists need to distinguish NIFTP/FVPTC from cPTC. Recent studies show various cytomorphological features that can assist in the cytologic differentiation of cPTC from EFVPTC (previously) and NIFTP. Howitt et al.[13] examined a cohort of 39 cases including 11 EFVPTCs and 28 cPTCs. Cases of cPTC were significantly more frequently associated with a predominance of tumor sheets, papillae, and nuclear pseudoinclusions compared with EFVPTCs. Maletta et al. also showed that although the most common diagnosis that is given in the cases of NIFTP is “follicular neoplasm,” a number of these cases do have nuclear features of PTC and these nuclear features are significantly different from those of benign follicular tumors and hyperplastic nodules, but not from those of invasive FVPTC. Therefore, the presence of PTC nuclear features in a follicular patterned nodule may indicate the possibility of both FVPTC and NIFTP.[14] Strickland et al. also reinforced the results of previous studies by indicating that simple cytologic and architectural features (i.e., papillae, nuclear pseudoinclusions, psammomatous calcifications, and microfollicular architecture) can be used to distinguish the majority of NIFTPs and other follicular-patterned lesions from cPTCs prospectively.[15]

In recent years, attempts have been made to use machine learning algorithms in digital pathology for computer-assisted diagnosis. Machine learning techniques often used in digital pathology image analysis include supervised learning (e.g., support vector machine [SVM], random forest, and convolutional neural networks) and unsupervised learning (e.g., k-means, autoencoders, and principal component analysis).[16] However, there have been few attempts on using machine learning algorithms on pathology reports. The benefit of such an algorithm is to help the pathologists with decision-making in challenging cases and as an aid to pattern recognition software for computer-assisted diagnosis.[17,18] Herein, we took an alternative approach and evaluated the performance of SVM, a machine learning algorithm, in differentiating NIFTP from cPTC by analysis of the microscopic description part of cytopathology reports. SVM is a supervised learning algorithm that is used in classification problems. It assigns the input data to one of two categories by building a model based on a set of training examples (learning) and then using that learned model to classify new examples.[19]

METHODS

Study population

The study was approved by our institution's institutional review board. The dataset comprised surgical pathology cases with the diagnosis of cPTC, NIFTP, or EFVPTC on the final pathology report, 1 cm in size or larger, obtained from the laboratory information system. Only cases with existing FNA matching the tumor and available microscopic description were included. NIFTP cases with ipsilateral micro-PTC were excluded. For cases with multiple nodules, clinical history, radiologic findings, and surgical pathology report were reviewed to ensure the FNA correlated with the correct nodule. The final cohort consisted of 59 cases (29 cPTCs and 30 NIFTP/EFVPTCs) resected in the time frame of 2010–2016. A total of nine different cytopathologists had diagnosed the FNAs in the department of pathology in that period.

Data processing and machine learning algorithms

The data processing was done using Matlab Text Analytics Toolbox. We preprocessed the cytomorphological descriptions, by making all text lowercase, removing common words, such as “the, a, at, to” and reduce words into their root forms by trimming their endings. We also removed language that pertained to the differential diagnosis and only focused on the description.

To digitize and convert the microscopic descriptions to a format usable by the machine learning algorithm, the preprocessed microscopic descriptions were reviewed, and 59 cytomorphologic keywords/phrases that were used by the pathologists were identified. The synonymous phrases were then grouped into 32 categories [Table 1]. The presence or absence of each category in each case was recorded by values of 1 or 0 respectively in arrays 32 bins in length.

Table 1.

Keywords or phrases found commonly in the microscopic descriptions

1. Follicular cells	12. Cystic	25. Nuclear enlargement
2. Small follicular cells	13. Metaplastic	25. Enlarged nuclei
2. Benign follicular cells	13. Metaplasia	25. Enlarged
2. Benign appearing follicular cells	14. Reactive	26. Pale chromatin
3. Oncocytic cells	15. Hypercellular	26. Powdery chromatin
3. Oncocyte	15. Cellular	26. Fine chromatin
4. Histiocytes	16. Crowded	27. Intranuclear inclusion
4. Macrophages	17. Syncytial	27. Nuclear pseudoinclusion
4. Pigmented macrophages	18. Overlap	27. Pseudoinclusions
4. Hemosiderin-laden macrophages	19. Cluster	27. Pseudoinclusion
4. Pigment-laden macrophages	20. Microfollicular pattern	28. Nuclear grooves
4. Hemosiderin macrophages	20. Microfollicular arrangement	28. Grooves
5. Multinucleated	20. Microfollicular	28. Irregular nuclei
6. Cyst-lining cells	20. Microfollicle	28. Irregular nuclear contour
7. Scant colloid	21. Macrofollicles	29. Hyperchromatic
8. Abundant colloid	22. Sheets	29. Nuclear hyperchromasia
9. Dense colloid	23. Papillary	29. Hyperchromasia
9. Thick colloid	24. Atypical	30. Nucleoli
10. Calcification	24. Atypia	31. Pinpoint nucleoli
11. Focal		32. Prominent nucleoli

The synonymous phrases are assigned to 32 different categories

We used a supervised machine learning algorithm, (SVM, specifically an L1 regularized SVM implementation through LIBLINEAR),[20] to train a linear classifier on the digitized microscopic descriptions, assigning them as NIFTP/EFVPTC or cPTC.

Linear SVM has one modifying parameter, C (i.e., misclassification cost and error penalty). When a small misclassification cost is used, the algorithm can find more global and generalizable trends in the data but is prone to making errors. A higher misclassification cost is associated with more specific solutions, a lower error rate, but less generalizability. The goal is a misclassification cost best balancing error and generalizability. We thus explored misclassifications (C) values ranging from 0.1 to 100 (log scale) to find the C with the best balance between error and generalizability. In addition, we used a regularized version of SVM here. This means that we used a penalty for model complexity and forced the SVM algorithm to use the fewest number of keywords in making the prediction. Limiting complexity would make model interpretation easier and also increases its generalizability.

The classifier was trained on a training set comprising all cases except for one random case of NIFTP/EFVPTC and one random case of cPTC. The trained model's ability to classify (performance) was then assessed on predicting the left-out cases (i.e., leave-2-out cross-validation). We repeated this process for all possible iterations of leave-2-out. As a post hoc analysis, we interrogated the classifier post training to identify the keywords that were used to assign cases to NIFTP/EFVPTC versus cPTC.

RESULTS

The demographic data are shown in Table 2. The Bethesda categories ranged from II to V in the NIFTP/EFVPTC cases with 66% being Bethesda categories III and IV. The Bethesda categories ranged from III to VI in the cPTC cases with category VI being the most common (59%) [Table 3]. The average length of the microscopic description was 22.17 ± 9.38 words for NIFTP/EFVPTC and 26.31 ± 10.60 words for cPTC. Word counts did not differ significantly between NIFTP/EFVPTC and cPTC microscopic descriptions (Student's t-test, P = 0.12). Frequency of various cytomorphologic features as mentioned in the microscopic description part of the cytology reports are shown in Table 4. Microfollicles were more commonly described in the NIFTP/EFVPTC cases, and sheets were more commonly described in the cPTC cases. Nuclear grooves, pseudoinclusions, and papillary structures were more commonly described in the cPTC cases compared to the cases of NIFTP/EFVPTC.

Table 2.

Age, gender, tumor location, and tumor size in patients with noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma

	Average age (range)	Male/female (%)	Right/left/isthmus (%)	Average size (range)
NIFTP/EFVPTC	47.45 (29-80)	3.33/67.77	56.67/43.33/0.00	3.71 cm (1.00-7.80 cm)
cPTC	45.29 (11-80)	6.90/93.10	41.38/44.83/13.79	2.57 cm (1.00-6.00 cm)

NIFTP: Noninvasive follicular thyroid neoplasm with papillary-like nuclear features, EFVPTC: Encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion, cPTC: Classic papillary thyroid carcinoma

Table 3.

The frequency of the Bethesda categories in cases of noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma (%)

Bethesda category	NIFTP/EFVPTC (%)	cPTC (%)
I	0 (0)	0 (0)
II	10 (33)	0 (0)
III	10 (33)	5 (17)
IV	7 (23)	2 (7)
V	3 (10)	5 (17)
VI	0 (0)	17 (59)

NIFTP: Noninvasive follicular thyroid neoplasm with papillary-like nuclear features, EFVPTC: Encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion, cPTC: Classic papillary thyroid carcinoma

Table 4.

The frequency of various cytomorphologic features noted in the microscopic descriptions of noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma cases (%)

	NIFTP/EFVPTC (%)	cPTC (%)
Microfollicles	37	10
Sheets	43	66
Nuclear grooves	43	90
Nuclear pseudoinclusions	17	69
Papillae	3	17

NIFTP: Noninvasive follicular thyroid neoplasm with papillary-like nuclear features, EFVPTC: Encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion, cPTC: Classic papillary thyroid carcinoma

The SVM-trained classifier correctly identified cPTC from NIFTP/EFVPTC in 76.05 ± 0.96% of cases in a cross-validated sample (predictions were done on test set that the model was not trained on). This performance was above chance (>50%, P < 0.001). For detection of cPTC, the sensitivity of prediction was 72.6% and the specificity was 81.6%. Area under the receiver operating characteristic curve was 0.79 (95% confidence interval 0.77–0.81) [Figure 1].

Figure 1.

Receiver operating characteristic curve, representing the support vector machine algorithm's performance in distinguishing classic papillary thyroid carcinoma from noninvasive follicular thyroid neoplasm with papillary-like nuclear features cases (cross-validated). The shaded area represents 95% confidence interval calculated through bootstrapping

We then conducted a post hoc analysis, assessing the strategy that was used by the machine learning algorithm to make the decision. Through its training, SVM assigns a weight to each of the phrases in the text: a positive weight means that the occurrence of that word is associated with cPTC diagnosis, and a negative weight implies that the occurrence of that phrase is associated with NIFTP/EFVPTC diagnosis. Figure 2 shows the categories with a nonzero weight that were used by the classifier to differentiate NIFTP/EFVPTC from cPTC.

Figure 2.

The plot represents the (normalized to the largest) weights that are used for the classification. Only categories that had a non-zero weight were plotted. The weights with a negative value were suggestive of noninvasive follicular thyroid neoplasm with papillary-like nuclear features, and the weights with a positive value were indicative of classic papillary thyroid carcinoma. Greater weights were more strongly associated with a noninvasive follicular thyroid neoplasm with papillary-like nuclear features versus classic papillary thyroid carcinoma diagnosis

CONCLUSIONS

Reclassification of EFVPTC as a neoplasm of low malignant potential (NIFTP) will result in a smaller number of diagnosed thyroid cancer cases and more conservative management of these patients.[6] Therefore, preoperative differentiation of cPTC from FVPTC is of utmost importance. Cytopathologic evaluation according to TBS is the most reliable diagnostic procedure used in the preoperative assessment of thyroid nodules. Due to subtle nuclear features of FVPTC, these tumors are usually classified in the indeterminate categories of TBS, and as described by Strickland et al. and Faquin et al.,[10,11] this reclassification will change the ROM associated with these diagnostic categories. Various cytomorphological features such as the predominance of tumor sheets versus microfollicular architecture, papillae, nuclear pseudoinclusions, and psammomatous calcifications[13,14,15] can assist in the cytologic differentiation of cPTC from FVPTC and NIFTP.

In this study, we evaluated the performance of SVM, a supervised machine learning algorithm, in differentiating NIFTP/EFVPTC from cPTC by analyzing the microscopic description part of cytopathology reports using a cohort of cPTCs and NIFTP/EFVPTCs with a corresponding FNA. We demonstrated that a machine learning algorithm was able to successfully distinguish microscopic descriptions of NIFTP/EFVPTC and cPTC FNAs in 76.05% ± 0.96% of cases with high sensitivity and specificity.

The significant aspect of this study was applying predictive analytics and machine learning to analyze cytomorphologic descriptions, written by practicing cytopathologists. In distinguishing cPTC versus NIFTP/EFVPTC, the model relied on keywords that were not unexpected, but the analysis provided relative weights (i.e., importance) for each of these keywords. Machine learning algorithms can be used as a decision-making tool in challenging cases or as an aid to pattern recognition software for computer-assisted diagnosis.[17] Although there is a lot of interest in applying machine learning algorithms in digital pathology, most studies have focused on machine vision and image analysis. We, however, employed semantic and text-based analysis methods. Semantic methods cannot be deployed as an independent tool in making diagnostic decisions, but they can help provide a better understanding of the intuitions and the tacit approaches that are used by pathologists when making diagnoses. These intuitions then can turn into tangible criteria, guiding others. They can also be used in providing feedback and confirmation when diagnoses are made. Although the diagnosis of NIFTP/EFVPTC is not a cytological diagnosis and relies on tumor excision to exclude capsular and vascular invasion and the presence of papillary structures, it will be helpful to cytologically distinguish NIFTP/EFVPTC from cPTC as these patients require a more conservative surgery (hemithyroidectomy versus total thyroidectomy). The model introduced here was able to correctly categorize NIFTP/EFVPTC and cPTC in three-quarter of the cases. Similar to many other medical diagnostic tools, this approach has predictive value, but the predictive power is not absolute. However, it can help better stratify the risk of cancer and can be used in making clinical decisions by providers and patients. This work is based on modest sample size, collected in one center. This raises the possibility that the performance we found could represent overfitting, which means that the algorithm learns trends in the data not generalizable to other samples. We addressed this by (1) using regularization (limiting model complexity), and (2) using cross-validation by training and testing the model on two separate datasets. Replication with larger samples, however, would be required to bolster the strength of our observation. Despite these methodological limitations, we found validated and clinically meaningful results. In conclusion, this study shows the power and potential of this method for clinical use. Ideally, this approach can be used to develop data-driven scoring systems, which can act as a guide for cytopathology and surgical pathology diagnosis.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.