Skip to main content
International Journal of Molecular Sciences logoLink to International Journal of Molecular Sciences
. 2019 Jun 26;20(13):3126. doi: 10.3390/ijms20133126

Differences in Mutational Profile between Follicular Thyroid Carcinoma and Follicular Thyroid Adenoma Identified Using Next Generation Sequencing

Martyna Borowczyk 1,*, Ewelina Szczepanek-Parulska 1, Szymon Dębicki 1, Bartłomiej Budny 1, Frederik A Verburg 2, Dorota Filipowicz 1, Barbara Więckowska 3, Małgorzata Janicka-Jedyńska 4, Lidia Gil 5, Katarzyna Ziemnicka 1, Marek Ruchała 1
PMCID: PMC6651591  PMID: 31248021

Abstract

We aimed to identify differences in mutational status between follicular thyroid adenoma (FTA) and follicular thyroid cancer (FTC). The study included 35 patients with FTA and 35 with FTC. DNA was extracted from formalin-fixed paraffin-embedded (FFPE) samples from thyroidectomy. Next-generation sequencing (NGS) was performed with the 50-gene Ion AmpliSeq Cancer Hotspot Panel v2. Potentially pathogenic mutations were found in 14 (40%) FTA and 24 (69%) FTC patients (OR (95%CI) = 3.27 (1.22−8.75)). The number of mutations was higher in patients with FTC than FTA (p-value = 0.03). SMAD4 and STK11 mutations were present only in patients with FTA, while defects in FBXW7, JAK3, KIT, NRAS, PIK3CA, SMARCB1, and TP53 were detected exclusively in FTC patients. TP53 mutations increased the risk of FTC; OR (95%CI) = 29.24 (1.64–522.00); p-value = 0.001. FLT3-positivity was higher in FTC than in the FTA group (51.4% vs. 28.6%; p-value = 0.051). The presence of FLT3 and TP53 with no RET mutations increased FTC detectability by 17.1%, whereas the absence of FLT3 and TP53 with a presence of RET mutations increased FTA detectability by 5.7%. TP53 and FLT3 are candidate markers for detecting malignancy in follicular lesions. The best model to predict FTA and FTC may consist of FLT3, TP53, and RET mutations considered together.

Keywords: follicular thyroid cancer, follicular thyroid adenoma, next-generation sequencing, whole-genome studies, genetics

1. Introduction

The differentiation between follicular thyroid adenoma (FTA) and follicular thyroid carcinoma (FTC) in preoperative diagnostics is not reliable when based on imaging or biopsy [1]. Therefore, in recent years, many studies have attempted to identify additional factors for the discrimination of these two related pathological entities [2], as well as to further clarify the extent of the relationship in their pathogenesis [3].

A main area on which research has focused is the identification of genetic alterations that may be present in FTC, but not in FTA. Molecular markers are increasingly used as pre-surgical diagnostic tools in the management of indeterminate thyroid nodules [4]. However, the panel of genes proposed to date has not demonstrated conclusive diagnostic results [5].

While in papillary thyroid cancer (PTC), next-generation sequencing (NGS) has yielded promising results [6,7], this technique has, thus far, rarely been applied to FTC [8]. The majority of studies in FTC have tested only well-recognized thyroid cancer-related genes using, e.g., a 7-gene [9], 24-gene panel, or standard PCR for single mutations [10]. Furthermore, for FTA, even fewer data about the genetic alterations have been published [11]. To the best of our knowledge, authors comparing the genetic background of FTA and FTC have so far rarely applied NGS [12], which enables detection of many genetic changes simultaneously [13] and the analysis of the complex genetic interplay of distinct mutations [14].

Recently, in oncological analyses, novel extensive panels have been successfully applied in a variety of solid cancers [15]. These have been used to identify point mutations, single nucleotide variants, polymorphisms, and rearrangements, which may be associated with the genesis of malignant tumors [16]. However, the genetic background of FTA and FTC has not yet been compared using this method.

The aim of this study was to compare FTA and FTC using the wide oncological molecular panel and to identify differences in mutational profile, which may aid in the preoperative differentiation between these two related thyroidal pathologies, as well as in an understanding of their pathways of origin.

2. Results

The molecular analyses allowed for the identification of at least one possibly pathogenic mutation in 14 out of 35 (40.0%) patients diagnosed with FTA and 24 (68.6%) out of 35 patients diagnosed with FTC (Table 1). The number of detected mutations was significantly higher in patients with FTC in comparison with those diagnosed with FTA (p-value = 0.03). Identification of any mutation from the panel indicated that the patient was 3.27-times more likely to be diagnosed with FTC (OR (95%CI) = 3.27 (1.22–8.75)).

Table 1.

Patients’ mutational status according to histopathological diagnosis.

Characteristics Follicular Thyroid Adenomas n = 35 Follicular Thyroid Carcinomas n = 35 p-Value OR (95%CI)
Presence of any mutation, n (%) 14 (40.0%) 24 (68.6%) 0.030 3.27 (1.22–8.75)
Presence of any mutation found in COSMIC database, n (%) 8 (22.9%) 9 (25.7%) 0.780 1.17 (0.39–3.49)
Number of mutated genes, n (%) 13 (37.1%) 18 (51.4%) 0.229 1.79 (0.69–4.65)
Mutations found in both groups of patients
APC, n (%) 1 (2.9%) 4 (11.4%) 0.356 4.39 (0.46–41.40)
CSFR1, n (%) 7 (20.0%) 6 (17.1%) 0.759 0.83 (0.25–2.77)
ERBB4, n (%) 2 (5.7%) 3 (8.6%) 1.000 1.55 (0.24–9.88)
FGFR3, n (%) 7 (20.0%) 7 (20.0%) 1.000 1.00 (0.31–3.23)
FLT3, n (%) 10 (28.6%) 18 (51.4%) 0.051 2.65 (0.99–7.11)
HRAS, n (%) 7 (20.0%) 6 (17.1%) 0.759 0.83 (0.25–2.77)
IDH1, n (%) 1 (2.9%) 1 (2.9%) 1.000 1.00 (0.06–16.65)
KDR, n (%) 2 (5.7%) 4 (11.4%) 0.673 2.13 (0.36–12.46)
MET, n (%) 2 (5.7%) 2 (5.7%) 1.000 1.00 (1.13–7.53)
PDGFRA, n (%) 5 (14.3%) 5 (14.3%) 1.000 1.00 (0.26–3.81)
RET, n (%) 11 (31.4%) 11 (31.4%) 1.000 1.00 (0.36–2.74)
COSMIC mutations found in both groups of patients
HRAS (COSM249860), n (%) 7 (20.0%) 5 (14.3%) 0.526 0.67 (0.19–2.35)
PDGFRA (COSM22413), n (%) 2 (5.7%) 2 (5.7%) 1.000 1.00 (1.13–7.53)
Mutations found in only one group of patients
SMAD4, n (%) 1 (2.9%) 0 (0.0%) 1.000 0.32 (0.01–8.23)
STK11, n (%) 1 (2.9%) 0 (0.0%) 1.000 0.32 (0.01–8.23)
FBXW7, n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
JAK3, n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
KIT, n (%) 0 (0.0%) 4 (11.4%) 0.114 10.14 (0.53–195.91)
NRAS, n (%) 0 (0.0%) 2 (5.7%) 1.000 5.30 (0.25–114.47)
PIK3CA, n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
SMARCB1, n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
TP53, n(%) 0 (0.0%) 10 (28.6%) 0.001 29.24 (1.64–522.00)
COSMIC mutations found in only one group of patients
IDH1 (COSM105), n (%) 1 (2.9%) 0 (0.0%) 1.000 0.32 (0.01–8.23)
JAK3 (COSM34213), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
KIT (COSM21983), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
KIT (COSM28026), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
MET (COSM710), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
NRAS (COSM584), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)
SMARCB1 (COSM1090), n (%) 0 (0.0%) 1 (2.9%) 1.000 3.09 (0.12–78.41)

Bold elements denote statistical significance (p-value < 0.05) for pairwise comparisons between FTC and FTA. p-values were based on chi-square test (or Fisher’s exact test when appropriate).

The median frequency and the depth of coverage of all detected variants in the whole cohort were 59.9% and 537 x, respectively, and ranged from 10.8% to 100% and from 260 to 11138 x. Twenty-six out of the mutation-positive patients (68%) had allele burden of over 51% and were considered homozygous for the mutation.

The majority of the mutations occurred with the same frequency in FTA and FTC. SMAD4 and STK11 mutations were present only in patients diagnosed with FTA. Defects in FBXW7, JAK3, KIT, NRAS, PIK3CA, SMARCB1, and TP53 were detected exclusively in FTC patients. However, only TP53 mutations indicated that patients were significantly more likely to be diagnosed with FTC (OR (95%CI) = 29.24 (1.64−522.00), p-value = 0.001).

The most common mutation in FTA was in RET, followed by FLT3 (31.4% and 28.6% of all FTA) whereas, conversely, the most common FTC mutation was in FLT3, followed by RET (51.4% and 31.4% of all FTC). The increased FLT3-positivity in FTC was found to be significant (p-value = 0.051). Figure 1 shows a list of mutations common to and exclusive for FTA and FTC and the co-occurrence of the mutations in different patients.

Figure 1.

Figure 1

Genetic landscape of follicular thyroid adenomas (n = 14/35)—marked in blue—and follicular thyroid carcinomas (n = 24/35)—marked in red. Columns of the table represent single patients, rows—type of mutation.

The variety of the mutations tended to occur with higher frequency in FTC in comparison with FTA although this was not significant.

Multivariate logistic regression analysis showed that FLT3, TP53, and RET mutations should be jointly considered to improve the prediction of FTA and FTC diagnosis (Table 2 and Table 3). For this model, the area under curve (AUC) in ROC curve was significantly higher for the prognosis of FTA and FTC than in a case of single mutation model (AUC (Model 1) = 0.64, AUC (Model 2) = 0.76, p-value = 0.016). These results were also confirmed by IDI (0.11, p-value = 0.003), as well as categorical net reclassification improvement (categorical NRI) (0.23, p-value = 0.007).

Table 2.

The coefficients necessary to determine the predicted risk of follicular thyroid carcinoma (FTC) and follicular thyroid adenoma (FTA) on the basis of single mutation TP53-Model 1 and the group of the mutations (comprising FLT3 + TP53 + RET) selected in the backward stepwise logistic regression-Model 2.

Mutations p-Value * β Coefficient *
Model 1
intercept 0.206 −0.33
TP53 0.028 3.38
Model 2
intercept 0.095 −0.57
FLT3 0.023 2.19
RET 0.058 −1.93
TP53 0.029 3.34

* Penalized maximum likelihood estimation method for bias correction (Firth’s method).

Table 3.

The comparison of two models of FTC and FTA prediction.

Measures of Prediction Accuracy Mutations in Logistic Regression Models Models Comparison
Model 1 (TP53 Only) Model 2 (FLT3 + TP53 + RET) p-Value
AUC (95%CI) 0.64 (0.57–0.72) 0.76 (0.65–0.86) 0.016
IDI (95%CI) 0.11 (0.04–0.19) 0.003
categorial NRI (95%CI) 0.23 (0.06–0.40) 0.007
categorial NRI [FTC] 0.171
categorial NRI [FTA] 0.057

The prevalence of FLT3 and TP53 with no RET mutations increased FTC detectability by 17.1% (NRI (FTC) = 0.171), whereas the absence of FLT3 and TP53 with a presence of RET mutations increased FTA detectability by 5.7% (NRI (FTA) = 0.057). Assignment of patients to FTC and FTA groups after reclassification due to the logistic regression model based on three indicated mutations is also shown in Figure 2.

Figure 2.

Figure 2

Reclassification of individuals predicted to follicular thyroid cancer (FTC) group and to follicular thyroid adenoma (FTA) group according to logistic regression model based on mutational status.

Ten patients (28.6%) were correctly classified in the FTC group based only on the TP53 mutation, whereas 16 (45.7%) patients were correctly classified based on TP53, FLT3, and RET. Similarly, the number of patients correctly classified as belonging to the FTA group increased by 8.6%. based on these three indicated mutations.

The characteristics of patients diagnosed with follicular thyroid cancer according to their mutational status is presented in the Table 4. The full list of mutations, which occurred in at least three FTA and/or FTC samples, is presented in Table S1.

Table 4.

The characteristics of patients diagnosed with follicular thyroid cancer according to their mutational status.

Characteristics TP53+ or TP53-, FLT3+, RET- n = 16 Other Mutational Status n = 19 p-Value
Male/female, n (%) 2/14 (12.5%/87.5%) 2/17 (10.5%/89.5%) 1.0000
Median age at diagnosis, years (range) 50 (27–81) 56 (27–82) 0.4560
Age group (≤60 years/>60 years), n (%) 3/13 (18.75%/81.25%) 7/12 (36.8%/63.2%) 0.2853
Median length of follow-up, months (range) 108 (17–130) 116 (28–144) 0.5674
Multifocality, n (%) 2 (12.5%) 2 (10.5%) 1.0000
Capsule invasion, n (%) 5 (31) 9 (47) 0.3322
Extracapsular extension, n (%) 9 (56.3%) 9 (47.4%) 0.6005
Nodal (N) involvement, n (%) 1 (6.3%) 3 (15.8%) 0.6081
Mean tumor size, mm (range) 32 (7–75) 29 (12–18) 0.6737
Tumor diameter ≤10 mm, n (%) 1 (6.3%) 1 (5.3%) 1.0000
Localization in the right/left/both lobes, n (%) 7/2/7 (43.8%/12.5%/43.8%) 10/1/8 (52.6%/5.3%/42.1%) 0.8844
Chronic lymphocytic thyroiditis n (%) 2 (12.5%) 5 (26.3%) 0.4150
Radioactive iodine-refractoriness n (%) 0 (0.0%) 2 (10.5%) 0.4891

The p-values were based on chi-square test (or Fisher’s exact test when appropriate) for categorical variables and Mann–Whitney U test for quantitative variables.

3. Discussion

Our study provides a new perspective in the debate on genetics of follicular lesions. Whether FTA and FTC constitute distinct entities or a continuum is still debatable [17,18,19,20]. The similarity of their genetic background found here may suggest the latter. The majority of mutations occur with similar frequency in FTA and FTC. However, the number of possible pathogenic mutations is higher in FTC than FTA. Also, the variety and number of the mutations found exclusively in FTC tend to be higher than in FTA. This demonstrates a more complex FTC genetics than FTA and may reflect a natural history of FTC with the accumulation of new genetic changes.

The most common finding in FTC was FLT3 mutation. A mutation in class III receptor tyrosine kinase that is involved in the regulation of apoptosis, proliferation, and differentiation of hematopoietic cells may result in constitutive autophosphorylation of the immature form of the FLT3 receptor, resulting in strong factor-independent activation of STAT5 [21], which was previously described in acute myeloid leukemia (AML) [22,23]. However, our study demonstrates that this may be similar to FTC. We demonstrated that the FLT3 mutation, most common in FTC, may be an interesting malignancy marker candidate as it was nearly twice as common in FTC than in FTA. As pre- and postsurgical diagnosis of FTC remains a challenge [24], FLT3-positivity as a possible marker of malignancy should be carefully evaluated in future prospective studies.

According to our results, TP53 may be another candidate malignancy marker, as this was the second most common mutation in FTC samples not detected in any FTA specimen. The presence of TP53 mutation indicated the patient was 29-times more prone to have FTC. The role of TP53 has been confirmed in previous studies [25]. TP53 damage in a differentiated carcinoma is likely a key factor of dedifferentiation and major chromosomal instability [26,27]. TP53 alterations causing inactivation of apoptosis and cell-cycle progression were also reported to be the most frequent (54.4%) in anaplastic thyroid carcinomas (ATCs) [28]. The wide confidence interval for the odds ratio does not allow us to precisely determine how many times higher the risk for FTC diagnosis will be when this mutation occurs, but it does not undermine the statistical significance of the relationship. The significant influence of the TP53 mutation on the occurrence of FTC was also confirmed by building a logistic regression model and determining the ROC curve (Table 3), where we can also see that the area under the ROC curve with the confidence interval (AUC (95% CI) = 0.64 (0.57−0.72)) is above 0.5, which increases the reliability of the findings.

RAS mutations are most common in FTA and FTC, where they are considered to be the second most common genetic alterations [29]. In our study, RAS was the third most commonly mutated gene. The RAS pathway is considered to be an indicator of follicular-derived thyroid lesions [30]. However, despite their high prevalence in FTA and FTC, their role as malignancy markers remains unclear [31]. The importance may lie in a type of RAS mutation. In our study, HRAS occurred both in FTA and FTC. However, NRAS was more prevalent in FTC. NRAS has previously been reported to be more frequent in carcinomas (24%) than adenomas (14%) [32], and significantly associated with mortality. There is a possibility that the NRAS mutation may indicate a higher malignant potential than other forms of RAS [33]. However, the presence of the NRAS mutation in only one patient sample in our study reduces the generalizability of this finding and requires further research. Moreover, the exclusive occurrence of an NRAS mutation in FTC, shown also in other studies [33], indicates that the diagnosis of adenoma alongside the presence of this mutation should be made cautiously. As opposed to previous findings [34,35], KRAS was not found in either FTA or FTC. The presence of HRAS mutation with the same frequency in both benign and malignant lesions shown in our study may suggest its role in early tumorigenesis [36], and suggests a common genetic background. Our study raises the possibility that FTAs with RAS mutations have an inherent malignant potential and suggests caution. However, the clinical significance of this finding should be further investigated in more patients and over a longer follow-up period. The results of our study should not drive the decision for surgery, although they may highlight a group of higher-risk patients that should be more vigilantly monitored, as RAS-positive FTAs may have a higher risk of lesion malignant transformation.

An interesting finding is the presence of SMAD4 and APC mutations in FTA, previously reported in ATC [28]. Surprisingly, RET mutations, occurring mostly in medullary thyroid carcinoma, were one of the most common genetic events in both FTAs and FTCs. As the mutation was previously found in both benign and malignant thyroid lesions [37], mainly advanced and poorly differentiated thyroid cancers, this may indicate a group of patients who should be subjected to special surveillance. Thus, the mutational profile may determine not only a choice of targeted treatment, but also the frequency of the follow-up visits.

The mutation in the MET gene described in both FTA and FTC may potentially be a druggable target. This demonstrates a role for NGS in treatment personalization, to determine treatment based on the mutational profile of the tumor and to choose a drug that would potentially target the detected cancer pathway.

A logistic regression model showed that the most accurate predictive model for FTC and FTA is based on the combination of FLT3, TP53, and RET mutations. The prevalence of FLT3 and TP53 with no RET mutations increased FTC risk by 17.1%, whereas the presence of RET mutations alone indicated an increased FTA detectability by 5.7%. Any of the described factors, apart from the mutational status, do not differ when comparing FTA and FTC—which makes the mutational status the only factor differentiating FTA and FTC in the multivariate regression analysis. When compared to different features of FTC, which is shown in Table 4, no significant clinico-pathological differences in a group of patients divided according to their mutational status was found. Again, the mutational status is the only factor to differ between groups and to be included in multivariate analysis. The prediction of FTA and FTC is, therefore, based on complex genetic interplay of the mutations and reflects heterogenic genetic background of follicular lesions.

Although our study group was relatively small, the research focused on detailed analysis of NGS data of FTA and FTC from homogenous population. This is the first study to compare genetic background in FTA and in FTC from the same population with NGS using a panel that comprises not only thyroid cancer-specific mutations [38], but an extended panel of 50 different types of cancer-related genes. Moreover, given a parameter value difference of 25%, a standard deviation of 30%, an alpha error level of 0.05, and a beta error level of 50%, both FTC and FTA groups should consist of at least 16 patients to obtain statistically significant conclusions. Another limitation was the absence of paired normal tissue samples. However, recent recommendations of the Association for Molecular Pathology and College of American Pathologists (AMPCAP) established that a 30 x medium coverage might be adequate for germline testing [39]. In the absence of paired normal tissue samples, a much higher coverage is necessary for somatic testing, and for targeted panels, the coverage recommended by AMPCAP to reliably report a somatic variant is at least 250 reads depth in the clinical setting [39]. Therefore, such a cut-off was established, although this decreased the number of mutations found.

Further studies are needed to understand the relationships between the presence of these genetic mutations, their impact on the development of thyroid tumors, and their clinical application. Follow-up studies in a larger cohort of patients with a longer follow-up period, including collection of data on radioiodine refractoriness and overall survival rate, are required to fully understand the significance of these mutations in thyroid follicular lesions, as well as their potential as new diagnostic and therapeutic targets.

FTA and FTC share many common mutations. However, the number of mutations is significantly higher in FTC as compared with FTA. This indicates that FTA and FTC may share common genetic background that may be more complex in the case of FTC. TP53 and FLT3 may be candidate markers for detecting malignancy in follicular lesions. However, the most accurate predictive model for FTA and FTC may comprise a combination of FLT3, TP53, and RET mutations. Further understanding of the importance of FLT3, HRAS, RET, and other mutations found in FTA is required to determine whether more careful follow-up or more aggressive treatment is necessary. Identification of new genetic factors involved in FTC pathogenesis could improve understanding of carcinogenesis and enable the development of new targeted drugs.

4. Materials and Methods

4.1. Patients’ Characteristics and Clinicopathological Analysis

We retrospectively analyzed 70 randomly selected patients at the single tertiary care department of endocrinology of our university hospital, diagnosed with follicular lesions (35 with FTA and 35 with FTC), according to the World Health Organization [40]. Both groups were adjusted for age and gender.

The study was approved by the Bioethical Committee of Poznan University of Medical Sciences (an approval no. 1061/15 from January 2015) and was conducted in accordance with the Declaration of Helsinki. Additional informed consent was not required due to the retrospective nature of our analysis and the use of stored materials.

Surgically obtained formalin-fixed paraffin-embedded (FFPE) samples from total or subtotal thyroidectomy and detailed clinical annotates were subjected to further analysis. The group consisted of 58 women and 12 men with a median age at diagnosis of 55 years (range 27 to 82). Patients enrolled in the study were Caucasians not suffering from any other endocrine disorders or cancer and were not receiving any treatment at the time of diagnosis. The analysis covered data collected 2008–2018. Patient characteristics are presented in Table 5.

Table 5.

Patients’ characteristics according to histopathological diagnosis.

Characteristics Follicular Thyroid Adenomas n = 35 Follicular Thyroid Carcinomas n = 35 p-Value
Male/female, n (%) 27/8 (77.1%/22.9%) 31/4 (88.6%/11.4%) 0.205
Median age at diagnosis, years (range) 55 (29–81) 52 (27–82) 0.549
Age group (≤60 years/>60 years), n (%) 24/11 (68.6%/31.4%) 23/12 (65.7%/34.3%) 0.799
Median length of follow-up, months (range) 89 (18–122) 112 (17–144) 0.687
Multifocality, n (%) 0 (0.0%) 4 (11.4%) 0.114
Capsule invasion, n (%) NA 14 (40.0%) NA
Extracapsular extension, n (%) NA 18 (51.4%) NA
Nodal (N) involvement, n (%) NA 3 (8.6%) NA
Mean tumor size, mm (range) 23 (6–50) 30 (7–80) 0.112
Tumor diameter ≤10 mm, n (%) 5 (14.3%) 2 (5.7%) 0.428
Localization in the right/left/both lobes, n (%) 18/16/1 (51.43%/45.7%/2.9%) 17/15/3 (48.6%/42.9%/8.6%) 0.739
Chronic lymphocytic thyroiditis n (%) 2 (5.7%) 7 (20.0%) 0.151
Radioactive iodine-refractoriness n (%) NA 2 (5.7%) NA

The p-values were based on chi-square test (or Fisher’s exact test when appropriate) for categorical variables and Mann–Whitney U test for quantitative variables. NA—not applicable.

The diagnosis of FTC was confirmed by histopathological reexamination of the specimen following thyroidectomy performed by qualified pathologists. For each patient, we recorded the age at diagnosis, gender, tumor size, multifocality, extra-thyroidal extension, the presence of histopathological signs of chronic lymphocytic thyroiditis, histopathological staging (pTNM) according to the 8th tumor-node-metastasis (TNM) classification [41], and radioiodine refractoriness.

Tumors were regarded as multifocal when two or more foci were found. In the case of multifocality, the size of the tumor was noted as the size of the largest focus. All samples were anonymized.

For the present study, FTC was defined as radioactive iodine (RAI) refractory if a cumulative activity of 22.2 GBq/600 mCi of RAI did not result in achieving complete remission.

4.2. Genomic DNA Extraction

A qualified pathologist indicated areas of interest from 70 FFPE specimens, and the unstained slides were manually microdissected. Genomic DNA was extracted using a QIAamp DNA FFPE Tissue Kit (Qiagen, Valencia, CA, USA), following the manufacturer’s instructions. Genomic DNA was quantified using a fluorometer Qubit platform (Invitrogen, Carlsbad, CA, USA), and the DNA quality and integrity were tested.

4.3. Next Generation Sequencing

The 50-gene Ion AmpliSeq Cancer Hotspot Panel v2 (CHPv2) (Thermo Fisher Scientific, Carlsbad, CA, USA) was used with the IonTorrentTM Personal Genome Machine platform (Life Technologies, Foster City, CA, USA) in all experiments. The panel enables the amplification of 207 amplicons covering approximately 2800 mutations deposited in the COSMIC database from 50 oncogenes and tumor-suppressor genes commonly mutated in human cancers (ABL1, AKT1, ALK, APC, ATM, BRAF, CDH1, CDKN2A, CSF1R, CTNNB1, EGFR, ERBB2, ERBB4, EZH2, FBXW7, FGFR1, FGFR2, FGFR3, FLT3, GNA11, GNAS, GNAQ, HNF1A, HRAS, IDH1, IDH2, JAK2, JAK3, KDR, KIT, KRAS, MET, MLH1, MPL, NOTCH1, NPM1, NRAS, PDGFRA, PIK3CA, PTEN, PTPN11, RB1, RET, SMAD4, SMARCB1, SMO, SRC, STK11, TP53, and VHL). The Ion AmpliSeq Library Kit, version 2.0 (Life Technologies) was used to amplify 10 ng of DNA according to the manufacturer’s instructions. Sequencing beads were templated and enriched using the Hi-Q Template OT2 200 Kit. The libraries were barcoded with Ion Xpress Barcode Adaptors Kit (Thermo Fisher Scientific, Carlsbad, CA, USA), clonally amplified by emulsion PCR in vitro on the Ion PGM Template OneTouch 2 system (Thermo Fisher Scientific, Carlsbad, CA, USA). Ionsphere particles with DNA were isolated and sequenced on an Ion 318v2 Chip using the Hi-Q Sequencing Kit (Thermo Fisher Scientific, Carlsbad, CA, USA) according to the manufacturer’s protocols.

4.4. Mutation Analysis

The obtained data from genomic experiments were analyzed using dedicated software. Signal processing, mapping, and quality control were performed with Torrent Suite Software, v.5.2 (Life Technologies). The sequence variants were called, and data were analyzed using the Ion Reporter with the AmpliSeq CHPv2 single-sample workflow and default settings. The Variant Caller plugin included in the Torrent Suite Software (v.3.6; Thermo Fisher Scientific, Carlsbad, CA, USA) and the MutationTaster2 algorithm were used to identify variations in target regions. Variants were categorized according to whether they comprised a nonsynonymous or frameshift mutation or stop codon in the exonic region. Each of the identified genetic variations was coded according to “plus strand” of the Human Genome assembly hg19. The limit of detection was a 5% mutational allelic frequency at 250 × coverage depth for each tested region.

To analyze a putative function of mutations as driver mutations, we employed four separate programs: SIFT [42], Polyphen-2 [43], and MutationTaster2 [44], as well as FATHMM (functional analysis through hidden Markov models (v2.3), which resulted in an index, calculated with a high- throughput web-server, able to predict the functional consequences of both coding variants, i.e., non-synonymous single nucleotide variants (nsSNVs), and non-coding variants to distinguish between cancer-promoting/driver mutations and other germline polymorphisms [45].

We checked for the presence of particular mutations and their previous reports in the catalogue of somatic mutations in cancer (COSMIC), the dbSNP, 1000 Genome Project, ClinVar, and ExAC databases.

4.5. Statistical Analysis

Parameters were recorded and entered into a dedicated database. Descriptive analysis was used to summarize the collected data. To determine the normality of continuous variables, data were tested by the D’Agostino and Pearson omnibus normality test. Variables that were found to be normally distributed were expressed as mean values. Data that were found to be distributed differently were expressed as median and minimum–maximum values.

To compare differences between the groups for categorical variables, the chi-square test was used if the Cochrane assumptions were met, or the Fisher’s exact test otherwise. Interval data were compared with the use of the Mann–Whitney U test as the data did not follow a normal distribution. Odds ratio (OR) and the 95% confidence interval (95% CI) were calculated using the group of patients diagnosed with FTAs as the reference population.

Multivariate logistic regression analysis included mutations that appeared more than twice. A backward stepwise logistic regression indicated mutations that should be considered jointly as predictors of FTC and FTA-Model 2. Firth’s method for eliminating the well-known small sample bias in maximum likelihood estimation was applied to calculate logistic regression coefficients. These were then compared with the predictive accuracy of the most significant single mutation alone (Model 1). Using the regression model, patients diagnosed with FTA were treated as a reference group, whereas high-risk values indicated a highly probable occurrence of FTC. The difference was calculated with the ROC curve (DeLong’s method), the integrated discrimination improvement (IDI), and the categorical net reclassification improvement (categorical NRI) [46], for which the patient was classified to the FTC group if the risk exceeded 70% and to the FTA group if it was lower than 30%. No decision taken if the risk was within 30–70%.

A p-value of less than 0.05 was regarded as significant. Statistical analyses were performed with StatSoft Statistica v13.0 (stepwise logistic regression), LogXact v11.1.0 (logistic regression coefficients based on penalized maximum likelihood estimation for bias correction-Firth’s method), and PQStat v1.6.8 software (ROC comparison, NRI, IDI, and one-dimensional analysis).

Acknowledgments

The authors would like to thank Glenn Flux for English editing and his important support, and Michał Gniot, Luiza Handschuch, Olaf Thalmann, and Łukasz Wolko for their valuable consultations.

Abbreviations

FTA Follicular thyroid adenoma
FTC Follicular thyroid cancer
PTC Papillary thyroid cancer
NGS Next-generation sequencing
AUC Area under curve
NRI Net Reclassification Improvement
IDI Integrated discrimination improvement
AML Acute myeloblastic leukocemia
TNM Tumor-node-metastasis
FFPE Formalin-fixed paraffin-embedded
OR Odds Ratio
AMPCAP Association for Molecular Pathology and College of American Pathologists
nsSNVs Synonymous single nucleotide variants
CI Confidence interval
COSMIC Catalogue of Somatic Mutations in Cancer
FATHMM Functional Analysis through Hidden Markov Models

Supplementary Materials

Supplementary materials can be found at https://www.mdpi.com/1422-0067/20/13/3126/s1. Table S1. The list of mutations which occurred at least in 3 FTA and/or FTC samples.

Author Contributions

Conceptualization, M.B. and E.S.-P.; methodology, M.B., S.D., B.B., and M.J.-J.; software, M.B., S.D., B.B., and B.W.; validation, S.D. and F.A.V.; formal analysis, L.G. and M.R.; investigation, M.B., S.D., and B.B.; resources, M.B.; data curation, D.F.; writing—original draft preparation, M.B.; writing—review and editing, E.S.-P., F.A.V., D.F., B.B., L.G., and K.Z.; visualization, K.Z.; supervision, E.S.-P.; K.Z., and M.R.; project administration, M.B.; funding acquisition, M.B.

Funding

This study was supported by a PRELUDIUM Grant of the Polish National Center for Science number 2015/19/N/NZ5/02257.

Conflicts of Interest

The authors declare no conflict of interest.

References

  • 1.McHenry C.R., Phitayakorn R. Follicular adenoma and carcinoma of the thyroid gland. Oncologist. 2011;16:585–593. doi: 10.1634/theoncologist.2010-0405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Pstrag N., Ziemnicka K., Bluyssen H., Wesoly J. Thyroid cancers of follicular origin in a genomic light: In-depth overview of common and unique molecular marker candidates. Mol. Cancer. 2018;17:116. doi: 10.1186/s12943-018-0866-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Jung S.H., Kim M.S., Jung C.K., Park H.C., Kim S.Y., Liu J., Bae J.S., Lee S.H., Kim T.M., Lee S.H., et al. Mutational burdens and evolutionary ages of thyroid follicular adenoma are comparable to those of follicular carcinoma. Oncotarget. 2016;7:69638–69648. doi: 10.18632/oncotarget.11922. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Nikiforov Y.E., Nikiforova M.N. Molecular genetics and diagnosis of thyroid cancer. Nat. Rev. Endocrinol. 2011;7:569–580. doi: 10.1038/nrendo.2011.142. [DOI] [PubMed] [Google Scholar]
  • 5.Borowczyk M., Szczepanek-Parulska E., Olejarz M., Wieckowska B., Verburg F.A., Debicki S., Budny B., Janicka-Jedynska M., Ziemnicka K., Ruchala M. Evaluation of 167 Gene Expression Classifier (GEC) and ThyroSeq v2 Diagnostic Accuracy in the Preoperative Assessment of Indeterminate Thyroid Nodules: Bivariate/HROC Meta-analysis. Endocr. Pathol. 2018 doi: 10.1007/s12022-018-9560-5. [DOI] [PubMed] [Google Scholar]
  • 6.Cancer Genome Atlas Research Network Integrated genomic characterization of papillary thyroid carcinoma. Cell. 2014;159:676–690. doi: 10.1016/j.cell.2014.09.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gerber T.S., Schad A., Hartmann N., Springer E., Zechner U., Musholt T.J. Targeted next-generation sequencing of cancer genes in poorly differentiated thyroid cancer. Endocr. Connect. 2018;7:47–55. doi: 10.1530/EC-17-0290. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Chen H., Luthra R., Routbort M.J., Patel K.P., Cabanillas M.E., Broaddus R.R., Williams M.D. Molecular Profile of Advanced Thyroid Carcinomas by Next-Generation Sequencing: Characterizing Tumors Beyond Diagnosis for Targeted Therapy. Mol. Cancer Ther. 2018;17:1575–1584. doi: 10.1158/1535-7163.MCT-17-0871. [DOI] [PubMed] [Google Scholar]
  • 9.Nikiforov Y.E., Ohori N.P., Hodak S.P., Carty S.E., LeBeau S.O., Ferris R.L., Yip L., Seethala R.R., Tublin M.E., Stang M.T., et al. Impact of mutational testing on the diagnosis and management of patients with cytologically indeterminate thyroid nodules: A prospective analysis of 1056 FNA samples. J. Clin. Endocrinol. Metab. 2011;96:3390–3397. doi: 10.1210/jc.2011-1469. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Song Y.S., Lim J.A., Min H.S., Kim M.J., Choi H.S., Cho S.W., Moon J.H., Yi K.H., Park D.J., Cho B.Y., et al. Changes in the clinicopathological characteristics and genetic alterations of follicular thyroid cancer. Eur. J. Endocrinol. 2017;177:465–473. doi: 10.1530/EJE-17-0456. [DOI] [PubMed] [Google Scholar]
  • 11.Topf M.C., Wang Z.X., Tuluc M., Pribitkin E.A. TERT, HRAS, and EIF1AX Mutations in a Patient with Follicular Adenoma. Thyroid. 2018;28:815–817. doi: 10.1089/thy.2017.0504. [DOI] [PubMed] [Google Scholar]
  • 12.Wojtas B., Pfeifer A., Oczko-Wojciechowska M., Krajewska J., Czarniecka A., Kukulska A., Eszlinger M., Musholt T., Stokowy T., Swierniak M., et al. Gene Expression (mRNA) Markers for Differentiating between Malignant and Benign Follicular Thyroid Tumours. Int. J. Mol. Sci. 2017;18:1184. doi: 10.3390/ijms18061184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Cha Y.J., Koo J.S. Next-generation sequencing in thyroid cancer. J. Transl. Med. 2016;14:322. doi: 10.1186/s12967-016-1074-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Giordano T.J. Follicular cell thyroid neoplasia: Insights from genomics and The Cancer Genome Atlas research network. Curr. Opin. Oncol. 2016;28:1–4. doi: 10.1097/CCO.0000000000000248. [DOI] [PubMed] [Google Scholar]
  • 15.Lee A., Lee S.H., Jung C.K., Park G., Lee K.Y., Choi H.J., Min K.O., Kim T.J., Lee E.J., Lee Y.S. Use of the Ion AmpliSeq Cancer Hotspot Panel in clinical molecular pathology laboratories for analysis of solid tumours: With emphasis on validation with relevant single molecular pathology tests and the Oncomine Focus Assay. Pathol. Res. Pract. 2018;214:713–719. doi: 10.1016/j.prp.2018.03.009. [DOI] [PubMed] [Google Scholar]
  • 16.Butler K.S., Young M.Y., Li Z., Elespuru R.K., Wood S.C. Performance characteristics of the AmpliSeq Cancer Hotspot panel v2 in combination with the Ion Torrent Next Generation Sequencing Personal Genome Machine. Regul. Toxicol. Pharmacol. 2016;74:178–186. doi: 10.1016/j.yrtph.2015.09.011. [DOI] [PubMed] [Google Scholar]
  • 17.Dom G., Frank S., Floor S., Kehagias P., Libert F., Hoang C., Andry G., Spinette A., Craciun L., de Saint Aubin N., et al. Thyroid follicular adenomas and carcinomas: Molecular profiling provides evidence for a continuous evolution. Oncotarget. 2018;9:10343–10359. doi: 10.18632/oncotarget.23130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Gupta N., Dasyam A.K., Carty S.E., Nikiforova M.N., Ohori N.P., Armstrong M., Yip L., LeBeau S.O., McCoy K.L., Coyne C., et al. RAS mutations in thyroid FNA specimens are highly predictive of predominantly low-risk follicular-pattern cancers. J. Clin. Endocrinol. Metab. 2013;98:E914–E922. doi: 10.1210/jc.2012-3396. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Karger S., Krause K., Engelhardt C., Weidinger C., Gimm O., Dralle H., Sheu-Grabellus S.Y., Schmid K.W., Fuhrer D. Distinct pattern of oxidative DNA damage and DNA repair in follicular thyroid tumours. J. Mol. Endocrinol. 2012;48:193–202. doi: 10.1530/JME-11-0119. [DOI] [PubMed] [Google Scholar]
  • 20.Krause K., Prawitt S., Eszlinger M., Ihling C., Sinz A., Schierle K., Gimm O., Dralle H., Steinert F., Sheu S.Y., et al. Dissecting molecular events in thyroid neoplasia provides evidence for distinct evolution of follicular thyroid adenoma and carcinoma. Am. J. Pathol. 2011;179:3066–3074. doi: 10.1016/j.ajpath.2011.08.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Stone R.M., Mandrekar S.J., Sanford B.L., Laumann K., Geyer S., Bloomfield C.D., Thiede C., Prior T.W., Dohner K., Marcucci G., et al. Midostaurin plus Chemotherapy for Acute Myeloid Leukemia with a FLT3 Mutation. N. Engl. J. Med. 2017;377:454–464. doi: 10.1056/NEJMoa1614359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Kindler T., Lipka D.B., Fischer T. FLT3 as a therapeutic target in AML: Still challenging after all these years. Blood. 2010;116:5089–5102. doi: 10.1182/blood-2010-04-261867. [DOI] [PubMed] [Google Scholar]
  • 23.Smith C.C., Wang Q., Chin C.S., Salerno S., Damon L.E., Levis M.J., Perl A.E., Travers K.J., Wang S., Hunt J.P., et al. Validation of ITD mutations in FLT3 as a therapeutic target in human acute myeloid leukaemia. Nature. 2012;485:260–263. doi: 10.1038/nature11016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Pfeifer A., Wojtas B., Oczko-Wojciechowska M., Kukulska A., Czarniecka A., Eszlinger M., Musholt T., Stokowy T., Swierniak M., Stobiecka E., et al. Molecular differential diagnosis of follicular thyroid carcinoma and adenoma based on gene expression profiling by using formalin-fixed paraffin-embedded tissues. BMC Med. Genomics. 2013;6:38. doi: 10.1186/1755-8794-6-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Nicolson N.G., Murtha T.D., Dong W., Paulsson J.O., Choi J., Barbieri A.L., Brown T.C., Kunstman J.W., Larsson C., Prasad M.L., et al. Comprehensive Genetic Analysis of Follicular Thyroid Carcinoma Predicts Prognosis Independent of Histology. J. Clin. Endocrinol. Metab. 2018;103:2640–2650. doi: 10.1210/jc.2018-00277. [DOI] [PubMed] [Google Scholar]
  • 26.Moretti F., Farsetti A., Soddu S., Misiti S., Crescenzi M., Filetti S., Andreoli M., Sacchi A., Pontecorvi A. p53 re-expression inhibits proliferation and restores differentiation of human thyroid anaplastic carcinoma cells. Oncogene. 1997;14:729–740. doi: 10.1038/sj.onc.1200887. [DOI] [PubMed] [Google Scholar]
  • 27.Pita J.M., Figueiredo I.F., Moura M.M., Leite V., Cavaco B.M. Cell cycle deregulation and TP53 and RAS mutations are major events in poorly differentiated and undifferentiated thyroid carcinomas. J. Clin. Endocrinol. Metab. 2014;99:E497–E507. doi: 10.1210/jc.2013-1512. [DOI] [PubMed] [Google Scholar]
  • 28.Bonhomme B., Godbert Y., Perot G., Al Ghuzlan A., Bardet S., Belleannee G., Criniere L., Do Cao C., Fouilloux G., Guyetant S., et al. Molecular Pathology of Anaplastic Thyroid Carcinomas: A Retrospective Study of 144 Cases. Thyroid. 2017;27:682–692. doi: 10.1089/thy.2016.0254. [DOI] [PubMed] [Google Scholar]
  • 29.Howell G.M., Hodak S.P., Yip L. RAS mutations in thyroid cancer. Oncologist. 2013;18:926–932. doi: 10.1634/theoncologist.2013-0072. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Yoo S.K., Lee S., Kim S.J., Jee H.G., Kim B.A., Cho H., Song Y.S., Cho S.W., Won J.K., Shin J.Y., et al. Comprehensive Analysis of the Transcriptional and Mutational Landscape of Follicular and Papillary Thyroid Cancers. PLoS Genet. 2016;12:e1006239. doi: 10.1371/journal.pgen.1006239. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Angell T.E. RAS-positive thyroid nodules. Curr. Opin. Endocrinol. Diabetes Obes. 2017;24:372–376. doi: 10.1097/MED.0000000000000354. [DOI] [PubMed] [Google Scholar]
  • 32.Vasko V., Ferrand M., Di Cristofaro J., Carayon P., Henry J.F., de Micco C. Specific pattern of RAS oncogene mutations in follicular thyroid tumors. J. Clin. Endocrinol. Metab. 2003;88:2745–2752. doi: 10.1210/jc.2002-021186. [DOI] [PubMed] [Google Scholar]
  • 33.Fukahori M., Yoshida A., Hayashi H., Yoshihara M., Matsukuma S., Sakuma Y., Koizume S., Okamoto N., Kondo T., Masuda M., et al. The associations between RAS mutations and clinical characteristics in follicular thyroid tumors: New insights from a single center and a large patient cohort. Thyroid. 2012;22:683–689. doi: 10.1089/thy.2011.0261. [DOI] [PubMed] [Google Scholar]
  • 34.Puzziello A., Guerra A., Murino A., Izzo G., Carrano M., Angrisani E., Zeppa P., Marotta V., Faggiano A., Vitale M. Benign thyroid nodules with RAS mutation grow faster. Clin. Endocrinol. (Oxf.) 2016;84:736–740. doi: 10.1111/cen.12875. [DOI] [PubMed] [Google Scholar]
  • 35.Swierniak M., Pfeifer A., Stokowy T., Rusinek D., Chekan M., Lange D., Krajewska J., Oczko-Wojciechowska M., Czarniecka A., Jarzab M., et al. Somatic mutation profiling of follicular thyroid cancer by next generation sequencing. Mol. Cell Endocrinol. 2016;433:130–137. doi: 10.1016/j.mce.2016.06.007. [DOI] [PubMed] [Google Scholar]
  • 36.Wiseman S.M., Griffith O.L., Gown A., Walker B., Jones S.J. Immunophenotyping of thyroid tumors identifies molecular markers altered during transformation of differentiated into anaplastic carcinoma. Am. J. Surg. 2011;201:580–586. doi: 10.1016/j.amjsurg.2011.01.010. [DOI] [PubMed] [Google Scholar]
  • 37.Giordano T.J., Beaudenon-Huibregtse S., Shinde R., Langfield L., Vinco M., Laosinchai-Wolf W., Labourier E. Molecular testing for oncogenic gene mutations in thyroid lesions: A case-control validation study in 413 postsurgical specimens. Hum. Pathol. 2014;45:1339–1347. doi: 10.1016/j.humpath.2014.03.010. [DOI] [PubMed] [Google Scholar]
  • 38.Nikiforova M.N., Wald A.I., Roy S., Durso M.B., Nikiforov Y.E. Targeted next-generation sequencing panel (ThyroSeq) for detection of mutations in thyroid cancer. J. Clin. Endocrinol. Metab. 2013;98:E1852–E1860. doi: 10.1210/jc.2013-2292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Jennings L.J., Arcila M.E., Corless C., Kamel-Reid S., Lubin I.M., Pfeifer J., Temple-Smolkin R.L., Voelkerding K.V., Nikiforova M.N. Guidelines for Validation of Next-Generation Sequencing-Based Oncology Panels: A Joint Consensus Recommendation of the Association for Molecular Pathology and College of American Pathologists. J. Mol. Diagn. 2017;19:341–365. doi: 10.1016/j.jmoldx.2017.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Szczepanek-Parulska E., Hernik A., Ruchala M. Anemia in thyroid diseases. Pol. Arch. Intern. Med. 2017;127:352–360. doi: 10.20452/pamw.3985. [DOI] [PubMed] [Google Scholar]
  • 41.Szczepanek-Parulska E., Zybek A., Biczysko M., Majewski P., Ruchala M. What might cause pain in the thyroid gland? Report of a patient with subacute thyroiditis of atypical presentation. Endokrynol. Pol. 2012;63:138–142. [PubMed] [Google Scholar]
  • 42.Kumar P., Henikoff S., Ng P.C. Predicting the effects of coding non-synonymous variants on protein function using the SIFT algorithm. Nat. Protoc. 2009;4:1073–1081. doi: 10.1038/nprot.2009.86. [DOI] [PubMed] [Google Scholar]
  • 43.Adzhubei I., Jordan D.M., Sunyaev S.R. Predicting functional effect of human missense mutations using PolyPhen-2. Curr Protoc Hum. Genet. 2013;76:7–20. doi: 10.1002/0471142905.hg0720s76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Schwarz J.M., Cooper D.N., Schuelke M., Seelow D. MutationTaster2: Mutation prediction for the deep-sequencing age. Nat. Methods. 2014;11:361–362. doi: 10.1038/nmeth.2890. [DOI] [PubMed] [Google Scholar]
  • 45.Shihab H.A., Rogers M.F., Gough J., Mort M., Cooper D.N., Day I.N.M., Gaunt T.R., Campbell C. An Integrative Approach to Predicting the Functional Consequences of Non-coding and Coding Sequence Variation. Bioinformatics. 2015;31:1536–1543. doi: 10.1093/bioinformatics/btv009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Pencina M.J., D’Agostino R.B., Sr., D’Agostino R.B., Jr., Vasan R.S. Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Stat. Med. 2008;27:157–172; discussion 207–112. doi: 10.1002/sim.2929. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials


Articles from International Journal of Molecular Sciences are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)

RESOURCES