Skip to main content
JCI Insight logoLink to JCI Insight
. 2021 Jun 8;6(11):e148182. doi: 10.1172/jci.insight.148182

Combination of CT and telomerase+ circulating tumor cells improves diagnosis of small pulmonary nodules

Wen Zhang 1,, Xinchun Duan 2, Zhenrong Zhang 3, Zhenrong Yang 4, Changyun Zhao 5, Chunzi Liang 6, Zhidong Liu 7, Shujun Cheng 4, Kaitai Zhang 4
PMCID: PMC8262359  PMID: 33905377

Abstract

BACKGROUND

Early diagnosis and treatment are key to the long-term survival of lung cancer patients. Although CT has significantly contributed to the early diagnosis of lung cancer, there are still consequences of excessive or delayed treatment. By improving the sensitivity and specificity of circulating tumor cell (CTC) detection, a solution was proposed for differentiating benign from malignant pulmonary nodules.

METHODS

In this study, we used telomerase reverse transcriptase–based (TERT-based) CTC detection (TBCD) to distinguish benign from malignant pulmonary nodules < 2 cm and compared this method with the pathological diagnosis as the gold standard. FlowSight and FISH were used to confirm the CTCs detected by TBCD.

RESULTS

Our results suggest that CTCs based on TBCD can be used as independent biomarkers to distinguish benign from malignant nodules and are significantly superior to serum tumor markers. When the detection threshold was 1, the detection sensitivity and specificity of CTC diagnosis were 0.854 and 0.839, respectively. For pulmonary nodules ≤ 1 cm and 1–2 cm, the sensitivity and specificity of CTCs were both higher than 77%. Additionally, the diagnostic ability of CTC-assisted CT was compared by CT detection. The results show that CT combined with CTCs could significantly improve the differentiation ability of benign and malignant nodules in lung nodules < 2 cm and that the sensitivity and specificity could reach 0.899 and 0.839, respectively.

CONCLUSION

TBCD can effectively diagnose pulmonary nodules and be used as an effective auxiliary diagnostic scheme for CT diagnosis.

FUNDING

National Key Research and Development Project grant nos. 2019YFC1315700 and 2017YFC1308702, CAMS Initiative for Innovative Medicine grant no. 2017-I2M-1-005, and National Natural Science Foundation of China grant no. 81472013.

Keywords: Oncology, Pulmonology

Keywords: Diagnostics, Lung cancer

Introduction

Lung cancer is the most common cancer worldwide in both incidence and mortality. It seriously affects public health and increases the socioeconomic burden. Early diagnosis and early treatment are key to the long-term survival of patients with lung cancer (1). The 5-year survival rate of patients with early lung cancer after surgical resection can reach 90% (2). However, early lung cancer mainly manifests as solitary pulmonary nodules (SPNs) in the lungs, which are mainly detected based on screening because they have no symptoms. The National Lung Screening Trial (NLST) shows that low-dose helical CT (LDCT) provides greater sensitivity for the early detection of lung cancer and reduces the risk of lung cancer mortality by 20% compared with chest radiography (3). Although 24% of participants were positive after LDCT screening, the benign proportion was as high as 96.4% after follow-up (3, 4).

At present, the clinical diagnosis of benign and malignant pulmonary nodules is mainly based on their size, density, morphology, composition ratio, and signs on CT. However, due to the diversity of pulmonary nodules, it is often difficult to distinguish benign and malignant diseases on CT. Positron emission tomography–CT (PET-CT) and CT-guided percutaneous biopsy are the methods used to further differentiate small lung nodules that are difficult to diagnose. Alternatively, the growth changes of these nodules can be followed up, and time is considered to be the best diagnostic approach. Although these treatment strategies follow guidelines (57), long waiting times cause anxiety in patients, with the attendant consequences of excessive or delayed treatment. Studies have shown that the number of benign pulmonary nodules after surgical treatment can be as high as 20% (8). Therefore, it is very important to accurately differentiate benign from malignant pulmonary nodules.

Liquid biopsy is a potentially novel and noninvasive method. Compared with traditional invasive biopsy, liquid biopsy has the advantages of better compliance, easy specimen acquisition, repeatability, and comprehensive tumor information. Therefore, it is expected to be an ideal technology for early auxiliary diagnosis, concomitant diagnosis, therapeutic monitoring, and prognostic assessment of cancer (9). The liquid part of liquid biopsy includes circulating tumor DNA (ctDNA), circulating tumor cells (CTCs), and exosomes. In the last 10 years, some studies have used CellSearch, folic acid receptor detection, and other detection methods based on CTCs to identify benign and malignant pulmonary nodules (10, 11), and some achievements have been made. However, there are still deficiencies in the sensitivity and specificity for the identification of smaller pulmonary nodules. Improving the sensitivity and specificity of CTC detection is also key to the identification of benign and malignant pulmonary nodules. Owing to CTC heterogeneity, the separation methods based on CTC surface biomarkers have some limitations (12). Therefore, other methods are needed to improve the detection ability of CTCs. In a previous study (13), we have reported a telomerase reverse transcriptase–based (TERT-based) CTC detection (TBCD) method. TERT is the basis for tumors to maintain their limitless replication potential, and upregulation of TERT activity has been detected in 80%–90% of malignancies compared with most normal cells, which lack telomerase activity (14, 15). TERT has also emerged as a potential diagnostic marker for tumors (16). The pathological type of most malignant lung nodules is adenocarcinoma, in which there is significant activity related to high TERT expression. Accordingly, there is intense interest and intuitive appeal in the discrimination of benign from malignant small pulmonary nodules with TBCD.

In this study, we used TBCD to detect CTCs in the peripheral blood (PB) of patients with pulmonary nodules within 2 cm in size, combined with tumor markers to assist comprehensive CT judgment, which improved the benign and malignant differential diagnosis of pulmonary nodules and helped patients benefit from diagnosis and treatment.

Results

Patient characteristics.

From May 2017 to March 2019, consecutive patients were enrolled in the study. All enrolled patients had been diagnosed by chest CT, were highly suspected of having malignant pulmonary lesions, and were prepared for surgery. The details of the enrolled patients are listed in Table 1. Of the 120 highly suspected lung cancer patients enrolled, 89 were pathologically diagnosed with primary lung cancer, and 31 were diagnosed with benign nodules. The majority of patients were diagnosed with pathological stage I (p-stage I) disease (79 of 89), 2 patients had p-stage II disease, and 8 patients had p-stage III disease. A flowchart of the diagnoses of the patients enrolled in the study is shown in Figure 1.

Table 1. Patient characteristics.

graphic file with name jciinsight-6-148182-g086.jpg

Figure 1. Flowchart of the diagnoses in the patients enrolled in this study.

Figure 1

GGN, ground-glass nodules; p-stage, pathological stage.

TBCD effectively detected CTCs in pulmonary nodule patients.

In this study, we used a live CTC detection approach based on TBCD. First, we used the online platform GEPIA2 (17) to analyze TERT expression in lung carcinoma and normal samples from The Cancer Genome Atlas (TCGA) and the Genotype-Tissue Expression (GTEx) project (http://gepia2.cancer-pku.cn/#index). The results show that TERT expression was different in lung adenocarcinoma (LUAD) samples, lung squamous cell carcinoma (LUSC) samples, and normal samples, and the expression level in LUAD or LUSC samples was higher than that in normal samples (Figure 2A). In addition, analysis of TERT at different stages of LUAD and LUSC showed that the TERT expression level was independent of stage and was higher at all stages (Figure 2, B and C). These results suggest that TBCD is feasible for detecting CTCs in patients with lung carcinoma.

Figure 2. Expression of TERT in lung cancer.

Figure 2

(A) TERT expression in lung carcinoma and normal samples from TCGA and the GTEx project. LUAD: tumor n = 483, normal n = 347; LUSC: tumor n = 486, normal n = 338. (B) Expression levels of TERT at different stages of LUAD; Pr > F = 0.0757. (C) Expression levels of TERT at different stages of LUSC; Pr > F = 0.581. One-way ANOVA was used for statistical analysis.

Subsequently, we performed a statistical analysis of the CTC detection results of 120 patients with pulmonary nodules. The results show that the number of CTCs in patients with malignant nodules was significantly different from that in patients with benign nodules (Figure 3A). The average number of CTCs in patients with malignant nodules was 6.05 ± 0.85 cells/4 mL PB, and the average number of CTCs in patients with benign nodules was 1.29 ± 0.43 cells/4 mL PB (P < 0.0001; Table 2). The details of CTC detection of enrolled patients are listed in Supplemental Table 1 (supplemental material available online with this article; https://doi.org/10.1172/jci.insight.148182DS1). Next, we analyzed the relationship between pulmonary nodule size and the number of CTCs. We divided the patients into the ≤ 1 cm nodule group and the 1–2 cm nodule group according to the preoperative CT detection results of the size of the pulmonary nodules. The results show that there was no statistically significant difference in the number of CTCs between patients with ≤ 1 cm nodules (6.27 ± 1.80 cells/4 mL PB for malignant and 1.11 ± 0.39 cells/4 mL PB for benign) and patients with 1–2 cm nodules (5.97 ± 0.96 cells/4 mL PB for malignant and 1.36 ± 0.58 cells/4 mL PB for benign), regardless of whether the patients had malignant or benign pulmonary nodules (P = 0.88 for malignant and P = 0.72 for benign; Figure 3B). We divided the patients into the ≤ 1 cm nodule group (31 of 120) and the 1–2 cm nodule group (89 of 120) according to the size of the pulmonary nodules. In the above 2 groups, there were significant differences in the number of CTC tests in patients with benign and malignant pulmonary nodules (P = 0.01 and P < 0.0001, respectively; Figure 3C). As shown in Figure 3D, receiver operating characteristic (ROC) curve analysis showed that the AUC was 0.843 (95% CI, 0.759–0.927). When the threshold value was 1, the sensitivity and specificity of detection were 0.854 and 0.839, respectively (Figure 3E). Moreover, in the analysis of groups by nodule size, TBCD had excellent detection efficiency (Figure 3, F and G). The AUCs were 0.798 (95% CI, 0.644–0.952) and 0.858 (95% CI, 0.753–0.963) in the ≤ 1 cm nodule group and the 1–2 cm nodule group, respectively. When the threshold value was 1, the sensitivity and specificity of the ≤ 1 cm nodule group were 0.773 and 0.778, and those of the 1–2 cm nodule group were 0.881 and 0.864, respectively (Figure 3, H and I). The details are listed in Supplemental Table 2. The results show that CTC detection could be defined as a positive result when the number of CTCs was greater than or equal to 2. Based on this threshold, this method had a high differentiating efficiency for benign and malignant pulmonary nodules.

Figure 3. CTCs effectively detected pulmonary nodules in patients.

Figure 3

(A) The average numbers of CTCs detected in the enrolled patients with benign and malignant nodules. Malignant, n = 89; benign, n = 31. Statistical analysis performed with nonparametric Mann–Whitney U test; ****P < 0.0001. (B) The average numbers of CTCs detected in the diameter groups in patients with benign and malignant nodules: malignant (≤ 1 cm, n = 22; 1–2 cm, n = 67); benign (≤ 1 cm, n = 9; 1–2 cm, n = 22; P = 0.88 and P = 0.72, respectively). Statistical analysis performed with nonparametric Mann–Whitney U test. (C) The average numbers of CTCs detected in patients with malignant or benign nodules with different diameters: ≤ 1 cm (malignant, n = 22; benign, n = 9); 1–2 cm (malignant, n = 67; benign, n = 22). Statistical analysis performed with nonparametric Mann–Whitney U test; *P = 0.01 and ****P < 0.0001. (D) ROC analysis of TBCD-CTCs in the enrolled patients. AUC = 0.843; 95% CI, 0.759–0.927. (E) Youden index of TBCD-CTCs in the enrolled patients. Sensitivity was 0.854 and specificity was 0.839 at the best threshold value of 1. (F) ROC analysis of TBCD-CTCs in patients with ≥ 1 cm pulmonary nodules. AUC = 0.798; 95% CI, 0.644–0.952. (G) ROC analysis of TBCD-CTCs in patients with 1–2 cm pulmonary nodules. AUC = 0.858; 95% CI, 0.753–0.963. (H) Youden index of TBCD-CTCs in patients with ≤ 1 cm pulmonary nodules. Sensitivity was 0.773 and specificity was 0.778 at the best threshold value of 1. (I) Youden index of TBCD-CTCs in patients with 1–2 cm pulmonary nodules. Sensitivity was 0.881 and specificity was 0.864 at the best threshold value of 1.

Table 2. Characteristics of patients and numbers of CTCs detected using TBCD.

graphic file with name jciinsight-6-148182-g087.jpg

FlowSight and FISH confirmed the CTCs detected by TBCD.

To verify that the CTCs were derived from the subsection junction, FlowSight imaging and FISH experiments were performed. The detection method of CTCs was modified, EpCAM antibodies were added in the detection stage, and finally, the fluorescence tracer of CTCs was displayed through the FlowSight system. As shown in Figure 4A, CTCs were slightly larger than WBCs, and GFP expression under control of the human TERT (hTERT) promoter was observed with or without EpCAM expression (CD45GFP+EpCAM+ or CD45GFP+EpCAM). Additionally, WBCs only expressed CD45 (CD45+GFPEpCAM). We found that there was a heterogeneity in CTCs among different patients or in the same patient. In patient 1, CD45GFP+EpCAM+ and CD45GFP+EpCAM CTCs could be found at the same time. However, in patient 2, we observed the presence of only CD45GFP+EpCAM+ CTCs. We used FISH to detect PTEN deletion or EGFR amplification in patients with CTCs. In patients with positive CTC test results, we found a typical signal configuration with heterozygous deletion of the PTEN gene (Figure 4B). The detected EGFR amplification was also confirmed by FISH (Figure 4B). These results suggest that the CTCs detected were derived from malignant nodules.

Figure 4. CTC confirmation via FISH and FlowSight.

Figure 4

(A) CD45-eFluor405 (purple), hTERTp-GFP (green), EpCAM (red), and bright-field digital images are shown for CTCs and WBCs. CTCs derived from malignant nodules are marked by GFP with or without the EpCAM marker. CTCs were defined as CD45/GFP+/EpCAM+ or CD45/GFP+/EpCAM cells. WBCs were only marked with CD45. Scale bar: 7 μm. (B) Dual-color FISH results of the PTEN (10q23.3) heterozygous deletion and EGFR (7p11.2) amplification in CTCs from patients with malignant pulmonary nodules. Scale bar: 10 μm.

TBCD CTCs were independent of serum tumor markers in malignant nodule patients.

Subsequently, we analyzed the correlation between the number of CTCs and serum tumor markers (CEA, NSE, pro-GRP, and CYFRA21-1) in malignant nodule patients. Patients with tumor marker recordings 5 times higher than the detection threshold (all of these patients were positive for CTCs) were excluded because severely deviated sample information would lead to a false linear relationship in the correlation analysis. The correlation results are shown in Figure 5, A–D. As shown in Figure 5E, by using logistic regression, we found that the number of CTCs was not correlated with the expression of serum tumor markers and was an independent factor (Supplemental Table 3). Compared with the ROC curve analysis of serum tumor markers, TBCD was superior in determining malignant and benign nodules. The AUC of our approach was 0.843 (95% CI, 0.759–0.927), and the AUCs for the serum CEA level, NSE level, pro-GRP level, and CYFRA21-1 level were 0.524 (95% CI, 0.412–0.637), 0.530 (95% CI, 0.414–0.646), 0.502 (95% CI, 0.375–0.629), and 0.559 (95% CI, 0.445–0.673), respectively (Figure 5F). TBCD was better than all other serum models and was significantly different (P < 0.0001, P < 0.0001, P < 0.0001, and P = 0.0001, respectively; Supplemental Table 4). The results also show that the number of CTCs detected by TBCD was independent of the diameter of pulmonary nodules. These results indicate that CTCs are useful markers of pulmonary nodule malignancy and that TBCD can effectively discriminate benign and malignant pulmonary nodules.

Figure 5. Correlation between CTCs and tumor biomarkers.

Figure 5

(A) There was no association between the levels of CTCs and CEA in patients with malignant pulmonary nodules (n = 86; Spearman’s correlation analysis). (B) There was no association between the levels of CTCs and NSE in patients with malignant pulmonary nodules (n = 89; Spearman’s correlation analysis). (C) There was no association between the levels of CTCs and pro-GRP in patients with malignant pulmonary nodules (n = 88; Spearman’s correlation analysis). (D) There was no association between the levels of CTCs and CYFRA21-1 in patients with malignant pulmonary nodules (n = 88; Spearman’s correlation analysis). (E) ROC curve analysis of the numbers of CTCs and the levels of CEA, NSE, pro-GRP, and CYFRA21-1 in the enrolled patients with pulmonary nodules. (F) Spearman’s correlations between CTCs and tumor biomarkers and nodule diameters in patients with malignant pulmonary nodules, displayed as heatmaps. Positive correlations are shown in blue-based colors, while negative correlations (anticorrelations) are shown in red-based colors.

TBCD could effectively assist CT in pulmonary nodule diagnosis.

Although the development of CT technology has made the detection of pulmonary nodules easier, approximately 20% of the pulmonary nodules resected in clinical surgery are still benign lesions, and the qualitative diagnosis of pulmonary nodules is still very difficult. As shown in Figure 6A, among the patients enrolled in this study, postoperative pathology showed benign nodules in patients with highly suspicious CT results and showed malignant tumors in patients with benign nodules in the same way. The positive rate of CTCs (CTC > 1 defined as positive) was 85.4% for malignant nodules and 16.1% for benign nodules (Supplemental Figure 1). Therefore, we further compared the correlation between TBCD and CT. According to the CT results, we divided the 120 enrolled patients into the solid nodule group (64 of 120) and the subsolid nodule group (56 of 120). The results show that there was no significant difference in the average number of CTCs detected in patients with solid nodules and in those with subsolid nodules (6.325 ± 1.485 versus 5.816 ± 0.955, P = 0.77) in the malignant nodule group (Figure 6B and Table 2). In the benign nodule group, the average number of CTCs detected was also not significantly different (1.125 ± 0.505 versus 1.875 ± 0.769, P = 0.44; Figure 6B and Table 2). By comparing malignant and benign nodules, the average number of CTCs detected was significantly different in both the solid nodule and subsolid nodule groups (P = 0.0018 and P = 0.003, respectively; Figure 6C and Table 2). ROC curve analysis showed that the AUC of the solid nodule group was 0.868 (95% CI, 0.769–0.967), while the AUC of the subsolid nodule group was 0.773 (95% CI, 0.599–0.946), indicating that TBCD was effective in differentiating benign from malignant nodules in different groups (Figure 6, D and E). To predict whether a pulmonary nodule is malignant, chest CT features including size, density, growth, and specific morphology features can be used. We obtained parameters for imaging (including spicular sign, lobulation, pleural indentation, vacuole sign, aerial bronchogram, vessel convergence, mean CT value, and node diameter), and logistic regression was used for model fitting. Then, ROC curve analysis was conducted for the independent CT evaluation and the CT combined with CTC evaluation of the enrolled patients. The results show that the AUC of CT alone was 0.830 (95% CI, 0.741–0.919), and the AUC of CT combined with CTCs was 0.918 (95% CI, 0.858–0.977), suggesting that CT combined with CTCs was a better approach for the diagnosis of pulmonary nodules (P = 0.040; Figure 6F and Table 3). Subsequently, in the solid nodule group and the subsolid nodule group, the AUCs of CT alone were 0.858 (95% CI, 0.770–0.947) and 0.653 (95% CI, 0.410–0.896), respectively, and the AUCs of CT combined with CTCs were 0.923 (95% CI, 0.857–0.989) and 0.786 (95% CI, 0.580–0.991), respectively, with no significant difference between the 2 methods (P = 0.16 and P = 0.37; Figure 6, G and H). Finally, decision curve analysis (DCA) was performed to compare the clinical effects of CT alone and CT combined with CTCs (Figure 6I). Interestingly, although CT evaluation showed that patients could benefit clinically when the threshold was 0.4, CT combined with CTC evaluation provided clinical benefit to patients at a lower threshold. In addition, within the threshold range of more than 0.4, the clinical benefit of patients evaluated by CT combined with CTCs was significantly higher than that evaluated by CT alone. These results show that TBCD is an effective method to assist CT in the determination of benign and malignant nodules in the lung and can significantly improve the accuracy of CT in the determination of benign and malignant nodules.

Figure 6. Comparison of TBCD with CT in the enrolled patient cohort.

Figure 6

(A) The results of a typical CT examination were inconsistent with those of postoperative pathology. Total original magnification, ×200. (B) The average numbers of CTCs detected in patients with malignant or benign nodules with different densities. Malignant (solid, n = 40; subsolid, n = 49); benign (solid, n = 24; subsolid, n = 7). Statistical analysis performed with nonparametric Mann–Whitney U test; P = 0.77 and P = 0.44, respectively. (C) The average numbers of CTCs detected in patients with different types of malignant or benign nodules. Solid (malignant, n = 40; benign, n = 24); subsolid (malignant, n = 49; benign, n = 7); P = 0.0018 and P = 0.003, respectively. Statistical analysis performed with nonparametric Mann–Whitney U test; **P < 0.01; ***P < 0.001. (D) ROC curve analysis of the numbers of CTCs in patients with solid nodules. AUC = 0.868; 95% CI, 0.769–0.967. (E) ROC curve analysis of the numbers of CTCs in patients with subsolid nodules. AUC = 0.773; 95% CI, 0.599–0.946. (F) ROC curve analysis of CT and CT combined with CTCs in the enrolled patients with pulmonary nodules. AUC-CT = 0.830; 95% CI, 0.741–0.919. AUC-CT + CTC = 0.918; 95% CI, 0.858–0.977. (G) ROC curve analysis of CT and CT combined with CTCs in patients with solid nodules. AUC-CT = 0.858; 95% CI, 0.770–0.947. AUC-CT + CTC = 0.923; 95% CI, 0.857–0.989. (H) ROC curve analysis of CT and CT combined with CTCs in patients with subsolid nodules. AUC-CT = 0.653; 95% CI, 0.410–0.896. AUC-CT + CTC=0.786; 95% CI, 0.580–0.991. (I) Decision curve analysis for pulmonary nodule diagnosis. Red line, CT diagnosis only; blue line, CT combined with CTC diagnosis.

Table 3. ROC curve analysis of CTCs, CT, and CT combined with CTCs in the enrolled patients with pulmonary nodules.

graphic file with name jciinsight-6-148182-g088.jpg

Discussion

In this study, we used the TBCD assay to distinguish benign from malignant nodules in patients with nodules < 2 cm in size and compared this method with the pathological diagnosis as the gold standard. Our results show that the number of CTCs detected based on TBCD was not correlated with serum tumor markers. Thus, TBCD could be used as an independent biological indicator for the determination of benign and malignant nodules and showed a significantly better differentiation ability than serum tumor markers. TBCD improved the detection capability in both the ≤ 1 cm and 1–2 cm pulmonary nodule groups based on the optimal threshold of 1 recommended by the ROC curve (patients with 2 or more CTCs were considered positive). In addition, logistic regression analysis with multiple CT test indexes showed that the CTC test as an auxiliary CT test could significantly improve the ability to distinguish benign from malignant nodules in lung nodules smaller than 2 cm.

Lung cancer is a malignant tumor with the highest morbidity and mortality in the world, and the most effective control methods are early screening, early diagnosis, and early treatment. As the initial stage of pulmonary disease progression, pulmonary nodules may be early lesions of lung cancer. The correct diagnosis of pulmonary nodules and the determination of benign and malignant nodules are of great significance for the corresponding clinical treatment (18).

With the emergence of liquid biopsy techniques represented by CTCs and ctDNA, an increasing number of studies have focused on the relationship between CTCs and the prediction and prognostic assessment of lung cancer (10, 11, 19, 20). At present, the methods of CTC capture and enrichment are basically divided into 2 categories (21). The first is based on physical methods. Such an approach can be used to screen CTCs by differences in cell size or density. The other is the capture of CTCs by immune binding based on cell surface markers. Most of the studies reported so far have adopted the combination of the above 2 approaches, such as positive and negative screening schemes through cell size combined with surface markers. In addition, further identification methods, such as reverse transcription PCR (RT-PCR) and immunofluorescence labeling, are often used after the capture of enriched CTCs (22, 23). These different approaches have their own advantages and disadvantages. The enrichment and separation methods based on physical characteristics tend to be biased, resulting in a decrease in detection sensitivity and specificity. However, the detection method based on surface markers is often highly sensitive but produces false-positive signals. The root cause is that there is no recognized accurate tumor surface marker at this stage. A CellSearch-based primary lung cancer diagnostic study reported that, although CTC counts were significantly different in lung cancer patients and nonmalignant patients, ROC curve analysis showed that the sensitivity and specificity of the test were lower than those of the serum marker CEA (30.4 versus 45.6 for sensitivity, 88.0 versus 92.0 for specificity; ref. 11). CellSearch-based CTCs are more suitable as biomarkers for predicting primary metastasis than for early identification. A recent study has also shown that the enrichment and separation of CTCs using the isolation by size of epithelial tumor cell technique was not suitable for lung cancer screening (24). Some studies on the diagnosis of lung cancer based on folic acid receptors have shown that the method has a certain determination efficacy in detection, with a sensitivity between 72.0 and 76.4 and a specificity between 73.8 and 84.1 (10, 25, 26). However, its detection ability is often related to tumor size and stage, and the detection efficiency in p-stage I patients is lower than that in other stage patients (sensitivity is 67.2; ref. 27). Some studies have also shown the clinical application potential of ctDNA in benign and malignant screening for small nodules of lung cancer. Chen et al. assessed the diagnostic accuracy of plasma samples for early lung cancer based on promoter methylation in 8 lung cancer-specific genes (CDO1, TAC1, SOX17, HOXA7, HOXA9, GATA4, GATA5, and PAX5) in plasma samples. Among these genes, a combination of CDO1, SOX17, and HOXA7 could achieve a sensitivity and specificity of 90% and 71%, respectively (27). Liang et al. investigated protocols of 10,560 prospective, observational, and multicenter clinical trials (28). They conducted a multicenter clinical study of methods for detecting ctDNA methylation in plasma samples with a sensitivity and specificity of 82.5% and 83.3%, respectively. Exosome-based benign and malignant screening for small nodules of lung cancer has rarely been reported because of its late start. Kuang et al. first reported the role of plasma exosomal fibrinogen β chain (FGB) and fibrinogen γ chain (FGG) in the prediction of benign or malignant pulmonary nodules (29). Since ctDNA and exosomes exist in plasma, the combined application of CTCs with ctDNA or exosomes to the same sample may be the way to further improve diagnostic accuracy between benign and malignant pulmonary nodules.

In our study, a CTC enrichment and separation approach based on TERT was adopted. The approach was achieved by using TERT activity, which maintains the unlimited replication potential of tumor cells, as a positive screening marker combined with the negative screening of CD45 antibody markers. The results show that, in LUAD, the expression level of TERT in tumor samples was significantly different from that in normal samples, and the expression level of TERT in tumor samples was not correlated with stage. These results suggest that TBCD is feasible for differentiating malignant and benign lung cancer patients and that the ability to differentiate between patients at different stages should be uniform. The detection results of the 120 enrolled patients with pulmonary nodules < 2 cm in size showed that, with a threshold of 1, the overall detection AUC was 84.3, and the detection sensitivity and specificity reached 85.4 and 83.9, respectively. The enrolled patients were further grouped. In the 1–2 cm nodule group, the detection AUC of this method was 85.8, and the sensitivity and specificity were 88.0 and 86.4, respectively. In the ≤ 1 cm nodule group, the AUC was 79.8, and the sensitivity and specificity were 77.3 and 77.8, respectively. The results show that TBCD exhibited robust efficacy in the screening of benign and malignant pulmonary nodules in diagnostic experiments using pathology as the gold standard.

For some patients with pulmonary nodules, conventional diagnostic methods such as fine-needle aspiration and transbronchial biopsy are often only able to obtain small tumor specimens, as it is difficult to obtain samples with these methods (30). Therefore, in PB, the assessment of tumor markers is of clinical value for the determination of benign and malignant tumors (31). Because of the low concentration of clinically common tumor markers in the PB of patients with pulmonary small nodules, it is difficult to use them as independent biomarkers for diagnosis (31). Some studies have shown that the combination of multiple tumor markers in serum can distinguish cancer patients from healthy people (32, 33). Although research on tumor markers for lung cancer has made great progress, there is still no breakthrough in tumor markers with high specificity and sensitivity, especially for early- or precancerous-stage lung cancer, and no suitable tumor markers can be used for diagnosis and application. In our study, we performed ROC curve analysis of serum tumor markers (CEA, NSE, pro-GRP, and CYFRA21-1) in the 120 enrolled patients with pulmonary nodules. The results show that there was little difference in the individual detection efficacy between these markers, and the AUC values were all less than 0.6, suggesting that they were not suitable as biomarkers for the diagnosis of lung cancer. In some studies, the CEA expression level was shown to have a certain application value in the determination of pulmonary nodule benignity and malignancy (11, 33, 34), while in this study, it was not shown to have an application value. This may be because serum CEA expression levels in patients with pulmonary nodules < 2 cm tend to be within a normal detection threshold range. Compared with serum markers, TBCD performed significantly better than the above 4 serum tumor markers (all P < 0.001). With a threshold of 1, the sensitivity and specificity reached 0.854 and 0.839, respectively. In addition, Pearson’s correlation coefficient showed that CTCs had no correlation with the above serum markers and could be used as independent biomarkers to determine benign and malignant lung cancer tumors. Compared with serum tumor markers, TBCD alone or in combination with other markers may be a more effective technical approach for differentiating benign from malignant pulmonary nodules.

At present, early-stage lung cancers are mainly diagnosed as isolated lung nodules by chest CT and are divided into solid and subsolid nodules based on their respective differentiation components — the latter containing pure ground-glass nodules (pGGNs) and part-solid nodules (PSNs). Previous studies have shown that chest CT plays an important role in the differential diagnosis of benign and malignant pulmonary nodules with specific morphological features or CT values (e.g., radiology) and even some CT characteristics, such as components, pathological subtypes, gene mutations, and prognostic correlation (35, 36). With the popularization and application of LDCT in early lung cancer screening, a large number of pulmonary nodules have been found, and the main problem is overdiagnosis and overtreatment, with studies showing false-positive rates of lung cancer exceeding 18.5% (37). This was due to CT radiologic features of benign and malignant pulmonary nodules overlapping, making them difficult to distinguish (Figure 6A). In theory, CTC detection by a noninvasive liquid biopsy approach can be used to avoid the interference of the complex signs of CT and serve as an adjunctive approach to help CT differentiate and diagnose pulmonary nodules. In our study, we not only evaluated the efficacy of CT alone in the enrolled patients, but we also conducted a retrospective evaluation of the combination of TBCD with CT. By regression fitting of imaging parameters (including spicular sign, lobulation, pleural indentation, vacuole sign, aerial bronchogram, vessel convergence, mean CT value, and tumor diameter), the sensitivity and specificity of CT as a single diagnostic method in this study reached 83.1 and 80.6 in all patients, respectively. The patient population was divided into a solid nodule group and a subsolid nodule group, and CT alone showed high specificity and low sensitivity (67.5 for sensitivity and 95.8 for specificity) in the solid nodule group or high sensitivity and low specificity (87.8 for sensitivity and 42.6 for specificity) in the subsolid nodule group. The diagnostic efficiency of TBCD combined with CT in the diagnosis of pulmonary nodules was better than that of CT alone (P = 0.039), and the sensitivity and specificity improved to 89.9 and 83.9, respectively. The diagnosis in different subgroups showed that TBCD combined with CT could achieve a more balanced detection sensitivity and specificity. Therefore, these results indicate that TBCD can be an excellent auxiliary technique to improve the accuracy of evaluating the nature of pulmonary nodules by CT, as well as to reduce the anxiety, further costly evaluation, cumulative radiation hazard, and pain caused by invasive examination.

To more comprehensively evaluate the diagnostic value of a test, it is necessary to consider all possible diagnostic thresholds. ROC curve analysis is widely used in the performance evaluation of medical diagnostic tests (38). When the AUC is 0.7–0.9, the diagnostic accuracy is moderate, and when the AUC is above 0.9, the diagnostic accuracy is high. DCA is also used in studies to evaluate the diagnostic value of diagnostic tests (39). To ensure the repeatability and accuracy of TBCD, we also conducted test validation using 3 batches. The simulated samples were tested for accuracy, precision, and specificity. The linear regression coefficients of different batches were all higher than 0.99 (data not shown). The coefficient of variation between test batches and batch precision ranged from 5% to 8% (data not shown). These results demonstrate that TCBD is an accurate and reproducible CTC assay.

In this study, there were some patients with false-positive or false-negative results. The false negatives may have been due to the principle of detection and the periodicity of CTCs, which should be further optimized in subsequent studies. Among the patients with false-positive results, 3 patients had atypical adenomatoid hyperplasia and should be followed up with to assess their true status. CD45GFP+ CTCs could also be detected in patients with benign lesions, which might have been due to the following reasons. (a) Limited specificity of the method. In the peripheral circulation, there are a large number of juvenile WBCs, which could cause false-positive results. (b) These patients may have had undetected tumor lesions that released the CTCs detected, resulting in positive results. Therefore, patients with benign lesions with positive results can be further excluded based on multiple tests and a follow-up.

This study has the following shortcomings and improvements. (a) As an exploratory study, the sample size of enrolled patients was small. It is necessary to further expand the sample size and carry out multicenter verification. (b) The patients enrolled in this study were highly suspected of having lung cancer by CT diagnosis, and TBCD can also be further verified in LDCT to further explore the ability of TBCD combined with LDCT in lung cancer screening. (c) With the development of artificial intelligence (AI) diagnostic technology, the combination of AI diagnostic technology with CT and TBCD may achieve a better diagnostic performance.

In conclusion, TBCD can improve the diagnosis of pulmonary nodules and can be used as a robust auxiliary diagnostic scheme for CT diagnosis.

Methods

Patients and sample collection.

In total, 120 patients newly diagnosed by CT were recruited, and the clinical information of the included patients was provided by the Department of Thoracic Surgery, Beijing Chest Hospital. Blood (4 mL) was collected from eligible patients with K2E (EDTA) tubes, kept at 4°C, and transported to the laboratory within 2 hours.

Blood sampling treatment and CTC identification.

The 4 mL blood samples were centrifuged at 500g for 5 minutes at room temperature, and the plasma was discarded. Then, RBC lysis buffer (NH4Cl, 0.15M; EDTA, 0.1 mM; KHCO3, 10 mM; pH 7.2) was added to the samples. The samples were centrifuged at 500g for 5 minutes at room temperature, the supernatant was discarded, and 5 mL of 1× PBS was added to wash the cells. The samples were centrifuged again at 500g for 5 minutes at room temperature, and 2 mL of serum-free medium was used to resuspend and seed all cells from a 4 mL blood sample. CTCs were detected using reagent (oHSV1-hTERTp-GFP) as previously described (13). Cells transduced with oHSV1-hTERTp-GFP were incubated in a humidified atmosphere of 5% CO2 at 37°C for 24 hours. The transduced cells were harvested and stained with an APC-CD45 antibody (HI30, Invitrogen). CD45GFP+ cells were recorded as TBCD-CTCs for TBCD (Supplemental Figure 2).

Identification of CTCs using FlowSight and FISH.

The samples were subjected to a standard CTC identification process. For flow imaging, all cells from 4 mL blood samples were collected and incubated with an eFluor405 anti-CD45 antibody (HI30, BioLegend) and APC anti-EpCAM antibody (CO17-1A, BioLegend). The incubation time was 30 minutes at room temperature. After washing, CTCs were detected by the ImageStreamX Mark II system (FlowSight, Amnis). CD45GFP+EpCAM+ cells in blood samples were considered CTC+ cells (Supplemental Figure 3).

For FISH, MACS magnetic human CD45 microbeads (Miltenyi Biotec) were used to isolate CD45 cells. After fixation and sectioning, the cells were labeled with a dual-color FISH probe set consisting of a Spectrum Red–labeled PTEN gene probe (CELNOVTE) in the chromosome 10q23.3 region and a Spectrum Green–labeled CEP10 gene probe (CELNOVTE FISH system). DAPI was used to stain the nuclei. The predominantly red and green signal numbers were recorded for each FISH probe. Heterozygous deletion of PTEN was defined as the presence of fewer PTEN signals compared with the centromere 10 probe signal. The cells were also labeled with a dual-color FISH probe set consisting of a Spectrum Red–labeled EGFR gene probe (CELNOVTE) in the chromosome 7p11.2 region and a Spectrum Green–labeled CEP7 gene probe (CELNOVTE FISH system). DAPI was used to stain the nuclei. The numbers of predominantly red and green signals were recorded for each FISH probe. A positive EGFR/CEP7 ratio ≥ 2.0 indicated EGFR gene amplification in a sample.

Statistics.

Statistical analysis of the data was carried out with standard software (IBM SPSS Statistics 26.0 and Prism 8, USA). The nonparametric Mann–Whitney U test was used to test 2 patient groups (categorical and continuous data). Pearson’s χ2 test was used to test the expected frequencies and the observed frequencies in 2 categories of a contingency table. ROC curves were constructed based on the diagnostic efficiency of tumor biomarkers and CTCs, and the AUC represented the diagnostic performance. Logistic regression was used to calculate the predictive probability for the combined methods of CT and CTCs. Clinical usefulness was evaluated by DCA. Data are expressed as the mean ± SEM. All P values were 2 sided, with P < 0.05 considered statistically significant.

Study approval.

This study was approved by the ethics committee of the Beijing Chest Hospital, Capital Medical University (KY-2018-004). Written informed consent was obtained from the enrolled patients.

Author contributions

WZ, XD, ZCY, and CZ performed the experiments. All authors participated in designing various parts of the study and in discussion and interpretation of the results. XD, ZZ, and ZL collected the samples and clinical information. WZ, XD, and ZY performed the data analysis. WZ and XD wrote the manuscript with input from all authors. WZ, SC, and KZ supervised the study.

Supplementary Material

Supplemental data
ICMJE disclosure forms

Acknowledgments

We thank the staff of the National Center for Protein Sciences Beijing (Peking University) and Fei Wang for her technical assistance with flow cytometry imaging. We also thank the Diatech Co. Ltd. technical support for the TCBD method. This research was funded by National Key Research and Development Project grant nos. 2019YFC1315700 and 2017YFC1308702, CAMS Initiative for Innovative Medicine grant no. 2017-I2M-1-005, and National Natural Science Foundation of China grant no. 81472013.

Version 1. 04/27/2021

In-Press Preview

Version 2. 06/08/2021

Electronic publication

Footnotes

Conflict of interest: The authors have declared that no conflict of interest exists.

Copyright: © 2021, Zhang et al. This is an open access article published under the terms of the Creative Commons Attribution 4.0 International License.

Reference information: JCI Insight. 2021;6(11):e148182.https://doi.org/10.1172/jci.insight.148182.

Contributor Information

Wen Zhang, Email: zhangwen@cicams.ac.cn.

Xinchun Duan, Email: duanxinchun45@163.com.

Zhenrong Zhang, Email: doctor_zzr@126.com.

Zhenrong Yang, Email: yangzr0629@outlook.com.

Changyun Zhao, Email: zhchy7172@163.com.

Chunzi Liang, Email: cliang1873@outlook.com.

Zhidong Liu, Email: lzdzrd@163.com.

Shujun Cheng, Email: chengshj@cicams.ac.cn.

Kaitai Zhang, Email: zhangkt@cicams.ac.cn.

References

  • 1.Hirsch FR, et al. Lung cancer: current therapies and new targeted treatments. Lancet. 2017;389(10066):299–311. doi: 10.1016/S0140-6736(16)30958-8. [DOI] [PubMed] [Google Scholar]
  • 2.Kodama K, et al. Oncologic outcomes of segmentectomy versus lobectomy for clinical T1a N0 M0 non-small cell lung cancer. Ann Thorac Surg. 2016;101(2):504–511. doi: 10.1016/j.athoracsur.2015.08.063. [DOI] [PubMed] [Google Scholar]
  • 3.National Lung Screening Trial Research Team, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409. doi: 10.1056/NEJMoa1102873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Aberle DR, et al. Results of the two incidence screenings in the National Lung Screening Trial. N Engl J Med. 2013;369(10):920–931. doi: 10.1056/NEJMoa1208962. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.MacMahon H, et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner Society 2017. Radiology. 2017;284(1):228–243. doi: 10.1148/radiol.2017161659. [DOI] [PubMed] [Google Scholar]
  • 6.Wood DE. National Comprehensive Cancer Network (NCCN) clinical practice guidelines for lung cancer screening. Thorac Surg Clin. 2015;25(2):185–197. doi: 10.1016/j.thorsurg.2014.12.003. [DOI] [PubMed] [Google Scholar]
  • 7. Gould MK, et al. Evaluation of individuals with pulmonary nodules: when is it lung cancer? Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2013;143(5 Suppl):e93S-e120S. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Madariaga ML, et al. Multidisciplinary selection of pulmonary nodules for surgical resection: diagnostic results and long-term outcomes. J Thorac Cardiovasc Surg. 2020;159(4):1558–1566. doi: 10.1016/j.jtcvs.2019.09.030. [DOI] [PubMed] [Google Scholar]
  • 9.De Rubis G, et al. Liquid biopsies in cancer diagnosis, monitoring, and prognosis. Trends Pharmacol Sci. 2019;40(3):172–186. doi: 10.1016/j.tips.2019.01.006. [DOI] [PubMed] [Google Scholar]
  • 10.Chen X, et al. Folate receptor-positive circulating tumor cell detected by LT-PCR-based method as a diagnostic biomarker for non-small-cell lung cancer. J Thorac Oncol. 2015;10(8):1163–1171. doi: 10.1097/JTO.0000000000000606. [DOI] [PubMed] [Google Scholar]
  • 11.Tanaka F, et al. Circulating tumor cell as a diagnostic marker in primary lung cancer. Clin Cancer Res. 2009;15(22):6980–6986. doi: 10.1158/1078-0432.CCR-09-1095. [DOI] [PubMed] [Google Scholar]
  • 12.Sharma S, et al. Circulating tumor cell isolation, culture, and downstream molecular analysis. Biotechnol Adv. 2018;36(4):1063–1078. doi: 10.1016/j.biotechadv.2018.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Zhang W, et al. Tumor-selective replication herpes simplex virus-based technology significantly improves clinical detection and prognostication of viable circulating tumor cells. Oncotarget. 2016;7(26):39768–39783. doi: 10.18632/oncotarget.9465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Shay JW, Wright WE. Telomeres and telomerase: three decades of progress. Nat Rev Genet. 2019;20(5):299–309. doi: 10.1038/s41576-019-0099-1. [DOI] [PubMed] [Google Scholar]
  • 15.Shay JW, Bacchetti S. A survey of telomerase activity in human cancer. Eur J Cancer. 1997;33(5):787–791. doi: 10.1016/S0959-8049(97)00062-2. [DOI] [PubMed] [Google Scholar]
  • 16.Dratwa M, et al. TERT-regulation and roles in cancer formation. Front Immunol. 2020;11:589929. doi: 10.3389/fimmu.2020.589929. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Tang Z, et al. GEPIA: a web server for cancer and normal gene expression profiling and interactive analyses. Nucleic Acids Res. 2017;45(w1):W98–W102. doi: 10.1093/nar/gkx247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Au-Yong ITH, et al. Pulmonary nodules. BMJ. 2020;371:m3673. doi: 10.1136/bmj.m3673. [DOI] [PubMed] [Google Scholar]
  • 19.Yin W, et al. Overcoming obstacles in pathological diagnosis of pulmonary nodules through circulating tumor cell enrichment. Small. 2020;16(25):e2001695. doi: 10.1002/smll.202001695. [DOI] [PubMed] [Google Scholar]
  • 20.Zhou Q, et al. Value of folate receptor-positive circulating tumour cells in the clinical management of indeterminate lung nodules: a non-invasive biomarker for predicting malignancy and tumour invasiveness. EBioMedicine. 2019;41:236–243. doi: 10.1016/j.ebiom.2019.02.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Zhong X, et al. Circulating tumor cells in cancer patients: developments and clinical applications for immunotherapy. Mol Cancer. 2020;19(1):15. doi: 10.1186/s12943-020-1141-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Liu X, et al. Circulating tumor cells detection in neuroblastoma patients by EpCAM-independent enrichment and immunostaining-fluorescence in situ hybridization. EBioMedicine. 2018;35:244–250. doi: 10.1016/j.ebiom.2018.08.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Wang J, et al. A circulating tumor cell-based digital assay for the detection of EGFR T790M mutation in advanced non-small cell lung cancer. J Mater Chem B. 2020;8(26):5636–5644. doi: 10.1039/D0TB00589D. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Marquette CH, et al. Circulating tumour cells as a potential biomarker for lung cancer screening: a prospective cohort study. Lancet Respir Med. 2020;8(7):709–716. doi: 10.1016/S2213-2600(20)30081-3. [DOI] [PubMed] [Google Scholar]
  • 25.Lou J, et al. Quantification of rare circulating tumor cells in non-small cell lung cancer by ligand-targeted PCR. PLoS One. 2013;8(12):e80458. doi: 10.1371/journal.pone.0080458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Yu Y, et al. Folate receptor-positive circulating tumor cells as a novel diagnostic biomarker in non-small cell lung cancer. Transl Oncol. 2013;6(6):697–702. doi: 10.1593/tlo.13535. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Chen C, et al. Ultrasensitive DNA hypermethylation detection using plasma for early detection of NSCLC: a study in Chinese patients with very small nodules. Clin Epigenetics. 2020;12(1):39. doi: 10.1186/s13148-020-00828-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Liang W, et al. Evaluating the diagnostic accuracy of a ctDNA methylation classifier for incidental lung nodules: protocol for a prospective, observational, and multicenter clinical trial of 10,560 cases. Transl Lung Cancer Res. 2020;9(5):2016–2026. doi: 10.21037/tlcr-20-701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Kuang M, et al. FGB and FGG derived from plasma exosomes as potential biomarkers to distinguish benign from malignant pulmonary nodules. Clin Exp Med. 2019;19(4):557–564. doi: 10.1007/s10238-019-00581-8. [DOI] [PubMed] [Google Scholar]
  • 30.McNulty W, et al. Investigating the solitary pulmonary nodule. BMJ. 2012;344:e2759. doi: 10.1136/bmj.e2759. [DOI] [PubMed] [Google Scholar]
  • 31.Duffy MJ, O’Byrne K. Tissue and blood biomarkers in lung cancer: a review. Adv Clin Chem. 2018;86:1–21. doi: 10.1016/bs.acc.2018.05.001. [DOI] [PubMed] [Google Scholar]
  • 32.Korkmaz ET, et al. Triple test with tumor markers CYFRA 21.1, HE4, and ProGRP might contribute to diagnosis and subtyping of lung cancer. Clin Biochem. 2018;58:15–19. doi: 10.1016/j.clinbiochem.2018.05.001. [DOI] [PubMed] [Google Scholar]
  • 33.Molina R, et al. Assessment of a combined panel of six serum tumor markers for lung cancer. Am J Respir Crit Care Med. 2016;193(4):427–437. doi: 10.1164/rccm.201404-0603OC. [DOI] [PubMed] [Google Scholar]
  • 34.Zheng J, et al. The combination of CTCs and CEA can help guide the management of patients with SPNs suspected of being lung cancer. BMC Cancer. 2020;20(1):106. doi: 10.1186/s12885-020-6524-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Uthoff J, et al. Post-imaging pulmonary nodule mathematical prediction models: are they clinically relevant? Eur Radiol. 2019;29(10):5367–5377. doi: 10.1007/s00330-019-06168-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Feng B, et al. Differentiating minimally invasive and invasive adenocarcinomas in patients with solitary sub-solid pulmonary nodules with a radiomics nomogram. Clin Radiol. 2019;74(7):570.e1–570.e11. doi: 10.1016/j.crad.2019.03.018. [DOI] [PubMed] [Google Scholar]
  • 37.Patz EF, Jr, et al. Overdiagnosis in low-dose computed tomography screening for lung cancer. JAMA Intern Med. 2014;174(2):269–274. doi: 10.1001/jamainternmed.2013.12738. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5(9):1315–1316. doi: 10.1097/JTO.0b013e3181ec173d. [DOI] [PubMed] [Google Scholar]
  • 39.Van Calster B, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. 2018;74(6):796–804. doi: 10.1016/j.eururo.2018.08.038. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental data
ICMJE disclosure forms

Articles from JCI Insight are provided here courtesy of American Society for Clinical Investigation

RESOURCES