Key Points
Question
Can the detection of lung cancer in symptomatic patients be improved by using circulating microRNAs as biomarkers?
Findings
This cohort study used genome-wide microRNA profiles from the blood samples of 3046 individuals to identify patients with lung cancer with 91.4% accuracy, 82.8% sensitivity, and 93.5% specificity.
Meaning
The findings of this study suggest that the identified patterns of circulating microRNAs may enable them to be used as biomarkers in a liquid biopsy to complement imaging tests, sputum cytology, and biopsies.
Abstract
Importance
The overall low survival rate of patients with lung cancer calls for improved detection tools to enable better treatment options and improved patient outcomes. Multivariable molecular signatures, such as blood-borne microRNA (miRNA) signatures, may have high rates of sensitivity and specificity but require additional studies with large cohorts and standardized measurements to confirm the generalizability of miRNA signatures.
Objective
To investigate the use of blood-borne miRNAs as potential circulating markers for detecting lung cancer in an extended cohort of symptomatic patients and control participants.
Design, Setting, and Participants
This multicenter, cohort study included patients from case-control and cohort studies (TREND and COSYCONET) with 3102 patients being enrolled by convenience sampling between March 3, 2009, and March 19, 2018. For the cohort study TREND, population sampling was performed. Clinical diagnoses were obtained for 3046 patients (606 patients with non–small cell and small cell lung cancer, 593 patients with nontumor lung diseases, 883 patients with diseases not affecting the lung, and 964 unaffected control participants). No samples were removed because of experimental issues. The collected data were analyzed between April 2018 and November 2019.
Main Outcomes and Measures
Sensitivity and specificity of liquid biopsy using miRNA signatures for detection of lung cancer.
Results
A total of 3102 patients with a mean (SD) age of 61.1 (16.2) years were enrolled. Data on the sex of the participants were available for 2856 participants; 1727 (60.5%) were men. Genome-wide miRNA profiles of blood samples from 3046 individuals were evaluated by machine-learning methods. Three classification scenarios were investigated by splitting the samples equally into training and validation sets. First, a 15-miRNA signature from the training set was used to distinguish patients diagnosed with lung cancer from all other individuals in the validation set with an accuracy of 91.4% (95% CI, 91.0%-91.9%), a sensitivity of 82.8% (95% CI, 81.5%-84.1%), and a specificity of 93.5% (95% CI, 93.2%-93.8%). Second, a 14-miRNA signature from the training set was used to distinguish patients with lung cancer from patients with nontumor lung diseases in the validation set with an accuracy of 92.5% (95% CI, 92.1%-92.9%), sensitivity of 96.4% (95% CI, 95.9%-96.9%), and specificity of 88.6% (95% CI, 88.1%-89.2%). Third, a 14-miRNA signature from the training set was used to distinguish patients with early-stage lung cancer from all individuals without lung cancer in the validation set with an accuracy of 95.9% (95% CI, 95.7%-96.2%), sensitivity of 76.3% (95% CI, 74.5%-78.0%), and specificity of 97.5% (95% CI, 97.2%-97.7%).
Conclusions and Relevance
The findings of the study suggest that the identified patterns of miRNAs may be used as a component of a minimally invasive lung cancer test, complementing imaging, sputum cytology, and biopsy tests.
This cohort study assesses whether circulating microRNA profiles could be used as biomarkers to improve the detection of lung cancer in symptomatic patients.
Introduction
Lung cancer is among the 3 most common cancers among both sexes and is the leading cause of cancer-related deaths worldwide.1,2 Despite decreasing smoking rates in industrial countries, millions of current or former heavy smokers have an elevated risk of developing lung cancer, including more than 90 million people in the United States.3 Because the symptoms of lung cancer may not be obvious in its early stages, more than two-thirds of all cases are detected in late and inoperable stages. Patients with metastatic stage IV lung cancer have a 5-year survival rate of only 4.7% compared with a 5-year survival rate of up to 56.3% for those with stage I cancer.4
In 2011, the National Lung Screening Trial (NLST) reported a 20% decrease in lung cancer mortality associated with screening with low-dose computed tomography (CT) vs annual chest radiography in a high-risk population.3 Low-dose CT is, however, characterized by a high rate of false-positive results and an elevated risk of overdiagnosis.3,5 To detect lung tumors, the use of liquid biopsy–based strategies have been explored.6,7,8,9 In addition to DNA and proteins, microRNAs (miRNAs) have been shown to have potential for detecting a wide variety of human pathologies.10,11,12,13 Several previous studies have examined miRNA signatures in patients with non–small cell lung cancer (NSCLC), those with chronic obstructive pulmonary disease (COPD),14,15,16,17,18,19 and those with metastases after tumor resection.20
Despite several publications on tumor biomarkers, few tissue-based biomarkers and virtually no new blood-borne biomarkers are used in clinical practice.21 The clinical application of biomarkers is largely hampered by the relatively small number of cases that have been analyzed in most preclinical studies. This limitation also applies to the studies10,11,14,15,16,17,18,19,20,21,22,23,24,25 of blood-borne miRNAs in patients with lung cancer. The problem is further aggravated by the fact that most studies only consider a small number of cases, limiting the validation of the miRNA signatures that were analyzed. In addition, different methodologies (for collection, storage, analysis) have a particularly strong influence on the results of small studies. This drawback also applies to small studies that use reverse transcriptase polymerase chain reaction. Alternatively, an extended number of cases are studied using few miRNA signatures mostly via reverse transcriptase polymerase chain reaction.16,17,22,23,24,26 For large studies that use reverse transcriptase polymerase chain reaction, a preselection bias from small studies can often be observed, resulting in a skewed miRNA signature. These problems cannot be overcome by meta-analyses that combine the results of studies that differ methodologically in terms of collection, storage, and analysis protocols.
The present study assessed the use of miRNA signatures in whole blood samples for diagnosis of lung cancer in symptomatic patients.
Methods
This study was approved by the local ethics committee, Ärztekammer des Saarlandes, in Saarbrücken, Germany. All participants provided verbal informed consent to participate in a study analyzing blood-borne RNA signatures. The study and the analysis of the data followed the Standards for Reporting of Diagnostic Accuracy (STARD) reporting guideline.
Participants
We assessed the diagnostic value of using blood-borne miRNAs to diagnose lung cancer in 3 different scenarios. First, patients with lung cancer were identified in a group encompassing lung cancer, other lung diseases, other diseases not affecting the lungs, and unaffected control participants. Second, patients with lung cancer were compared with patients with nontumor lung diseases. Third, patients with lung cancer (stage I or II lung tumors) were compared with patients with other lung diseases, patients with other diseases not affecting lungs, and unaffected control participants.
This retrospective, multicenter cohort study included 3046 blood samples obtained from 4 different groups: patients with (1) lung cancer, (2) patients with nontumor lung diseases, (3) patients with diseases not affecting the lungs, and (4) unaffected control participants. The patients with lung cancer included those with non–small cell lung cancer (NSCLC) and small cell lung cancer (SCLC). Patients with other lung diseases were mostly diagnosed with COPD. The samples from patients with COPD were provided by the COSYCONET cohort study. Patients in the group with diseases not affecting the lungs had multiple sclerosis, Parkinson disease, breast cancer, endometriosis, various heart diseases, were undergoing abdominal surgery, or were developing sepsis. Physicians diagnosed the diseases according to the standard diagnostic procedure established at the hospital at the time of diagnosis. Samples from unaffected control participants were provided by the TREND cohort study. This heterogeneous background set was selected to discover miRNA markers that are not associated with the development of lung cancer but are general disease markers.10,11,12,13
The samples were collected between March 3, 2009, and March 19, 2018. Six years of follow-up data were available for patients from the TREND and COSYCONET cohort studies (336 and 538 patients, respectively), but only the data from the point of inclusion in the respective studies were used for the present study. For the other 2172 patients who were not included in the TREND and COSYCONET studies, samples were collected at 11 different sites (eTable 2 in the Supplement), but no follow-up data were collected. An overview of the patient groups, group sizes, and comparisons is presented in eFigure 1 in the Supplement. Table 1 and Table 2 provide the age, sex, and smoking history of the analyzed groups. Patient characteristics, including the diagnosis and assignment to 1 of the 4 groups for each sample are described in eTable 1 in the Supplement. The distribution of the samples per site is aggregated in eTable 2 in the Supplement. A total of 3102 patients were enrolled in the study. For 1.8% of these patients (56), no definitive clinical diagnosis was available; therefore, they were excluded. The collected data were analyzed between April 2018 and November 2019.
Table 1. Demographic Characteristics of the Participants.
Characteristica | Lung Cancer | Nontumor Lung Diseases | Other Diseases | Controls | All Participants |
---|---|---|---|---|---|
Sample size, No. | 606 | 593 | 883 | 964 | 3046 |
Age, mean (SD), y | 65.4 (9.1) | 65.8 (10.1) | 63.7 (15.1) | 52.4 (20.7) | 61.1 (16.3) |
Age group, No. (%), y | |||||
<55 | 84 (13.9) | 60 (10.2) | 206 (24.1) | 357 (40.5) | 707 (24.1) |
55-59 | 72 (11.9) | 60 (10.2) | 90 (10.5) | 56 (6.4) | 278 (9.5) |
60-64 | 127 (21.0) | 105 (17.8) | 102 (11.9) | 135 (15.3) | 469 (16.0) |
65-69 | 120 (19.8) | 129 (21.9) | 104 (12.2) | 151 (17.1) | 504 (17.2) |
70-75 | 115 (19.0) | 147 (25.0) | 121 (14.2) | 93 (10.6) | 476 (16.3) |
>75 | 87 (14.4) | 88 (14.9) | 231 (27.0) | 89 (10.1) | 495 (16.9) |
Totalb | 605 (99.8) | 589 (97.8) | 854 (96.7) | 881 (91.4) | 2929 (96.2) |
Sex, No. (%) | |||||
Male | 390 (67.1) | 386 (65.5) | 484 (61.9) | 467 (51.7) | 1727 (60.5) |
Female | 191 (32.9) | 203 (34.5) | 298 (38.1) | 437 (48.3) | 1129 (39.5) |
Totalb | 581 (95.9) | 589 (99.3) | 782 (88.6) | 904 (93.8) | 2856 (93.8) |
Smoking status, No. (%) | |||||
Never | 31 (9.1) | 27 (4.8) | 302 (51.8) | 346 (64.4) | 706 (35.0) |
Past | 113 (33.3) | 398 (71.3) | 181 (31.0) | 161 (30.0) | 853 (42.3) |
Present | 195 (57.5) | 133 (23.8) | 100 (17.2) | 30 (5.6) | 458 (22.7) |
Total | 339 (55.9) | 558 (94.1) | 583 (66.0) | 537 (55.7) | 2017 (66.2) |
Cumulative tobacco use (among smokers), mean (SD), pack-years | 42.3 (23.9) | 49.3 (36.6) | 19.6 (21.0) | 13.1 (14.1) | 40.5 (32.1) |
Information about age was available in 96.2% of cases (n = 2929), sex in 93.8% (n = 2856) smoking history in 66.2% (n = 2017), and the TNM classification and/or corresponding Union for International Cancer Control stage in 74.9% of cases (n = 454).
Total values do not reflect the sample size because some patients did not provide information on specific characteristics.
Table 2. Demographic Characteristics of Patients With Lung Cancer.
Characteristic | Non–Small Cell Lung Cancer | Small Cell Lung Cancer | Stage I or II Lung Cancer | Stage III or IV Lung Cancer | All Patients |
---|---|---|---|---|---|
Sample size, No. | 405 | 157 | 194 | 260 | 606 |
Age, mean (SD), y | 66.3 (9.1) | 63.8 (8.8) | 66.0 (9.7) | 64.9 (8.4) | 65.4 (9.1) |
Age, No. (%), y | |||||
<55 | 48 (11.9) | 25 (15.9) | 24 (12.4) | 37 (14.3) | 84 (13.9) |
55-59 | 44 (10.9) | 24 (15.3) | 27 (13.9) | 28 (10.8) | 72 (11.9) |
60-64 | 87 (21.5) | 31 (19.7) | 40 (20.6) | 61 (23.6) | 127 (21.0) |
65-69 | 74 (18.3) | 38 (24.2) | 24 (12.4) | 60 (23.2) | 120 (19.8) |
70-75 | 83 (20.5) | 22 (14.0) | 42 (21.6) | 39 (15.1) | 115 (19.0) |
>75 | 68 (16.8) | 17 (10.8) | 37 (19.1) | 34 (13.1) | 87 (14.4) |
Totala | 404 (99.8) | 157 (100.0) | 194 (100.0) | 259 (99.6) | 605 (99.8) |
Sex, No. (%) | |||||
Male | 282 (69.6) | 78 (58.6) | 138 (71.5) | 149 (63.1) | 390 (67.1) |
Female | 123 (30.4) | 55 (41.4) | 55 (28.5) | 87 (36.9) | 191 (32.9) |
Total | 405 (100.0) | 133 (84.7) | 193 (99.5) | 236 (90.8) | 581 (95.9) |
Smoking status, No. (%) | |||||
Never | 29 (12.0) | 0 | 20 (11.6) | 10 (7.1) | 31 (9.1) |
Past | 70 (29.0) | 28 (41.2) | 47 (27.3) | 52 (37.1) | 113 (33.3) |
Present | 142 (58.9) | 40 (58.8) | 105 (61.0) | 78 (55.7) | 195 (57.5) |
Total | 241 (59.5) | 68 (43.3) | 172 (88.7) | 140 (53.8) | 339 (55.9) |
Cumulative tobacco use, mean (SD), pack-years | 42.4 (22.2) | 42.6 (16.5) | 43.3 (23.3) | 41.7 (19.9) | 42.3 (23.9) |
Total values do not reflect the sample size because some patients did not provide information on age.
RNA Extraction and Quality Control
RNA was isolated as described previously.11,25 In brief, RNA from 3046 whole blood samples in PAXgene Tubes (BD Biosciences) was isolated using the PAXgene blood miRNA Kit (QIAGEN). All isolations were performed either manually or semiautomatically using the QIAcube robot (QIAGEN). The individuals who extracted blood samples (N.L., C.D., and S.D.) were familiar with the manufacturer’s instructions of the PAXgene blood tubes. RNA was quantified using a NanoDrop spectrophotometer (Thermo Fisher Scientific) and RNA integrity was checked on a bioanalyzer using the RNA Nano Kit (Agilent Technologies). The mean (SD) RNA integrity value was 7.9 (0.93).
Arrays
The miRNA expression profile of mature human miRNAs was assessed as described previously using human miRNA microarrays and the miRNA Complete Labeling and Hyb Kit (Agilent Technologies).25,27,28 Labeled RNA was hybridized to the array slides for 20 hours at 55 °C with 20-rpm rotation. Arrays were washed twice, air dried, and scanned in the microarray scanner with a 3-μm resolution in double-path mode. Raw data were extracted using Agilent Feature Extraction software. The data are freely available under ArrayExpress accession E-MTAB-8026.29 All analyses were performed in 2 labs at Hummingbird Diagnostics in Heidelberg and at Saarland University. The larger number of samples (2219 of 3046) was measured in Heidelberg, and the remaining 827 samples were measured at Saarland University.
Statistical Analysis
The miRNA expression intensities were background subtracted and the median intensity of replicates was computed. The resulting matrix was filtered for expressed miRNAs and normalized using quantile normalization (eAppendix in the Supplement). Analyses were performed on 1183 expressed miRNAs. As the hypothesis test, an unpaired 2-tailed t test was applied and Benjamini-Hochberg multiple testing correction was performed. In addition to the complete cohort, which is largely matched and contains patients with similar age, sex, and smoking status distribution, we performed perfect patient matching according to sex, smoking status (past, present, and never), and age, thereby yielding equally sized cohorts. Sex and smoking status were matched perfectly, and for age, a 1-year difference was allowed (eAppendix in the Supplement). To assess and compare the diagnostic performance of individual miRNAs and miRNA signatures, the area under the receiver operating characteristics curve (AUROC) was computed. The general distribution of miRNAs was investigated using analysis of variance (ANOVA). This analysis allowed identification of markers that were specifically dysregulated in only 1 or in several groups. Statistical analysis was performed using R, version 3.3 (R Foundation).
Unless stated otherwise, the significance threshold was .05 and tests were 2-sided. Positive predictive values were calculated for all scenarios according to the formula ((sensitivity × prevalence)/(sensitivity × prevalence + 1-specifity) × (1-prevalence)). If not specified otherwise, prevalence was defined as the fraction of individuals in the respective scenario. Reproducibility of measurements was assessed by computing the average Pearson correlation between repeated measurements of the same individual. We also corrected for technical batch effects as follows: (1) study or sampling site, (2) the site where the sample was profiled, and (3) the biochip identification number, using the limma package.30
To identify the groups with highest or lowest median expression we focused on the 100 most statistically significant miRNAs with an adjusted P < .05 obtained from the ANOVA.
The machine learning approach that we used is based on gradient boosted trees with a filter-based feature selection that relies on the feature importance reported by LightGBM31 (eAppendix in the Supplement). Two analysis approaches were tested and compared to evaluate predictive performance: models were trained on a training set (50% of data) and evaluated on a validation set (remaining 50% of the data), with equal distribution of individuals to the disease groups. The CIs were estimated by performing 10 times repeated 2-fold cross-validation. To further test stability, resampling was carried out. With a fixed seed used for the random number generator, 5 times repeated 5-fold cross-validation was performed. To test for potential overtraining, nonparametric permutation tests were performed. Machine learning analysis was performed using Python (Python Software Foundation) with the LightGBM 2.1.0,31 scikit-learn 0.19.1, and scipy 1.1 packages.
Results
In this study, 3102 individuals were enrolled; 56 individuals were excluded because of missing clinical diagnoses. Clinical diagnoses were obtained for 3046 participants (mean [SD] age of 61.1 (16.3) years) who were included in the analysis; data on sex were available for 2856 individuals, with 1727 (60.5%) being men. Of 3046 participants, 606 had non–small cell or small cell lung cancer, 593 had nontumor lung diseases, 883 had diseases not affecting the lung, and 964 were unaffected control participants. Additional demographic data are provided in Table 1, Table 2, and eFigure 1 in the Supplement. Although the cohorts were matched for sex, similar age, and similar smoking history, referred to as largely matched cases, all analyses were also carried out for subcohorts, with perfect matching for age, sex, and smoking history (Table 3).
Table 3. Performance of Single MicroRNAs (miRNAs) With the Highest AUROC and the miRNA Signature in Each Scenarioa .
Scenario | No. of Participants | Single miRNA | miRNA Signature | Improvement AUROC (95% CI) | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Name | AUROC | Size | Accuracy, Mean (SD) | Specificity, Mean (SD) | Sensitivity, Mean (SD) | PPV, Mean (SD) | NPV, Mean (SD) | AUROC, Mean (SD) | |||
Lung cancer vs all others | 606 vs 2440 | hsa-miR-660-5p | 0.745 | 14 | 93.1 (0.2) | 96.1 (0.3) | 81.0 (1.4) | 83.8 (0.8) | 95.3 (0.3) | 0.965 (0.03) | + 0.22 (0.19-0.24) |
Lung cancer vs NTLDs | 606 vs 593 | hsa-miR-17-3p | 0.899 | 14 | 93.1 (0.4) | 89.6 (0.7) | 96.6 (0.7) | 90.5 (0.6) | 96.3 (0.8) | 0.977 (0.02) | + 0.078 (0.06-0.10) |
Stage I or II lung cancer vs all non-lung cancer | 194 vs 2440 | hsa-miR-374b-5p | 0.830 | 9 | 96.2 (0.4) | 97.5 (0.3) | 80.1 (2.2) | 73.5 (2.9) | 98.3 (0.2) | 0.960 (0.06) | + 0.13 (0.08-0.18) |
Abbreviations: AUROC, area under the receiver operating characteristics curve; NPV, negative predictive value; NTLDs, nontumor lung diseases; PPV, positive predictive value.
Performance was assessed by applying resampling with 5 times 5-fold cross-validation. The AUROC of the best single miRNA was compared with the AUROC of an miRNA signature of the machine-learning algorithm for the largely matched cohorts. For each scenario, the improvement in the AUROC using a signature is shown in the right column.
Distribution of the miRNA Expression Levels in the 4 Comparison Groups
First, ANOVA was performed for miRNA expression in the 4 groups. The 5 most statistically significant markers, hsa-miR-17-3p, hsa-miR-21-3p, hsa-miR-193a-3p, hsa-miR-18b-5p, and hsa-miR-18a-5p had adjusted P values less than 10−150 after the Benjamini-Hochberg adjustment. All 5 miRNAs showed the highest mean (SD) expression in patients with nontumor lung diseases vs those with lung cancer (hsa-miR-17-3p, 161.36 [92.99] vs 51.21 [55.45]; hsa-miR-21-3p, 36.32 [21.77] vs 15.72 [11.14]; hsa-miR-193a-3p, 14.00 [4.21] vs 10.50 [3.56]; hsa-miR-18b-5p, 98.21 [65.66] vs 35.46 [37.11]; and hsa-miR-18a-5p 202.94 [144.72] vs 65.21 [73.70]), and 3 of the 5 miRNAs showed the lowest mean (SD) expression in cases of lung cancer.
The minimal median expression value among the 100 most statistically significant miRNAs was observed for 47 miRNAs in the lung cancer group (maximal median expression, 41952.030; minimal median expression, 7.766; SD, 6268.537), 25 miRNAs in nontumor lung diseases (maximal median expression, 2675.910; minimal median expression, 7.080; SD, 809.671), 15 miRNAs in diseases not affecting the lung (maximal median expression, 984.194; minimal median expression, 8.008; SD, 258.121), and 13 miRNAs in unaffected control participants (maximal median expression, 1836.074; minimal median expression, 7.418; SD, 647.110). These findings suggest a statistically significantly lower expression of miRNAs in lung cancer and a higher expression in nontumor lung diseases (P = .01 assessed using the Fisher exact test). Raw and adjusted P values of the ANOVA are given in eTable 3 in the Supplement, and the Figure shows an upregulation of hsa-miR-17-3p in patients with nontumor lung diseases, but not in patients with lung cancer as well as a downregulation of hsa-miR-140-5p in patients with lung cancer only and a downregulation of hsa-miR-628-3p and hsa-miR-374c-5p in those with lung diseases in general.
Scenario 1: Patients With Lung Cancer vs All Other Individuals
First, blood samples from symptomatic patients diagnosed with lung cancer (n = 606) were compared with those from all other individuals (n = 2440). From the training set, a signature of 15 miRNAs was identified. When applied to the validation set, the signature distinguished blood samples from patients diagnosed with lung cancer from all other individuals with an accuracy of 91.4% (95% CI, 91.0%-91.9%), a sensitivity of 82.8% (95% CI, 81.5%-84.1%), and a specificity of 93.5% (95% CI, 93.2%-93.8%).
Next, the training and validation cohorts were jointly evaluated. Overall, hsa-miR-660-5p had the highest AUROC (0.745; 95% CI, 0.723-0.767; P = 1.27 × 10−55). Raw and adjusted P values as well as the AUROC values for all miRNAs are given in eTable 4 in the Supplement. Resampling of the full cohort confirmed these results by predicting the validation set: the 5 times repeated 5-fold cross-validation highlighted a mean AUROC value of 0.965 (95% CI, 0.962-0.967). The receiver operating characteristic curve is shown in eFigure 2 in the Supplement. For the perfect matched scenario (matched age, sex, smoking status, and cohort size), 11 miRNAs were required (eTable 5 in the Supplement). The results yielded a similar mean AUROC value of 0.944 (95% CI, 0.938-0.950). In comparison, the mean permutation test AUROC was 0.499 (95% CI, 0.497-0.500) in the largely matched and in the perfectly matched comparison.
Scenario 2: Patients With Lung Cancer vs Nontumor Lung Disease
Blood samples from patients with lung cancer (n = 606) were compared with those from patients with a nontumor lung disease (n = 593). Although both cohorts had nearly the same mean (SD) age (65.4 [9.1] years for lung cancer vs 65.8 [10.1] years for patients with nontumor lung disease), the cohort of patients with lung cancer had a lower mean (SD) number of pack-years than patients with nontumor lung disease (42.3 [23.9] vs 49.3 [36.6], respectively).
Training our models on the training set yielded a 14-miRNA signature. When applied to the validation set, the model distinguished patients with lung cancer from those with a nontumor lung disease with an accuracy of 92.5% (95% CI, 92.1%-92.9%), a sensitivity of 96.4% (95% CI, 95.9%-96.9%), and a specificity of 88.6% (95% CI, 88.1%-89.2%).
The joint analysis of the training set and the validation set revealed that hsa-miR-17-3p distinguished patients with lung cancer (mean [SD] on log2 scale, 5.151 [1.215] [n = 606]) from those with nontumor lung diseases (mean [SD] on log2 scale, 7.084 [0.918] [n = 593]) with the highest significance (P = 6.94 × 10−151) and an AUROC value of 0.899 (95% CI, 0.881-0.917) for this comparison. In addition to hsa-miR-17-3p, several other miRNAs had high AUROC values for this comparison (eTable 6 in the Supplement). The miRNA signatures from the 5 times repeated 5-fold cross-validation yielded a mean AUROC value of 0.977 (95% CI, 0.975-0.978). The same comparison in the perfectly matched cohort yielded a mean AUROC value of 0.974 (95% CI, 0.971-0.977) (eTable 7 in the Supplement).
Scenario 3: Patients With Early-Stage Lung Cancer vs Individuals Without Lung Cancer
Blood samples from patients with early-stage lung cancer, including Union for International Cancer Control stages I and II (n = 194), were compared with those from all patients without lung cancer, including patients with nontumor lung diseases (n = 593), patients with diseases other than lung cancer (n = 883), and control participants (n = 964). On the training set, a signature of 14 miRNAs was established. When applied to the validation set, the signature identified patients with early-stage lung cancer with an accuracy of 95.9% (95% CI, 95.7%-96.2%), a sensitivity of 76.3% (95% CI, 74.5%-78.0%), and a specificity of 97.5% (95%CI, 97.2%-97.7%).
The joint analysis of the samples revealed that hsa-miR-374b-5p distinguished patients with early-stage lung cancer (stage I or II) (mean [SD] on log2 scale, 6.993 [1.726]) from those without lung cancer (log2 scale mean [SD] on log2 scale, 8.946 [1.280]) with high significance (P = 4.17 × 10−34) and had an AUROC of 0.830 (95% CI, 0.800-0.859) (eTable 8 in the Supplement). The miRNA signatures for the 5 times repeated 5-fold cross-validation yielded a 9-miRNA signature with a mean AUROC of 0.960 (95% CI, 0.954-0.965) (eTable 9 in the Supplement). The same computation for the perfectly matched cohort identified a signature including 5 miRNAs with a mean AUROC value of 0.936 (95% CI, 0.929-0.944).
Association of miRNA Signatures With Age and Smoking Status
In the scenarios described herein, we found similar results between the largely and the perfectly matched subcohorts, indicating an association between the miRNA signatures and age, sex, and smoking status. To identify how different inclusion criteria for age and smoking status change the test performance, the AUROC values of the signatures were assessed separately for different age groups and different groups of tobacco use. We defined 6 age-related thresholds (<55, 55-59, 60-64, 65-69, 70-75, and >75 years) and 4 thresholds for the numbers of pack-years (individuals having a smoking history of at least 30, 40, 50, or 60 pack-years). The highest AUROC value (0.977; 95% CI, 0.971-0.984) was computed for smokers with a history of more than 50 pack-years without any age restriction. The patient group of particular interest (based on NLST criteria), that is patients who are older than 55 years with a smoking history of at least 30 pack-years, had a high AUROC value of 0.974 (95% CI, 0.971-0.976) (eFigure 3 and eTable 10 in the Supplement). To further assess the influence of confounding variables, models were built using the resampling approach combining the miRNA expression and the confounding variables age, sex, and smoking status. All models yielded slightly higher AUROC values (compared with the models without confounders) for the 3 scenarios presented above and for nonmatched and matched cohorts (scenario 1: 0.972 [95% CI, 0.970-0.974] vs 0.952 [95% CI, 0.949-0.955]; scenario 2: 0.977 [95% CI, 0.975-0.978] vs 0.979 [95% CI, 0.977-0.982]; scenario 3: 0.964 [95% CI, 0.959-0.969] vs 0.953 [95% CI, 0.948-0.957]).
Considering Predictive Values
The positive predictive value (PPV) and negative predictive value (NPV) were computed for different prevalence rates for symptomatic lung cancer range from 0.645% (for the group in the NLST3) to 50% (eTable 11 in the Supplement). For the resampling approach, the NPV approaches 100% with decreasing prevalence while the PPV declines. For example, scenario 3 has a PPV of 97.0% (95% CI, 96.6%-97.4%) and an NPV of 83% (95% CI, 81.6%-84.5%) for a prevalence of 50% and a PPV of 62.8% (95% CI, 59.7%-66.5%) and an NPV of 98.9% (95% CI, 98.8%-99.0%) for a prevalence of 5%. The results based on the split in training and validation data sets revealed comparable results (eTable 11 in the Supplement).
NSCLC vs SCLC
Although the goal of this study was to identify general biomarkers for lung cancer, NSCLC and SCLC are biologically so different that we performed a classification in NSCLC and SCLC cases. The single miRNA best able to distinguish between NSCLC and SCLC was hsa-miR-30a-5p (mean [SD] on log2 scale, 6.696 [1.317] for NSCLC [n = 405] vs 5.125 [0.846] for SCLC [n = 157]) (adjusted P = 1.02 × 10−45), which had an AUROC of 0.844 (95% CI, 0.809-0.878). Using an 9-miRNA signature had an accuracy of 84.6% (95% CI, 83.7%-85.6%), a sensitivity of 90.1% (95% CI, 88.7%-91.4%), and a specificity of 70.6% (95% CI, 68.3%-72.9%) with the resampling approach (AUROC, 0.882; 95% CI, 0.870-0.893). We found a substantial overlap in the miRNAs present among the 3 diagnostic scenarios presented above, but none of the 9 miRNAs from the NSCLC vs SCLC comparison overlapped with any of the miRNAs from the 3 scenarios (Table 4). The results of the comparison are given in eTable 12 in the Supplement and the miRNA signature is provided in eTable 13 in the Supplement.
Table 4. Signature Overlap Among Scenarios and Signatures That Distinguished Non–Single Cell Lung Cancer (NSCLC) From Single Cell Lung Cancer (SCLC) Tumorsa.
MicroRNA | Scenario 1 | Scenario 2 | Scenario 3 | NSCLC vs SCLC | Sum |
---|---|---|---|---|---|
miR-1285-3p | x | x | x | 3 | |
miR-205-5p | x | x | x | 3 | |
miR-1260a | x | x | 2 | ||
miR-1260b | x | x | 2 | ||
miR-3152-3p | x | x | 2 | ||
miR-378b | x | x | 2 | ||
miR-17-3p | x | x | 2 | ||
miR-1202 | x | x | 2 | ||
miR-139-5p | x | 1 | |||
miR-16-2-3p | x | 1 | |||
miR-18a-3p | x | 1 | |||
miR-23b-3p | x | 1 | |||
miR-3907 | x | 1 | |||
miR-551b-3p | x | 1 | |||
miR-93-3p | x | 1 | |||
miR-564 | x | 1 | |||
miR-374b-5p | x | 1 | |||
let-7g-3p | x | 1 | |||
miR-193a-5p | x | 1 | |||
miR-21-3p | x | 1 | |||
miR-3610 | x | 1 | |||
miR-4282 | x | 1 | |||
miR-4286 | x | 1 | |||
miR-452-3p | x | 1 | |||
miR-516a-3p | x | 1 | |||
miR-572 | x | 1 | |||
miR-625-5p | x | 1 | |||
miR-122-5p | x | 1 | |||
miR-181c-3p | x | 1 | |||
miR-30a-5p | x | 1 | |||
miR-3615 | x | 1 | |||
miR-3680-3p | x | 1 | |||
miR-375 | x | 1 | |||
miR-378a-5p | x | 1 | |||
miR-589-3p | x | 1 | |||
miR-665 | x | 1 |
For each microRNA contained in any of the resampling signatures, the signatures in which the miRNA is included are marked. The table is sorted in the decreasing order of the number of signatures contained.
Reproducibility
To identify potential batch effects and to test the reproducibility across time, we continuously measured 1 individual 41 times uniformly distributed more than 1000 consecutive measurements. An average Pearson correlation of 0.995 (95% CI, 0.9948-0.9952) was calculated, demonstrating the stability of this approach (eFigure 4 in the Supplement). We observed a moderate decrease in the diagnostic performance, with resampling AUROC values for patients with lung cancer vs all other individuals, those with lung cancer vs nontumor lung disease, and those with early-stage lung cancer vs non-lung cancer of 0.917 (95% CI, 0.912-0.922), 0.908 (95% CI, 0.904-0.912), and 0.886 (95% CI, 0.880-0.892), respectively.
Discussion
This study on 3046 blood samples (from patients with lung cancer, nontumor lung diseases, diseases not affecting the lung, and unaffected controls) presents a comprehensive analysis of blood-borne miRNA changes in lung cancer. We identified miRNA signatures that have potential for (1) the differentiation of patients with lung cancer vs patients with nontumor lung diseases, those with diseases not affecting the lungs, and unaffected control participants; (2) the differentiation of patients with lung cancer vs patients with nontumor lung diseases; and (3) the differentiation of patients with early-stage lung cancer (Union for International Cancer Control stages I and II) vs patients with diseases other than lung cancer, and unaffected control participants. Although the present study focuses on symptomatic patients with lung cancer and the results cannot be directly translated to screening tests (eg, the low-dose CT screening established in the United States), screening represents a future application. However, this application requires studies with large cohorts to confirm miRNA signatures that yield high PPV and NPV in cohorts with low prevalence rates (<1%).
A significant overlap was observed in the signatures selected using the resampling approach (Table 4); we expected signatures used in resampling to be more stable than those selected by the models trained on only 50% of the data, which show only marginal overlap in signatures (eTable 14 in the Supplement). The signatures identified in scenarios 1 and 2 share 3 miRNAs, similarly, scenarios 2 and 3 had 3 miRNAs in common, and 6 common miRNAs were observed in scenarios 1 and 3. Several of the miRNAs that were found with altered abundances in at least 1 of the 3 scenarios have previously been associated with lung cancer, including hsa-miR-205-5p, hsa-miR-564, hsa-miR-1260b, and hsa-miR-1285-3p.19,32,33,34 Specific miRNAs have been previously found to have functional relevance for lung cancer (eg, hsa-miR-660-5p35) and other cancers (eg, miR-120236). However, one must be cautious in comparing studies that used different specimens, including whole blood or serum, different miRNA identifiers, and different approaches. eTable 15 in the Supplement provides an overview of selected studies, including a previous study with a much smaller cohort size,17 that shows that many of the miRNAs in the present study were relevant in previous studies.
Clinical Application
Several challenges need to be overcome toward a clinical application of miRNA signatures for detection of lung cancer. Prospective studies with large cohorts of patients with specific diseases, a format that is readily applicable in clinical in vitro diagnostic tests (RT-qPCR or enzyme-linked immunosorbent assay [ELISA]37,38), and an evaluation of the extent to which the miRNA signatures may complement imaging, sputum cytology, or biopsy are needed for clinical application of miRNA for diagnosis of lung cancer. The general limitations of retrospective studies warrant a final prospective validation for such analyses.
Alternative strategies measuring molecules in exhaled breath condensate toward an early diagnosis of lung cancer.39,40 Bronchial genomic classifiers are commercially available, but they require invasive bronchoscopy for obtaining examination material. In addition, bronchial genomic classifiers yield high sensitivity but lack specificity (as shown in AEGIS trials).41,42 Besides miRNAs, there are also other promising blood-borne biomarkers, such as Zyxin or marker panels of serum proteins, all with their own advantages.43,44
Limitations
This study has some limitations. One limitation is its retrospective design. The resulting patient distribution is different from a clinical application scenario. Although we estimated the influence of the prevalence on the expected PPVs and NPVs, these values need to be confirmed with a prospective validation study. Another limitation is the format used to measure the miRNA expression values. Microarrays are not readily applicable in clinical in vitro diagnostic tests and thus other formats need to be tested as discussed in the Clinical Application section.
Conclusions
We believe this study presents a standardized approach that could be used to identify symptomatic patients with lung cancer based on blood-borne miRNA signatures. Of note, patients with early-stage cancer may be distinguished from other patients, including those with nontumor lung disease and diseases other than cancer. The circulating biomarker test may complement imaging, sputum cytology, and biopsy tests in the future, after being validated by prospective studies.
References
- 1.Stewart BW, Wild C; International Agency for Research on Cancer, World Health Organization . World Cancer Report 2014. Lyon, France and Geneva, Switzerland: International Agency for Research on Cancer WHO Press; 2014. [Google Scholar]
- 2.Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394-424. doi: 10.3322/caac.21492 [DOI] [PubMed] [Google Scholar]
- 3.Aberle DR, Adams AM, Berg CD, et al. ; National Lung Screening Trial Research Team . Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395-409. doi: 10.1056/NEJMoa1102873 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.National Cancer Institute, Surveillance, Epidemiology, and End Results Program. Cancer stat facts: lung and bronchus cancer. https://seer.cancer.gov/statfacts/html/lungb.html. Accessed January 31, 2020
- 5.Hoffmann H, Heußel CP, Eichhorn M. Screening for lung cancer: current status [in German]. Zentralbl Chir. 2017;142(S 01):S11-S16. [DOI] [PubMed] [Google Scholar]
- 6.Carozzi FM, Bisanzi S, Carrozzi L, et al. ; ITALUNG Working Group . Multimodal lung cancer screening using the ITALUNG biomarker panel and low dose computed tomography: results of the ITALUNG biomarker study. Int J Cancer. 2017;141(1):94-101. doi: 10.1002/ijc.30727 [DOI] [PubMed] [Google Scholar]
- 7.Nardi-Agmon I, Peled N. Exhaled breath analysis for the early detection of lung cancer: recent developments and future prospects. Lung Cancer (Auckl). 2017;8:31-38. doi: 10.2147/LCTT.S104205 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Pérez-Callejo D, Romero A, Provencio M, Torrente M. Liquid biopsy based biomarkers in non-small cell lung cancer for diagnosis and treatment monitoring. Transl Lung Cancer Res. 2016;5(5):455-465. doi: 10.21037/tlcr.2016.10.07 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Manser RL, Irving LB, Byrnes G, Abramson MJ, Stone CA, Campbell DA. Screening for lung cancer: a systematic review and meta-analysis of controlled trials. Thorax. 2003;58(9):784-789. doi: 10.1136/thorax.58.9.784 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Keller A, Leidinger P, Vogel B, et al. . miRNAs can be generally associated with human pathologies as exemplified for miR-144. BMC Med. 2014;12:224. doi: 10.1186/s12916-014-0224-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Keller A, Leidinger P, Bauer A, et al. . Toward the blood-borne miRNome of human diseases. Nat Methods. 2011;8(10):841-843. doi: 10.1038/nmeth.1682 [DOI] [PubMed] [Google Scholar]
- 12.Zafari S, Backes C, Meese E, Keller A. Circulating biomarker panels in Alzheimer’s disease. Gerontology. 2015;61(6):497-503. doi: 10.1159/000375236 [DOI] [PubMed] [Google Scholar]
- 13.Shah AA, Leidinger P, Backes C, et al. . A set of specific miRNAs is connected with murine and human gastric cancer. Genes Chromosomes Cancer. 2013;52(3):237-249. doi: 10.1002/gcc.22024 [DOI] [PubMed] [Google Scholar]
- 14.Leidinger P, Keller A, Meese E. MicroRNAs - important molecules in lung cancer research. Front Genet. 2012;2:104. doi: 10.3389/fgene.2011.00104 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Keller A, Leidinger P, Borries A, et al. . miRNAs in lung cancer - studying complex fingerprints in patient’s blood cells by microarray experiments. BMC Cancer. 2009;9:353. doi: 10.1186/1471-2407-9-353 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Leidinger P, Brefort T, Backes C, et al. . High-throughput qRT-PCR validation of blood microRNAs in non-small cell lung cancer. Oncotarget. 2016;7(4):4611-4623. doi: 10.18632/oncotarget.6566 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Leidinger P, Keller A, Borries A, et al. . Specific peripheral miRNA profiles for distinguishing lung cancer from COPD. Lung Cancer. 2011;74(1):41-47. doi: 10.1016/j.lungcan.2011.02.003 [DOI] [PubMed] [Google Scholar]
- 18.Halvorsen AR, Bjaanæs M, LeBlanc M, et al. . A unique set of 6 circulating microRNAs for early detection of non-small cell lung cancer. Oncotarget. 2016;7(24):37250-37259. doi: 10.18632/oncotarget.9363 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jiang M, Zhang P, Hu G, et al. . Relative expressions of miR-205-5p, miR-205-3p, and miR-21 in tissues and serum of non-small cell lung cancer patients. Mol Cell Biochem. 2013;383(1-2):67-75. doi: 10.1007/s11010-013-1755-y [DOI] [PubMed] [Google Scholar]
- 20.Leidinger P, Backes C, Blatt M, et al. . The blood-borne miRNA signature of lung cancer patients is independent of histology but influenced by metastases. Mol Cancer. 2014;13:202. doi: 10.1186/1476-4598-13-202 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Diamandis EP. Cancer biomarkers: can we turn recent failures into success? J Natl Cancer Inst. 2010;102(19):1462-1467. doi: 10.1093/jnci/djq306 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pan J, Zhou C, Zhao X, et al. . A two-miRNA signature (miR-33a-5p and miR-128-3p) in whole blood as potential biomarker for early diagnosis of lung cancer. Sci Rep. 2018;8(1):16699. doi: 10.1038/s41598-018-35139-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hetta HF, Zahran AM, Shafik EA, et al. . Circulating miRNA-21 and miRNA-23a expression signature as potential biomarkers for early detection of non-small-cell lung cancer. Microrna. 2019;8(3):206-215. doi: 10.2174/1573399815666190115151500 [DOI] [PubMed] [Google Scholar]
- 24.Keller A, Leidinger P, Gislefoss R, et al. . Stable serum miRNA profiles as potential tool for non-invasive lung cancer diagnosis. RNA Biol. 2011;8(3):506-516. doi: 10.4161/rna.8.3.14994 [DOI] [PubMed] [Google Scholar]
- 25.Keller A, Fehlmann T, Ludwig N, et al. ; COSYCONET Study Group . Genome-wide microRNA expression profiles in COPD: early predictors for cancer development. Genomics Proteomics Bioinformatics. 2018;16(3):162-171. doi: 10.1016/j.gpb.2018.06.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Bagheri A, Khorshid HRK, Tavallaie M, et al. . A panel of noncoding RNAs in non-small-cell lung cancer. J Cell Biochem. 2019;120(5):8280-8290. doi: 10.1002/jcb.28111 [DOI] [PubMed] [Google Scholar]
- 27.Ludwig N, Leidinger P, Becker K, et al. . Distribution of miRNA expression across human tissues. Nucleic Acids Res. 2016;44(8):3865-3877. doi: 10.1093/nar/gkw116 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kahraman M, Röske A, Laufer T, et al. . MicroRNA in diagnosis and therapy monitoring of early-stage triple-negative breast cancer. Sci Rep. 2018;8(1):11584. doi: 10.1038/s41598-018-29917-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Athar A, Fullgrabe A, George N, et al. Data from: ArrayExpress update—from bulk to single-cell expression data. Nucleic Acids Res 2019;47(D1):D711-D715. ArrayExpress Archive of Functional Genomics Data https://www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-8026/ [DOI] [PMC free article] [PubMed]
- 30.Ritchie ME, Phipson B, Wu D, et al. . limma powers differential expression analyses for RNA-sequencing and microarray studies. Nucleic Acids Res. 2015;43(7):e47. doi: 10.1093/nar/gkv007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ke G, Meng Q, Finley T, et al. LightGBM: a highly efficient gradient boosting decision tree. Paper presented at: The 31st Conference on Neural Information Processing Systems; December 4-9, 2017. [Google Scholar]
- 32.Xu L, Li L, Li J, et al. . Overexpression of miR-1260b in non-small cell lung cancer is associated with lymph node metastasis. Aging Dis. 2015;6(6):478-485. doi: 10.14336/AD.2015.0620 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Yang B, Jia L, Guo Q, et al. . MiR-564 functions as a tumor suppressor in human lung cancer by targeting ZIC3. Biochem Biophys Res Commun. 2015;467(4):690-696. doi: 10.1016/j.bbrc.2015.10.082 [DOI] [PubMed] [Google Scholar]
- 34.Gao X, Wang Y, Zhao H, et al. . Plasma miR-324-3p and miR-1285 as diagnostic and prognostic biomarkers for early stage lung squamous cell carcinoma. Oncotarget. 2016;7(37):59664-59675. doi: 10.18632/oncotarget.11198 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Fortunato O, Boeri M, Moro M, et al. . Mir-660 is downregulated in lung cancer patients and its replacement inhibits lung tumorigenesis by targeting MDM2-p53 interaction. Cell Death Dis. 2014;5:e1564. doi: 10.1038/cddis.2014.507 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Du B, Zhang P, Tan Z, Xu J. MiR-1202 suppresses hepatocellular carcinoma cells migration and invasion by targeting cyclin dependent kinase 14. Biomed Pharmacother. 2017;96:1246-1252. doi: 10.1016/j.biopha.2017.11.090 [DOI] [PubMed] [Google Scholar]
- 37.Kappel A, Keller A. miRNA assays in the clinical laboratory: workflow, detection technologies and automation aspects. Clin Chem Lab Med. 2017;55(5):636-647. doi: 10.1515/cclm-2016-0467 [DOI] [PubMed] [Google Scholar]
- 38.Kappel A, Backes C, Huang Y, et al. . MicroRNA in vitro diagnostics using immunoassay analyzers. Clin Chem. 2015;61(4):600-607. doi: 10.1373/clinchem.2014.232165 [DOI] [PubMed] [Google Scholar]
- 39.Brussino L, Culla B, Bucca C, et al. . Inflammatory cytokines and VEGF measured in exhaled breath condensate are correlated with tumor mass in non-small cell lung cancer. J Breath Res. 2014;8(2):027110. doi: 10.1088/1752-7155/8/2/027110 [DOI] [PubMed] [Google Scholar]
- 40.Carpagnano GE, Spanevello A, Curci C, et al. . IL-2, TNF-alpha, and leptin: local versus systemic concentrations in NSCLC patients. Oncol Res. 2007;16(8):375-381. doi: 10.3727/000000006783980900 [DOI] [PubMed] [Google Scholar]
- 41.Ferguson JS, Van Wert R, Choi Y, et al. . Impact of a bronchial genomic classifier on clinical decision making in patients undergoing diagnostic evaluation for lung cancer. BMC Pulm Med. 2016;16(1):66. doi: 10.1186/s12890-016-0217-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Silvestri GA, Vachani A, Whitney D, et al. ; AEGIS Study Team . A bronchial genomic classifier for the diagnostic evaluation of lung cancer. N Engl J Med. 2015;373(3):243-251. doi: 10.1056/NEJMoa1504601 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kim YJ, Sertamo K, Pierrard MA, et al. . Verification of the biomarker candidates for non-small-cell lung cancer using a targeted proteomics approach. J Proteome Res. 2015;14(3):1412-1419. doi: 10.1021/pr5010828 [DOI] [PubMed] [Google Scholar]
- 44.Ma S, Wang W, Xia B, et al. . Multiplexed serum biomarkers for the detection of lung cancer. EBioMedicine. 2016;11:210-218. doi: 10.1016/j.ebiom.2016.08.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.