A Multiplexed Serum Biomarker Immunoassay Panel Discriminates Clinical Lung Cancer Patients from High-Risk Individuals Found to be Cancer-Free by CT Screening

Abstract

Introduction

Clinical decision-making in the setting of CT screening could benefit from accessible biomarkers that help predict the level of lung cancer risk in high-risk individuals with indeterminate pulmonary nodules.

Methods

To identify candidate serum biomarkers, we measured 70 cancer-related proteins by Luminex xMAP® multiplexed immunoassays in a training set of sera from 56 patients with biopsy-proven primary non small cell lung cancer and 56 age-, sex- and smoking-matched CT-screened controls.

Results

We identified a panel of 10 serum biomarkers – prolactin, transthyretin, thrombospondin-1, E-selectin, C-C motif chemokine 5, macrophage migration inhibitory factor, plasminogen activator inhibitor, receptor tyrosine-protein kinase, Cyfra 21.1, and serum amyloid A – that distinguished lung cancer from controls with an estimated balanced accuracy (average of sensitivity and specificity) of 76.0%±3.8% from 20-fold internal cross-validation. We then iteratively evaluated this model in independent test and verification case/control studies confirming the initial classification performance of the panel. The classification performance of the 10-biomarker panel was also analytically validated using ELISAs in a second independent case/control population further validating the robustness of the panel.

Conclusions

The performance of this 10-biomarker panel based model was 77.1% sensitivity/76.2% specificity in cross-validation in the expanded training set, 73.3% sensitivity/93.3% specificity (balanced accuracy 83.3%) in the blinded verification set with the best discriminative performance in Stage I/II cases: 85% sensitivity (balanced accuracy 89.2%). Importantly, the rate of misclassification of CT-screened controls was not different in most control subgroups with or without airflow obstruction or emphysema or pulmonary nodules. These biomarkers have potential to aid in the early detection of lung cancer and more accurate interpretation of indeterminate pulmonary nodules detected by screening CT.

INTRODUCTION

Lung cancer is the leading cause of cancer deaths in the US, with more than 190,000 deaths per year. Nearly 60% of patients diagnosed with lung cancer die within one year of their diagnosis; nearly 75% die within two years with five-year survival less than 16%. Poor survival is largely due to the late stage at which lung cancer is currently detected. The cost to the US health care system due to lung cancer is about $12 billion (in 2009 dollars) annually, representing 2% of health care costs.¹ Despite these statistics, lung cancer screening is not currently recommended.² Early detection is complicated by the inaccessibility of the lungs and the consequent risks involved in obtaining lung tissue for pathologic diagnosis. Chest radiographs and sputum cytology were previously examined for use in screening, but clinical trials utilizing these low sensitivity screening methods failed to show a benefit for overall survival. More sensitive computed tomography (CT) imaging technology makes detection of early lung cancer feasible.³

Since the publication of the Early Lung Cancer Action (ELCAP) study in 1999⁴, CT screening for detecting lung cancer in clinically asymptomatic, high risk subjects (e.g. individuals age >50 years with a substantial smoking history) has generated both interest and controversy, and there is continuing debate regarding the benefits and risks of lung cancer screening.⁵ Thoracic CT scans are much more sensitive than chest radiographs but a common confounding observation is detection of benign pulmonary nodules with a reported a range of 20–50% on initial screening of participants in single-arm CT screening trials.⁶ In our ongoing CT screening study, the Pittsburgh Lung Screening Study (PLuSS), 2.2% of the more than 3,600 screened high-risk smokers were diagnosed with primary lung cancer in the first three years of the study, however 40.6% of the PLuSS participants had a noncalcified nodule detected by CT scan, and 821 of these subjects underwent additional CT and/or positron emission tomography (PET) scans.⁷ Of all PLuSS subjects, 1% had a major invasive thoracic procedure such as thoracotomy or video-assisted thoroscopic surgery to remove what turned out to be a benign pulmonary nodule during the first three years of follow-up.⁷ As in previously published CT screening studies, PLuSS did detect a large number of lung cancers at an early, curable stage (64 Stage I, 7 Stage II, 36 Stage III, 12 Stage IV) in the first five years but the screening also resulted in considerable medical risks and medical costs. An initial report from the National Lung Screening Trial (NLST) also documented a high rate of invasive procedures for CT-detected pulmonary nodules that were benign.⁸ This observation was confirmed and extended in the recent full NLST publication⁹, that demonstrated, for the first time, a significant 20.0% relative reduction in lung cancer mortality with low-dose CT screening, which was accompanied, however, with a combined rate of positive low-dose CT screening tests of 24.2%. In subjects with a positive CT screening finding of a pulmonary nodule, 96.4% were false positive results for lung cancer [Berg et al NEJM 2011]. The risk of death due to complications as a result of a major thoracic procedure approaches 1%, indicating the great need to avoid unnecessary thoracic procedures. However, the relatively favorable survival associated with early, particularly very early (Stage IA), lung cancer strongly motivates the need and search for effective early detection methods. A recent review reported that Stage IA patients, with tumor size <10 mm, who underwent complete resection, experienced an 86% overall and 100% cancer specific 5-year survival.¹⁰

We hypothesized that addition of biomarker analysis to CT screening results can improve discrimination between individuals with and without lung cancer. Sensitive and specific lung cancer biomarkers, measured in non-invasively collected biospecimens such as serum, could help guide clinical decision-making regarding the level of lung cancer risk in high-risk subjects, particularly in patients with CT-detected indeterminate pulmonary nodules. Application of robust biomarkers could potentially reduce risks and costs of CT screening, while allowing for detection of lung cancer more often at an early stage where cure is much more likely. A small number of individual serum biomarkers have been reported in lung cancer; however none has been demonstrated to provide clinical utility, mainly because of the lack of sufficient SN and SP. Published studies have demonstrated elevated serum levels of Cyfra 21.1, carcinoembryonic antigen (CEA) and tissue plasminogen activator (TPA) in NSCLC. However, these individual biomarkers were found to have poor SN and SP.^11,12 In addition, most biomarkers achieve better SN in advanced stage disease compared to Stage I lung cancer and thus their use for early diagnosis or screening has not had an impact on patient care. The current lack of robust lung cancer serum biomarkers drives current research efforts, including this reported study, to identify and validate new biomarkers with potential clinical utility. Toward this end, we used serum samples collected from PLuSS cancer-free subjects, matched to serum from lung cancer cases of adenocarcinoma or squamous cell histology based on age, sex and smoking status, to evaluate 70 cancer-associated protein biomarkers representing a spectrum of biological functions selected on the basis of published studies documenting an association with epithelial cancer development and progression in a training set and evaluated the performance of the resulting 10-biomarker panel in independent test and verification case/control sample sets including subjects with a clinical spectrum of potentially confounding non-malignant lung disease including airflow obstruction, emphysema or pulmonary nodules.

MATERIALS AND METHODS

Lung Cancer Cases

Patients with clinically ascertained and biopsy-proven untreated primary lung cancer were consented to the University of Pittsburgh Cancer Institute (UPCI) Lung Research Registry, a University of Pittsburgh Institutional Review Board (IRB) approved clinical research protocol in the UPCI SPORE in Lung Cancer, and provided demographic data, including sex, age at diagnosis, and smoking status, clinical information including histology and stage of tumor, and results of pre-surgical pulmonary function tests, and blood collections for research. Blood samples were collected within four weeks of the first biopsy-proven lung cancer diagnosis, and prior to removal of the cancer by a surgical procedure. All cases used in this study were confirmed to be primary lung cancer by pathology review. Cancer cases were classified by pulmonary function tests for evidence of airflow obstruction. Using a standardized phlebotomy procedure, a 50 mL non-fasting peripheral blood sample was collected without anticoagulent from each consented patient to yield serum following a rigorous validated protocol based on prior recommendations from the NIH and the NCI Early Detection Research Network (EDRN). Processing and final cryopreservation at −80°C were completed within one hour of blood collection. Serum aliquots used in the study were not thawed prior to the study assays.

Controls – The Pittsburgh Lung Screening Study (PLuSS)

PLuSS is a community-based IRB- approved study of lung cancer screening with low-dose multi-detector helical computed tomography (CT), funded by the Lung Cancer SPORE.⁷ Beginning in early 2002, we recruited and screened 3,642 volunteers primarily from southwestern PA at high risk for lung cancer. The PLuSS participants also underwent spirometry for pulmonary function testing (PFT) because of the known relationship between chronic obstructive pulmonary disease (COPD) and lung cancer.¹³ Airflow obstruction was classified by standard GOLD criteria. All chest CT scans were obtained on multi-detector helical CT scanners during a single-breath-hold at full inspiration.^7,13 Three physician readers, a pulmonologist, a general radiologist, and a chest radiologist, visually scored the baseline CT scans for presence and type of pulmonary nodules as well as radiographic emphysema presence and severity. PLuSS subjects were classified as having no nodule, a benign nodule (≤3 mm), a low suspicion (4–7 mm), or a moderate/high suspicion nodule (8–20 mm). Patients with low suspicion nodules or worse were followed for at least 3 years and all PLuSS subjects with nodules used in this study had benign outcomes. Scoring procedures for emphysema used a five level semi-quantitative scale, based on National Emphysema Treatment Trial (NETT) criteria, to represent no, trace, mild, moderate, and severe emphysema. We have demonstrated that radiographic emphysema is an important, independent risk factor for lung cancer within the PLuSS cohort.¹³ Peripheral blood was collected from PLuSS subjects within two weeks of the baseline CT scan and processed and stored under the same rigorous conditions described above.

Lung Cancer Case/Control Training, Testing, and Verification Sets

For our initial discovery study, sera from 56 NSCLC lung cancer patients were individually matched to a serum sample from 56 PLuSS participants who were known to be cancer-free a minimum of 3 years following the baseline CT scan. Matching was based on age at serum collection (±5 years), sex, and smoking status (current or ex-smoker). A nested concurrent test set constituted ten additional clinically ascertained and confirmed primary lung cancer cases, and 83 randomly selected, unmatched PLuSS subjects known to be cancer-free after a minimum of 3 years of follow-up. The samples comprising the training and testing sets were run together in a single laboratory run. An initial blinded verification set consisted of an independent randomly selected set of 30 primary lung cancer cases with a range of histologies and 30 unmatched PLuSS controls. These controls were also known to be cancer free after a minimum 3-year follow-up. The samples in the verification set were run blinded as an independent laboratory analysis subsequent to the training and testing set samples. The clinical and demographic characteristics of the cases and control subjects comprising these sets are summarized in Table 1.

Table 1.

Clinical and Demographic Characteristics of the Primary Lung Cancer Cases and PLuSS Control Subjects Utilized in the Initial Training, Test and Verification Sets.

Characteristic	Training Set	Test Set	Verification Set
Primary Lung Cancer Cases	N=56	N=10	N=30
Age (Years)
38–44	0	0	3
46–49	2	1	0
50–59	6	2	3
60–69	21	3	8
70–79	22	3	10
80+	5	1	6
Sex
Male	30	4	13
Female	26	6	17
Smoking
Never	0	0	1
Previous	43	2	17
Current	13	8	12
Histology
Adenocarcinoma	28	5	10
Squamous cell	28	4	7
Neuroendocrine	0	1	0
Pleomorphic	0	0	2
NSCLC, Undifferentiated	0	0	8
Small Cell	0	0	3
Stage
IA/IB/Limited*	29	6	17
IIA/IIB	7	2	3
IIIA/IIB/Extensive*	16	2	9
IV	4	0	1
PLuSS Control Subjects	N=56	N=83	N=30
Age (Years)
38–44	0	0	0
46–49	0	0	0
50–59	10	34	19
60–69	21	32	9
70–79	25	17	2
80+	0	0	0
Sex
Male	30	50	12
Female	26	33	18
Smoking
Never	0	0	0
Previous	43	51	13
Current	13	32	17

^*Limited (N=2) and extensive (N=1) staging refer to small cell carcinoma only

Luminex Multianalyte Profiling (xMAP®)

Multiplexed serum immunoassays were performed using the Luminex Corporation xMAP® technology platform (Luminex Corp, Austin, TX) that facilitates the simultaneous quantitation of up to 100 soluble analytes in a single sample. A total of 70 cancer-associated candidate serum biomarkers (Table 2) were analyzed in samples from lung cancer patients and matched controls. Together, these biomarkers incorporate a wide range of host and tumor derived factors that allow a broad analysis of the lung cancer/host interaction, and includes a number of previously described epithelial cell cancer-associated serological markers. Although some of these biomarkers have been previously analyzed in lung cancer, no integrated analysis of the performance of these combined biomarkers has been previously performed. The initial goal of this discovery study was to identify the most robust subset of these biomarkers to discriminate lung cancer and matched control samples.

Table 2.

Candidate Luminex xMAP® Lung Cancer Serum Biomarkers (N=70)

Biological Group	Protein Analytes	Assay Source
Cytokines	IL-6, IL-8, TNFα, TNF-RI, TNF-RII, G-CSF G-CSF-R, M-CSF, IL-2R, IL-6R, IL-1RT1 sCD40-L, GROα	Invitrogen/Biosource, Camarillo, CA Bio-Rad Laboratories, Hercules, CA
Chemokines	CCL5 (RANTES), MCP-1,3; eotaxin, LIF, MIF, IP-10	Invitrogen/Biosource Bio-Rad Laboratories
Growth/angiogenic factors	EGF, VEGF, BFGF, EGFR, c-ErbB-2, IGFBP-1 angiostatin, THBS1, HGF, NGF, PDGF, SCGF-B SDF-1α, SCF	Invitrogen/Biosource Bio-Rad Laboratories UPCI LPL
Cancer antigens	CEA, AFP, CA 72-4, TTR, HE4, SCC	UPCI LPL
Apoptotic proteins	Cyfra 21.1, TRAIL, sDR5, sFas, sFasL	UPCI LPL
Proteases	Kallikrein 10, MMP-1,7,8,9,12	UPCI LPL R&D Systems, Minneapolis, MN
Adhesion molecules	sICAM-1, sVCAM-1, E-selectin	Millipore/Linco, St. Louis, MO
Hormones	PRL, TSH, LH, ACTH, GH, FSH	Millipore/Linco
Adipokines	Adiponectin, leptin, resistin	Millipore/Linco
Other biomarkers	Mesothelin, Hsp70, ULBP-1,2, MICA, SAA PAI-1 (SERPINE1), MPO, thrombospondin-1, ULBP-1,2	UPCI LPL Millipore/Linco

For these analytes, we utilized commercially available bead-based immunoassays together with custom research assays in xMAP^® format developed and validated in the Luminex Platform Laboratory (LPL) of the University of Pittsburgh Cancer Institute (http://www.upci.upmc.edu/luminex/). Generation, optimization, and multiplexing of these bead-based serum protein immunoassays were performed as previously described.^14,15 All assays were research grade. Multiplexed analyses were performed according to the manufacturers’ protocols as previously described.¹⁶ Samples were analyzed using the Bio-Plex suspension array system (Bio-Rad Laboratories, Hercules, CA). For each analyte, 100 labeled beads were analyzed for each sample and mean fluorescence intensities (FIs) were calculated using the system software. Analysis of the experimental data and extrapolation to the standard curves was performed using four-parameter logistic curve fitting to derive the analyte concentrations in each sample. The intra-assay variability of each assay was 1.5–6%. The inter-assay variability for assays performed on the same day was 3–9%; for assays performed on different days the inter-assay variability was 5–20% depending on whether the same lots of reagents were utilized.^17–19 Each bead-based assay was previously validated in the UPCI LPL against the corresponding dedicated ELISA using the same capture and detection antibody pairs with 89–98% correlation for the LCF assays. The performance of the purchased assays (Table 2) was in agreement with that claimed by the manufacturer.

ELISAs

ELISA kits for human thrombospondin-1 (THBS1), plasminogen activator inhibitor (PAI-1, SERPINE1), macrophage migration inhibitory factor (MIF), were purchased from R&D Systems (Minneapolis, MN). ELISA kits for carcinoma embryonic antigen (CEA), receptor tyrosine-protein kinase (erbB-2) and transthyretin (TTR) were purchased from Immunology Consultants Laboratory, Inc (Minneapolis, MN), Calbiochem (Gibbstown, NJ), ALPCO (Salem, NH), respectively. The assays were performed according to protocols provided by the manufacturers. Both standard samples containing recombinant proteins, as well as the selected serum samples, were assayed in duplicate to reduce variation.

Data Analysis, Biomarker Panel, and Model Development Approaches

Our main approach for the initial data analysis was to apply feature selection and model development methods we had previously utilized for biomarker discovery to search for candidate multianalyte panels with estimated classification performance above 80% SN and SP over cross-validation on the training dataset. This was used to determine parameter settings that would yield robust models likely to generalize to the test and verification set data. We then developed a model from the combined data using these parameter settings, and applied it to the independent verification set to evaluate its classification performance. There are three main components in our rule learning approach: (a) Efficient Bayesian Discretization (EBD), (b) Rule Learner (RL), and (c) Inference Engine (IE). Feature selection is automatically performed through the use of univariate Bayesian discretization by the rule learning toolkit that we used for modeling the immunoassay data. We have previously successfully applied utilized this rule learning algorithm^20–23 to biomarker discovery from proteomic mass spectra obtained from cerebrospinal fluid for screening for amyotrophic lateral sclerosis (ALS)^24,25 and in a verification study of biomarkers for ALS.²⁶ A full description of our RL-based data analysis and prediction model development methods is included in Supplemental Material.

RESULTS

Selected Biomarkers and Rule Models

Using our methods, we first derived a rule model from the initial training set comprising 11 rules that included 8 biomarkers – prolactin (PRL), transthyretin (TTR), thrombospondin-1, E-selectin, C-C motif chemokine 5 (CCL5, RANTES), macrophage migration inhibitory factor (MIF), plasminogen activator inhibitor 1 (PAI-1, SERPINE1), and receptor tyrosine-protein kinase erbB-2 (Table 3). This rule model distinguished the lung cancer case samples from the control samples in the training set with a SN of 92.9% and SP of 87.5%. The BACC from 20-fold internal cross-validation of the training set was 82.5%±4.8%. The rule model was then applied to the nested concurrent test set samples, achieving 90.0% SN and 77.1% SP (BACC 83.6%).

Table 3.

Candidate Luminex xMAP® Lung Cancer Serum 10-Biomarker Panel

Protein Biomarker	Gene	Function	Literature*
Upregulated in lung cancer sera
Cyfra 21.1 (Cytokeratin-19) [P08727]	KRT19	Apoptosis	↑	[27]
Macrophage migration inhibitory factor (MIF) [P14174]	MIF	Lymphokine	↑	[28–30]
Prolactin (PRL) [P01236]	PRL	Hormone	↑	[31,32]
Serum amyloid A Protein (SAA) [P02735]	SAA1	Acute phase reactant	↑	[33]
Downregulated in lung cancer sera
C-C motif chemokine 5 (CCL5, RANTES) [P13501]	CCL5	Chemokine	—
E-selectin [P16581]	SELE	Cell adhesion	↑	[34–37]
Receptor tyrosine-protein kinase erbB-2 [P04626]	ERBB2	Growth factor	↑-	[38–41]
Serpine 1 (PAI-1) [P05121]	SERPINE1	Protease inhibitor	—
Thrombospondin-1 [P07996]	THBS1	Cell adhesion	↓	[42]
Transthyretin (TTR) [P02766]	TTR	Protein transporter	↓	[43]

^*Serum biomarker lung cancer case-control associations reported in literature, — literature not informative, ↑ biomarker elevated in cases, ↓ biomarker depressed in cases, ↑-biomarker elevated in cases according to some studies and unassociated with lung cancer according to other studies

Further Training Led to a 10-Biomarker Model

We then used RL to learn a model using all the data for the combined training and test sets (205 subjects, 66 with lung cancer and 139 controls), to determine if a more informative model could be found using all of the data in this larger set of cases and controls. In this expanded training set, the original 8 proteins together with 2 additional biomarkers [cytokeratin fragment 19-9 (Cyfra 21.1) and serum amyloid A protein (SAA) provided improved discriminative performance (Table 3). Both of these proteins have been previously described as lung cancer serum biomarkers.^11,12,33 The results from using positive predictive value as the certainty factor on this combined training set were 77.1% SN and 76.2% SP over 20-fold cross-validation (BACC 76.0±3.8%). This analysis yielded a final model consisting of a set of 12 rules with the 10 biomarkers. The 10-biomarker rule model is included as Supplementary Figure 2. The classification performance of the individual selected biomarkers in both the training and test sample sets is detailed in the Supplementary Table.

Verification Set

A set of 60 independent serum samples, comprising 30 randomly selected clinical lung cancer cases and 30 PLuSS controls was processed and analyzed on the multiplex platform as a blinded verification dataset. The verification set samples were analyzed as a subsequent laboratory analysis to the initial run of the training and testing set samples. The clinical and demographic characteristics of these lung cancer cases and PLuSS control subjects comprising the verification set are summarized in Table 1. Because all types of lung cancer have been detected in the PLuSS cohort by CT screening, the verification set included a range of lung cancer histologies including small cell carcinoma not included in the training and test samples. The previously identified 10-biomarker panel analytes were measured as described. The previously collected data were calibrated together with the new data generated in the blinded verification set using the same procedure as described previously. This calibration procedure relies on the inclusion of quality control (QC) samples in each experimental plate in addition to the required concentration standards (CS).

Applying the 10-biomarker panel and associated RL rules to the verification set data yields an overall classification performance of 73.3% SN and 93.3% SP with only 10 misclassifications among the 60 total predictions made (Tables 4 and 5), and may provide clinical utility in guiding interpretation of screening CT scans, even in tobacco-exposed persons with COPD or emphysema. Formal validation in larger patient cohorts will be needed to confirm these initial findings.

Table 4.

Misclassification of Lung Cancer Cases in the Verification Set

Characteristic	Verification Set 10 Biomarker Model
Total	8/30 (26.7%)
Sex
Male	3/13 (23.1%)
Female	5/17 (29.4%)
Age
38–44	2/3 (66.7%)
46–49	ND
50–59	1/3 (33.3%)
60–69	1/8 (12.5%)
70–79	3/10 (30%)
80+	1/6 (16.7%)
Histology
Adenocarcinoma	4/10 (40%)
Squamous Cell	2/7 (28.6%)
Neuroendocrine	ND
Pleomorphic	1/2 (50%)
NSCLC, Undiff	1/8 (12.5%)
Small Cell	0/3 (0%)
Stage
IA/IB/Limited*	3/17 (17.6%)
IIA/IIB	0/3 (0%)
IIIA/IIB/Extensive*	5/9 (55.6%)
IV	0/1 (0%)
Smoking Status
Never	0/1 (0%)
Previous	5/17 (29.4%)
Current	3/12 (25%)
Airflow Obstruction
Yes	5/18 (27.8%)
No	2/5 (40%)
Unknown	1/7 (14.3%)

^*Limited (N=2) and Extensive (N= 1) Stage refer to staging of small cell carcinomas only ND = not done

Table 5.

Misclassification of PLuSS Controls in the Verification Set According to Demographic and Pulmonary Function Variables and Nodule Status

Lung Cancer Risk Factor	Verification Set 10 Biomarker Model
Total	2/30 (6.7%)
Sex
Male	1/12 (9.1%)
Female	1/18 (5.6%)
Age
50–59	1/19 (5.3%)
60–69	1/9 (11.1%)
70–79	0/2 (0%)
Smoking Status
Previous	0/13 (0%)
Current	2/17 (11.8%)
Emphysema
None	0/21 (0%)*
Trace	2/6 (33.3%)
Mild	0/3 (0%)
Moderate/Severe	ND
Airflow Obstruction
GOLD 0 (FVC>80%)	1/11 (9.1%)
GOLD 0 (FVC<80%)	1/4 (25%)
GOLD I	0/4 (0%)
GOLD II	0/6 (0%)
GOLD III	0/5 (0%)
GOLD IV	ND
Combined
None-Mild & GOLD 0-I	2/19 (10.6%)
Moderate/Severe & GOLD II–IV	ND
All Others	0/11 (0%)
CT Screening Result
No Nodule or Benign	2/15 (13.3%)
Low Suspicion Nodule	0/15 (0%)
Moderate/High Suspicion Nodule	ND

^*P=0.04 (Fisher’s Exact Test)

Analytical Validation of the 10-Biomarker Panel Using ELISAs

As a final evaluation of the candidate 10-biomarker panel, we performed individual ELISA measurements using commercially available kits to determine the level of these biomarkers in serum in a second independent set of 38 lung cancer cases and 76 controls representative of the previous case/control populations. We also included several biomarkers, specifically CRP, HGF, and CEA, for individual ELISA measurement that had been reported in the literature to be differentially abundant in the serum of lung cancer patients, and which had been previously included in our initial Luminex discovery analysis. Using the serum level of each of the 10-biomarker panel proteins as determined by ELISA, Naïve Bayes classification analysis with 10-fold cross-validation was able to discriminate cases from controls in this new set of subjects with an average accuracy of 78.95% (average SN 55.3% and average SP 90.8%), consistent with our previous verification findings thus confirming the classification performance of the panel using an independent platform analysis. Addition of the ELISA data for HGF, CRP, and CEA to the analysis did not improve classification performance as these 3 biomarkers were found to highly correlated with others in our 10-biomarker panel. We conclude that the 10-biomarker panel has robust classification performance and can be used as the basis for further refinement, e.g. including novel biomarkers not assayed to date, to produce an optimized serum-based biomarker panel for detection of lung cancer.

Impact of Demographic and Clinical Variables on Misclassifications by the Models

An important feature of the RL approach is that it assigns individual classifications to each case or control subject, allowing for examination of possible confounders. The observed misclassification rates of the lung cancer cases and PLuSS controls in the models stratified by demographic and clinical variables are summarized in Tables 4 and 5. Overall rates of misclassification by the 10-biomarker panel were 26.7% of cases and 6.7% of controls (Tables 4 and 5). The 10-biomarker panel yields equal performance in correctly distinguishing males and females as cases or controls, and smoking status was also not a factor in classification. Age overall was not a significant factor in misclassification of cases or controls, although 2 of 3 cases aged 38–44 were misclassified as controls by the 10-biomarker model. This inaccuracy may result from the absence of younger subjects in the training set that included no cases younger than 46 years at diagnosis and no controls younger than 50 years. Against all adenocarcinomas and squamous cell carcinomas in the verification set, the only 2 histologies of lung cancer in the training set, the 10-biomarker model performed at an overall misclassification rate of 35.3% (BACC 63.8%, Tables 4 and 5). It appeared that the model tended to misclassify adenocarcinomas to a greater degree than squamous cell carcinomas (Table 4) although this difference was not statistically significant. The overall BACC in the verification set for all histologic types of lung cancer examined was 83.3% (SP 93.3%) for the 10-biomarker panel and, although the sample size was small, the model correctly classified all 3 small cell carcinomas (Table 4).

Among Stage I/II lung tumors, the 10-biomarker panel misclassified 15% of Stage I/II tumors in the verification set, compared to 50% of the Stage III/IV tumors (Table 4), suggesting the model performs well in discriminating early stage lung cancer which was the predominant case group in the training and testing sets. With a SP of 93.3%, the 10-biomarker model BACC was 89.2% in Stage I/II disease. Application of Fisher’s Exact Test to these results reveals that none of the observed differences by sex, age, histology, stage, or smoking status are statistically significant given the relatively small sample sizes in each subgroup.

Examination of Classification Confounding by Airway Disease

Inflammatory response and immune cell functions were identified in the pathway analyses for the 10-biomarker panel (see below), and are known to contribute to chronic obstructive pulmonary disease (COPD). Since COPD is a known risk factor for lung cancer, it could be a confounding clinical variable in classification by the panel although airflow obstruction was not a factor in misclassification in the 10-biomarker model (Tables 4 and 5). Pulmonary function test results and measurements of radiographic emphysema were available for all of the PLuSS controls (Table 5). Misclassification rates in these PLuSS controls showed no significant association with presence or degree of airflow obstruction in the model. The model had a higher misclassification rate in subjects with trace emphysema compared to no or mild emphysema (3 class comparison, P=0.04, Fisher’s Exact Test). Those controls with the best overall lung health (none to mild radiographic emphysema and GOLD scores of 0 or 1) did not show a significantly different misclassification rate than others (Table 5). In the model, the only misclassifications of controls were in subjects with no or minimal airway disease, suggesting that the presence of COPD is not significantly contributing to incorrect predictions of controls as cases. Importantly, the presence or type of pulmonary nodules detected by CT screening also did not appear to contribute to misclassifications. In fact, those PLuSS subjects with a suspicious nodule were more often correctly classified as controls than those with no nodule or a benign nodule (Table 5). All nodules found in these subjects remained clinically noncancerous at least 3 years after initial detection, based on either resolution or no further growth on subsequent CT scans.

Lastly, we examined the model predictions in the verification set of the subset of clinical lung cancer cases whose invasive diagnostic procedures were triggered by CT pulmonary nodule findings that were less than 3.5 cm in diameter. This type of patient is most comparable to those who might undergo routine CT screening with a resulting indeterminate pulmonary nodule. Twenty of the 30 cases fell in this category (Table 6). These patients were referred for CT for a number of clinical indications, including incidental findings due to workup for a non-pulmonary condition or pulmonary symptoms. Four of the 20 subjects with small CT nodules were being followed as the result of participation in PLuSS, so are among the CT-screened population. These 20 patients all received prompt invasive diagnostic procedures after a worrisome CT finding, at which time their blood was drawn for this study. The 10-biomarker panel predicted cancer correctly in 15 of 20 of these cases (75.0%), including 3 of the 4 PLuSS participants. Of the remaining 10 subjects in the verification set with larger CT masses (>3.5 cm), the model correctly predicted cancer in 7 (Table 6). For CT findings of <2.0 cm, the model correctly predicted 5 of 8 (62.5%). These findings suggest that the model has robust predictive performance in patients with small tumors that would be detected by CT screening as well as in CT screened subjects without cancer.

Table 6.

Predictions Made in Lung Cancer Cases with Small (<3.5 cm) and Large (>3.5 cm) Pulmonary Masses Found on CT

Study No	Referral Reason	Size of CT Mass (cm)1	Time to Diagnosis (Days)2	Staging	Prediction3
<3.5 cm
37	Cough	3.2	13	IIA	Cancer
46	Sclerosis	1.6	8	IIIA	Control
52	F/Up for lipoma	1.3	96	IA	Cancer
88	Incidental	3.1	42	III1	Cancer
97	Pneumonia	2.1	5	IIIA	Cancer
107	Cough	<0.5 (2)	80	Limited4	Cancer
126	PLuSS	1.6	5	IIIA	Control
189	Incidental	1.6	62	I5	Cancer
273	Incidental	2.4	5	I5	Cancer
297	Cough	1.3	43	IIIA	Control
327	Incidental	3.0	1	III5	Control
335	Incidental	2.7	50	IIIA	Control
358	PLuSS	2.3	36	IIB	Cancer
364	PLuSS	1.6	14	IA	Cancer
380	Dyspnea	2.8	62	I5	Cancer
390	COPD exacerbation	2.5	42	I5	Cancer
410	PLuSS	1.5	90	IA	Cancer
424	Dyspnea	2.0	37	I5	Cancer
469	Incidental	1.6	33	I5	Cancer
471	Dyspnea	2.7	49	Limited4	Cancer
>3.5 cm
23	Cough	10	40	IIB	Cancer
31	Flank Pain	3.7	10	IB	Cancer
62	Hemoptysis	3.7	7	IB	Control
245	Incidental	10	3	IB	Cancer
311	Hemoptysis	4	1	IB	Cancer
370	Cough	4.2	110	I5	Control
388	Shoulder pain	3.6	30	IB	Control
403	Chest pain	6	7	IV	Cancer
412	Incidental	3.5	71	IA	Cancer
483	Cough	3.06	15	Extensive4	Cancer

¹Diameter of CT mass finding that triggered invasive diagnostic procedure

²Time interval between suspicious CT finding and biopsy-proven diagnosis

³Case-control classification prediction from 10-biomarker panel model

⁴Small cell lung carcinoma only

⁵Clinical staging; patient deemed inoperable

⁶With enlarged lymph nodes

Pathways Identified in the 10-Biomarker Model

Ingenuity Pathway Analysis (IPA) software version 8.6 was used to determine cellular functions and diseases that might by associated with the informative biomarkers. All 10 biomarkers were eligible for pathway analysis by the IPA software. A network was found encompassing all of the biomarkers, excluding TTR, while TTR was assigned to a separate network. The top diseases encompassed by the 10-biomarker panel were cancer, genetic disorder, metabolic disease, and inflammatory response. The top cellular functions identified were cell movement, cell signaling, and cell death. The top physiological functions identified were hematological system development, immune cell trafficking, tumor morphology, and tissue development. Other molecules that were linked in a network with the biomarker panel included: EGFR, caspase, focal adhesion kinase, IL1, IL12, NFκB, TGFβ, and the Fox family of transcription factors. In summary, these 10 proteins interconnect five major functions, including two functions related to tumor biology (cancer and tumor morphology) and three functions related to the host response (inflammatory response, cell movement, and immune cell trafficking).

DISCUSSION

Although many individual serum biomarkers, or combinations of biomarkers, which have been reported to distinguish cancer patients from individuals without cancer have been reported, few are in clinical use. The major limitation has been lack of sufficient sensitivity (presence of false negatives) or specificity (presence of false positives). In addition to the previously referenced individual serum biomarkers Cyfra 21.1, CEA and TPA, the published literature contains a number of reports of the evaluation of panels of serum protein biomarkers associated with NSCLC. Khan et al had previously reported 2 of the serum biomarkers in our panel, SAA and MIF, as NSCLC serum biomarkers⁴⁴. Patz et al identified a 4 serum protein panel comprising CEA, retinol binding protein (RBP), 1-antitrypsin (ATT), and squamous cell carcinoma antigen (SCC) which together correctly classified lung cancer patients in a training set with 89.3% SN and 84.7% SP and with 77.8% SN and 75.4% SP in an independent validation set⁴⁵. Yee et al demonstrated that circulating protein biomarkers connective tissue-activating peptide III/neutrophil activating protein-2 (CTAP III/NAP-2) when combined with haptoglobin in a model that included age, smoking status, and FEV1 yielded an area under the curve (ROC) of 0.84 illustrating the value of including clinical and demographic variables in diagnostic and risk prediction models for lung cancer⁴⁶. Patel et al published a 6-analyte serum test for NSCLC that included Cyfra 21.1 and E-selectin, 2 additional biomarkers included in our 10-biomarker panel, with observed high specificity against high risk subjects without lung cancer except for individuals with lung nodules⁴⁷. Pine et al reported elevated levels of CRP, IL-6, and IL-8 in lung cancer patients in the National Cancer Institute-Maryland (NCI-MD) study and demonstrated that elevated levels of serum IL-8 and CRP together were a better prediction classifier for lung cancer diagnosis than either marker considered alone⁴⁸. These associations with serum IL-6 and IL-8 levels appeared to be robust being independent of smoking, age, sex, lung tumor histology, stage, presence or absence of systemic inflammation and whether the lung cancers were clinically ascertained or diagnosed as a result of CT screening. Most recently, candidate biomarkers and panels of these biomarkers have been suggested from a study of 4 mouse models of human lung cancer by Taguchi et al⁴⁹. Also, of potential complementary diagnostic utility to circulating protein biomarkers, the analysis of lung cancer serum autoantibodies has also been reported by Qiu et al⁵⁰ and Wu et al⁵¹. Combinations of these autoantibody panels yield similar AUROC performance to our results and previously published protein biomarker panels.

The performance of a cancer biomarker in screening the general population must be at an extremely high stringency, on the order of 99.5% accuracy. In contrast, in a clinical context using a high-risk population, significant improvement in clinical work-up for a specific disease could be achieved with tests that discriminate with substantially lower accuracy. Toward this goal of clinical application, we developed a highly performing 10-biomarker panel in a 2-stage process by first identifying 8 biomarkers that could discriminate NSCLC patients from those of matched tobacco-exposed controls in a training set and demonstrating its performance in a test set. We then combined all the data from the training and test sets to train a new model. Not surprisingly, the original 8 biomarkers remained in the model, but 2 additional markers were identified that showed considerable SP (93.3%) with minimal loss of SN (73.3%) in the blinded verification set, and showed ability to correctly classify lung cancers of divergent histologic subtypes. Training on a larger diverse set of cases and controls appears to have produced a more discriminatory model, despite the fact that the larger training set contained unmatched individuals.

In the blinded study which constituted an independent test set for the new model, the 10-biomarker panel operated at a BACC of 83.3% in all cases and CT screened controls and 89.2% in Stage I/II lung cancers and CT screened controls, those tumors which are most likely to be identified on serial CT screens. Confounding by airflow obstruction and emphysema appear to be minimal in this panel, based on similar misclassification rates in individuals with and without airway disease. Interestingly, although the model was developed by training only on lung adenocarcinomas and squamous cell carcinomas, in the blinded study the panel showed ability to correctly classify small cell carcinoma, and undifferentiated NSCLC, and also classified 1 of 2 pleomorphic carcinomas correctly. The biomarker panel will need to be formally validated in larger studies that would include examining sera from a wide range of lung cancer histologies, from patients who were diagnosed with lung cancer as the result of a screening CT nodule, as well as a larger group of CT-screened controls. The validation should also include subjects from another institution and a study of the temporal stability of these protein analytes in serum and the lead-time found in their association with lung cancer.

CT-screening detection of an indeterminate pulmonary nodule, a non-specific but frequent finding in high-risk subjects with a smoking history, creates a diagnostic dilemma. This diagnostic challenge was highlighted in the recent publication of the NLST⁹ results in which high-risk subjects, defined for this trial as individuals between 55 and 74 years of age with a 30 pack-year smoking history and, if a former smoker, having quit with the previous 15 years, were screened using low-dose CT three times at 1-year intervals, resulting in 24.2% of the tests classified as positive by virtue of any size nodule or nodules of 4 mm or greater being detected. The vast majority (96.4%) of these screening findings were false positives for lung cancer, reflecting the poor specificity of present CT imaging techniques. The smaller the nodule, the less the likelihood that it is malignant, and most of these nodules are classified as low suspicion upon follow up, and are not considered for biopsy or surgery. In our clinical experience, ~20% of the 1–2 cm nodules that are concerning enough to be considered for biopsy are actually malignant. Given the substantial risks of invasive diagnostic thoracic procedures, unselective biopsy of every person with a small nodule is clinically unacceptable. Most CT screening protocols delay lung biopsy until a small nodule appears to grow when monitored with repeated CT scans over time. At a population level, maximum benefit from early lung cancer detection through CT screening requires prompt diagnosis and treatment for individuals with cancer, while limiting the frequency of radiographic follow-up and unnecessary lung biopsy for persons without cancer. In principle, immediate intervention for a small nodule could be restricted to individuals with validated risk factors. Risk factors we have used to construct a model in the PLuSS cohort include advanced age, cigarette smoking history, family history of lung cancer, severe airflow obstruction, and severe emphysema¹³. However, the ability of this risk model to predict lung cancer in the PLuSS cohort operates with only 50% SP at 80% SN. That is, of every 100 individuals with 1–2 cm suspicious nodules, our predictive model, operating at 80% SN, theoretically would identify 16 of the 20 persons for immediate biopsy who actually have lung cancer in this group. However, in a setting where only 1 of 5 subjects with a solitary nodule >1 cm but <2 cm truly has lung cancer, an additional 40 individuals out of the 100 with 1–2 cm suspicious nodules would be incorrectly classified as needing an immediate biopsy because the model operates at only 50% SP.

To improve interpretation of CT images in the setting of a suspicious pulmonary nodule, the SN and SP needed is ~80% and 85–90% respectively, similar to the observed performance of the 10-biomarker panel for classifying Stage I/II cancer in our blinded study. Under these conditions of classification performance, of the same 100 individuals with suspicious nodules, 16 (67%) of 24 persons selected for immediate biopsy would be expected to have lung cancer and only 8 would be biopsied needlessly. Such a strategy could reduce the number of futile invasive procedures by 80% (40 of 80 without lung cancer vs. 8 of 80 without lung cancer). Although the biomarker model we described could not detect every lung cancer, it offers a significant clinical improvement over CT imaging alone. It remains to be proven in a validation study that patients with lung cancer who are identified by small pulmonary nodules can be correctly classified by our model. However, even a SN of 75% for this group would be an improvement over CT alone. Also, patients with nodules not identified as cancer by the model would continue to receive follow up clinical monitoring and would be biopsied if the nodules grew in size, which is the current standard of care.