A multi-analyte panel consisting of extracellular vesicle miRNAs and mRNAs, cfDNA, and CA19-9 shows utility for diagnosis and staging of pancreatic adenocarcinoma

Abstract

Purpose

To determine whether a multi-analyte liquid biopsy can improve the detection and staging of pancreatic adenocarcinoma (PDAC).

Experimental Design

We analyzed plasma from 204 subjects (71 healthy, 44 non-PDAC pancreatic disease, and 89 PDAC) for the following biomarkers: Tumor-associated extra-cellular vesicle (EV) miRNA and mRNA isolated on a nanomagnetic platform that we developed and measured by next-generation sequencing or qPCR, circulating cell-free DNA (ccfDNA) concentration measured by qPCR, ccfDNA KRAS G12D/V/R mutations detected by droplet digital PCR, and CA19–9 measured by ECLIA. We applied machine learning to training sets and subsequently evaluated model performance in independent, user-blinded test sets.

Results

To identify patients with PDAC versus those without, we generated a classification model using a training set of 47 subjects (20 PDAC and 27 non-cancer). When applied to a blinded test set (N = 136), the model achieved an area under the curve (AUC) of 0.95 and accuracy of 92%, superior to the best individual biomarker, CA19–9 (89%). We next used a cohort of 20 PDAC patients to train our model for disease staging and applied it to a blinded test set of 25 patients clinically staged by imaging as metastasis-free, including 9 subsequently determined to have had occult metastasis. Our workflow achieved significantly higher accuracy for disease staging (84%) than imaging alone (accuracy = 64%; P <0.05).

Conclusions

Algorithmically combining blood-based biomarkers may improve PDAC diagnostic accuracy and pre-operative identification of non-metastatic patients best suited for surgery, although larger validation studies are necessary.

Introduction

Pancreatic ductal adenocarcinoma (PDAC) is the third leading cause of cancer-related death in the United States, with an overall five-year survival of 9%(1). Diagnosis and staging currently rely on endoscopic ultrasound-guided biopsy, computerized tomography (CT), and magnetic resonance imaging (MRI)(2). Most patients are diagnosed at an advanced stage, and sufficiently sensitive and specific screening tests for early disease remain elusive. While curative-intent surgery remains an option for patients whose disease is confined to the pancreas, distinguishing these patients from those with metastases, who are unlikely to benefit from surgery, remains challenging due to the presence of occult metastases not detectable by standard of care imaging(3–5).

To address these challenges, several blood-based liquid biopsy biomarkers have been developed but show low sensitivity for detection of early stage disease(6–8). Carbohydrate antigen 19–9 (CA19–9), a longstanding PDAC-associated biomarker, is clinically utilized to monitor response to therapy but its role in screening or determining surgical resectability is unclear(9). More recently, several liquid biopsy biomarkers have shown potential for the diagnosis and staging of PDAC. PDAC patients with detectable circulating tumor cells (CTCs) had significantly reduced progression-free (PFS) and overall survival (OS) (10–11), although CTCs are often undetectable in early-stage disease. Circulating cell-free DNA (ccfDNA) concentration has been shown to correlate with disease burden(12,13); KRAS mutations in ccfDNA have been detectable at various stages of disease although at lower rates in early stage disease(14); soluble protein biomarkers have demonstrated diagnostic value(15), and tumor-associated extracellular vesicles (EVs) have generated enthusiasm for their potential to improve diagnosis of the disease(7,15–17).

In our previous work, we showed that by enriching tumor-associated EVs from plasma using an immunomagnetic nanofluidic chip, and analyzing RNA cargo, we could identify transcriptional signatures that accurately classify metastatic PDAC patients from healthy controls in clinical cohorts(18). However, although we have demonstrated promising results for detection of early-stage disease in a murine model of pancreatic cancer (KPCY)(18), we have not yet demonstrated the accuracy of our approach for detection of early disease in human patients. Work from other groups has shown that the performance of a biomarker can be improved by combining it with different types of circulating biomarkers, such as combining ccfDNA and soluble proteins(19,20). Here we build on previous work and describe a multi-analyte panel that algorithmically combines tumor-associated EV mRNA and miRNA, ccfDNA concentration and KRAS mutation detection, and CA19–9 using machine learning. Using training sets of samples from patients, disease controls, and healthy individuals as well as independent, blinded test sets, we first apply the approach to distinguish cancer versus non-cancer patient samples. We then re-train the model for disease staging and the detection of metastatic disease for PDAC patients originally staged by standard of care imaging.

Materials and Methods

Patients and Sample Collection and Processing

Whole blood was collected at baseline (therapy-naïve) from 204 total patients at the Hospital of the University of Pennsylvania under IRB Protocol #822028 after obtaining written informed consent. The study was conducted in accordance with the Declaration of Helsinki. Among the 89 patients with PDAC, 58 were clinically staged on the basis of baseline imaging as having local disease only (M0), including 37 resectable patients and 21 patients with locally advanced disease. The remaining 31 patients had evidence of metastatic disease on baseline imaging (M1; Table 1). For the staging analysis, retrospective chart review was conducted to determine whether 34 patients originally staged by imaging as metastasis-free (M0) and resectable might have harbored metastatic disease below the level of detection for standard of care imaging. Ten patients were categorized as having had occult metastases, including 4 with metastases detected intra-operatively and 6 with very early recurrence, here defined as within 4 months of baseline blood draw (Figure S1). Time to metastasis (TTM) was defined with respect to the date of baseline blood draw, censoring patients based on the date of last follow-up. Imaging data and clinical staging were obtained by chart abstraction. The 115 subjects serving as non-cancer controls included 44 patients with non-cancer pancreatic diseases such as intraductal papillary mucinous neoplasm (IPMN) and pancreatitis, as well as 71 healthy individuals enrolled at the time of routine screening procedures such as endoscopy. Patients with an active malignancy at the time of blood draw were excluded from the control cohorts. All non-cancer control patients were followed for a minimum of 4 months to verify that no patient received a PDAC diagnosis subsequent to blood draw. Venous blood was collected in K₂EDTA vacutainers (Becton Dickinson) or Streck cfDNA BCT (Streck) and processed to plasma as previously described(18). K₂EDTA and Streck cfDNA whole blood was processed within 3 or 24 hours after blood draw, respectively. Plasma was aliquoted and stored at −80°C for future use. All subjects had sufficient total plasma from a single blood draw such that all assays described below could be performed. In addition to the 204 samples for which results are reported, a batch of 10 additional samples was processed but yielded results for 4 biomarkers that were significantly different than the training set. Remeasurement was not possible due to the plasma sample having been exhausted. This batch was excluded from the blinded test set before being classified using machine learning(21). This highlights a limitation of machine learning based approaches, in that the model can only be trusted when the test data is consistent with the data the model was trained with. An active area of research is automated outlier analysis to avoid errors in machine learning based on spurious data. The study was designed and conducted in accordance with the Reporting recommendations for tumor MARKer prognostic studies (REMARK) guidelines(22).

Table 1.

Clinical characteristics of study population.

Discovery Set (N=29)
	Age Range (Median)	Gender	non-PDAC pathology	TNM stage	Clinical Stage
Healthy Controls	62.0–69.4 (64.7)	n=4 Male n=3 Female
Disease Control	43.4–72.0 (70.7)	n=4 Male n=1 Female	n=4 Pancreatitis n=1 Billary Stricture
M0	60.5–71.5 (68.3)	n=3 Male n=1 Female		n=2 cT2cN0M0 n=1 cT3cN0M0 n=1 cTxN0M0	n=2 IB n=1 IIA n=1 X
M1	51.0–67.0 (64.0)	n=7 Male n=6 Female		n=4 cT3N1M1 n=1 cT4N1M1 n=8 cTxNxM1	n=13 IV
Training Set (N=47)
	Age Range (Median)	Gender	non-PDAC pathology	TNM stage	Clinical Stage
Healthy Controls	45.6–75.3 (62.3)	n=8 Male n=7 Female
Disease Control	43.4–82.2 (65.0)	n=10 Male n=2 Female	n=9 Pancreatitis n=3 IPMN
M0	54.4–77.7 (65.2)	n=3 Male n=6 Female		n=3 cT1cN0M0 n=1 cT1cNxM0 n=5 cT2N0M0	n=3 IA n=5 IB n=1 X
M1	51.0–81.5 (67.5)	n=5 Male n=6 Female		n=1 cT1cN0M1 n=2 cT2N0M1 n=1 cT2N1M1 n=1 cT2N0M1 n=1 cT3N1M1 n=2 cT4N1M1 n=1 cTxN0M1 n=2 cTxN1M1	n=11 IV
Test Set (N=136)
	Age Range (Median)	Gender	non-PDAC pathology	TNM stage	Clinical Stage
Healthy Controls	41.6–85.8 (66.0)	n=20 Male n=29 Female
Disease Control	19.9–83.1 (63.7)	n=13 Male n=17 Female	n=3 IPMN n=12 Pancreatitis n=2 Biliary Stricture n=1 Benign Neurofibroma n=11 Pancreatic Cyst n=1 Pancreatic Duct Dilation
M0	50.5–85.4 (66.5)	n=26 Male n=19 Female		n=3 cT1cN0M0 n=16 cT2N0M0 n=4 cT2N1M0 n=7 cT3N0M0 n=4 cT3N1M0 n=3 cT4N0M0 n=1 cT4N0M1 n=5 cT4N1M0 n=2 cT4N1M1	n=3 IA n=15 IB n=7 IIA n=9 IIB n=8 III n=3 IV
M1	48.6–71.2 (62.5)	n=7 Male n=5 Female		n=1 cT0N0M1 n=1 cT2N0M1 n=1 cT2N1M1 n=2 cT3N0M1 n=1 cT3NxM1 n=2 cT4N0M1 n=4 cT4N1M1	n=12 IV

^*indicates 8 patients are included in the discovery as well as training sets. Designation of M0 versus M1 is based on baseline imaging.

Tumor derived EV miRNA and mRNA isolation by track etched magnetic nanopore (TENPO) device

EVs from each patient’s K₂EDTA-collected plasma (1.5mL) were magnetically labeled using biotinylated antibodies and anti-biotin ultrapure 50nm diameter nanoparticles (Miltenyi Biotec). Antibodies used in this study included anti-human CD326 (EpCAM) (BioLegend), anti-human CD104 (ThermoFisher Scientific), anti-human c-Met Monoclonal (ThermoFisher Scientific), anti-human CD44v6 antibody (ThermoFisher Scientific), and anti-human TSPAN8 (Miltenyi Biotec). These surface markers have been previously shown to enrich pancreatic tumor-associated EVs from plasma(18,23). These five biotinylated antibodies (1.25 μL each) were pipetted into the human plasma samples and incubated for 20 minutes at room temperature on a shaking mixer. Subsequently, anti-biotin magnetic nanoparticles (20μL, Miltenyi Biotec) were added to the samples and incubated for another 20 minutes at room temperature on the shaking mixer. Next, the plasma samples were loaded into the reservoir of the TENPO device which was connected to a programmable syringe pump (Braintree Scientific) to provide the negative pressure driving the sample through the device.

Details on the design and fabrication of TENPO have been previously reported(18). Briefly, a permanent magnet (NdFeB disc magnet, d=1.5 inches, h=0.75 inches, K&J Magnetics) was placed beneath the TENPO device to magnetize TENPO’s paramagnetic Ni₈₀Fe₂₀ film and the superparamagnetic nanoparticles used to label the EVs. While samples were pulled through the device, EVs that were labeled with a sufficient number of magnetic nanoparticles were captured at the edges of the chip’s nanopores, while background EVs flowed through and were discarded. The positively selected EVs were subsequently lysed on the chip by directly loading QIAzol lysis reagent (700mL, Qiagen), incubated for 3 minutes, and collected the lysate. The RNA was then extracted from this lysate off-chip (ExoRNeasy serum/plasma kit, Qiagen). The EV miRNAs and mRNAs were eluted and stored at −80°C or immediately processed for further analysis.

EV miRNA sequencing and candidate discovery

A discovery cohort of 29 samples (Table 1, Figure S2) was analyzed by next-generation sequencing to identify miRNAs in the enriched tumor associated EVs that might be differentially expressed among patient cohorts. QIAseq miRNA library kit (Qiagen) was used to make a library from isolated EV miRNA. A BioAnalyzer was used to quantify RNA prior to sequencing. The library was sequenced using a HiSeq 2500 kit (Illumina, Next-Generation Sequencing Core, University of Pennsylvania). A modified version of the UPenn SCAP-T RNA-Seq expression pipeline (Fisher, S A., “Safisher/Ngs.” GitHub, 2017) was used for expression quantification by aligning to the hg38 genomes. The minimum fragment length allowed past the TRIM module was adjusted to 16 bases for miRNA analysis. The number of allowed mismatches was capped at one and unannotated splices were prohibited. Expression counts were normalized by DESeq2(24) and quantified using VERSE(25), using Gencode 25 and UCSD mm10 gene annotations, combined with MirBase v21 annotations for 3p and 5p microRNA.

Selection of EV RNA panel

To identify potential EV miRNA candidates for PDAC diagnosis, we first applied the feature selection algorithm Least Absolute Shrinkage and Selection Operator (LASSO) on EV miRNA sequencing results to find the most informative miRNAs (Figure S2A). The resulting eight miRNA candidates were: hsa.miR.103b, hsa.miR.23a.3p, hsa.miR.432.5p, hsa.miR.409.3p, hsa.miR.224.5p, hsa.miR.1299, hsa.miR.4782.5p, and hsa.miR.4772.3p (Figure S2B). We next validated the miRNA candidates by qPCR, and identified 3 miRNAs (hsa.miR.4772.3p, hsa.miR.4782.5p, and hsa.miR.432.5p) with Cq≥40, which were considered to not be adequately abundant and were therefore excluded from further analysis (Figure S2C). The remaining five miRNAs were measured by qPCR within the training set (N=47) and were compared with the EV miRNA sequencing data (Figure S2D) within each patient subset (non-cancer and PDAC). The qPCR and sequencing data corresponded well with one another (R² = 0.6, Figure S2D). We also included six EV mRNAs (CD63, CK18, GAPDH, H3F3A, KRAS, ODC1) which had previously been used to distinguish stage IV PDAC patients from healthy controls(18) to form a panel of 11 potential EV RNA biomarkers. These 11 EV RNA biomarkers combined with CA19–9, ccfDNA concentration (qPCR for ALU), and ctDNA (KRAS mutation allele fraction) formed the final 14-biomarker-candidates for later classification. The workflow of multi-analyte panel generation is shown in Figure S3.

EV miRNA and mRNA qPCR

The miScript SYBR Green PCR kit (Qiagen) and miScript primers (Qiagen) were used to quantify EV miRNAs. A master mix containing miScript SYBR Green, miScript primer, universal primer, and RNase-free water was prepared at a 5:1:1:2 ratio. 9μl of the master mix was added to each well of a 384-well plate, followed by 1μl of cDNA. 40 cycles were run with a default setting using CFX384 Touch Real-Time PCR machine (Bio-Rad). The SsoAdvanced Universal SYBR Green Supermix (Bio-Rad) and primers (Integrated DNA Technologies) were used for EV mRNA quantification. The SYBR Green supermix, primers, and RNase-free water were combined at a 5:0.5:3.5 ratio for the master mix. 9μl of the master mix was added to each well, followed by 1μl of cDNA. 40 cycles were run with a default setting using CFX384 Touch Real-Time PCR machine (Bio-Rad). Duplicates were performed for each sample. The melting curves for the amplified DNA were manually validated before subsequent analysis.

ccfDNA Extraction and Concentration

ccfDNA was isolated from K₂EDTA- or Streck-collected plasma. If necessary to ensure a consistent input volume across all samples, the volume was adjusted with Phosphate Buffered Saline and the measured ccfDNA concentration was corrected for original input. Extraction was performed using the QIAamp Circulating Nucleic Acid Kit (Qiagen #55114) with two modifications to the manufacturers protocol. First, incubation of the buffer-lysate solution was increased to 1 hour at 60°C. Second, the final elution was carried out twice with 30μL of Buffer AVE for a total of 60μL. The extracted ccfDNA from 1mL of plasma was used for downstream assays with extracted ccfDNA stored at 4°C for short-term use or at −20°C for long-term storage. The concentration of extracted ccfDNA was quantified by qPCR for a 115 bp amplicon of the ALU repetitive element(26). Briefly, qPCR was carried out on 1μL of extracted ccfDNA, in quadruplicate, using Power SYBR Green PCR Master Mix (Applied Biosystems #4367659) according to the manufacturer’s instructions on a ViiA 7 Real-Time PCR System (Applied Biosystems). Results were normalized to a standard curve of reference DNA (Promega #PAG3041) using QuantStudio Real-Time PCR Software (Applied Biosystems).

Pre-amplification ddPCR for detection of circulating KRAS G12D/V/R mutations

Pre-amplification PCR of the KRAS G12 locus was performed using 15μL of ccfDNA eluate in a 50μL reaction. Pre-amplified material was diluted 1:4 with TE buffer and stored for short-term use at 4°C and at −20°C for long-term storage. Multiplex ddPCR to detect KRAS G12D/V/R/WT or duplex ddPCR (KRAS G12D/WT, G12V/WT, or G12R/WT) was prepared as a 30μL reaction mix containing 2x TaqMan Genotyping Master Mix, 1x droplet stabilizer, and 200nM primers (Table 2), probes at 50nM (multiplex G12R only) or 100nM (multiplex G12D and WT, both probes in duplex assays), and 10μL of diluted pre-amplification reaction. Multiplex ddPCR for KRAS G12D/V/R/WT was initially used to identify positive samples; these findings were verified and quantified by testing with identified variant’s specific duplex assay. 25μL of each reaction mix was loaded onto the RainDrop Source instrument (RainDance Technologies, Inc.) for droplet production. Mutant allele fraction was calculated as the mutant allele copy number divided by the total (wild-type + mutant) copy number. Samples that failed to meet mutant copy number thresholds or with a mutant allele fraction <0.01% were considered undetectable and assigned a value of 0.001%. Of the samples with a detectable KRAS mutation, the allele fraction was analyzed as a continuous variable, with values ranging from 0.01%−39.08% (median 0.405%).

Table 2.

Primers and Probes for KRAS mutation analysis

Primer/Probe	Sequence
KRAS G12 forward primer	AGGCCTGCTGAAAATGACTGAATAT
KRAS G12 reverse primer	GCTGTATCGTCAAGGCACTCTT
KRASWT-VIC probe	VIC-TTGGAGCTGGTGGCGT-MGBNFQ
KRAS G12D-FAM probe	FAM-TGGAGCTGATGGCGT-MGBNFQ
KRAS G12R-FAM probe	FAM-TTGGAGCTCGTGGCGT-MGBNFQ
KRAS G12V FAM probe	FAM-GAGCTGTTGGCGT-MGBNFQ

CA19–9 measurement

The Hospital of the University of Pennsylvania Clinical Immunology Laboratory was provided a 200ul aliquot of K2EDTA plasma that had been banked at −80C. CA19–9 was measured as a research assay by electrochemiluminescence immunoassay (ECLIA) using the Elecsys CA19–9 Immunoassay on a cobas e601 platform (Roche), per the manufacturer’s instructions. The resulting CA19–9 values ranged from 0–793,700 U/mL (median 18.165U/mL).

Machine learning data analysis

Our machine learning-based development of a PDAC diagnostic includes a feature selection step, a training step, and a validation step using a blinded test set. To mitigate the effects of overfitting, the blinded test sets are separate and completely independent from the data used to discover features or to train the model. First, to select the features used in our model we performed feature selection using Least Absolute Shrinkage and Selection Operator (LASSO) on the 14-biomarker-candidates from the training set of data, which is labeled with each subject’s true state (for example, PDAC versus Non-Cancer). Using these identified features, we then trained a classifier model. During the development of this model, we evaluated its performance using cross validation within the training set. Finally, this machine learning model was evaluated by classifying subjects in a separate, user-blinded test set.

The following additional steps were taken to mitigate the effects of overfitting in the development of and the evaluation of our machine learning model. Instead of using only a single machine learning algorithm, we instead used an ensemble of classifier models (including K-Nearest-Neighbors, SVM, linear discriminate analysis, logistic regression, and Naive Bayes) and averaged their results. By performing model averaging, the overfitting by any single algorithm can be mitigated, as each model will overfit the data differently and thus be averaged out, providing a more accurate model than any single method alone(27). We additionally applied a bootstrapping method to randomly select multiple subgroups of the training set to train the ensemble model, and thus mitigate the effects of outlier data in the training set. Most importantly, the model was evaluated using an independent, blinded data set only once, avoiding the possibility of the model overfitting the test set. The classifier model implemented in Python and LASSO was carried out in Matlab 2017a. The 95% confidence interval for sensitivity, specificity, and accuracy of PDAC diagnosis and occult metastasis detection were calculated based on binomial proportion confidence interval. McNemar’s asymptotic test (Matlab 2017a) was used to test concordance between our panel with CA19–9 and imaging for PDAC diagnosis and occult metastasis detection, respectively(28). Mann-Whitney test was used to evaluate the statistical significance of differences in individual biomarker profiles between two groups.

Results

Biomarker panel development

We constructed a biomarker panel including multiple blood-based analytes with the aim of improving sensitivity and specificity of disease diagnosis and staging (Fig. 1). We included previously reported tumor-associated markers such as ccfDNA concentration and ccfDNA-based detection of the KRAS G12D, V, and R mutations present in about 90% of PDAC tumors(29). CA19–9 is a routinely ordered laboratory test for PDAC monitoring and thus could readily be applied in the setting of disease detection. Although we previously showed that a panel of EV miRNAs could detect PDAC in a transgenic mouse model, we wanted to determine which miRNAs would be optimal for analyzing human samples. To do this, we isolated EVs and their miRNA cargo from the plasma of a discovery cohort of 29 patients (Table 1, Fig.S1), including 7 healthy controls, 5 disease controls (1 non-malignant biliary stricture and 4 pancreatitis), and 17 PDAC patients of various disease stages. Next-generation sequencing was performed on extracted EV miRNA and we applied the LASSO feature to the results to identify the most informative miRNAs (Fig.S2A, S2B). Among the 8 most informative, only 5 were selected to move forward based on their abundance as detected by qPCR (Cq≤40, Fig. S2C). To validate qPCR-based detection of the 5 miRNAs, matched samples were run by qPCR and the results compared to sequencing results, resulting in a correlation coefficient of R²=0.6. We then added six EV mRNA candidates (CD63, CK18, GAPDH, H3F3A, KRAS, ODC1) which we had previously used to distinguish metastatic PDAC patients from healthy controls(18). Altogether, including ccfDNA concentration, circulating mutant KRAS allele fraction, and CA19–9 concentration, we analyzed a total of 14 biomarker candidates for each subject.

Figure 1. Combining multiple circulating biomarkers to diagnose and stage PDAC.

Our biomarker panel consists of the mRNA and miRNA cargo of tumor-derived EVs enriched from plasma, circulating CA19–9, circulating cell-free DNA concentration (as determined by qPCR to detect the ALU repeat element), and circulating mutant KRAS allele fraction. This multiplex panel is combined algorithmically using machine learning. The system is trained using supervised learning on a cohort of 47 patients including 15 healthy individuals, 12 non-cancer disease controls, and 20 with various stages of PDAC. Finally, the developed classifiers are evaluated using an independent, blinded test set of 136 individuals to quantify performance.

Using this panel of 14 biomarkers, we trained our machine learning model with a set of 15 healthy controls, 12 disease controls (3 IPMN and 9 pancreatitis), and 20 patients with PDAC of various stages (Fig. 2A, Table 1). The best individual marker at distinguishing PDAC patients from non-cancer controls was CA19–9 (Fig. 2C, Fig.S4), which also showed the highest fold change between the PDAC and non-PDAC cohort among the 14 biomarker candidates (Fig. 2B). CA19–9 achieved an accuracy of A = (TP + TN) / total = 84% (95% CI 82–85%), where TP is the number of true positives and TN is the number of true negatives, using the clinical threshold of 36 U/mL(30–32). The best performing individual EV mRNA marker was CK18 (A = 66%, 95% CI 58–73%), which also was shown to be a predictive marker in our previous study on EV mRNA biomarkers(18). The best performing EV miRNA marker was miR.409 (A=59%, 95% CI 55–63%), a marker that has been associated with pancreatic oncogenesis(33,34). The accuracy of ccfDNA concentration was A=62% (95% CI 52–73%), and that of circulating mutant KRAS allele fraction was A=66%.

Figure 2. Development of the biomarker panel using the training set.

(A) Heatmap shows values for the 14 circulating biomarkers from each patient in the training set, which included 15 healthy controls, 12 disease controls, and 20 PDAC patients. (B) Fold changes of all biomarkers are plotted comparing PDAC vs. Non-Cancer patients. Error bars are standard deviation. ΔC_q is calculated as C_q,PDAC - C_q,NC (C) Accuracy of each individual biomarker for PDAC diagnosis. Clinical threshold of 36 U/mL was used for CA19–9. Other biomarkers’ thresholds were determined by Linear Discriminant Analysis. Error bars are standard error from bootstrapping 10 times from the training set. (D) A colormap shows the Pearson correlation coefficient (R) between each circulating biomarker. The inset colormap shows the average Pearson correlation coefficient among EV-miRNAs (by averaging R from all possible EV-miRNA pairs), EV-mRNAs (by averaging R from all possible EV-mRNA pairs) with the CA19–9, ccfDNA concentration, and KRAS mutation detection in ccfDNA designated ctDNA, for circulating tumor DNA, in the figure).

To generate a predictive panel of biomarkers, each biomarker needs predictive power and the constituent biomarkers should not correlate with one another, such that each biomarker carries some unique information about the state of the patient. Pairwise correlation coefficients (R) between biomarkers were calculated and revealed that individual biomarkers were generally not well correlated with one another, except CA19–9 and circulating mutant KRAS allele fraction (|R|=0.73) (Fig. 2D), and were therefore suitable to be combined together in a panel. More specifically, we found that CA19–9 did not correlate with either ccfDNA concentration or EV RNAs (|R|<0.4). Moreover, ccfDNA concentration did not correlate with EV RNAs (|R|<0.5) and was weakly correlated with circulating mutant KRAS allele fraction |R|=0.55. Tumor derived EV miRNAs weakly correlated with one another (averaged |R| among EV miRNAs is 0.65) but not with other biomarkers (|R|<0.40). Tumor derived EV mRNAs weakly correlated with one another (averaged |R|=0.66) but not with other biomarkers (|R|<0.40). Interestingly, EV-CK18, in addition to having the greatest accuracy of any individual EV mRNA biomarker, was also particularly uncorrelated with any other measured biomarkers (|R|<0.55).

Distinguishing PDAC patients from non-cancer controls

We next sought to identify the optimal panel of biomarkers from the 14 discussed above to distinguish PDAC patients from non-cancer controls. To achieve this, we applied LASSO to our training set of data (Fig. 2A, 3A) and determined that the best performing panel (AUC=0.93), as measured using 10-fold cross-validation, included five diverse biomarkers: EV-CK18 mRNA, EV-CD63 mRNA, EV-miR.409, ccfDNA concentration, and CA19–9 (Fig. 3A–C). Next, we addressed the question of whether we had included enough subjects to properly train our model by generating a learning curve (Fig. 3D). We found that the model’s performance plateaued beyond 25 patients, indicating that our training set sample of 47 subjects was sufficient for the patient population in this study.

Figure 3. Applying the biomarker panel to distinguish PDAC from non-cancer.

(A) A summary of the patient cohort used to train our platform to classify PDAC vs. Non-PDAC. (B) We selected the panel using least absolute shrinkage and selection operator (LASSO). The best performing panel was selected based on its area under the curve (AUC) using 10-fold cross validation within the training set repeated 5 times. Error bars are standard error. (C) The resulting PDAC vs non-PDAC (PDAC-NC) panel consists of 5 biomarkers. (D) A learning curve generated by bootstrapping 10 times within the training set. Error bars are standard error. (E) A summary of the independent patient cohort used to evaluate the classification of PDAC-NC in a blinded study. (F) The confusion matrix on the blinded test set showing that 75 of 79 non-cancer samples (95%) and 50 of 57 PDAC samples (86%) were correctly identified. TPR: true positive rate; TNR: true negative rate; PPV: positive predictive value; NPV: negative predictive value; (G) Receiver operating characteristic (ROC) curve comparison between the PDAC-NC panel and the best individual biomarker CA19–9, plus a control experiments of unselected biomarkers, where the training set was used to generate a model without using feature selection. (H) Comparison of accuracy of our PDAC-NC panel and the best individual biomarkers, plus the same control experiments described above. Error bars are standard error from bootstrapping 10 times

To further evaluate our approach, we applied our 5-marker panel to an independent blinded test set of 136 subjects (Fig. 3E) and achieved an accuracy of A=92% (86%−96%, 95% confidence interval), with sensitivity of 88% (76%−95%, 95% confidence interval) and specificity of 95% (88%−99%, 95% confidence interval) (Fig. 3F). We also calculated an AUC of 0.95 (Fig. 3G). Comparing to CA19–9 alone, our panel had a higher accuracy although the added benefit of our panel did not reach statistical significance (P = 0.103, using McNemar’s test, Fig. 3H). To validate that the performance is specific to the set of biomarkers that we had selected, we compared results to a control experiment where we randomly chose sets of 5 biomarkers (AUC =0.62). Our model’s performance was significantly better than using randomly selected features (P <0.01, McNemar’s test). Taken together, these results suggest that a multi-analyte panel can accurately predict detection of PDAC.

Distinguishing metastatic from non-metastatic PDAC

Imaging is a widely used but imperfect technique for detecting metastases and determining whether a PDAC patient’s disease is sufficiently localized for consideration of curative-intent surgery. We hypothesized that we could train our model to identify a biomarker panel that, in conjunction with imaging, could better stage PDAC patients by distinguishing metastatic from non-metastatic disease. To train the model we selected 20 PDAC patients originally staged by imaging, which included 9 resectable patients with no detectable metastasis (M0), and 11 patients with metastasis (M1) (Fig. 4A). Since some patients originally identified as M0 may have had occult metastases below the level of imaging detection, we conducted chart review and retrospectively re-stratified the M0 patients into two groups: 1) M0s: those with no evidence of metastatic disease intraoperatively or within 4 months of follow-up and 2) Occult metastases: those who had metastases detected intraoperatively or had metastatic recurrence within 4 months of blood draw. We performed a sensitivity analysis of time-to-distant-failure among our patient cohort (Fig. S5) to select the cutoff of 4 months, a time that is far shorter than the median recurrence-free, relapse-free, or metastasis-free survivals reported in both experimental and control arms in large randomized trials(35–37). This stratification resulted in the training set of 8 M0 and 12 M1 (11 with imaging-confirmed metastases and one with occult metastases) (Fig. 4A). Using LASSO, a biomarker panel of 4 markers, including EV-miR.1299, EV-GAPDH, circulating mutant KRAS allele fraction, and CA19–9 was selected as having the highest Accuracy (A=91%; Fig. 4B, C). A learning curve using 8-fold cross validation showed that the curve plateaued by 15 subjects, indicating that the 20 subjects in our training set were sufficient for this study (Fig. 4D).

Figure 4. Retraining the model to distinguish metastatic from non-metastatic PDAC.

(A) Patient cohort used to train our platform to classify occult or imaging-confirmed metastatic patients from non-metastatic PDAC patients. Dotted line indicates one PDAC patient who was originally determined by imaging to be M0 but was subsequently determined to have harbored occult metastases due to metastatic outgrowth less than 4 months from blood draw, hence was considered as occult metastases. (B) We selected the panel using least absolute shrinkage and selection operator (LASSO). The best performing panel was selected based on its AUC using 8-fold cross-validation within the training set and repeated 10 times. The inset shows the comparison of the accuracy between our panel (red) and the clinical diagnosis (grey). Error bars are standard error from bootstrapping 10 repeats. (C) The panel for metastatic PDAC detection consists of 4 biomarkers. (D) Learning curve of metastatic PDAC detection generated by bootstrapping N = 10 times within the training set. Error bars represent standard error. (E) Proposed clinical workflow to combine liquid biopsy with imaging for a test set of 37 PDAC patients, including 9 patients who were determined to have a time to metastases of <4 months. Baseline imaging was used to classify patients as either metastatic (M1; N=12, top arm) or no detectable metastases (M0_imaging; N=25, bottom arm). For the 25 M0_imaging patients, the liquid biopsy panel was then performed, resulting in 2 patient classifications, those called by the model as M1 (occult metastases; top arm) or those called as M0 (M0_LB; bottom arm). LB: liquid biopsy; ML: machine learning. (F) Shown are the confusion matrices for the 25 M0_imaging PDAC patients by imaging alone (bottom) and our method combining liquid biopsy with imaging (top). Our panel achieved accuracy = 84% with 78% sensitivity and 88% specificity. TPR: true positive rate; TNR: true negative rate; PPV: positive predictive value; NPV: negative predictive value. (G) Receiver operating characteristic (ROC) curve analysis on N = 25 M0_imaging PDAC patients in the blinded test set. Inset shows the accuracy comparison between imaging only (grey, accuracy=64%), control experiment using unselected biomarkers (yellow, accuracy=48%), and liquid biopsy (red, accuracy=84%) panel. Error bars are standard error from bootstrapping 10 repeats.

To further evaluate our panel’s ability to identify occult metastatic disease, we applied our approach to an independent blinded test set of 37 subjects with PDAC as part of a clinical workflow starting with standard of care diagnostic imaging and followed by liquid biopsy (Fig. 4E). Twelve of 37 patients were identified by imaging alone as having metastases, were classified as M1, and had no further evaluation. The remaining 25 patients were determined by baseline imaging to be resectable have no detectable metastases (M0-imaging). Upon retrospective chart review, 16 of 25 had no evidence of metastases within 4 months. Nine of 25 patients were determined to have had occult metastases, including 4 who had surgery aborted due to intraoperative detection of metastatic disease and another 5 who completed surgery but had distant metastases detected on imaging within 4 months of their baseline blood draw. Our liquid biopsy workflow correctly identified 7 of 9 patients as having occult metastatic disease, and 14 of 16 patients as being metastasis-free (Fig. 4E). Thus, by comparing the liquid biopsy prediction to the true state of the patients, we found that our test had an accuracy of detecting distant metastasis of A = 84% (64%−95%, 95% confidence interval) with sensitivity of 78%(40%−97%, 95% confidence interval) and specificity of 88% (62%−98%, 95% confidence interval) with an AUC=0.85, which compares favorably to the accuracy of imaging alone (A=65%; P<0.05, McNemar’s test) among 25 patients originally identified as M0 by imaging (Fig. 4G). We also ran a control experiment to confirm the performance is specific to the biomarkers identified from our training set. In the control experiment, we randomly selected biomarkers and the resulting AUC=0.53 with accuracy of 48% for metastatic disease detection. Our model’s performance was significantly better than the control experiment (P <0.01, McNemar’s test) as well. Taken together, these results suggest that a multi-analyte panel outperforms conventional imaging for metastatic disease detection.

Discussion

In this study, we applied a multi-analyte liquid biopsy approach to clinical baseline blood samples obtained from patients with PDAC of all stages, as well as healthy and disease controls. We demonstrate that this platform can accurately identify patients with PDAC (A=92%) and, for patients with pathologically confirmed PDAC, improve the detection of occult metastases that are not initially detected by standard of care imaging but are found intraoperatively or shortly after surgery (A=84%). Surgical resection remains the only curative therapy for PDAC(3), but is limited to patients without detectable metastases. At time of diagnosis, only about 15 – 20% of PDAC patients will be deemed candidates for surgical resection based on imaging and clinical status(1,3). Even in this subgroup, the intraoperative detection of metastases, prompting the surgery to be aborted, or rapid emergence of distant metastases within months of surgery, can still occur(1,3,38–40). Those patients with recurrent disease demonstrate survival similar to a de novo metastatic patient(41) thus questioning the potential benefit of surgery in that setting. This yields two important clinical problems that our approach addresses: 1) detecting disease at an early enough stage for surgery to be feasible, and 2) once diagnosed with PDAC, accurately determining which patients would or would not benefit from surgery.

Our work differentiates itself most significantly from previous work in the following aspects: 1) it combines a diverse set of non-invasive markers, 2) our panel can not only diagnose PDAC, but also improve staging accuracy; and 3) it uses machine learning approaches that are resilient against overfitting and can continue to be trained and improved in future studies. To construct our multi-analyte panel, we selected the marker CA19–9, which is routinely ordered as a clinical blood test for PDAC patients, with existing liquid biopsy approaches for measuring ccfDNA concentration(12,26); ccfDNA allele fraction of mutant KRAS(14,29), and mRNA and miRNA isolated from tumor-associated EVs. We and others have shown that the mRNA and miRNA cargo of tumor derived EVs can be readily detected in pre-clinical and clinical samples(39). In the present work, we additionally demonstrate that EV transcriptional profiling provides orthogonal diagnostic information, thus providing the rationale for adding EV-based measures to those from protein- and DNA-based markers.

In our work, and in other studies, multi-analyte panels have demonstrated several advantages compared to single markers(17,19). Individual EV biomarkers have previously demonstrated promising results for PDAC(39,40,42,43), but faced challenges when applied to patient cohorts in different institutions(44). Melo et al. reported that GPC1⁺ exosomes were informative for distinguishing PDAC patients from healthy and disease controls with an AUC=1(40). However, independent studies reported markedly different performance of GPC1⁺ EVs for PDAC diagnosis(42,44). CTCs have shown promise(10,11,45) but detecting CTCs in early stage PDAC is challenging because of their low concentration. Recent publications have also shown a benefit of combining multiple biomarkers for PDAC diagnosis, however, biomarkers in most publications tend to come from a single category, e.g., from EV cargo nucleic acids including miRNAs(46,47), mRNAs(18), DNAs(48), or from EV surface protein profiling(16). Few studies combined biomarkers from different categories: Cohen et al combined CA19–9 with circulating tumor DNA and plasma proteins(20); Madhavan et al combined EV cargo proteins and miRNAs(23), but both focused on PDAC diagnosis only. Assays that identify signatures across multiple biomarkers have the potential to be more robust for diverse patient populations and are less dependent on any single reagent than single marker assays. An additional concern could be the complexity of conducting multiple tests. To address this, we have used widely available commercial platforms for ccfDNA analysis. Potential drawbacks to a multi-analyte panel could be the requirement for multiple blood draws, a large blood volume, or multiple collection types. However, our panel can be performed utilizing only 3 mL appropriately processed EDTA preserved plasma, less than the typical yield from a standard 10mL blood collection tube. While the magnetic nanofluidic-based approach used to isolate tumor-derived EVs is not yet commercially available, it is high-throughput, robust, and inexpensive to manufacture and thus well-suited to eventual clinical adaptation.

Here, we demonstrate proof of concept for a liquid biopsy-based multi-analyte panel, using a baseline blood sample. However, there are several limitations of our study offering opportunities for future study. For the occult metastasis cohort, we included those who had metastases detected intraoperatively or had recurrence within 4 months of baseline blood draw, a cutoff determined by a sensitivity analysis of time-to-distant-failure (Fig. S5). While 4 months is far shorter than the median recurrence-free, relapse-free, or metastasis-free survivals reported in experimental and control arms of large randomized trials(35–37), this cutoff should be re-examined in a larger cohort. In addition, the biomarkers in our panel are largely tumor-derived, and the model would likely benefit by the addition of tumor extrinsic-factors, including the immune compartment. Work is underway to address these limitations in the setting of a larger cohort of PDAC patients that includes an external validation cohort, and extend the model’s utility to detection and staging of other solid tumors.

Statement of Translational Relevance.

PDAC is a highly lethal disease, partly because most cases are not diagnosed until disease is widespread. There is, therefore, an urgent need for sensitive, non-invasive diagnostics. However, even for patients with pathologically confirmed PDAC, standard of care imaging can have low sensitivity to detect early metastatic disease. This complicates disease staging and therapy selection, including curative-intent surgery. Here we describe a multi-analyte liquid biopsy to better detect and stage PDAC from a single blood sample. This approach was able to distinguish patients with PDAC from those without. Moreover, among patients with PDAC, the model could improve detection of occult metastatic disease that was imaging-negative at baseline and only discovered intraoperatively or by subsequent imaging within 4 months of baseline blood draw. Although a larger validation study is needed, this test may improve early disease detection and, when performed in addition to diagnostic imaging, patient selection for curative intent surgery.