Using probe electrospray ionization mass spectrometry and machine learning for detecting pancreatic cancer with high performance

Abstract

A rapid blood-based diagnostic modality to detect pancreatic ductal adenocarcinoma (PDAC) with high accuracy is an unmet medical need. The study aimed to validate a unique diagnosis system using Probe Electrospray Ionization Mass Spectrometry (PESI-MS) and Machine Learning to the diagnosis of PDAC. Peripheral blood samples were collected from a total of 322 consecutive PDAC patients and 265 controls with a family history of PDAC. Five µl of serum samples were analyzed using PESI-MS system. The mass spectra from each specimen were then fed into machine learning algorithms to discriminate between control and cancer cases. A total of 587 serum samples were analyzed. The sensitivity of the machine learning algorithm using PESI-MS profiles to identify PDAC is 90.8% with specificity of 91.7% (95% CI 83.9%-97.4% and 82.8%-97.7% respectively). Combined PESI-MS profiles with age and CA19-9 as predictors, the accuracy for stage 1 or 2 of PDAC is 92.9% and for stage 3 or 4 is 93% (95% CI 86.3-98.2; 87.9-97.4 respectively). The accuracy and simplicity of the PESI-MS profiles combined with machine learning provide an opportunity to detect PDAC at an early stage and must be applicable to the examination of at-risk populations.

Introduction

The outlook for pancreatic ductal adenocarcinoma (PDAC) remains dismal due to lack of diagnostics effective for early detection, which leads to the diagnosis at advanced stage. As the majority of patients presented with advanced disease the 5-year survival rates remain below 10% [1]. To improve the prognosis of PDAC will require earlier detection, ideally utilising diagnostic technique that could be employed in at risk populations such as those with diabetes [2]. Currently serum carbohydrate antigen 19-9 (CA19-9) is the most widely used diagnostic test but, due to false-positivity in patients with cirrhosis, chronic pancreatitis, cholangitis, other malignancies and false-negativity in Lewis blood-type negative patients, cannot be recommended for general screening [3]. CA19-9 is reported to discriminate between PDAC patients and healthy controls or benign pancreatic disease with a sensitivity of 78.2-80.3% and a specificity of 80.2-82.8% [4,5]. To improve survival and reduce healthcare expenditure, it is estimated that a new diagnostic method will require a minimum sensitivity of 88% at a specificity of 85% [6].

Advances in cancer diagnosis can be facilitated by the parallel development of different techniques which when conflated produce a significant improvement in diagnostic accuracy. Both mass spectrometry and artificial intelligence are now established techniques in the medical armamentarium and machine learning, an application of artificial intelligence (AI) that endows systems with the ability to automatically learn and improve from experience, has overcome some of the prejudices inherent within medicine and is proving capable of solving some of the difficulties associated with large and complex datasets. Mass spectrometry is a powerful technique for the rapid molecular diagnosis of cancer and probe electrospray ionization (PESI) analysis uses a very fine low invasive needle to achieve direct mass spectrometry. The probe needle directly collects a small amount of tissue without pretreatment [7] and detects sets of low-molecular-weight metabolites and lipids in specimens which provide important information for disease detection. For biological samples it is one of the commonly used and successful ambient ionization techniques [8,9]. PESI-MS has been shown to discriminate renal cell carcinoma, hepatocellular carcinoma and head and neck squamous cell carcinoma from surrounding normal tissue [10]. Those results support the premise that PESI-MS is a versatile and promising technique for the identification of a number of different malignancies and when combined with AI as a potential screening approach in high risk populations. Recent studies from Germany, the USA and France demonstrated that AI algorithms perform better than dermatologists at detecting skin cancer [11]. We recently demonstrated the potential of PESI-MS combined with partial least squares-logistic regression as a diagnostic system in animal studies and squamous cell carcinoma [10]. The major attraction of PESI-MS is that it is a simple, rapid and inexpensive technique which can be easily automated and requires minimal sample preparation [12]. The present study aims to demonstrate that PESI-MS combined with AI can diagnose PDCA using human serum samples, facilitating early diagnosis in the clinical setting, potentially improving patient outcomes and reducing healthcare costs.

Materials and methods

Study population

Between January 2005 and December 2017 after obtaining written informed consent and before treatment at the National Taiwan University Hospital (NTUH), peripheral blood samples were collected from a total of 322 patients (age 63.6 ± 13.0, female = 45.3%, male = 54.7%) with cytological and/or pathological confirmation of PDAC. All the patients’ demographic data, including age, gender, serological studies, image studies, survival data, and clinical presentation were collected. Peripheral blood was collected from 265 high risk individuals (HRI) age 46.8 ± 14.8 (female 59.6%, male 40.4%) with a family history of PDAC, participating in a pancreatic cancer screening program at the NTUH between January 2005 and December 2015 [13]. Data from control subjects included a detailed family history, a full physical examination, blood sampling and magnetic resonance imaging (MRI). All of the control subjects were followed up for at least 2 years and confirmed free of pancreatic malignancy. Serum collected from PDAC patients and controls was stored at -80°C until PESI-MS analysis. The study was reviewed and approved by the Institutional Review Board of NTUH (No. 201301048RIND) and University of Yamanashi (No. 645). Table 1 shows demographic data of the study subjects.

Table 1.

The demographic data of 322 patiensts with pancreatic ductal adenocarcinoma (PDAC) and 265 high risk controls (HRCs)

	PDAC (n = 322)	HRC (n = 265)
Age (years)	63.675 (24.4-91.7)	46.847 (17.7-88.2)
Gender
Female	146 (45.3%)	158 (59.6%)
Male	176 (54.7%)	107 (40.4%)
Body-mass index	22.04 (13.3-42.7)	23.24 (15.6-37.7)
Pancreatic cancer location
Head (head/neck/uncinate)	150 (46.6%)
Body	111 (34.5%)
Tail	61 (18.9%)
Stage
I	11 (3.4%)
II	66 (20.5%)
III	60 (18.6%)
IV	185 (57.5%)
CA-19-9	4558.46 (0.5-66526.69)	15.21 (0-269.4)
CEA	63.309 (0.1-5783.59)	1.734 (0-9.37)
OS (month)	11.35 (0.07-94.22)

PESI-MS analysis

A PESI-MS system (installation of PESI on a single quadrupole mass spectrometer compartment of LCMS-2020; Shimadzu, Kyoto, Japan) was used in the study. Five µl of serums were added to 95 µl of 50% ethanol in a 1.5-ml tube and vortexed for 2 min. Samples were centrifuged at 150,000×g for 1 minute, and the resultant supernatant collected. Sample analysis by PESI-MS was performed as described previously [14]. All analyses were performed in both positive and negative ion modes for each specimen. The mass spectra from each specimen was generated using LabSolutions software (Ver. 5.82 SP1; Shimadzu). PESI is a discontinuous ionization method and differs from other ESI methods. The measurement was performed for 2 minutes with a time window of 10 sec that demonstrates a stable continuous ionization at maximal intensity (ca.50,000). This window was spread out into 10 consecutive sets of spectral data. The m/z of our analysis ranges from 10 to 2,000, which is divided into 1990 bins, each of which corresponds to a unitary mass by taking account of the mass spectral accuracy of Shimadzu PESI-MS 2020. Each bin contains information on m/z as well as the peak intensity and they are used to construct the database.

Data processing and machine learning

To demonstrate the suitability of PESI-MS to detect significant biochemical differences between PDAC and control subjects, a specialized machine learning (ML) algorithm was trained and tested on each PESI-MS ion mode dataset. The positive ion mode dataset contained 583 subjects (318 PDAC and 265 controls) and the negative ion mode 587 subjects (322 PDAC and 265 controls). The first objective of the algorithm was to accurately distinguish between cancer and control samples. The candidate models were trained and tested using 1,000 independent repetitions of a bootstrap cross-validation process (the average over the 1,000 independent models is reported). Bootstrap is a re-sampling technique that can be applied as cross-validation to estimate the performance of a model. The method randomly splits the data into training and test sets (see appendix for more information). An independent and automated ML classification model is then trained on the training partition of the data and tested on the test partition. This whole process including the random splitting of the data plus model training and validation, is independently repeated 1,000 times. The performance of each model is recorded and the average classification results over the 1,000 repetitions is calculated and reported in the results tables.

The ML algorithm used to build the classification model was a support vector machines (SVM) [15] and all algorithms have been implemented in R Statistical Package. During the model selection phase of the data analysis, other ML algori-thms, such as random forest [16], partial least squares-discriminant analysis (PLS-DA) [17] and convolutional neural networks (CNN) [18] have been tested but SVM produced the best overall performance on the datasets available. In particular, the CNN models seemed to be overkill for binary classification (see results in suplementary appendix Figure S1 and Table S1). SVM are supervised learning algorithms used for regression analysis and classification. By using different kernels, SVM can easily perform non-linear classification. SVM results reported here are all based on a radial basis kernel as it produced the best performance results on the present dataset. As the number of subject samples is large enough and the data do not present too many missing values, the different partitions of the data used for analysis have always been selected in such a way that there are no subjects with missing values (listwise deletion) [19].

For each ion mode data (positive and negative) two different types of models were extensively trained and tested. The first model type used only the PESI-MS peaks as predictors (independent variables) to discriminate cancer and control cases. The objective is to show that the information from PESI-MS analysis of peripheral blood samples alone can discriminate between healthy controls and all 4 stages of PDAC at least as well as the tumor marker CA19-9. The second used the PESI-MS peaks (as model type 1) plus age and CA19-9 values as predictors. The sensitivity, specificity, and accuracy of each model to discriminate PDAC from controls were calculated. Figure 1 depicts the workflow of sample collection, sample analysis and data analysis processes used in this study.

Figure 1.

Workflow of the data collection, sample analysis and data analysis processes.

Sample exclusion criteria

Serum samples from all 587 subjects were analyzed via PESI-MS in both negative (n = 587 samples) and positive (n = 583 samples) ion modes - 4 samples (spectra) from the positive ion mode were considered inconsistent and excluded from the analysis. When adding CA19-9 as predictors for diagnosis, samples with missing CA19-9 values were excluded. The numbers then become 515 samples in negative ion mode (i.e., 587 excluding 20 PDAC & 52 control samples) and 512 samples in positive ion mode (i.e., 583 excluding 19 PDAC & 52 control samples). To form the combined positive and negative ion mode data, we only considered samples with CA19-9 values recorded in both ion modes (n = 512).

Results

High-performance method for diagnosing PDAC

The data analysis based on this cohort shows that PESI-MS coupled with SVM classifier identified over 90% of PDAC cases as well as over 90% of the non-cancer cases with a 91.7% of sensitivity (95% CI 91.6-91.8) and a 90.4% of specificity (95% CI 90.2-90.6) respectively based only on the mass spectra from PESI-MS (Table 2 “spectra only as predictors”, negative ion mode). The performance of this diagnostic approach exceeds the expectation required for novel methods for PDAC screening and represents a valuable tool for the screening of PDAC.

Table 2.

Control vs Cancer: average automated cancer diagnosis classification results obtained over 1,000 independent models built using bootstrap cross-validation

Ion mode		SPECTRA ONLY AS PREDICTORS		SPECTRA + AGE + CA19-9 AS PREDICTORS
		Negative	Positive	Negative	Positive
Control		265	265	213	213
Cancer		322	318	302	299
Total		587	583	515	512
CLASSIFICATION	Accuracy % (95% C.I.)	91.1 (91, 91.2)	89.1 (89, 89.2)	92.8 (92.7, 92.9)	92.7 (92.6, 92.8)
	Sensitivity % (95% C.I.)	91.7 (91.6, 91.8)	91.2 (91, 91.4)	95.1 (95, 95.2)	95.9 (95.8, 96)
	Specificity % (95% C.I.)	90.4 (90.2, 90.6)	86.7 (86.4, 87)	89 (88.8, 89.2)	87.3 (87, 87.6)

In order to improve the accuracy, we have added several parameters as predictors for diagnosis (Table 2). In this mode, we considered the PESI-MS spectra and the factors “age” and “CA19-9” level as predictors. Inclusion of these two parameters drastically improved the sensitivity to 95.1% (95% C.I: 95.0-95.2) at the cost of a decreased specificity to 89% (95% C.I: 88.8-89.2) (Table 2).

Discriminating between the controls and different stages of PDAC

All the results presented in this section are based only on PESI-MS peaks as predictors (age and CA19-9 excluded). We tested if our system is also able to discern the changes in metabolites according to the cancer stage. A partial least squares-discriminant analysis (PLS-DA) algorithm (Figure 2A) was employed here to discriminate between (a) “Control”, (b) “Earlier PDAC stages” (stages 1 + 2) and (c) “Advanced PDAC stages” (stages 3 + 4) based on PES-MS spectra only as predictors. The dashed lines drawn around the cluster groups represent the 95% confidence interval for the respective cluster. PLS-DA was used because it also computes scores vectors that can be used to display a scatter plot of the general clustering of samples. The model was built using the combination of both negative and positive ion modes to compensate each other for achieving more accurate discrimination. Based on the biochemical information captured by PESI-MS, the subjects spontaneously grouped into control, early and advanced stage PDAC. At a glance, the control (black triangles) and cancer samples (blue and red circles) seem to form two well defined and clearly distinct clusters. The results suggest that PESI-MS is able to detect subtle alterations in spectra due to progression of PDAC. Comparing the separation of earlier (blue circles) with advanced (red circles) cancer stages, the scatter plot also shows two distinct clusters but with a slightly higher degree of overlap. This implies that different levels of metabolic changes between earlier and advanced cancer stages have also been detected by PESI-MS.

Figure 2.

A. PLS-DA scores plot for the combined data (- and + ion modes). Separation between samples from healthy controls and stage I + II and stage II + IV of PDAC. The model was based on the PESI-MS spectra only as predictors. The numbers between parentheses represent % of variance from the data explained by each principal discriminant. The dashed lines drawn around the clusters represent 95% CI for the respective cluster. B. SVM discrimination results (%) for healthy controls vs individual stages of PDAC with 95% CI between parentheses.

SVM was also applied to discriminate between healthy control patients and individual stages of PDAC (Figure 2B). The results show that, on average, 91.4% of the PDAC stage 1 patients were identified and that the higher the PDAC stage the higher the sensitivity of the model; over 92% to 97.6% for stages 2, 3 and 4. Table S2 in Supplementary Information shows results of SVM models discriminating between control and PDAC stages.

Combining information from both mass spectrometry ion modes

As MS ion mode is likely to capture slightly different molecular signatures from samples and the data analysis of each dataset presented slightly different classification results, a combination of both datasets should be beneficial for an automated cancer diagnostic process. Therefore, PESI-MS data from both ion modes were combined and a new specialized ML algorithm, based on SVM, was individually trained and tested to diagnose PDAC. Table 3A presents the average classification results obtained for the automated cancer diagnosis over 1,000 independent bootstrap cross-validations. The results shown in Table 3A suggest that the combination of both ion mode datasets is beneficial and the information gained improves the performance of the models. On this combined dataset, PESI-MS coupled with SVM detected with cancer cases 90.8% of sensitivity and 91.7% of specificity (95% C.I. 83.9-97.4 and 82.8-97.7) respectively.

Table 3.

A. Prediction Combining Decisions from Negative and Positive Ion Modes: average automated classification results obtained over 1,000 independent bootstrap cross-validations. B. Cancer Stage Prediction Combining Decisions from Negative and Positive Ion Modes: average automated classification results obtained over 1,000 independent bootstrap cross-validations

A.		COMBINED SPECTRA (Negative + Positive ion modes) ONLY AS PREDICTORS
Control		265
Cancer		318
Total		583
CLASSIFICATION	Accuracy % (95% C.I.)	91.2 (86.7, 95.8)
	Sensitivity % (95% C.I.)	90.8 (83.9, 97.4)
	Specificity % (95% C.I.)	91.7 (82.8, 97.7)
B.		COMBINED SPECTRA (Negative + Positive ion modes) + AGE + CA19-9 AS PREDICTORS
		*Control vs Earlier PDAC stages	Control vs Advanced PDAC stages
Control		213	213
Cancer		68	231
Total		281	444
CLASSIFICATION	Accuracy % (95% C.I.)	92.9 (86.3, 98.2)	93 (87.9, 97.4)
	Sensitivity % (95% C.I.)	81.2 (57.6, 95.4)	92.7 (84.2, 100)
	Specificity % (95% C.I.)	96.8 (92.5, 100)	93.4 (86.2, 100)

^*Cohen’s kappa coefficient for the “Control vs Earlier PDAC stages” model is k = 0.8.

Healthy controls vs cancer stages on the combined mass spectrometry ion modes

Surgical resection is the only potentially curative treatment for PDAC [20]. Based on the TNM cancer staging system developed by the American Joint Committee on Cancer, pancreatic cancers detected in stages 1 and sometimes 2 are potentially operable. The results in Table 3B show that PESI-MS coupled with machine learning distinguished between healthy controls and subjects with earlier stage of PDAC (stages 1 or 2) with sensitivity of 81.2% and specificity of 96.8% (95% CI: 57.6-95.4 and 92.5-100) respectively. Although our method does not formally stage the disease, it is clearly able to detect differences between early and advanced tumors. The results for the discrimination between healthy controls and subjects with advanced stage PDAC (stages 3 or 4) are also reported in Table 3B.

Discussion

In this study we have successfully developed a novel screening system for pancreatic cancer by simultaneously attaining high sensitivity and specificity that exceeds the criteria required for PDAC diagnosis and screening. This was realized by combination of unique ionization method for mass spectrometry (PESI-MS) and machine learning that utilizes all the spectral data obtained by PESI-MS. Since our system does not extract the conspicuous spectral peaks from the datasets, judgment of cancer is based on the broad collections of spectra even if there are dozens of peaks relatively higher contributing to the diagnosis. Taking the diagnostic accuracy up to 91.2%, this method must open a new avenue in the diagnosis of PDAC ever attained by other methods. Another important point that has to be paraphrased here is the inexplicability of exact pathophysiological mechanism for drawing a diagnosis. Since this method relies on the collections of spectral peaks telling us the diagnosis possibly due to changes in metabolism, we cannot definitely narrow down the molecules responsible for explaining the molecular mechanisms underlying the cancer. While another important direction is to identify the molecules for diagnosis the PDAC, the focus of this study is validating the performance of new diagnostic technique. As machine learning does not necessarily require us to annotate the molecules underlying biologic pathways or modifiable risk factors that are associated with PDAC [21]. We can employ simple and cost-effective mass spectrometer whose resolution is not high enough for identifying the molecules.

Data-driven machine learning produces consistent results if significant “learning” of biochemical changes is achieved by a large cohort of PDAC and healthy controls (N = 587 in this study). PDAC has been shown to take up to a decade following the initial mutation that gives opportunity for earlier diagnosis [22]. Furthermore, 2-3 year window is open to detect PDAC’s at stage 0 and 1. Although we do not register the patients at stage 0 or carcinoma in situ in our cohort, this technique should be applicable at these stages, considering this method can achieve an accuracy of 92.2% for the detection of early stages of PDAC including stage 1. Therefore, our system is promising in PDAC diagnosis that will revolutionize the routine of pancreatic diagnosis.

Metabolomics, including lipidomics, is a feasible way to identify metabolites responsible for PDAC detection [23]. Mayers and co-workers had shown that a metabolic biomarker signature with 9 plasma metabolites plus CA19-9 differentiated PDAC from chronic pancreatitis with diagnostic accuracy of 90.6% (95% CI 84.9% to 94.6%) [24]. In this study although PESI-MS principally measures metabolites and lipids we also identified the top 30 discriminating factors, in negative and positive modes in the supplementary appendix (Figures S2 and S3). Those peaks include lipid profiles, phospholipids, sphingolipids and cholesterol sulfates that have also been identified in other studies [25]. Our system does not deviate from molecular prediction method while it focuses on the fingerprint of responsible molecules.

The serum samples analyzed by PESI-MS do not require any pretreatments such as desalting, fractionation or enrichment. In addition, PESI-MS does not require large amount of sample for diagnosis, even sub-picoliters of samples are sufficient for analysis. In addition to these superior attributes, the advantages of our system lie in low invasiveness, robustness and rapidity of measurements, and comprehensive resolution of substances in the samples.

There are limitations associated with the study mainly related to its retrospective design. It must be acknowledged that given a current accuracy of 92.9% there is still potential for further improvement of machine learning in PDAC diagnosis. Because our study population was a unique cohort from Taiwan, in other populations specific optimization will be required. The new diagnostic technique needs validation in several independent prospective world-wide cohorts.

Improving the outcome of PDAC will require a method which enables early and accurate diagnosis. Although the incidence of pancreatic cancer is too low to justify whole population screening, the accuracy of 92.9% for earlier PDAC stage combined PESI-MS and machine learning potentially justifies the evaluation of high-risk groups. Inherited pancreatic cancer syndromes and familial pancreatic cancer are the logical targets but other asymptomatic groups could be considered [26,27]. The most obvious of these is new-onset diabetes and a study from the Mayo Clinic demonstrated that diabetes has a 40% prevalence in PDAC and is frequently new-onset [28].

In conclusion, AI and machine learning have begun to enter the field of cancer diagnostics. Our study clearly demonstrates the feasibility of developing a diagnostic test with a comprehensive metabolite profiling MS platform plus machine learning that can detect PDAC with greater accuracy than has previously been achieved with either conventional tumor markers or a metabolic signature. There is a need for prospective, real-world clinical evaluation of the diagnostic approach rather than only retrospective assessment of performance. The next stage is a large-scale diagnostic accuracy study among the at-risk populations where the test is intended to be employed.

Disclosure of conflict of interest

None.