Abstract
Purpose:
Early diagnosis of cancer is critical for improving patient outcomes, but cancers may be hard to diagnose if patients present with nonspecific signs and symptoms. We have previously shown that nuclear magnetic resonance (NMR) metabolomics analysis can detect cancer in animal models and distinguish between differing metastatic disease burdens. Here, we hypothesized that biomarkers within the blood metabolome could identify cancers within a mixed population of patients referred from primary care with nonspecific symptoms, the so-called “low-risk, but not no-risk” patient group, as well as distinguishing between those with and without metastatic disease.
Experimental Design:
Patients (n = 304 comprising modeling, n = 192, and test, n = 92) were recruited from 2017 to 2018 from the Oxfordshire Suspected CANcer (SCAN) pathway, a multidisciplinary diagnostic center (MDC) referral pathway for patients with nonspecific signs and symptoms. Blood was collected and analyzed by NMR metabolomics. Orthogonal partial least squares discriminatory analysis (OPLS-DA) models separated patients, based upon diagnoses received from the MDC assessment, within 62 days of initial appointment.
Results:
Area under the ROC curve for identifying patients with solid tumors in the independent test set was 0.83 [95% confidence interval (CI): 0.72–0.95]. Maximum sensitivity and specificity were 94% (95% CI: 73–99) and 82% (95% CI: 75–87), respectively. We could also identify patients with metastatic disease in the cohort of patients with cancer with sensitivity and specificity of 94% (95% CI: 72–99) and 88% (95% CI: 53–98), respectively.
Conclusions:
For a mixed group of patients referred from primary care with nonspecific signs and symptoms, NMR-based metabolomics can assist their diagnosis, and may differentiate both those with malignancies and those with and without metastatic disease.
Translational Relevance.
The accurate and timely diagnosis of cancer is critical for reducing the morbidity and mortality associated with late-stage cancer diagnoses, but early diagnosis can be difficult to achieve if patients present with nonspecific symptoms that may be due to cancer, the so-called “low-risk, but not no-risk” patient group. This often presents a challenge to physicians, and tests that can help aid diagnosis are urgently needed. Here we have shown that biofluid metabolomics analysis of peripheral blood from a cohort of such patients can detect cancers with high sensitivity and specificity. Biofluid metabolomics is a cheap and easy-to-administer test, requiring only a small blood sample, and has the potential to act as an effective triage tool to prioritize patients with nonspecific symptoms for more invasive investigations or to rule out cancer in patients with nonspecific symptoms.
Introduction
There is a strong correlation between earlier cancer diagnoses and improved outcomes (1). If cancers are detected early, they are more easily treated and have better outcomes as they are likely to be at an early stage, with less nodal involvement, and are less likely to have metastasized (2, 3).
Current cancer referral pathways, such as the “2-week wait” pathways in the UK, are designed predominantly around organ-specific symptoms, such as hemoptysis or hematuria, or clinically palpable abnormalities, such as breast lumps or an abdominal mass. This process is ideal for cancers that present with specific symptoms but is problematic when patients present with nonspecific symptoms, such as fatigue. If there are no organ-specific symptoms or signs it may be difficult to know to which specialist the patient should be referred. Time and resources may be wasted, leading to delays in diagnoses, in turn leading to increases in the proportion of patients presenting with advanced tumors (4).
To help identify cancers in patients with nonspecific symptoms, the Suspected CANcer (SCAN) pathway was established in Oxfordshire, UK, as part of the Accelerate, Coordinate, Evaluate (ACE) Wave 2 initiative, which developed multidisciplinary diagnostic center (MDC) based pathways for these patients (5). SCAN is a referral pathway from primary care to the hospital for patients with nonspecific symptoms such as fatigue and weight loss. All patients referred to SCAN undergo a contrast-enhanced CT of the chest, abdomen, and pelvis, blood biochemistry and hematology analysis. If an obvious pathology is not detected on initial investigations, consultant physician review of the patient at an MDC is used in an attempt to reach a diagnosis.
Biofluid metabolomics is potentially an alternative for this patient group. Biofluid metabolomics is a conceptually simple and inexpensive technique that relies upon the simultaneous determination of the levels of small-molecule constituents within a biological sample, which are analyzed to establish disease-specific patterns. These metabolomic profiles are produced by an analytical technique such as nuclear magnetic resonance (NMR) spectroscopy or mass spectrometry. We have previously demonstrated that we can use NMR-based biofluid metabolomic analysis to sensitively and specifically detect the presence of brain metastases in a mouse model of breast cancer (6). Moreover, our data indicate that there is a distinct pattern of metabolites that allows discrimination between brain and systemic tumor burdens (6). Other groups have further been able to identify different types of cancers using NMR metabolomics, including lung (7), colorectal (8), pancreatic (9), liver (10, 11), breast (12), and bladder (13) cancers.
Here, we hypothesized that biofluid metabolomics would (i) be able to distinguish patients with cancer from those without cancer, in a mixed cohort with nonspecific symptoms, and (ii) be able to distinguish patients with and without metastases. We have tested these hypotheses using NMR analysis of blood samples from patients recruited to the SCAN pathway.
Materials and Methods
Patient recruitment
Patients (n = 304) were recruited from the Oxfordshire (UK) SCAN pathway (5) between May 2017 and August 2018. Patients were recruited to the SCAN pathway from the general Oxfordshire population if they met three essential referral criteria: (i) there was no other “2-week wait” cancer-specific referral pathway suitable for the patient; (ii) the patient was ≥40 years old; (iii) the patient had at least one of the following: unexplained weight loss, severe unexplained fatigue, persistent nausea or appetite loss, new atypical pain, an unexplained laboratory test finding that did not suggest a specific diagnosis, a primary care physician clinical suspicion of cancer or serious disease (“gut feeling”; ref. 5). All patients gave written informed consent prior to study admission. The study was conducted in accordance with the UK policy framework for health and social care research and was approved by the Oxford Radcliffe Biobank (ORB) research tissue bank ethics committee (reference 19/SC/0173). Patients were included in the study if (i) both a blood sample and CT scan were available, (ii) a confirmed cancer/noncancer diagnosis was reached during the SCAN follow-up, and (iii) no contaminant peaks were observed in the NMR spectrum.
Patient pipeline and biofluid sample collection
Patients were referred from primary care to the SCAN pathway without diagnoses known, ensuring random sampling from the population. Referred patients underwent a contrast-enhanced thorax, abdomen, and pelvis CT, a full blood count and blood-based biochemical analyses, after a minimum fasting period of 2 hours. If a significant pathology was not detected by initial investigations, each patient then underwent medical consultation in a MDC. The MDC recommended further investigations if indicated, to reach a definitive diagnosis (5). If a patient received a cancer diagnosis within 62 days (2 months) of initial assessment they were considered to have had their cancer at the time of initial investigation (14). Patients were followed up for 12 months in total after SCAN assessment, both to record the diagnosis from the SCAN pathway and to determine whether any cancers developed that were not diagnosed within the first 62 days.
Blood for metabolomics was collected into lithium-heparin tubes immediately prior to CT imaging and was left to stand at room temperature before separation by centrifugation (2,200 × g, 10 minutes). Plasma was immediately separated, aliquoted, and stored at −80°C.
NMR acquisition
NMR metabolomics analysis of plasma was carried out as previously described (15, 16). Plasma samples from patients (n = 299, 150 μL each) were defrosted on ice and mixed with NMR buffer (450 μL, 70 mmol/L sodium phosphate at pH7.4 in D2O). Samples were clarified by centrifugation (16,000 × g, 3 minutes) to remove particulate matter before transferring to a 5-mm NMR tube.
All NMR spectra were acquired using a 700MHz Bruker AVIII spectrometer operating at 16.4T equipped with a 1H (13C/15N) TCI cryoprobe. Sample temperature was stable at 310K. 1H NMR spectra were acquired using both a 1D NOESY presaturation scheme with a 2-second presaturation for attenuation of the water resonance, as well as a Carr-Purcell-Meiboom-Gill (CPMG) spin-echo sequence to suppress broad signals arising from large molecular weight blood components (40-millisecond total effective filter time, 32 data collections, acquisition time of 1.5 seconds, relaxation delay of 2 seconds, fixed receiver gain; ref. 15). 2D 1H-1H total correlation spectroscopy (TOCSY) spectra were acquired on at least one sample in each classification to assign the metabolites. Assignments were confirmed by reference to literature values (17, 18) and online databases (19, 20), with spiking experiments. For quality control, pooled samples were spread throughout the run to monitor technical variation. Approximately 70 soluble metabolites, including a range of lipoprotein species, amino acids, carbohydrates, and ketone bodies, were detected, as previously described (21).
NMR data preprocessing
All spectra were phased, baseline corrected, and referenced to lactate at δ = 1.33 ppm, followed by visual inspection for errors, spectral distortion, or contamination. Only CPMG spectra were used for statistical analyses. The region 0.20 to 9.68 ppm, excluding the region covering the residual water peak, was divided into 0.01-ppm width ‘buckets’, integrated, and Pareto scaled (more details in Supplementary Methods). Mean NMR spectra for groups of patients were prepared by summing individual processed spectra from a patient group, then dividing the summed spectrum by the number of patients. Difference spectrum is the mean solid tumor spectrum minus the mean noncancer spectrum.
Statistical analysis
Prior to metabolomics analyses, patients were randomized into a modeling set (two thirds of patients, n = 192) and an independent test set (one third of patients, n = 92), based on referral order. The independent test set was reserved for determining the ability of the models to classify new patients.
NMR bucket integrals were imported into R (R Foundation for Statistical Computing, Vienna, Austria; RRID:SCR_001905; ref. 22). All multivariate statistical analysis was conducted using in-house R scripts and the ropls package (RRID:SCR_016888; ref. 23). Orthogonal partial least squares discriminatory analysis (OPLS-DA) was used to generate diagnostic mathematical models to classify patients in the modeling set employing the nonlinear iterative partial least squares (NIPALS) algorithm. The number of orthogonal components was computed using a seven-fold internal cross-validation to identify the number of components to produce the optimal Q2 up to a maximum of nine. The independent test set was not used at this stage. Within the modeling set, the quality of classification was assessed using a 10-fold external cross-validation scheme with 1,000 repetitions in total (to correct for unequal class sizes). This validation scheme involves multiple iterations of splitting the data into training and external-validation sets. The training data are used to estimate the model parameters and learn the underlying discriminatory patterns between the groups under consideration, whereas the external-validation sets are employed to assess the accuracy and generalizability of the trained models. We quantified the response of the ensemble of models by calculating the accuracy, sensitivity, and specificity of each model from the predicted classifications of the external-validation sets (i.e., samples that were not used in model building). It is important to appreciate that the classifier (OPLS-DA) was blinded to the external-validation set during the process of model training. These values were compared with those of a null distribution (obtained from randomly permuting the classifications) using the two-sided Kolmogorov–Smirnov test (significant if P value < 0.05). Discriminators were identified by calculating the variable importance of projection (VIP) score. Metabolites with a VIP > 2 were considered significant in the multivariate modeling. This validation scheme tends to avoid overfitting and helps assess the generalizability of the model to previously unseen datasets. For an exhaustive discussion on validation see ref. 24. The generalizability of these models was further confirmed using the independent test set which was left out of both model training and validation.
Following this exhaustive validation, receiver operator characteristic (ROC) curves were constructed to determine the optimal diagnostic threshold within the OPLS-DA model. The point closest to the top left corner of the ROC curve was used to determine the optimal overall sensitivity, specificity, and balanced accuracy [ (sensitivity + specificity)/2]. F1 values were calculated from to provide an overall assessment of accuracy suitable for models with uneven group sizes.
This classification threshold was then used with the independent test set (i.e., the one third of patients reserved at the beginning of the study, unused until now). Classifications were made for each patient in the independent test set and 2 × 2 contingency tables constructed to determine efficacy of the models for predicting the identity of unknown samples. Fisher exact test was used to determine if this classification was significantly better than chance. Confidence intervals(CI) for classification metrics were calculated using the Wilson score interval (details in Supplementary Methods). Finally, patients who received a changed diagnosis within the first year were assessed to see if the model predicted them as having cancer.
Comparisons of ROC curves and male:female sex ratios were carried out by calculation of the Z-statistic for each pair of curves or ratio and comparing with the expected normal distributions (details in Supplementary Methods). After multivariate modeling, univariate statistical analyses were conducted by summing integrals for each metabolite resonance identified as significant in the multivariate models. Differences between means were calculated using the Student t test. Correlation analyses were carried out using Pearson's correlation.
Data availability statement
The data generated in this study are available upon request from the corresponding author.
Results
Patient demographics and NMR acquisitions
In total, 304 patients were recruited from the SCAN pathway during the study period. Eleven patients (3.6%) were excluded from analyses owing to missing biofluid samples, lack of CT scan, loss to follow-up, NMR spectrum contamination, or lack of confirmed cancer/noncancer diagnosis. A flow chart of patient recruitment is given in Fig. 1.
The total numbers of patients with cancer diagnosed by the SCAN pathway during the study period was 27 of 294 (9.2%). SCAN was more successful at detecting new cancer diagnoses than existing organ-specific urgent cancer referral pathways (6.6% in 2019–20; ref. 25). Solid tumors were diagnosed in 24 of the 294 patients (8.2%). Hematologic malignancies were diagnosed in the remaining 3 patients, but these were removed from modeling owing to both the low number and alternate diagnosis strategies available for these diseases. The most common solid tumors diagnosed were large bowel (n = 8) and lung (n = 5). The remaining cancers were found in numerous organs and are listed in Supplementary Table S1. Within the population that received solid tumor diagnoses, 8 patients (33%) had nonmetastatic cancers, and 16 patients (66%) had metastatic cancer. Seven patients who received a noncancer diagnosis from the MDC assessment received a cancer diagnosis within 1 year. Of these, 5 received solid tumor diagnoses and 2 received hematologic malignancy diagnoses.
The mean patient age was 68 years. Patients receiving solid tumor diagnoses were older (73 ± 10 years) than unwell patients that received a noncancer diagnosis (68 ± 12 years, P < 0.05). No other subgroup analysis showed significant age differences. More women than men were recruited (P < 0.001), but no difference was evident in the distribution of men and women between the model-building and test sets. The mean patient body mass index (BMI) was 26.0 ± 6.2 kg/m2; and no differences in BMI were found between different patient groups. Full patient demographic details are summarized in Table 1 and a visual representation of BMI across the patient population is included in Supplementary Fig. S1. The most common reasons for patient referral were weight loss (64%), general practitioner gut feeling (63%), unexplained laboratory results (37%), fatigue (29%), nonspecific pain (28%), and nausea/appetite loss (27%). On average, patients presented with symptoms in 2 ± 1 of these six categories.
Table 1.
Variables | Total | Solid tumor diagnoses | Noncancer diagnoses | Modeling set | Independent test set |
---|---|---|---|---|---|
Number of patients | 284 (100%) | 24 (8%) | 260 (92%) | 192 (68%) | 92 (32%) |
Age | |||||
Mean ± SD | 68 ± 12 | 73 ± 10a | 68 ± 12 | 69 ± 11 | 67 ± 13 |
(Minimum, maximum) | (40, 93) | (53, 93) | (40, 90) | (40, 90) | (44, 93) |
BMI (kg/m2) | |||||
Mean ± SD | 26.0 ± 6.2 | 28.1 ± 4.6 | 25.8 ± 6.3 | 25.8 ± 6.3 | 26.3 ± 6.0 |
(Minimum, maximum) | (13.2, 57.1) | (18.3, 38.4) | (13.2, 57.1) | (13.2, 57.1) | (15.4, 45.7) |
Sex | |||||
Male | 123 (43%) | 10 (42%) | 113 (43%) | 78 (41%) | 45 (49%) |
Female | 161 (57%)b | 14 (58%) | 147 (57%)b | 114 (59%)b | 47 (51%) |
Note: Data are presented as n (%), or as mean ± SD with range (minimum, maximum) for age and BMI (BMI data were available for 278 out of the 284 patients). Differences between groups were compared using Student t test for age and calculation of Z-statistic for sex.
a P < 0.05 compared with noncancer diagnoses.
b P < 0.001, male:female ratio differs from 1:1.
Plasma metabolomics identifies cancer in individuals with nonspecific symptoms with high sensitivity and specificity
Mean NMR spectra from unwell patients with nonspecific symptoms receiving solid tumor and noncancer diagnoses are shown in Fig. 2, along with the difference spectrum between the two groups. OPLS-DA was able to separate unwell patients with solid tumor diagnoses from unwell patients with noncancer diagnoses (Fig. 3A and B) using the plasma metabolome with an AUC of 0.91 (95% CI: 0.83–0.99; P < 0.001; cut-off value Fig. 3B; ROC curve Fig. 3C). Cross-validation and permutation testing confirmed that the model significantly out-performed random chance (P < 0.001; Supplementary Fig. S2) confirming that the model performance is not a result of overfitting and that the developed model should be robust to novel data. Indeed, the model was able to identify cancer in the modeling set with sensitivity of 94% (95% CI: 73–99), a specificity of 82% (95% CI: 75–87), a negative predictive value of 99% (95% CI: 96–100), and a balanced accuracy of 88% (95% CI: 85–91). Two by two contingency tables are given in Supplementary Table S2 and complete summaries of model quality metrics at different classification thresholds are presented in Supplementary Fig. S3.
Inspection of model VIPs revealed that patients with solid tumors had significantly decreased plasma lipoprotein levels [–CH3, (–CH2–)n, =CH–CH2–CH2–, and saturated lipid resonances] along with an increase in plasma glucose, N-acetylglucosamine (NAC1), and threonine concentrations compared with unwell patients without cancer in the nonspecific symptom cohort (Fig. 4A; individual metabolite graphs with univariate analyses are shown in Supplementary Fig. S4). Correlation analyses, in the unwell with no cancer cohort, between age and each identified metabolite found no correlation with more than 5% of variation explained by age. Maximum R2 for any correlation was 0.05 for the mobile −CH3 chylomicrons/very low-density lipoprotein (CM/VLDL) resonance (r = −0.23).
Robustness of validated metabolomics models to independent test data
Following identification of the optimal diagnostic cutoff for the validated “solid tumor versus noncancer” model, classification performance was investigated on the independent test set.
Overall area under the ROC curve for the independent samples was 0.83 (95% CI: 0.72–0.95), which is not significantly different to that of the modeling set (P > 0.05). Although the decreased number of patients in the independent set can increase error, this area under the ROC translated to a sensitivity of 71% (five out of seven cancers detected), a specificity of 70% (60 out of 85 noncancer patients identified as such), and a negative predictive value of 97% (60 noncancer patients of 62 predicted noncancer). These classification results were significantly better than chance (Fisher exact test, P < 0.05; Supplementary Table S2).
Plasma metabolite profile of patients with metastatic cancers are distinct from patients with nonmetastatic cancer
Within the population of 24 patients who had solid tumors, OPLS-DA models were also able to separate patients with a metastatic cancer from those with nonmetastatic cancer with an AUC of 0.91 (95% CI: 0.77–1.00; Fig. 3C and D). This model also had sensitivity, specificity, and overall accuracy significantly better than random chance (P < 0.05; Supplementary Fig. S5). Model sensitivity was 94% (95% CI: 72–99), specificity was 88% (95% CI: 53–98), and balanced accuracy was 91% (95% CI: 83–98). Two by two contingency tables are given in Supplementary Table S2 and complete summaries of model quality metrics at different classification thresholds are presented in Supplementary Fig. S6.
Inspection of the VIPs highlighted as significant by the multivariate model indicated that patients with metastatic cancer had elevated levels of threonine, β-hydroxybutyrate, NAC1, and N-acetylneuraminic acid (NAC2) resonances along with modest decreases in lipoprotein resonances (–CH3 CM/VLDL and unsaturated lipid) relative to patients with nonmetastatic cancers (changes and univariate statistics shown in Fig. 4B). A summary of the directions of change of key metabolites in the metastatic and nonmetastatic cancer cohorts relative to the unwell noncancer patient cohort is given in Fig. 4C.
Identification of cancer before conventional imaging; two case studies
Next, we investigated whether the metabolomics model was able to identify early-stage cancers before conventional imaging. The patient cohort was followed up for 1 year to determine whether any new cancer diagnoses were made in patients who had received noncancer diagnoses in the initial 62-day diagnostic window. From the 267 initial noncancer diagnoses, 5 patients developed solid tumors and 2 developed hematological malignancies (identified at a mean of 7 ± 2 months after initial assessment). Of the 5 patients who developed solid tumors, 2 patients were identified as having cancer by metabolomics assessment of baseline blood samples (taken at the point of CT scan and MDC assessment; Fig. 5). Full prediction scatter plots are given in Supplementary Fig. S7.
In the first case, a patient in their 60s was referred to the SCAN pathway because of abdominal pain, anemia, and a raised C-reactive protein level. The radiology identified a nonspecific lung nodule, but metabolomics predicted cancer at baseline. The patient entered routine CT surveillance for the lung nodule. Suspicious features on the surveillance scans prompted a tissue biopsy which confirmed adenocarcinoma. The patient subsequently had a lobectomy 9 months after the initial MDC assessment.
In the second case, a patient in their 80s was referred to the SCAN pathway because of unexplained weight loss. The patient had a past medical history of gastric cancer, treated with distal gastrectomy 11 years prior to referral to SCAN. There was no evidence of recurrence on either the CT imaging or any of the other diagnostic tests conducted as part of the MDC assessment and the patient received a noncancer diagnosis. Recurrence of the gastric cancer was subsequently identified by endoscopy 7 months after initial referral, and it was discovered that the baseline metabolomics data for this patient also clearly predicted cancer.
Discussion
In this study we have shown that metabolomic analysis of blood can sensitively and specifically identify solid tumors in patients with nonspecific symptoms. We have demonstrated that our models perform consistently under an extensive cross-validation scheme and can predict independent sample classifications, despite heterogeneity in cancer types represented. We have further shown that we can distinguish between nonmetastatic and metastatic cancers, within the solid tumor patient population. Finally, we present two cases in which biofluid metabolomics identified the presence of cancer earlier than the initial CT scan.
Metabolomics detects cancer in patients with nonspecific symptoms
The diagnosis of malignancy in patients presenting with nonspecific symptoms is a problem that is not solved by established population-level screening programs, such as those used for breast cancer, or by specific symptom referral pathways, such as those used to refer a patient with a persistent cough. In many centers there remains no clinical referral pathway for patients with nonspecific symptoms.
Although multidisciplinary approaches, such as that being used in Oxfordshire, have shown promise for cancer diagnosis in patients with nonspecific symptoms (26) they are not established in many centers in the UK or around the world. Without such a pathway, most of the patients with cancer in the cohort studied would probably have remained undetected until they presented at a later stage. Metabolomics analysis of blood is both rapid and inexpensive, and may enable accurate, timely, and cost-effective triaging of patients with suspected cancer.
We have also shown that we can distinguish between metastatic and nonmetastatic cancers, within the solid tumor patient population. This ability to diagnose metastasis is independent of our ability to be able to diagnose the primary tumor and shows promise for the use of metabolomics as a potential cancer staging tool. The ability to identify metastatic patients at first presentation may change the investigations performed and the pathway for these patients.
Metabolomics has the potential to identify cancers before they are detectable by CT
In our study cohort, 5 patients developed solid tumors within 1 year of receiving a noncancer diagnosis. In 2 of these 5, our baseline metabolomics analysis indicated the presence of cancer before conventional imaging and investigation enabled a diagnosis to be made. This suggests that metabolomics analysis has the potential to detect cancer earlier than the current conventional investigation pathways, and may be of value for both new presentations of malignancy and recurrence. Although the metabolomics did not predict cancer in all 5 of the patients who later went on to a cancer diagnosis, this may reflect one of many factors, including insufficient training data for specific tumor types or tumor size at baseline assessment. Our modeling set was heterogeneous, including a broad range of cancers, but at the expense of depth for individual cancer types. Future studies, with increased numbers of patients in specific cancer subsets, are likely to be needed to accurately determine the sensitivity of our models for early cancer diagnoses.
The case of the patient with the new primary lung tumor offers an interesting insight into how metabolomics may be deployed clinically. This patient's CT demonstrated a small lung nodule, which is a relatively common scenario as these nodules are commonly identified as an incidental abnormality on CT scans performed for a variety of clinical indications. Fortunately, most of these lung nodules are benign, but 2% to 3% will develop into lung cancers (27). Identifying which nodules will develop into cancers in a timely fashion is important for improving outcomes. The British Thoracic Society Guidelines advise serial CT scans for solid lung nodules >5 mm at either 3-, 12-, or potentially 24-monthly intervals over a period of 1 to 2 years (27). By observing changes in the characteristics and size of the nodule, those nodules that are likely to be malignant can be detected. Diagnosis is confirmed with further imaging, such as PET-CT scanning, and potentially a CT-guided lung biopsy, which has a small but significant morbidity and is rightly reserved for those where there is a significant suspicion of malignancy. Interval scanning places an increased burden on radiology departments, and an additional radiation dose and multiple hospital visits for the patient. In this study, we had metabolomic evidence that a small lung nodule of indeterminate etiology at presentation was malignant. This required repeat CT scans to identify nodule enlargement suggestive of malignancy which was then confirmed on biopsy.
Comparison of metabolomics with alternative technologies
The mainstay of modern clinical cancer diagnosis is imaging, primarily CT scanning, and if positive this is frequently coupled with biopsy. However, it is clear from observing the numbers of late-stage cancer diagnoses (3) that not all patients with cancer are being referred for imaging investigations soon enough. Here we have demonstrated that metabolomics is one technology that could identify these patients sooner, but it is not the only technology in development. One notable alternative technology is the analysis of circulating tumor DNA (ctDNA), which relies on the sequencing of DNA shed from tumors into the blood stream (reviewed in ref. 28). The abundance of particular tumor-associated mutations in the isolated DNA gives an indication of tumor burden. Owing to its genetic approach, ctDNA analysis has great potential for distinguishing tumor subtypes (29) and for monitoring tumor evolution (30). However, for initial detection of tumors the case is less clear, as ctDNA is released only by tumor cells. Consequently, there is no indirect amplification of the signal as is seen with metabolomics. Instead ctDNA analysis must detect the minute traces of DNA released directly by tumor cells which has intrinsic sensitivity limitations. A second factor is that all mutations must be known a priori, meaning that some mutations may be missed, even if they are being released by tumor cells.
Theoretical mechanism for cancer-mediated metabolic disruption
Dysregulated metabolism is so inextricably linked with cancer that it is included as one of the key hallmarks of cancer (31), and the study of oncometabolites and altered metabolism are major areas of research (32, 33). As such, it may be logical to assume that the metabolic changes we observe in this study are direct consequences of intracellular changes in cancer cells, for example there is a growing body of evidence that lipid metabolism is disrupted directly in tumor cells (reviewed in refs. 34, 35), which also has a direct impact on intracellular glucose concentrations. However, the small size of some of the tumors would limit their direct effect on the overall whole-body metabolic profile. Therefore, we expect that the metabolic profile changes we observe are the combined sum of changes associated with tumor metabolism and altered stromal metabolism—the so-called reactive stroma (36, 37). Others have shown that stromal signaling pathways can be activated by cancerous epithelial cells to promote the transition to a cancer-associated reactive stroma, which is thought to generate a supporting microenvironment for tumor growth and progression (38). Unfortunately, in most investigations metabolites have been studied in samples from individuals with specific types of cancer and then compared with those in healthy controls, which makes direct comparison between our findings and those of others problematic. However, some generalizations are possible and largely agree with our observations.
In this study, we see clear trends of change underpinning separations. Given that patients were fasted for at least 2 hours prior to sample collection and that there's no link to BMI, it's likely that these changes are metabolic responses. One clear trend is for decreased lipid concentrations being associated with solid tumors. Although intracellular changes in lipid metabolism do occur, we believe that the systemic response to the presence of the tumor is a more likely source of the altered lipid metabolism (39). For example, it is often argued that lipoproteins are an important source of metabolites for tumors and this demand may account, at least in part, for the lipid changes we have observed (39, 40). Another example is the association of low levels of low-density lipoprotein (LDL) with poor outcomes in patients with liver cancer (41) and lower levels of VLDL with breast carcinoma compared with healthy controls (42). Inflammatory tumor-associated cytokines like TNF, IL1β, IL6, and monocyte chemoattractant protein-1 (MCP-1) are all known to be involved in altered lipid metabolism at a systemic level (43). In particular, IL6 has been shown to reduce insulin-induced lipogenesis (44), whilst IL1β stimulated lipid accumulation by the liver (45). This involvement of cytokines lends credence to the hypothesis that it is indirect systemic changes, both within the local tumor microenvironment and across the whole body, e.g., changes in liver metabolism, that may be driving the magnification of the cancer metabolome signal. This indirect magnification hypothesis agrees with previous mouse studies, in which very small tumors growing in different locations in the body produced markedly different metabolic profile changes that cannot be simply attributed to any ‘direct’ metabolic effects they could have produced (6). These systemic effects have been postulated for some time (46), but it is only relatively recently that suitable whole-organism studies in cancer have been carried out to explicitly investigate the link between tumor and systemic metabolism (47, 48).
The increase in glucose concentrations in cancer and the increase in threonine in metastatic disease are more counterintuitive based on the Warburg effect. However, we are looking at systemic, not intratumoral glucose concentrations, and increased circulating glucose concentrations have often been associated with an increased risk of developing cancer (49, 50). Moreover, changes in plasma glucose concentration may reflect release from muscle glycogen stores in response to situations of stress (48).
Overall, we believe that the profile of altered metabolism we observe is a combination of multiple interactions involving the tumor, the local microenvironment, and distant regions such as the liver and adipose tissue, responding to the altered inflammatory profile induced by the tumor.
Clinical application of metabolomics
For metabolomics to make the transition from a research project to a clinically useful tool, it needs to: (i) have sufficient sensitivity and specificity to detect tumors in a cohort of patients; (ii) enable alterations in investigation pathways for patients with suspected malignancies, including the cohort investigated here of “low-risk, but not no-risk” patients.
Incorporating metabolomics into the initial investigation of patients with nonspecific symptoms may assist in the triage of patients for further referral or investigation. If metabolomic analysis is ordered from primary care with other initial blood analyses, it would provide valuable information for next diagnostics steps. Practically speaking a metabolomics blood test would require a blood sample collection and then analysis at a specialist center, meaning results could be returned before imaging appointments. Overall cost would be considerably lower than imaging-based investigations, with a likely price around that of other moderately specialist blood investigations.
At the optimum classification threshold, our negative predictive value of 99% provides a valuable “rule-out” test for cancer that would be useful for primary care physicians, letting them reassure their patients. Although specificity in our model is high, the low percentage of true cancers in the at-risk population would lead to false positives. A positive test, therefore, is a strong indication for referral for imaging investigation, but not a guarantee of cancer. In such cases, it will be necessary for the physician to take into consideration both the metabolomic prediction of cancer and other symptoms. In a larger study, it may be possible to formalize this framework into a risk score to help diagnoses. In any case, even a strong suspicion of cancer often justifies the nonnegligible radiation dose from a CT scan. In our study, for every patient with cancer that we would recommend for CT scanning, we would also recommend CT scans for 2 patients without cancer. However, we would also eliminate 9 patients from cancer imaging pathways.
In a more general sense, metabolomic analysis may help interpret the significance of incidental findings on imaging studies, such as lung nodules or pancreatic cysts, prompting earlier interventions than the current watch-and-wait approach. This may enable an earlier diagnosis to be reached, reducing the need for surveillance imaging, radiation exposure, and costs. The ability of metabolomics to differentiate metastatic from nonmetastatic cancer may be clinically important, providing a guide on the most appropriate investigations, improving diagnostic pathway efficiency, with potential significant cost and time savings.
Conclusions and future directions
In this study we have shown that the application of NMR metabolomics to a small blood sample offers a complementary approach to the current pathway for investigating patients with a clinical suspected possible malignancy. It is sensitive, specific, and of low cost, requiring nothing more than a blood sample in the clinic and an inexpensive NMR analysis, and can identify patients with solid tumors when referred with nonspecific symptoms—a traditionally hard-to-diagnose cohort.
NMR metabolomics now needs testing in a larger cohort of patients, potentially linked to multiple specific cancer referral pathways, as well as the pathway for the “low-risk, but not no-risk” patient group. By including potential time and cost-savings in the analysis, as well as the conventional cancer detection utility metrics, the utility of NMR metabolomics can be evaluated in a broader fashion.
Supplementary Material
Acknowledgments
We acknowledge the contribution to this study made by the Oxford Centre for Histopathology Research and the Oxford Radcliffe Biobank, which are funded by the University of Oxford, the Oxford CRUK Cancer Centre, the NIHR Oxford Biomedical Research Centre (BRC; Molecular Diagnostics Theme/Multimodal Pathology Sub-theme), and the NIHR CRN Thames Valley network. Many thanks to Wiktoria Jazwierska for assistance with clinical collections. Financial support: This work was supported by CRUK (C5255/A15935 awarded to N.R. Sibson), and the OxCODE and CRUK Oxford Centre (grant number C5255/A18085, which funded J.R. Larkin, A.G. Yates, F. Gleeson, and S. Anthony), the CRUK and EPSRC Cancer Imaging Centre in Oxford (grant number C5255/A16466, which funded V.A. Johanssen), the National Institute for Health Research (NIHR) Oxford Medtech and In-Vitro Diagnostics Co-operative (MIC-2018-018), the Oxford-Wellcome Institutional Strategic Support Fund (ISSF - 0005670), and a Dorothy Hodgkin Early Career Fellowship in Association with Somerville College, which funded F. Probert.
The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.
Footnotes
Note: Supplementary data for this article are available at Clinical Cancer Research Online (http://clincancerres.aacrjournals.org/).
Authors' Disclosures
J.R. Larkin reports grants from Cancer Research UK (CRUK) and Oxford Centre for Early Cancer Detection (OxCODE) during the conduct of the study; in addition, J.R. Larkin has a patent for Diagnosis of Cancer pending and is a cofounder and shareholder in Oxomics Ltd. S. Anthony reports grants from OxCODE during the conduct of the study. V.A. Johanssen reports grants from the CRUK and EPSRC Cancer Imaging Centre in Oxford during the conduct of the study. T.D.W. Claridge reports grants from OxCODE and CRUK during the conduct of the study; in addition, T.D.W. Claridge has a patent for Diagnosis of Cancer pending and is a cofounder and shareholder in Oxomics Ltd. F. Gleeson reports grants from CRUK during the conduct of the study as well as grants from CRUK, InnovateUK, NIHR, Roche Ltd., and GE Healthcare and personal fees from Optellum Ltd. and AstraZeneca outside the submitted work. N.R. Sibson reports grants from CRUK and the OxCODE during the conduct of the study as well as grants from the Medical Research Council outside the submitted work; in addition, N.R. Sibson has a patent for Diagnosis of Cancer pending and is a cofounder and shareholder in Oxomics Ltd. F. Probert reports grants from Dorothy Hodgkin Early Career Fellowship during the conduct of the study; in addition, F. Probert has a patent for Diagnosis of Cancer pending and is a cofounder and shareholder in Oxomics Ltd. No disclosures were reported by the other authors.
Authors' Contributions
J.R. Larkin: Conceptualization, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. S. Anthony: Conceptualization, data curation, funding acquisition, project administration, writing–review and editing. V.A. Johanssen: Resources, data curation, writing–review and editing. T. Yeo: Investigation, writing–review and editing. M. Sealey: Investigation, writing–review and editing. A.G. Yates: Resources, data curation, project administration, writing–review and editing. C. Friedemann Smith: Data curation. T.D.W. Claridge: Resources, software, investigation, methodology, writing–review and editing. B.D. Nicholson: Resources, project administration, writing–review and editing. J.-A. Moreland: Resources, data curation, writing–review and editing. F. Gleeson: Resources. N.R. Sibson: Conceptualization, supervision, funding acquisition, investigation, methodology, project administration, writing–review and editing. D.C. Anthony: Conceptualization, resources, formal analysis, supervision, funding acquisition, investigation, methodology, project administration, writing–review and editing. F. Probert: Conceptualization, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing.
References
- 1. Hawkes N. Cancer survival data emphasise importance of early diagnosis. BMJ 2019;364:l408. [DOI] [PubMed] [Google Scholar]
- 2. World Health Organization. Guide to cancer early diagnosis. World Health Organization; 2017. Available from: https://www.who.int/publications/i/item/guide-to-cancer-early-diagnosis. [Google Scholar]
- 3. Elliss-Brookes L, McPhail S, Ives A, Greenslade M, Shelton J, Hiom S, et al. Routes to diagnosis for cancer – determining the patient journey using multiple routine data sets. Br J Cancer 2012;107:1220–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Porta M, Fernandez E, Belloc J, Malats N, Gallén M, Alonso J. Emergency admission for cancer: a matter of survival? Br J Cancer 1998;77:477–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Nicholson BD, Oke J, Friedemann Smith C, Phillips J-A, Lee J, Abel L, et al. The Suspected CANcer (SCAN) pathway: protocol for evaluating a new standard of care for patients with non-specific symptoms of cancer. BMJ Open 2018;8:e018168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Larkin JR, Dickens AM, Claridge TDW, Bristow C, Andreou K, Anthony DC, et al. Early diagnosis of brain metastases using a biofluids-metabolomics approach in mice. Theranostics 2016;6:2161–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Hu J-M, Sun H-T. Serum proton NMR metabolomics analysis of human lung cancer following microwave ablation. Radiat Oncol 2018;13:40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Gu J, Xiao Y, Shu D, Liang X, Hu X, Xie Y, et al. Metabolomics analysis in serum from patients with colorectal polyp and colorectal cancer by 1H-NMR spectrometry. Dis Markers 2019;2019:1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Michálková L, Hornιk Š, Sýkora J, Habartová L, Setnicka V. Diagnosis of pancreatic cancer via1H NMR metabolomics of human plasma. Analyst 2018;143:5974–8. [DOI] [PubMed] [Google Scholar]
- 10. Guo W, Tan HY, Wang N, Wang X, Feng Y. Deciphering hepatocellular carcinoma through metabolomics: from biomarker discovery to therapy evaluation. Cancer Manag Res 2018;10:715–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Fages A, Duarte-Salles T, Stepien M, Ferrari P, Fedirko V, Pontoizeau C, et al. Metabolomic profiles of hepatocellular carcinoma in a European prospective cohort. BMC Med 2015;13:242. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Yang L, Wang Y, Cai H, Wang S, Shen Y, Ke C. Application of metabolomics in the diagnosis of breast cancer: a systematic review. J Cancer 11:2540–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Cheng X, Liu X, Liu X, Guo Z, Sun H, Zhang M, et al. Metabolomics of non-muscle invasive bladder cancer: Biomarkers for early detection of bladder cancer. Front Oncol 2018;8:494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. NHS Interim Management and Support. Delivering cancer waiting times - a good practice guide. NHS Interim Management and Support; 2015. Available from: https://www.england.nhs.uk/wp-content/uploads/2015/03/delivering-cancer-wait-times.pdf. [Google Scholar]
- 15. Jurynczyk M, Probert F, Yeo T, Tackley G, Claridge TDW, Cavey A, et al. Metabolomics reveals distinct, antibody-independent, molecular signatures of MS, AQP4-antibody and MOG-antibody disease. Acta Neuropathol Commun 2017;5:95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Probert F, Walsh A, Jagielowicz M, Yeo T, Claridge TDW, Simmons A, et al. Plasma nuclear magnetic resonance metabolomics discriminates between high and low endoscopic activity and predicts progression in a prospective cohort of patients with ulcerative colitis. J Crohns Colitis 2018;12:1326–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Tang H, Wang Y, Nicholson JK, Lindon JC. Use of relaxation-edited one-dimensional and two dimensional nuclear magnetic resonance spectroscopy to improve detection of small metabolites in blood plasma. Anal Biochem 2004;325:260–72. [DOI] [PubMed] [Google Scholar]
- 18. Lenz EM, Bright J, Wilson ID, Morgan S, Nash A. A 1H NMR-based metabonomic study of urine and plasma samples obtained from healthy human subjects. J Pharm Biomed Anal 2003;33:1103–15. [DOI] [PubMed] [Google Scholar]
- 19. Wishart DS, Tzur D, Knox C, Eisner R, Guo AC, Young N, et al. HMDB: the human metabolome database. Nucleic Acids Res 2007;35:D521–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Wishart DS, Feunang YD, Marcu A, Guo AC, Liang K, Vázquez-Fresno R, et al. HMDB 4.0: the human metabolome database for 2018. Nucleic Acids Res 2018;46:D608–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Gowda GAN, Gowda YN, Raftery D. Expanding the limits of human blood metabolite quantitation using NMR spectroscopy. Anal Chem 2014;87:706–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Team RC. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. Available from: http://www.R-project.org/. [Google Scholar]
- 23. Thévenot EA, Roux A, Xu Y, Ezan E, Junot C. Analysis of the human adult urinary metabolome variations with age, body mass index, and gender by implementing a comprehensive workflow for univariate and OPLS statistical analyses. J Proteome Res 2015;14:3322–35. [DOI] [PubMed] [Google Scholar]
- 24. Arlot S, Celisse A. A survey of cross-validation procedures for model selection. Statistics Surveys 2010;4:40–79. [Google Scholar]
- 25. National Cancer Registration and Analysis Service. Urgent suspected cancer referrals: referral, conversion and detection rates. NCRAS; 2020. Available from: http://www.ncin.org.uk/cancer_type_and_topic_specific_work/topic_specific_work/tww_conversion_and_detection. [Google Scholar]
- 26. Chapman D, Poirier V, Vulkan D, Fitzgerald K, Rubin G, Hamilton W, et al. First results from five multidisciplinary diagnostic centre (MDC) projects for non-specific but concerning symptoms, possibly indicative of cancer. Br J Cancer 2020;123:722–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Callister MEJ, Baldwin DR, Akram AR, Barnard S, Cane P, Draffan J, et al. British Thoracic Society guidelines for the investigation and management of pulmonary nodules: accredited by NICE. Thorax 2015;70:ii1–ii54. [DOI] [PubMed] [Google Scholar]
- 28. Keller L, Belloum Y, Wikman H, Pantel K. Clinical relevance of blood-based ctDNA analysis: mutation detection and beyond. Br J Cancer 2020;124:345–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Vandekerkhove G, Lavoie J-M, Annala M, Murtha AJ, Sundahl N, Walz S, et al. Plasma ctDNA is a tumor tissue surrogate and enables clinical-genomic stratification of metastatic bladder cancer. Nat Commun 2021;12:184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Cresswell GD, Nichol D, Spiteri I, Tari H, Zapata L, Heide T, et al. Mapping the breast cancer metastatic cascade onto ctDNA using genetic and epigenetic clonal tracking. Nat Commun 2020;11:1446. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Hanahan D, Weinberg RA. Hallmarks of cancer: the next generation. Cell 2011;144:646–74. [DOI] [PubMed] [Google Scholar]
- 32. Khatami F, Aghamir SMK, Tavangar SM. Oncometabolites: a new insight for oncology. Mol Genet Genomic Med 2019;7:e873. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Madeddu C, Maccio A, Mantovani G. Metabolic changes in cancer patients. Anti-Inflammatory & Anti-Allergy Agents in Medicinal Chemistry 2011;10:281–5. [Google Scholar]
- 34. Cheng C, Geng F, Cheng X, Guo D. Lipid metabolism reprogramming and its potential targets in cancer. Cancer Commun 2018;38:27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Munir R, Lisec J, Swinnen JV, Zaidi N. Lipid metabolism in cancer cells under metabolic stress. Br J Cancer 2019;120:1090–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Andersen MK, Rise K, Giskeødegård GF, Richardsen E, Bertilsson H, Størkersen Ø, et al. Integrative metabolic and transcriptomic profiling of prostate cancer tissue containing reactive stroma. Sci Rep 2018;8:14269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Karta J, Bossicard Y, Kotzamanis K, Dolznig H, Letellier E. Mapping the metabolic networks of tumor cells and cancer-associated fibroblasts. Cells 2021;10:304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Shen S, Li J, Huo S, Ma M, Zhu X, Rasam S, et al. Parallel, High-quality proteomic and targeted metabolomic quantification using laser capture microdissected tissues. Anal Chem 2021;93:8711–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Maran L, Hamid A, Hamid SBS. Lipoproteins as markers for monitoring cancer progression. J Lipids 2021;2021:1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Kuemmerle NB, Rysman E, Lombardo PS, Flanagan AJ, Lipe BC, Wells WA, et al. Lipoprotein lipase links dietary fat to solid tumor cell proliferation. Mol Cancer Ther 2011;10:427–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Saito N, Sairenchi T, Irie F, Iso H, Iimura K, Watanabe H, et al. Low serum LDL cholesterol levels are associated with elevated mortality from liver cancer in Japan: the Ibaraki Prefectural Health Study. Tohoku J Exp Med 2013;229:203–11. [DOI] [PubMed] [Google Scholar]
- 42. Mosapour A, Tehrani FSK, Atri M. Expression level of VLDL receptor and VLDL-c levels in the malignant and benign breast tumors: the correlation with miRNA-4465 and miRNA-1297. Mol Cell Probes 2020;53:101624. [DOI] [PubMed] [Google Scholar]
- 43. Shi J, Fan J, Su Q, Yang Z. Cytokines and abnormal glucose and lipid metabolism. Front Endocrinol 2019;10:703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Lagathu C, Bastard J-P, Auclair M, Maachi M, Capeau J, Caron M. Chronic interleukin-6 (IL-6) treatment increased IL-6 secretion and induced insulin resistance in adipocyte: prevention by rosiglitazone. Biochem Biophys Res Commun 2003;311:372–9. [DOI] [PubMed] [Google Scholar]
- 45. Miura K, Kodama Y, Inokuchi S, Schnabl B, Aoyama T, Ohnishi H, et al. Toll-like receptor 9 promotes steatohepatitis by induction of interleukin-1 in mice. Gastroenterology 2010;139:323–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Douglas RG, Shaw JHF. Metabolic effects of cancer. Br J Surg 1990;77:246–54. [DOI] [PubMed] [Google Scholar]
- 47. Naser FJ, Jackstadt MM, Fowle-Grider R, Spalding JL, Cho K, Stancliffe E, et al. Isotope tracing in adult zebrafish reveals alanine cycling between melanoma and liver. Cell Metab 2021;33:1493–504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Goodwin ML, Pennington Z, Westbroek EM, Cottrill E, Ahmed AK, Sciubba DM. Lactate and cancer: a “lactatic” perspective on spinal tumor metabolism (part 1). Ann Transl Med 2019;7:220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Mathews EH, Mathews GE, Meyer AA. A hypothetical method for controlling highly glycolytic cancers and metastases. Med Hypotheses 2018;118:19–25. [DOI] [PubMed] [Google Scholar]
- 50. Liao W-C, Tu Y-K, Wu M-S, Lin J-T, Wang H-P, Chien K-L. Blood glucose concentration and risk of pancreatic cancer: systematic review and dose-response meta-analysis. BMJ 2015;349:g7371. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data generated in this study are available upon request from the corresponding author.