Abstract
Simple Summary
Recently, immunotherapy has presented new opportunities for clinical development in the treatment of non-small cell lung cancer (NSCLC). Although effective in sustaining overall survival in several clinical trials, not all the NSCLC patients respond to these treatments. Thus, a better patient selection, as well as the identification of predictive biomarkers of treatment efficacy, are of paramount importance. In this work, metabolomics was used with the aim of identifying responder with respect to non-responder subjects. We show that the metabolomic fingerprint of serum samples, collected before therapy, acts as a predictive biomarker to treatment response. Prospective identification of subjects that will benefit from immunotherapy could improve patient stratification, thus optimizing the treatment and avoiding unsuccessful strategies.
Abstract
In the treatment of advanced non-small cell lung cancer (NSCLC), immune checkpoint inhibitors have shown remarkable results. However, not all patients with NSCLC respond to this drug treatment or receive durable benefits. Thus, patient stratification and selection, as well as the identification of predictive biomarkers, represent pivotal aspects to address. In this framework, metabolomics can be used to support the discrimination between responders and non-responders. Here, metabolomics was used to analyze the sera samples from 50 patients with NSCL treated with immune checkpoint inhibitors. All the samples were collected before the beginning of the treatment and were analyzed by NMR spectroscopy and multivariate statistical analyses. Significantly, we show that the metabolomic fingerprint of serum acts as a predictive “collective” biomarker to immune checkpoint inhibitors response, being able to predict individual therapy outcome with > 80% accuracy. Metabolomics represents a potential strategy for the real-time selection and monitoring of patients treated with immunotherapy. The prospective identification of responders and non-responders could improve NSCLC treatment and patient stratification, thus avoiding ineffective therapeutic strategies.
Keywords: immune checkpoint inhibitors, non-small cell lung cancer, metabolomics, nuclear magnetic resonance
1. Introduction
Lung cancer (LC) is the leading cause of cancer-related mortality worldwide. Eighty-five percent of patients with LC are affected by non-small cell lung cancer (NSCLC), with the majority of patients presenting with advanced, unresectable disease at the time of diagnosis [1]. Current treatment strategies for advanced NSCLC include chemotherapy, targeted agents, immunotherapy, or chemo-immunotherapy. Treatment choices are made according to tumor histology as well as PDL-1 TPS (tumor proportion score) and the presence of specific genomic alterations. However, the majority of patients does not attain durable disease control, and the five-year survival rates remain very low. Thus, more effective strategies are needed.
The ability of LC to evade the body’s immune surveillance by utilizing certain “immune checkpoints” that normally protect against autoimmunity and inflammation has been well characterized [2]. Immune checkpoint inhibitors (ICIs) have become an attractive treatment modality for chemo-refractory solid neoplasms. These agents reactivate T lymphocyte-mediated immune response against the tumor in the microenvironment by blocking immune checkpoint molecules, including cytotoxic T-lymphocyte antigen-4 (CTLA-4), programmed cell death protein 1 (PD-1), and its ligand (PDL-1) [3]. Antibody-directed therapies against these checkpoints have shown remarkable early success in many malignancies, and play a critical role in the management of advanced lung cancer and other tumour types [4].
Several monoclonal antibodies directed to the PD-1 receptor or its ligand PDL-1 (for example nivolumab, pembrolizumab, and atezolizumab) are already established in daily practice, with others in preclinical development. Clinical trials with these agents have shown rapid and durable responses in about 14–20% of pre-treated patients with advanced NSCLC [5,6,7,8]. Importantly, although progression-free survival data are modest (20% progression-free survival at one year), reported sustained survival outcomes are remarkable. In the 27-month follow-up of a cohort of 129 patients with NSCLC treated with nivolumab, two-year survival was 24% in the overall population and 42% in the subset of patients treated at the dose selected for further development (3 mg/kg every 2 weeks); three-year survival was 18% in the overall population and 27% in the development dose subset of patients [9]. Based on these data, immunotherapy has established an important role in the first line of treatment of advanced disease as a monotherapy or in combination with chemotherapy [10], as well as in limited-stage disease [11].
However, not all patients respond to ICIs, with modest response rates (approximately 20% or less in LC) and associated high treatment costs. Thus, there is crucial interest in the discovery and development of biomarkers that can predict which patients are most likely to respond and benefit from ICIs, thus improving clinical decision-making and disease management [12].
Clinical efficacy of ICIs appears independent of histology, but in most trials, a greater benefit was observed in smokers and in patients with positive PDL-1 expression. PDL-1 expression has proved to be an effective patient selector but is far from being a true predictive marker. Patients with PDL-1-positive NSCLC do not derive universal benefit from these agents, while some tumors with low or negative expression of PDL-1 do show response [4]. Additionally, the utilization of tumor mutational burden (TMB; the overall quantity of aberrant nucleotide sequences a given tumor may harbor) has produced conflicting results, casting doubt on its ability to be a predictor of response to immunotherapy [13].
Routine clinical practice can be complicated by the fact that radiological indications of tumor response and progression associated with immune checkpoint blockade therapy can be different from those observed in patients receiving a conventional chemotherapeutic and/or molecular-targeted agents. Pseudo-progression can manifest itself as an increase in the size of target lesions, or even as the development of new lesions on imaging. However, such paradoxical changes are not necessarily due to resistance to treatment, but rather a sign of treatment effect [14].
Metabolomics deals with the characterization of the metabolome, i.e., the ensemble of metabolites presents in a cell, a tissue, a biofluid or in an entire organism [15]. The human metabolome is composed by several thousands of small molecules produced by the organism and by its microflora, as well as deriving from environmental factors [16]. It is a dynamic entity that may change as a result of lifestyle, stress and, most importantly, onset of pathologies and medical treatment [17,18,19,20,21,22]. In the field of oncology, metabolomics has shown potential for early diagnosis, prognosis, individual monitoring and drug therapy design [23,24,25]. Nuclear magnetic resonance (NMR)-based metabolomics is an efficient and highly reproducible platform for the analysis of biofluids, such as blood serum or plasma [26,27]. The use of biofluids for screening and monitoring has the advantage of being minimally invasive, can be applied on a large scale, and may provide significant information on tumor -specific phenotype [26,27]. In this framework, NMR metabolomic fingerprinting of biofluids has been increasingly used to establish a metabolomic signature both before and after a given drug therapy that might inform on treatment outcomes [28,29,30].
In this study, we conducted an NMR-based metabolomic investigation of sera samples from 50 patients treated with immune checkpoint inhibitors for NSCLC. All the analyzed samples were collected before the beginning of the treatment with the aim to find predictive metabolomic profiles of the response to immune checkpoint inhibitors.
2. Results
Serum sample from 53 patients with advanced NSCLC were analyzed. Patient and tumor characteristics and type of treatment are reported in Table S1. Thirty-four patients were treated with nivolumab as second line therapy after chemotherapy, and 19 patients were treated with pembrolizumab as first-line therapy. Overall, 19 patients presented with a squamous cell carcinoma (SqCC), 31 with adenocarcinoma (non-SqCC), and three with NOS (not otherwise specified) carcinoma.
Due to the higher number of samples collected, our analyses mainly focused on nivolumab-treated subjects.
2.1. NSCLC Nivolumab-Treated Cohort
Firstly, we investigated if the metabolomic profiles of baseline T0 serum samples from nivolumab-treated patients were significantly different according to histological profile (Figure 1). Applying principal component analysis (PCA) and orthogonal partial least squares (O-PLS) models, we compared the sera from patients with squamous cell (N = 19), adenocarcinoma (N = 13) and NOS carcinoma (N = 2). The T0 metabolomic phenotypes of the patients with different histology types were not significantly different (2-group discrimination accuracy of 48.0% (p-value: 0.86), squamous vs. adenocarcinoma, DQ2 of −0.02) (Figure 1).
As there were no significant differences according to histological subtype, all the tumor types were analysed together from here on.
Multivariate statistical analysis was then used to analyse T0 samples to extract a serum metabolomic fingerprint able to discriminate between responder and non-responder subjects, with the final aim of obtaining a priori information about treatment outcome.
The first attempt to establish a classification between responder and non-responder subjects was performed according to the first radiological assessment (Table 1). In this framework, iRECIST criteria were used. “Responders” were defined as those subjects who achieved complete response (iCR), partial response (iPR), or stable disease (iSD), whereas “non-responders” were those with unconfirmed progressive disease (iUPD), confirmed progressive disease (iCPD) or hyper-progression (iHP). Subjects who died before the first radiological assessment were classified as iHP.
Table 1.
Subject Number | N° Cycles | TTF (Weeks) |
OS (Weeks) |
1 RA | 2 RA | 3 RA | 4 RA | 5 RA | 6 RA | 7 RA | 8 RA | 9 RA | 10 RA | 11 RA | 12 RA |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
MetL01 | 4 | 8 | 10 | na (HP) | |||||||||||
MetL05 | 24 | 59 | 94 | iSD/PR | iSD | iUPD | iCPD | ||||||||
MetL06 | 4 | 8 | 13 | iUPD | |||||||||||
MetL07 | 84 | 186+ | 193+ | iPR | iPR | iPR | iPR | iPR | iPR | iPR | iPR | iPR | iPR | iPR | iPR |
MetL09 | 19 | 38 | 67 | iSD | iUPD | iUPD | iCPD | ||||||||
MetL10 | 10 | 18 | 20 | iSD | iSD | ||||||||||
MetL11 | 9 | 23 | 60 | iUPD | iCPD | ||||||||||
MetL12 | 23 | 45 | 60 | iUPD | iPR | iSD | iPD | ||||||||
MetL13 | 29 | 140+ | 167+ | iUPD | iUPD | iPR | iPR | iUPD | iUPD | iPR | iSD | iSD | iSD | ||
MetL15 | 2 | 2 | 5 | na (HP) | |||||||||||
MetL17 | 19 | 41 | 103 | iUPD | iUPD | iCPD | |||||||||
MetL19 | 27 | 66 | 104 | iUPD | iUPD | iSD | iSD | iCPD | |||||||
MetL20 | 4 | 6 | 8 | na (HP) | |||||||||||
MetL24 | 6 | 10 | 48 | iPD | |||||||||||
MetL26 | 6 | 10 | 36 | iPD | |||||||||||
MetL29 | 3 | 5 | 60 | na (HP) | |||||||||||
MetL30 | 2 | 4 | 14 | HP | |||||||||||
MetL32 | 8 | 17 | 19 | iUPD | |||||||||||
MetL33 | 13 | 35 | 41 | iSD | iSD | iUPD | iCPD | ||||||||
MetL34 | 2 | 2 | 4 | na (HP) | |||||||||||
MetL36 | 8 | 16 | 21 | iUPD | iCPD | ||||||||||
MetL37 | 17 | 38 | 44 | iUPD | iUPD | iCPD | |||||||||
MetL39 | 4 | 7 | 14 | iUPD | |||||||||||
MetL40 | 47 | 99+ | 102+ | iPR | iPR | iPR | iPR | iPR | iPR | iPR | |||||
MetL41 | 6 | 12 | 23 | iUPD | iCPD | ||||||||||
MetL42 | 18 | 39 | 100+ | iUPD | iUPD | iUPD | iCPD | ||||||||
MetL44 | 11 | 20 | 92+ | iUPD | iCPD | ||||||||||
MetL45 | 8 | 15 | 19 | iUPD | iCPD | ||||||||||
MetL48 | 19 | 37 | 75+ | iUPD | iPR | iPD | |||||||||
MetL49 | 1 | 1 | 1 | na (HP) | |||||||||||
MetL50 | 35 | 70+ | 73+ | iPR | iPR | iPR | iPR | ||||||||
MetL52 | 29 | 58+ | 70+ | iUPD | iUPD | iUPD | iUPD | iUPD | iCPD | ||||||
MetL53 | 19 | 54+ | 59+ | iPR | iPR | iSD | iPR | ||||||||
MetL57 | 7 | 15 | 52+ | iUPD | iCPD |
O-PLS analysis was not able to discriminate partial/complete responders (9 subjects) from non-responders (25 subjects), obtaining a discrimination accuracy lower than 60% (p-value: 0.76) and a DQ2 of −0.30 (Figure 2A). In the case of immunotherapy, the therapy response estimation through radiological assessment may not be easy to evaluate, due to characteristic patterns of response to treatment, including the phenomenon of the pseudo-progression. Pseudo-progression does not reflect tumour cell growth but may be misclassified as disease progression (Figure 2B).
To overcome the difficulties of evaluating treatment responses to include clinical judgment, we decided to take into account the second radiological assessment (Table 1). In daily clinical practice, if a minimum or asymptomatic progression of disease is observed, treatment is continued until the patient becomes symptomatic, as suggested by expert consensus [1].
All subjects who discontinued the treatment at the second radiological assessment, or before, due to confirmed progressive disease, hyper-progression, or death, were considered as non-responders. Conversely, all subjects continuing therapy until the third or further radiological assessments were considered as responders. Thus, 15 subjects were considered as responders, and 19 subjects as non-responders. As expected, the time to treatment failure (TTF) values of responder and non-responder subjects (Table 1) are significantly different (median TTF in non-responders: 10 weeks versus median TTF in responders: 54 weeks; p-value < 0.001). Moreover, looking at overall survival (OS), the OS median values of responder and non-responder subjects are significantly different, Table 1 and Figure 2C (median OS in non-responders: 19.5 weeks versus median OS in responders: 75 weeks; p-value < 0.001).
Using this criterion, the discriminatory power of the O-PLS analysis significantly improved, the model being able to discriminate between responders and non-responders with an accuracy of 82% (DQ2: 0.16); accordingly, the ROC curve derived from O-PLS cross-validation showed an area under the curve (AUC) of 0.79. In this model only six out of 33 subjects were misclassified (Figure 2D). The O-PLS predicting power was assessed for significance against the null hypothesis of no prediction accuracy in the data, by means of 500 randomized class permutation tests. The average discrimination accuracy obtained after randomization was 58.3%, demonstrating the significance of our results (p-value: 0.006).
No apparent common characteristics are shared among the six misclassified subjects (MetL13, MetL19, MetL41, MetL45, MetL50, and MetL57): four were male (two had squamous carcinoma and two adenocarcinoma), and two were female—one with squamous carcinoma and the other with adenocarcinoma.
Due to the relatively small number of samples, these results were obtained using a leave-one-out cross-validation scheme. By using a Monte Carlo cross-validation scheme, the discrimination accuracy remained very high, at 78% (p-value of 0.006), with a DQ2 of 0.17.
2.2. NSCLC Pembrolizumab-Treated Cohort
The serum samples of 19 patients treated with pembrolizumab as first line therapy were also collected (Table S1). Due to the small sample number size, these cannot be used to build a statistically significant model aimed at the identification of responders to pembrolizumab. Thus, to explore the possibility that a metabolomic signature able to discriminate between responders and non-responders to anti PD-1 therapy can be established in the context of pembrolizumab therapy, we used the discriminatory power of the O-PLS model created in nivolumab-treated subjects to predict the outcome of pembrolizumab-treated patients (Table 2).
Table 2.
Subject Number | N° Cycles | TTF (Week) |
OS (Week) |
PD-L1 | 1 RA | 2 RA | 3 RA | 4 RA | 5 RA | 6 RA | 7 RA | 8 RA | 9 RA |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
MetL22 | 35 | 115+ | 118+ | 80 | iPR | iPR | iPR | iPR | iPR | iSD | iSD | iUPD | iSD |
MetL23 | 36 | 115+ | 116+ | 60 | iPR | iPR | iCR | iCR | iCR | iSD | iSD | iSD | |
MetL25 | 33 | 106+ | 109+ | 70 | iSD | iSD | iSD | iSD | iSD | iSD | iSD | iSD | iSD |
MetL27 | 3 | 6 | 14 | 80 | |||||||||
MetL28 | 8 | 24 | 39 | 70 | iPR | iUPD | iCPD | ||||||
MetL31 | 3 | 8 | 10 | 70 | |||||||||
MetL35 | 11 | 37 | 39 | 70 | iSD | iUPD | iUPD | iCPD | |||||
MetL38 | 26 | 82+ | 87+ | 60 | iPR | iPR | iSD | iSD | iSD | ||||
MetL43 | 1 | 5 | 5 | 60 | |||||||||
MetL46 | 17 | 62+ | 66+ | 60 | iUPD | iPR | iPR | iPR | iPR | ||||
MetL47 | 17 | 54+ | 56+ | 80 | iPR | iUPD | iSD | ||||||
MetL51 | 14 | 45+ | 49+ | 70 | iPR | iPR | iSD | ||||||
MetL55 | 1 | 1 | 1 | 80 | |||||||||
MetL56 | 3 | 9 | 17 | 90 | iPD | ||||||||
MetL59 | 3 | 6 | xx | 80 | iPD | ||||||||
MetL60 | 5 | 15 | 19 | 60 | iSD | ||||||||
MetL63 | 4 | 20 | 23 | 50 | iUPD | iCPD | |||||||
MetL65 | 8 | 22 | 30+ | 90 | iPR | iUPD | |||||||
MetL66 | 7 | 26 | 90 | iUPD | iCPD |
Firstly, we investigated if the metabolomic profiles of T0 samples from nivolumab- and pembrolizumab-treated patients were significantly different. Applying PCA and O-PLS models, we detected that the serum metabolomic phenotypes were not significantly different according to treatment group, with a 2-group discrimination accuracy of 52.0% (p-value: 0.84) with a DQ2 of −0.43 (Figure 3A,B). Then, the O-PLS model created on nivolumab-treated patients was used to predict the pembrolizumab-treated subjects. Fifteen out of the nineteen pembrolizumab-treated subjects were correctly classified, obtaining a total prediction accuracy of 79% (Figure 3C). These results indicated that a serum fingerprint able to discriminate responders exists both for nivolumab and pembrolizumab treatment.
2.3. Metabolite Analysis
NMR spectra were analyzed to identify which metabolites were the main contributers to the discrimination between responders and non-responders (Table S2 and Figure S1). Considering nivolumab-treated patients, non-responder and responder subjects were characterized by significantly (p-value < 0.05, FDR < 0.05) different levels of the amino acid alanine and of pyruvate. In particular, responders are characterized by lower serum levels of the two above mentioned metabolites (Figure 4A). Interestingly, the same behavior of pyruvate is confirmed in pembrolizumab-treated patients. Pyruvate serum levels, even not significantly, are lower in non-responders with respect to responders (Figure 4B).
3. Discussion
Cancer immunotherapy has changed conventional treatment paradigms by expanding treatment options for patients with NSCLC. However, despite current success, the response rate to ICIs in advanced NSCLC is around 30%. Thus, there is a growing need to identify predictive and prognostic biomarkers to guide patient selection. The basic principles underlying a good biomarker include analytical validity (reliability and reproducibility), as well as clinical utility. Patient selection based on the expression of PDL-1, as detected by immunohistochemistry and the tumour mutational burden, is used to augment prediction of the likelihood of response to immunotherapy [31,32,33]. In recent years, efforts have been made to identify more reliable methods for the prediction of the response to immune checkpoint inhibitors with controversial results [34,35]. The molecular basis of tumor immunogenicity and cancer immune-escape involves complex mechanisms, and are not yet well understood. The extreme dynamism of the immune system likely explains in part the difficulties encountered in defining a true predictive marker. The study of cellular metabolism performed at the beginning of treatment may overcome the obstacles described above.
Here, we explored the role of metabolomic fingerprinting of serum samples as a predictive biomarker of response to immunotherapy, and we identified a serum metabolomic “signature” that differentiates responder and non-responder patients with NSCLC to nivolumab and pembrolizumab. To our knowledge, this represents the first study that applied NMR-based metabolomics of serum to predict responsiveness to anti PD-1 therapy in NSCLC. A mass spectrometry (MS)-based metabolomic analysis in plasma samples from 55 patients with NSCLC treated with nivolumab has been recently performed [36]. In that paper Hatae et al. demonstrated that a combination of 4 plasma metabolites and several T cell markers could serve as good biomarkers for responder identification (AUC = 0.96). The four metabolite markers include molecules related to gut microbiota (hippuric acid), fatty acid oxidation (butyryl-carnitine), and two redox-related metabolites (cystine and glutathione disulfide) [36]. Using the same methodological approach, in another recent investigation, metabolomic profiling of gut microbiota of a small cohort of patients with NSCLC treated with nivolumab were analysed [37]. In this case, two metabolites—2-pentanone (ketone) and tridecane (alkane)—were significantly associated with early progression. Conversely, short chain fatty acids propionate and butyrate, as well as lysine and nicotinic acid were significantly associated with long-term beneficial effects [37]. Moreover, MS-based metabolomic analysis in serum samples has also recently been used as a tool to investigate metabolic alterations in response to immune checkpoint blockade in advanced melanoma and renal cell carcinoma patients treated with nivolumab (17). In this setting, an increase in serum kynurenine/tryptophan ratio was identified as an adaptive resistance mechanism associated with worse overall survival [38]. All these studies underline the multifactorial and complex nature of immune systems and immunotherapy efficacy, highlighting individual gut microflora as one of the important players. Some evidence has demonstrated that the tumor microenvironment may also play and important role in inhibiting T cell functionality and immunotherapy efficacy [39]. The tumor microenvironment, in fact, can be metabolically hostile due to insufficient vascular exchange and cancer cell metabolism. Cancer cells rapidly consume nutrients and produce lactic acid, leading to hypoxia and high acidity [40]. In this framework, several studies have shown the association of hypoxia with a more aggressive cancer phenotype [41,42]. In particular, lack of oxygen stabilizes HIF-1α, which increases glycolysis and decreases oxidative phosphorylation [41,42]. Interestingly, in the serum samples of non-responder subjects we detected significantly higher levels of pyruvate and alanine along with, even not statistically significant, higher lactate and glycine levels, and lower citrate levels. Higher levels of pyruvate, ala, gly and lactate are all indicative of increased glycolysis and lower levels of citrate are indicative of decreased TCA pathway [43,44]. Thus, we can hypothesize that, in non-responder patients, the tumor microenvironment is particularly hostile and T-cells are not able to overcome these metabolic challenges. It is important to underline that the significance of the univariate analysis performed on single metabolite levels strongly relies on the number of subjects included in the study. Thus, further investigations to confirm this hypothesis are needed.
The strength of our work relies on the fact that, although the sensitivity of NMR is low compared to MS, the 1H NMR fingerprint approach take advantage of the its intrinsically untargeted nature and high reproducibility that allowed us to identify metabolomic signatures that are independent from metabolite assignment and act as stronger biomarkers with respect to a single molecules or a panel of few molecules [27].
Our study is strengthened by the use of the sera-derived metabolomic fingerprints of patients treated with pembrolizumab as a small independent validation group, which confirmed the results obtained in nivolumab-treated patients. We chose to combine patients receiving nivolumab and pembrolizumab because both treatments have the same mechanism of action, since they both interfere with the interaction between PD-1 and PDL-1, an interaction that downregulates T cells, allowing cancer cells to evade immune surveillance [45].
4. Materials and Methods
4.1. Patient Recruitment and Serum Sample Collection
Patients with advanced NSCLC, older than 18 years, were enrolled at the Hospital of Prato, Prato, Italy (Table S1). We collected blood samples from all patients just prior to the initiation of the first administration of an immune checkpoint inhibitor treatment according to normal first and second or third line clinical practice [5,6,33]. All patients gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki.
This research is approved by Comitato Etico Regionale per la Sperimentazione Clinica della Toscana - sezione AREA VASTA CENTRO on 28 March 2017, the ethic code is “Rif CEAVC—2018-438 (ID 10812)”.
The enrolled patients are consecutive and received pembrolizumab alone as first line treatment [33] if their tumors had a PDL1 expression greater than or equal to 50% or nivolumab as second- or third-line treatment [5,6] regardless of the PDL1 expression value after platinum failure.
Nivolumab (240 mg total dose) was administered as 30 min infusion every 2 weeks and continued until disease progression, unacceptable toxicity, withdrawal of consent, or loss to follow up. Pembrolizumab (200 mg total dose) was administered every 3 weeks and continued until disease progression, unacceptable toxicity, withdrawal of consent, or lost to follow up.
During treatment, each subject was monitored, and efficacy assessment was periodically evaluated through radiological evaluations (and according to iRECIST [46] criteria) every 8 or 9 weeks, for nivolumab and pembrolizumab, respectively. Each patient was classified as complete/partial responder or non-responder.
Overall survival (OS), calculated as the time from randomization to death, and time to treatment failure (TTF) [47], calculated as the time from randomization to treatment discontinuation for any reason, including disease progression, treatment toxicity, patient preference, or death [48], were also calculated for each patient.
Blood samples were collected from each patient before starting nivolumab or pembrolizumab treatment (T0 samples). Serum samples were prepared, collected, and stored following standard operating procedures guaranteeing high quality biological samples for metabolomic analysis [49,50]. Briefly, blood samples were collected from after a minimum of 8 h fasted patients and processed within two hours of blood collection. The blood was allowed to clot in an upright position for 30−60 min at room temperature (RT) before centrifugation (1500 RCF for 10 min at RT). Serum samples were immediately stored at −80 °C.
4.2. NMR Analysis
NMR samples were prepared according to standard procedures [50]. Frozen serum samples were thawed at room temperature and shaken before use.
A total of 350 μL of sodium phosphate buffer (70 mM Na2HPO4; 20% (v/v) 2H2O; 6.1 mM NaN3, 4.6 mM sodium trimethylsilyl [2,2,3,3−2H4] propionate (TMSP), pH 7.4) was added to 350 μL of each serum sample, and the mixture was homogenized by vortexing for 30 s. A total of 600 μL of each mixture was transferred into a 5.00 mm NMR tube (Bruker BioSpin) for the analysis.
1H-NMR spectra for all samples were acquired using a Bruker 600 MHz spectrometer (Bruker BioSpin) operating at 600.13 MHz proton Larmor frequency and equipped with a 5 mm PATXI 1H−13C−15N and 2H-decoupling probe including a z axis gradient coil, an automatic tuning-matching (ATM) and an automatic and refrigerated sample changer (SampleJet, Bruker BioSpin). A BTO 2000 thermocouple served for temperature stabilization at the level of approximately 0.1 K at the sample. Before measurement, samples were kept for 5 min inside the NMR probe head, for temperature equilibration at 310 K.
For each serum sample, three monodimensional 1H NMR spectra were acquired at 600 MHz with water peak suppression and different pulse sequences that allowed the selective observation of different molecular components:
-
(i)
A standard NOESY 1Dpresat (noesygppr1d.comp; Bruker BioSpin) pulse sequence (using 32 scans, 98,304 data points, a spectral width of 18,028 Hz, an acquisition time of 2.7 s, a relaxation delay of 4 s and a mixing time of 0.1 s.) to obtain a spectrum in which signals of both metabolites and high molecular weight molecules (lipids and lipoproteins) are visible.
-
(ii)
A standard CPMG (cpmgpr1d.comp; Bruker BioSpin) pulse sequence (using 32 scans, 73,728 data points, a spectral width of 12,019 Hz and a relaxation delay of 4 s.), designed for the selective observation of small molecule components in solutions containing macromolecules.
-
(iii)
A standard diffusion-edited (ledbgppr2s1d.comp; Bruker BioSpin) pulse sequence (using 32 scans, 98,304 data points, a spectral width of 18,028 Hz and a relaxation delay of 4 s.), for the selective observation of macromolecule components in solutions containing small molecules.
4.3. Spectral Processing
Free induction decays were multiplied by an exponential function equivalent to a 0.3 Hz line-broadening factor before applying Fourier transform. Transformed spectra were automatically corrected for phase and baseline distortions and calibrated (glucose doublet at δ 5.24 ppm) using TopSpin 3.5 (Bruker BioSpin). Each spectrum in the region 10.00–0.2 ppm was segmented into 0.02 ppm chemical shift bins, and the corresponding spectral areas were integrated using the AMIX software. Binning is a means to reduce the number of total variables and to compensate for small shifts in the signals, making the fingerprint analyses more robust and reproducible. The area of each bin was normalized to the total spectral area, calculated with exclusion of the water region (4.50–5.00 ppm).
4.4. Statistical Analysis
Multivariate analyses were applied on binned spectra using R software using in-house scripts (the scripts could be provided upon reasonable request).
Principal component analysis (PCA) was used as unsupervised exploratory analysis to obtain a preliminary outlook of the data (presence of clusters or outliers). Orthogonal Partial Least Squares Discriminant Analysis (OPLS-DA) was used to increase supervised data reduction and obtain the best discrimination between the analyzed groups.
The global accuracy for classification was assessed by means of both a Leave-one-out cross-validation scheme and a Monte Carlo cross-validation scheme. Accordingly, each dataset was randomly divided into a training set (90% of the data) and a test set (10% of the data). The training set was used to build the model, whereas the test set was used to validate its discriminant and predictive power; this operation was repeated 500 times.
For each model, the resultant confusion matrix was reported, and its discrimination accuracy, specificity and sensitivity were estimated according to standard definitions. Each classification model was also validated using a permutation test; the permutation was repeated 500 times and the resulting p-value was calculated. Discriminant Q2 values (DQ2) were calculated according to Westerhuis et al. [51].
The metabolites, whose peaks in the spectra were well resolved, were assigned and their concentration levels analyzed, as shown in Table S2. The assignment was performed using an internal NMR spectral library of pure organic compounds, public databases such as the Human Metabolome Database, stored reference NMR spectra of metabolites, and using literature data. Matching between new NMR data and databases was performed using the Assure NMR software (Bruker, Billerica, MA, USA).
The nonparametric Wilcoxon–Mann–Whitney test was used for the determination of the meaningful metabolites. A p-value < 0.05 was considered statistically significant. In order to reduce false discoveries, false discovery rate correction (FDR) was then applied using the Benjamini and Hochberg method.
5. Conclusions
In conclusion, we showed that the individual outcome of anti PD-1 therapy in patients with NSCLC can be predicted by metabolomic analysis of serum collected before commencing treatment, with >80% accuracy. Moreover, metabolomic prediction appears to be independent of ICI agent, making metabolomics a potential strategy for the real-time selection and monitoring of patients treated with immunotherapy. This may address the demands of modern personalized medicine, in which treatment decisions are tailored based on a subject’s individual subtype, with the final aim of avoiding ineffective therapy and improving patient care. A larger confirmatory study is ongoing.
Acknowledgments
VG is the recipient of a post-doctoral fellowship 2018 provided by Fondazione Veronesi. In 2019−2020, VG was supported by AIRC fellowship for Italy.
Supplementary Materials
The following are available online at https://www.mdpi.com/2072-6694/12/12/3574/s1, Table S1: Patient and tumor characteristics for NSCLC cohort, Table S2: List of the metabolites assigned and analyzed in serum samples of NSCLC patients, Figure S1: Bar plot reporting the values of Log2 (Fold Change, FC) of quantified metabolites. Metabolites with Log2 (FC) positive values have higher concentration in plasma samples from responder patients with respect to non-responders. Metabolites with Log2 (FC) negative values have lower concentration in plasma samples from responder patients with respect to non-responders. Red bars represent p-values < 0.05.
Author Contributions
Conceptualization, D.P. and C.L.; methodology, V.G. and T.L.; software, V.G.; validation, V.G. and T.L.; formal analysis, V.G.; investigation, D.P., B.F., and L.L.; resources, F.d.M., B.F., and L.L.; writing—original draft preparation, V.G.; writing—review and editing, V.G., C.L., D.P., A.M., and M.B.; supervision, D.P. and C.L. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Brahmer J.R., Govindan R., Anders R.A., Antonia S.J., Sagorsky S., Davies M.J., Dubinett S.M., Ferris A., Gandhi L., Garon E.B., et al. The Society for Immunotherapy of Cancer consensus statement on immunotherapy for the treatment of non-small cell lung cancer (NSCLC) J. Immunother. Cancer. 2018;6 doi: 10.1186/s40425-018-0382-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Postow M.A., Callahan M.K., Wolchok J.D. Immune Checkpoint Blockade in Cancer Therapy. J. Clin. Oncol. 2015;33:1974–1982. doi: 10.1200/JCO.2014.59.4358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hamada T., Kosumi K., Nakai Y., Koike K. Surrogate study endpoints in the era of cancer immunotherapy. Ann. Transl. Med. 2018;6 doi: 10.21037/atm.2018.09.31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hirsch F.R., Scagliotti G.V., Mulshine J.L., Kwon R., Curran W.J., Wu Y.-L., Paz-Ares L. Lung cancer: Current therapies and new targeted treatments. Lancet. 2017;389:299–311. doi: 10.1016/S0140-6736(16)30958-8. [DOI] [PubMed] [Google Scholar]
- 5.Brahmer J., Reckamp K.L., Baas P., Crinò L., Eberhardt W.E.E., Poddubskaya E., Antonia S., Pluzanski A., Vokes E.E., Holgado E., et al. Nivolumab versus Docetaxel in Advanced Squamous-Cell Non–Small-Cell Lung Cancer. N. Engl. J. Med. 2015;373:123–135. doi: 10.1056/NEJMoa1504627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Borghaei H., Paz-Ares L., Horn L., Spigel D.R., Steins M., Ready N.E., Chow L.Q., Vokes E.E., Felip E., Holgado E., et al. Nivolumab versus Docetaxel in Advanced Non-squamous Non-small Cell Lung Cancer. N. Engl. J. Med. 2015;373:1627–1639. doi: 10.1056/NEJMoa1507643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Herbst R.S., Baas P., Kim D.-W., Felip E., Pérez-Gracia J.L., Han J.-Y., Molina J., Kim J.-H., Arvis C.D., Ahn M.-J., et al. Pembrolizumab versus docetaxel for previously treated, PD-L1-positive, advanced non-small-cell lung cancer (KEYNOTE-010): A randomised controlled trial. Lancet. 2016;387:1540–1550. doi: 10.1016/S0140-6736(15)01281-7. [DOI] [PubMed] [Google Scholar]
- 8.Rittmeyer A., Barlesi F., Waterkamp D., Park K., Ciardiello F., von Pawel J., Gadgeel S.M., Hida T., Kowalski D.M., Dols M.C., et al. Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): A phase 3, open-label, multicentre randomised controlled trial. Lancet. 2017;389:255–265. doi: 10.1016/S0140-6736(16)32517-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gettinger S.N., Horn L., Gandhi L., Spigel D.R., Antonia S.J., Rizvi N.A., Powderly J.D., Heist R.S., Carvajal R.D., Jackman D.M., et al. Overall Survival and Long-Term Safety of Nivolumab (Anti–Programmed Death 1 Antibody, BMS-936558, ONO-4538) in Patients with previously Treated Advanced Non–Small-Cell Lung Cancer. J. Clin. Oncol. 2015;33:2004–2012. doi: 10.1200/JCO.2014.58.3708. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gandhi L., Rodríguez-Abreu D., Gadgeel S., Esteban E., Felip E., De Angelis F., Domine M., Clingan P., Hochmair M.J., Powell S.F., et al. Pembrolizumab plus Chemotherapy in Metastatic Non-Small-Cell Lung Cancer. N. Engl. J. Med. 2018;378:2078–2092. doi: 10.1056/NEJMoa1801005. [DOI] [PubMed] [Google Scholar]
- 11.Antonia S.J., Villegas A., Daniel D., Vicente D., Murakami S., Hui R., Yokoi T., Chiappori A., Lee K.H., de Wit M., et al. Durvalumab after Chemoradiotherapy in Stage III Non-Small-Cell Lung Cancer. N. Engl. J. Med. 2017;377:1919–1929. doi: 10.1056/NEJMoa1709937. [DOI] [PubMed] [Google Scholar]
- 12.Brahmer J.R. Immune checkpoint blockade: The hope for immunotherapy as a treatment of lung cancer? Semin. Oncol. 2014;41:126–132. doi: 10.1053/j.seminoncol.2013.12.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wood M.A., Weeder B.R., David J.K., Nellore A., Thompson R.F. Burden of tumor mutations, neoepitopes, and other variants are weak predictors of cancer immunotherapy response and overall survival. Genome Med. 2020;12:33. doi: 10.1186/s13073-020-00729-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wang Q., Gao J., Wu X. Pseudoprogression and hyperprogression after checkpoint blockade. Int. Immunopharmacol. 2018;58:125–135. doi: 10.1016/j.intimp.2018.03.018. [DOI] [PubMed] [Google Scholar]
- 15.Nicholson J.K., Lindon J.C. Systems biology: Metabonomics. Nature. 2008;455:1054–1056. doi: 10.1038/4551054a. [DOI] [PubMed] [Google Scholar]
- 16.Nicholson G., Rantalainen M., Maher A.D., Li J.V., Malmodin D., Ahmadi K.R., Faber J.H., Hallgrímsdóttir I.B., Barrett A., Toft H., et al. Human metabolic profiles are stably controlled by genetic and environmental variation. Mol. Syst. Biol. 2011;7:525. doi: 10.1038/msb.2011.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ghini V., Tenori L., Capozzi F., Luchinat C., Bub A., Malpuech-Brugere C., Orfila C., Ricciardiello L., Bordoni A. DHA-Induced Perturbation of Human Serum Metabolome. Role of the Food Matrix and Co-Administration of Oat β-glucan and Anthocyanins. Nutrients. 2019;12:86. doi: 10.3390/nu12010086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Ghini V., Saccenti E., Tenori L., Assfalg M., Luchinat C. Allostasis and Resilience of the Human Individual Metabolic Phenotype. J. Proteome Res. 2015;14:2951–2962. doi: 10.1021/acs.jproteome.5b00275. [DOI] [PubMed] [Google Scholar]
- 19.Antonaros F., Ghini V., Pulina F., Ramacieri G., Cicchini E., Mannini E., Martelli A., Feliciello A., Lanfranchi S., Onnivello S., et al. Plasma metabolome and cognitive skills in Down syndrome. Sci. Rep. 2020;10:10491. doi: 10.1038/s41598-020-67195-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ghini V., Unger F.T., Tenori L., Turano P., Juhl H., David K.A. Metabolomics profiling of pre-and post-anesthesia plasma samples of colorectal patients obtained via Ficoll separation. Metabolomics. 2015;11:1769–1778. doi: 10.1007/s11306-015-0832-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Meoni G., Lorini S., Monti M., Madia F., Corti G., Luchinat C., Zignego A.L., Tenori L., Gragnani L. The metabolic fingerprints of HCV and HBV infections studied by Nuclear Magnetic Resonance Spectroscopy. Sci. Rep. 2019;9:4128. doi: 10.1038/s41598-019-40028-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dani C., Bresci C., Berti E., Ottanelli S., Mello G., Mecacci F., Breschi R., Hu X., Tenori L., Luchinat C. Metabolomic profile of term infants of gestational diabetic mothers. J. Matern. Fetal Neonatal Med. 2014;27:537–542. doi: 10.3109/14767058.2013.823941. [DOI] [PubMed] [Google Scholar]
- 23.Hart C.D., Vignoli A., Tenori L., Uy G.L., Van To T., Adebamowo C., Hossain S.M., Biganzoli L., Risi E., Love R.R., et al. Serum Metabolomic Profiles Identify ER-Positive Early Breast Cancer Patients at Increased Risk of Disease Recurrence in a Multicenter Population. Clin. Cancer Res. 2017;23:1422–1431. doi: 10.1158/1078-0432.CCR-16-1153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Bertini I., Cacciatore S., Jensen B.V., Schou J.V., Johansen J.S., Kruhøffer M., Luchinat C., Nielsen D.L., Turano P. Metabolomic NMR Fingerprinting to Identify and Predict Survival of Patients with Metastatic Colorectal Cancer. Cancer Res. 2012;72:356–364. doi: 10.1158/0008-5472.CAN-11-1543. [DOI] [PubMed] [Google Scholar]
- 25.Spratlin J.L., Serkova N.J., Eckhardt G.S. Clinical Applications of Metabolomics in Oncology: A Review. Clin. Cancer Res. 2009;15:431–440. doi: 10.1158/1078-0432.CCR-08-1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Takis P.G., Ghini V., Tenori L., Turano P., Luchinat C. Uniqueness of the NMR approach to metabolomics. TrAC Trends Anal. Chem. 2019;120:115300. doi: 10.1016/j.trac.2018.10.036. [DOI] [Google Scholar]
- 27.Vignoli A., Ghini V., Meoni G., Licari C., Takis P.G., Tenori L., Turano P., Luchinat C. High-Throughput Metabolomics by 1D NMR. Angew. Chem. Int. Ed. Engl. 2019;58:968–994. doi: 10.1002/anie.201804736. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Wishart D.S. Emerging applications of metabolomics in drug discovery and precision medicine. Nat. Rev. Drug Discov. 2016;15:473–484. doi: 10.1038/nrd.2016.32. [DOI] [PubMed] [Google Scholar]
- 29.Everett J.R. NMR-based pharmacometabonomics: A new paradigm for personalised or precision medicine. Prog. Nucl. Magn. Reson. Spectrosc. 2017;102-103:1–14. doi: 10.1016/j.pnmrs.2017.04.003. [DOI] [PubMed] [Google Scholar]
- 30.Vignoli A., Tenori L., Giusti B., Valente S., Carrabba N., Balzi D., Barchielli A., Marchionni N., Gensini G.F., Marcucci R., et al. Differential Network Analysis Reveals Metabolic Determinants Associated with Mortality in Acute Myocardial Infarction Patients and Suggests Potential Mechanisms Underlying Different Clinical Scores Used to Predict Death. J. Proteome Res. 2020;19:949–961. doi: 10.1021/acs.jproteome.9b00779. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Garon E.B., Rizvi N.A., Hui R., Leighl N., Balmanoukian A.S., Eder J.P., Patnaik A., Aggarwal C., Gubens M., Horn L., et al. Pembrolizumab for the treatment of non-small-cell lung cancer. N. Engl. J. Med. 2015;372:2018–2028. doi: 10.1056/NEJMoa1501824. [DOI] [PubMed] [Google Scholar]
- 32.Hellmann M.D., Ciuleanu T.-E., Pluzanski A., Lee J.S., Otterson G.A., Audigier-Valette C., Minenza E., Linardou H., Burgers S., Salman P., et al. Nivolumab plus Ipilimumab in Lung Cancer with a High Tumor Mutational Burden. N. Engl. J. Med. 2018;378:2093–2104. doi: 10.1056/NEJMoa1801946. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Reck M., Rodríguez-Abreu D., Robinson A.G., Hui R., Csőszi T., Fülöp A., Gottfried M., Peled N., Tafreshi A., Cuffe S., et al. Pembrolizumab versus Chemotherapy for PD-L1-Positive Non-Small-Cell Lung Cancer. N. Engl. J. Med. 2016;375:1823–1833. doi: 10.1056/NEJMoa1606774. [DOI] [PubMed] [Google Scholar]
- 34.De Vries R., Muller M., van der Noort V., Theelen W.S.M.E., Schouten R.D., Hummelink K., Muller S.H., Wolf-Lansdorf M., Dagelet J.W.F., Monkhorst K., et al. Prediction of response to anti-PD-1 therapy in patients with non-small-cell lung cancer by electronic nose analysis of exhaled breath. Ann. Oncol. 2019;30:1660–1666. doi: 10.1093/annonc/mdz279. [DOI] [PubMed] [Google Scholar]
- 35.Duruisseaux M., Martínez-Cardús A., Calleja-Cervantes M.E., Moran S., de Moura C.M., Davalos V., Piñeyro D., Sanchez-Cespedes M., Girard N., Brevet M., et al. Epigenetic prediction of response to anti-PD-1 treatment in non-small-cell lung cancer: A multicentre, retrospective analysis. Lancet Respir. Med. 2018;6:771–781. doi: 10.1016/S2213-2600(18)30284-4. [DOI] [PubMed] [Google Scholar]
- 36.Hatae R., Chamoto K., Kim Y.H., Sonomura K., Taneishi K., Kawaguchi S., Yoshida H., Ozasa H., Sakamori Y., Akrami M., et al. Combination of host immune metabolic biomarkers for the PD-1 blockade cancer immunotherapy. JCI Insight. 2020;5 doi: 10.1172/jci.insight.133501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Botticelli A., Vernocchi P., Marini F., Quagliariello A., Cerbelli B., Reddel S., Del Chierico F., Di Pietro F., Giusti R., Tomassini A., et al. Gut metabolomics profiling of non-small cell lung cancer (NSCLC) patients under immunotherapy treatment. J. Transl. Med. 2020;18:49. doi: 10.1186/s12967-020-02231-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Li H., Bullock K., Gurjao C., Braun D., Shukla S.A., Bossé D., Lalani A.-K.A., Gopal S., Jin C., Horak C., et al. Metabolomic adaptations and correlates of survival to immune checkpoint blockade. Nat. Commun. 2019;10:1–6. doi: 10.1038/s41467-019-12361-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Lim A.R., Rathmell W.K., Rathmell J.C. The tumor microenvironment as a metabolic barrier to effector T cells and immunotherapy. Elife. 2020;9:e55185. doi: 10.7554/eLife.55185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Hanahan D., Weinberg R.A. Hallmarks of Cancer: The Next Generation. Cell. 2011;144:646–674. doi: 10.1016/j.cell.2011.02.013. [DOI] [PubMed] [Google Scholar]
- 41.Harris A.L. Hypoxia—A key regulatory factor in tumour growth. Nat. Rev. Cancer. 2002;2:38–47. doi: 10.1038/nrc704. [DOI] [PubMed] [Google Scholar]
- 42.Wilson W.R., Hay M.P. Targeting hypoxia in cancer therapy. Nat. Rev. Cancer. 2011;11:393–410. doi: 10.1038/nrc3064. [DOI] [PubMed] [Google Scholar]
- 43.Bernacchioni C., Ghini V., Cencetti F., Japtok L., Donati C., Bruni P., Turano P. NMR metabolomics highlights sphingosine kinase-1 as a new molecular switch in the orchestration of aberrant metabolic phenotype in cancer cells. Mol. Oncol. 2017;11:517–533. doi: 10.1002/1878-0261.12048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.D’Alessandro G., Quaglio D., Monaco L., Lauro C., Ghirga F., Ingallina C., De Martino M., Fucile S., Porzia A., Di Castro M.A., et al. 1H-NMR metabolomics reveals the Glabrescione B exacerbation of glycolytic metabolism beside the cell growth inhibitory effect in glioma. Cell Commun. Signal. 2019;17:108. doi: 10.1186/s12964-019-0421-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Prasad V., Kaestner V. Nivolumab and pembrolizumab: Monoclonal antibodies against programmed cell death-1 (PD-1) that are interchangeable. Semin. Oncol. 2017;44:132–135. doi: 10.1053/j.seminoncol.2017.06.007. [DOI] [PubMed] [Google Scholar]
- 46.Seymour L., Bogaerts J., Perrone A., Ford R., Schwartz L.H., Mandrekar S., Lin N.U., Litière S., Dancey J., Chen A., et al. iRECIST: Guidelines for response criteria for use in trials testing immunotherapeutics. Lancet Oncol. 2017;18:e143–e152. doi: 10.1016/S1470-2045(17)30074-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Sridhara R., Zhou J., Theoret M.R., Mishra-Kalyani P.S. Time to treatment failure (TTF) as a potential clinical endpoint in real-world evidence (RWE) studies of melanoma. J. Clin. Oncol. 2018;36:9578. doi: 10.1200/JCO.2018.36.15_suppl.9578. [DOI] [Google Scholar]
- 48.Pazdur R. Endpoints for assessing drug activity in clinical trials. Oncologist. 2008;13:19–21. doi: 10.1634/theoncologist.13-S2-19. [DOI] [PubMed] [Google Scholar]
- 49.Ghini V., Quaglio D., Luchinat C., Turano P. NMR for sample quality assessment in metabolomics. New Biotechnol. 2019;52:25–34. doi: 10.1016/j.nbt.2019.04.004. [DOI] [PubMed] [Google Scholar]
- 50.Bernini P., Bertini I., Luchinat C., Nincheri P., Staderini S., Turano P. Standard operating procedures for pre-analytical handling of blood and urine for metabolomic studies and biobanks. J. Biomol. NMR. 2011;49:231–243. doi: 10.1007/s10858-011-9489-1. [DOI] [PubMed] [Google Scholar]
- 51.Westerhuis J.A., van Velzen E.J.J., Hoefsloot H.C.J., Smilde A.K. Discriminant Q2 (DQ2) for improved discrimination in PLSDA models. Metabolomics. 2008;4:293–296. doi: 10.1007/s11306-008-0126-2. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.