Skip to main content
PMC home page
eLife logoLink to eLife
. 2017 Jun 27;6:e26030. doi: 10.7554/eLife.26030

Replication Study: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate

Megan Reed Showalter 1,†, Jason Hatakeyama 2,3,†, Tomas Cajka 1,†, Kacey VanderVorst 2,3, Kermit L Carraway III 2,3, Oliver Fiehn 1; Reproducibility Project: Cancer Biology*
Editor: Jessica K Tyler4
PMCID: PMC5487214  PMID: 28653623

Abstract

In 2016, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Fiehn et al., 2016), that described how we intended to replicate selected experiments from the paper "The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate" (Ward et al., 2010). Here, we report the results of those experiments. We found that cells expressing R172K mutant IDH2 did not display isocitrate-dependent NADPH production above vector control levels, in contrast to the increased production observed with wild-type IDH2. Conversely, expression of R172K mutant IDH2 resulted in increased alpha-ketoglutarate-dependent consumption of NADPH compared to wild-type IDH2 or vector control. These results are similar to those reported in the original study (Figure 2; Ward et al., 2010). Further, expression of R172K mutant IDH2 resulted in increased 2HG levels within cells compared to the background levels observed in wild-type IDH2 and vector control, similar to the original study (Figure 3D; Ward et al., 2010). In primary human AML samples, the 2HG levels observed in samples with mutant IDH1 or IDH2 status were higher than those observed in samples without an IDH mutation, similar to what was observed in the original study (Figure 5C; Ward et al., 2010). Finally, we report meta-analyses for each result.

DOI: http://dx.doi.org/10.7554/eLife.26030.001

Research Organism: Human

Introduction

The Reproducibility Project: Cancer Biology (RP:CB) is a collaboration between the Center for Open Science and Science Exchange that seeks to address concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology (Errington et al., 2014). For each of these papers a Registered Report detailing the proposed experimental designs and protocols for the replications was peer reviewed and published prior to data collection. The present paper is a Replication Study that reports the results of the replication experiments detailed in the Registered Report (Fiehn et al., 2016) for a paper by Ward et al., and uses a number of approaches to compare the outcomes of the original experiments and the replications.

In 2010, Ward et al. demonstrated that a shared feature of isocitrate dehydrogenase (IDH) mutations identified in gliomas and acute myeloid leukemia (AML) is the acquisition of a neomorphic enzyme activity. This study built upon a previous study demonstrating cancer-associated IDH1 mutations resulted in increased production of the oncometabolite 2-hydroxyglutarate (2HG) (Dang et al., 2009). Similar to the IDH1 R132 mutation, the cancer-associated IDH2 R172 mutation was demonstrated to be deficient in isocitrate-dependent NADPH production (Yan et al., 2009; Ward et al., 2010), while acquiring a gain of function for alpha-ketoglutarate-dependent NADPH consumption that resulted in the production of 2HG (Ward et al., 2010). 2HG was also reported to be detectable in AML patient samples and predictive of IDH mutational status (Ward et al., 2010).

The Registered Report for the paper by Ward et al. described the experiments to be replicated (Figures 2, 3D and 5), and summarized the current evidence for these findings (Fiehn et al., 2016). Since that publication, additional studies have reported R172 mutant IDH2 exhibiting decreased NADPH production, as well as increased intracellular accumulation of 2HG compared to wild-type IDH2 (Carbonneau et al., 2016; Robinson et al., 2016). Similarly, there have been numerous studies that have observed this with IDH1 mutations (Chaturvedi et al., 2017; Ward et al., 2013). There have also been numerous studies reporting the utility of using 2HG as a predictive biomarker for the presence of IDH mutations. This includes the detection of 2HG in the peripheral blood of patients with AML (DiNardo et al., 2013; McGehee et al., 2016), serum samples from patients with a myeloid sarcoma diagnosis (Willekens et al., 2015), serum samples from patients with angioimmunoblastic T-cell lymphoma (Lemonnier et al., 2016), as well as plasma, urine, and the cerebrospinal fluid in glioma patients (Fathi et al., 2016; Kalinina et al., 2016; Lombardi et al., 2015). Additional strategies have also been reported to detect 2HG noninvasively in glioma that are based on advanced imaging and spectroscopy approaches (An et al., 2016; de la Fuente et al., 2016; Ganji et al., 2016; Verma et al., 2016). Clinical trials are also underway to target mutant IDH1 and IDH2 in AML, gliomas, myelodysplastic syndrome, and other solid tumors (Mondesir et al., 2016; Vander Heiden and DeBerardinis, 2017).

The outcome measures reported in this Replication Study will be aggregated with those from the other Replication Studies to create a dataset that will be examined to provide evidence about reproducibility of cancer biology research, and to identify factors that influence reproducibility more generally.

Results and discussion

Assessing the isocitrate-dependent NADPH production and alpha-ketoglutarate-dependent NADPH consumption of wild-type or R172K mutant IDH2

We sought to independently replicate whether the cancer-associated IDH2 R172 mutation, exhibited a loss-of-function for isocitrate-dependent NADPH production and a gain-of-function for alpha-ketoglutarate-dependent NADPH consumption compared to wild-type IDH2. This experiment is similar to what was reported in Figure 2 of Ward et al. (2010) and described in Protocol 1 in the Registered Report (Fiehn et al., 2016). HEK293T cells, which endogenously express wild-type IDH2, were transfected to ectopically express wild-type or R172 mutant IDH2 (Figure 1E). Extracts from cells expressing wild-type IDH2 displayed an increase in isocitrate-dependent NADPH production (0.49 [n = 7, SD = 0.090] change in OD340 after 30 min); while, extracts from cells expressing R172K mutant IDH2 reached a 0.12 [n = 7, SD = 0.069] change in OD340 over the same period, similar to the levels observed in extracts from vector control transfected cells (0.094 [n = 7, SD = 0.055] change in OD340) (Figure 1A). This compares to the original study that reported extracts from wild-type IDH2 expressing cells increased isocitrate-dependent NADPH production (~1.3 change in OD340 after 30 min), while R172K mutant IDH2 and vector control resulted in an ~0.1 and ~0.02 change in OD340, respectively (Ward et al., 2010). The same extracts were also tested for alpha-ketoglutarate-dependent NADPH consumption (Figure 1C). Expression of R172K mutant IDH2 resulted in a 0.022 [n = 7, SD = 0.0077] change in OD340 after 180 min, while wild-type IDH2 transfected cell extracts had minimal activity (0.0034 [n = 7, SD = 0.0019] change in OD340), similar to vector control (0.0019 [n = 7, SD = 0.0040] change in OD340). This compares to the original study that reported an ~0.06 change in OD340 for R172K mutant IDH2 expression, an ~0.014 change in OD340 for wild-type IDH2, and an ~0.003 change in OD340 for vector control after 171 min (Ward et al., 2010). The variation in the change in OD340 observed between the original and replication studies might be due to methodological differences such as the adjustment in the amount of extract and substrate added to each reaction to generate a sufficient signal to noise, as well as other critical experimental and plate reader settings (Chavez et al., 2017; Giehm and Otzen, 2010).

Figure 1. Isocitrate-dependent NADPH production and alpha-ketoglutarate-dependent NADPH consumption of wild-type or R172 mutant IDH2.

Figure 1.

Open in a new tab

IDH oxidative and reductive activity assays were performed on lysates of HEK293T cells transfected with wild-type or R172K mutant IDH2, or empty vector. (A) Lysates were assessed for generation of NADPH from NADP+ in the presence of 0.4 mM isocitrate over the indicated time course. Mean change in OD340 from the beginning of each assay (time = 0) is reported for each biological repeat performed [n = 7] and error bars represent s.e.m. (B) Linear regression slopes were determined for each biological repeat of the isocitrate-dependent NADPH production assay. Box and whisker plot with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Means as black dot and bold error bars represent 95% CI. Linear regression slope determined from the data estimated from the representative experiment reported in Figure 2A of Ward et al. (2010) is displayed as a single point (red circle) for comparison. Statistical analysis was performed on data generated during this replication attempt. Wild-type IDH2 compared to vector control: Wilcoxon-Mann-Whitney test; Z = 3.13, uncorrected p=0.00058, Bonferroni corrected p=0.0023. R172K mutant IDH2 compared to vector control: Wilcoxon-Mann-Whitney test; Z = 1.60, uncorrected p=0.13, Bonferroni corrected p=0.51. (C) Lysates were assessed for consumption of NADPH in the presence of 1 mM alpha-ketoglutarate over the indicated time course. Mean change in OD340 from the beginning of each assay (time = 0) is reported for each biological repeat performed [n = 7] and error bars represent s.e.m. (D) Linear regression slopes were determined for each biological repeat of the alpha-ketoglutarate-dependent NADPH consumption assay. Box and whisker plot with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Means as black dot and bold error bars represent 95% CI. Linear regression slope determined from the data estimated from the representative experiment reported in Figure 2B of Ward et al. (2010) is displayed as a single point (red circle) for comparison. Statistical analysis was performed on data generated during this replication attempt. Wild-type IDH2 compared to vector control: Welch’s t-test; t(9.42) = 0.29, uncorrected p=0.78, Bonferroni corrected p>0.99. R172K mutant IDH2 compared to vector control: Welch’s t-test; t(6.95) = 5.97, uncorrected p=0.00058, Bonferroni corrected p=0.0023. (E) Representative Western blots probed with an anti-IDH1 antibody, an anti-IDH2 antibody, and an anti-alpha-Tubulin antibody. Additional details for this experiment can be found at https://osf.io/6ve4d/.

DOI: http://dx.doi.org/10.7554/eLife.26030.002

There are multiple approaches that could be taken to explore these data; however, to provide a direct comparison to the original data, we are reporting the analysis specified a priori in the Registered Report (Fiehn et al., 2016). As outlined in the Registered Report, we planned to determine the linear regression slope for each biological repeat and compare vector control to wild-type IDH2 or R172K mutant IDH2 for both assays. To account for these multiple comparisons, the Bonferroni correction was used, making the a priori Bonferroni adjusted significance threshold 0.0125. For the isocitrate-dependent NADPH production assay, we performed a test on the slopes from vector control compared to wild-type IDH2, which was statistically significant (Wilcoxon-Mann-Whitney test; Z = 3.13, uncorrected p=0.00058, corrected p=0.0023). Thus, the null hypothesis that wild-type IDH2 and vector control have similar rates of isocitrate-dependent NADPH production can be rejected, while the test on the slopes from vector control compared to R172K mutant IDH2 was not statistically significant (Wilcoxon-Mann-Whitney test; Z = 1.60, uncorrected p=0.13, corrected p=0.51). Conversely, for the alpha-ketoglutarate-dependent NADPH consumption assay, the test on the slopes from vector control compared to wild-type IDH2 was not statistically significant (Welch’s t-test; t(9.42) = 0.29, uncorrected p=0.78, corrected p>0.99), while the test on the slopes from vector control compared to R172K mutant IDH2 was statistically significant (Welch’s t-test; t(6.95) = 5.97, uncorrected p=0.00058, corrected p=0.0023). To summarize, for this experiment we found results that were in the same direction as the original study and statistically significant where predicted.

Production of 2HG from wild-type or R172K mutant IDH2 transfected cells

HEK293T cells expressing ectopic wild-type IDH2, R172K mutant IDH2, or vector control were analyzed for accumulation of the metabolite 2HG to test if R172K mutant IDH2 acquired enhanced conversion of alpha-ketoglutarate to 2HG. This experiment is similar to what was reported in Figure 3D of Ward et al. (2010), and described in Protocol 2 in the Registered Report (Fiehn et al., 2016), with cells analyzed 24 and 48 hr after transfection. Following expression of the ectopic proteins, the accumulation of 2HG, as well as other organic acids including aspartate and glutamate, were assessed by gas-chromatography mass spectrometry (GC-MS) after MTBSTFA derivatization. Metabolites were identified based on retention times and electron ionization (EI) spectra that matched with derivatized commercial standards. The intracellular levels of 2HG in HEK293T cells expressing R172K mutant IDH2 were determined using the total ion chromatograms (TIC) and found to be over 600 times the levels detected in vector control and wild-type IDH2 expressing cells 48 hr after transfection (Figure 2A–B). This compares to the original study, which reported an approximately 100 times increase in intracellular 2HG levels in cells expressing R172K mutant IDH2 compared to vector control and wild-type IDH2 (Ward et al., 2010). The difference in magnitude between the original and replication studies is not due to the observed intracellular 2HG/glutamate levels in R172K mutant IDH2 expressing cells, but rather the lower bounds of detection in the vector control and wild-type IDH2 conditions (Figure 2A–B). A similar observation was observed 24 hr after transfection as well as analysis using the extracted ion chromatograms (EIC) (Figure 2C–D, Figure 2—figure supplement 1).

Figure 2. 2HG levels within cells expressing wild-type or R172K mutant IDH2.

HEK293T cells transfected with wild-type or R172K mutant IDH2, or empty vector, were analyzed for intracellular metabolites. (A) Cells harvested 48 hr after transfection had organic acids extracted, purified, and derivatized with MTBSTFA before analysis by GC-MS. Quantitation of 2HG signal intensity relative to glutamate was determined using the TIC for each biological repeat [n = 10]. Box and whisker plot with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Means as black dot and bold error bars represent 95% CI. Data estimated from the representative experiment reported in Figure 3D of Ward et al. (2010) is displayed as a single point (red circle) for comparison. Statistical analysis was performed on log10 transformed data generated during this replication attempt. R172K mutant IDH2 values were compared to the largest value observed in cells expressing vector control or wild-type IDH2 (0.005). One-sample t-test; t(9) = 21.9, uncorrected p=4.14×10−9, Bonferroni corrected p=8.28×10−9. (B) Representative TIC from samples harvested at 48 hr after transfection for vector control (top panel), wild-type IDH2 (middle panel), and R172K mutant IDH2 (bottom panel). The derivatized organic acids eluting between 31.4 and 34.4 min are shown, including aspartate (31.7 min), 2HG (33.1 min), and glutamate (34.3 min) based on spectra of derivatized commercial standards. (C) Cells harvested 24 hr after transfection were processed and analyzed similar to the 48 hr samples with [n = 10] biological repeats. Box and whisker plot with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Means as black dot and bold error bars represent 95% CI. R172K mutant IDH2 values were compared to the largest value observed in cells expressing vector control or wild-type IDH2 (0.001). One-sample Wilcoxon signed-rank test on log10 transformed data; V = 55, uncorrected p=0.0020, Bonferroni corrected p=0.0039. (D) Representative TIC from samples harvested at 24 hr after transfection for vector control (top panel), wild-type IDH2 (middle panel), and R172K mutant IDH2 (bottom panel). Additional details for this experiment can be found at https://osf.io/9ge2a/.

DOI: http://dx.doi.org/10.7554/eLife.26030.003

Figure 2.

Figure 2—figure supplement 1. 2HG levels within cells based on EIC analysis.

Figure 2—figure supplement 1.

This is the same experiment as in Figure 2, but with quantitation of 2HG signal intensity relative to glutamate determined using the EIC for each biological repeat. (A) Box and whisker plot with mean and 95% CI reported for each sample harvested 48 hr after transfection. R172K mutant IDH2 values were compared to the largest value observed in cells expressing vector control or wild-type IDH2 (0.004). One-sample t-test; t(9) = 22.4, uncorrected p=3.38×10−9, Bonferroni corrected p=6.77×10−9. (B) Representative EIC from samples harvested at 48 hr after transfection for vector control (left panels), wild-type IDH2 (middle panels), and R172K mutant IDH2 (right panels) with aspartate, glutamate, and 2HG shown. (C) Box and whisker plot with mean and 95% CI reported for each sample harvested 24 hr after transfection. R172K mutant IDH2 values were compared to the largest value observed in cells expressing vector control or wild-type IDH2 (0.001). One-sample Wilcoxon signed-rank test on log10 transformed data; V = 55, uncorrected p=0.0020, Bonferroni corrected p=0.0039. (D) Representative EIC from samples harvested at 24 hr after transfection for vector control (left panels), wild-type IDH2 (middle panels), and R172K mutant IDH2 (right panels) with aspartate, glutamate, and 2HG shown. Additional details for this experiment can be found at https://osf.io/9ge2a/.
DOI: http://dx.doi.org/10.7554/eLife.26030.004
Open in a new tab

The analysis plan specified in the Registered Report (Fiehn et al., 2016) proposed to compare the 2HG/glutamate levels among the different conditions. However, because of the near background levels of vector control and wild-type IDH2, we performed an exploratory analysis using the R172K mutant IDH2 values compared to the largest value observed in cells expressing vector control or wild-type IDH2. This resulted in a statistically significant result for the TIC analysis (one-sample t-test; t(9) = 21.9, uncorrected p=4.14×10−9, Bonferroni corrected p=8.28×10−9) and the EIC analysis (one-sample t-test; t(9) = 22.4, uncorrected p=3.38×10−9, Bonferroni corrected p=6.77×10−9) 48 hr after transfection. Similar results were determined on the samples analyzed 24 hr after transfection (one-sample Wilcoxon signed-rank test; V = 55, uncorrected p=0.0020, Bonferroni corrected p=0.0039). To summarize, for this experiment we found results that were in the same direction as the original study and statistically significant where predicted.

Assessing 2hg levels in AML patient samples

To test if 2HG is elevated in AML patients with IDH mutations, we performed GC-MS analysis on samples of AML cells from patients. This is comparable to the experiment that was reported in Figure 5 of Ward et al. (2010), and described in Protocol 3 in the Registered Report (Fiehn et al., 2016), except the mutational status of IDH1 and IDH2 were determined prior to conducting the metabolite analysis to have adequate samples for each genetically distinct group. Similar to the original study, all wild-type IDH samples displayed a 2HG/glutamate level less than 1%, while samples that had an IDH1 or IDH2 mutation had a 2HG/glutamate level greater than 1% (Figure 3, Figure 3—figure supplement 1). Like the metabolite analysis conducted with HEK293T cells, the lower limits of detection in the patient samples without an IDH mutation were lower in this replication attempt (mean 2HG/glutamate ratio of 0.001) than what was reported in the original study (mean 2HG/glutamate ratio of ~0.005). However, the average ratio in the samples with IDH mutations reported in this replication attempt (mean 2HG/glutamate ratio of 0.20) were lower than what was reported in the original study (mean 2HG/glutamate ratio of ~0.8). Although these ratios do not allow for a true comparison of the absolute 2HG levels, a comparison of the 2HG peaks between samples with or without IDH mutations resulted in a similar trend. Of interest, detection of 2HG in wild-type IDH samples, albeit at much lower levels than in samples with an IDH mutation, has also been reported in other studies measuring 2HG levels in patient samples (Borger et al., 2014; DiNardo et al., 2013; Fathi et al., 2012; Gross et al., 2010; Janin et al., 2014; Lemonnier et al., 2016; Natsumeda et al., 2014). Cells without IDH mutations produce small quantities of 2HG, although the mechanism is poorly understood and thought to be largely due to error of normal metabolism (Andronesi et al., 2013; Intlekofer et al., 2015). While detection of 2HG levels in serum and urine in AML and intrahepatic cholangiocarcinoma has been reported by various studies as a biomarker for IDH mutational status, detection in glioma samples does not appear as straightforward with no difference between serum samples, likely due to the blood–brain barrier, and further evaluation needed with the use of urinary samples (Fathi et al., 2016; Lombardi et al., 2016, 2015).

Figure 3. 2HG levels in AML patient samples.

AML patient peripheral blood or bone marrow samples were analyzed for intracellular metabolites. Cells had organic acids extracted, purified, and derivatized with MTBSTFA before analysis by GC-MS. Patient samples were prescreened for IDH genotypic status prior to metabolite analysis. (A) Quantitation of 2HG signal intensity relative to glutamate was determined using the TIC for each sample. Dot plot with means reported as crossbars. One-sample t-tests on log10 transformed data comparing 2HG/glutamate level of samples to a constant of 0.0024 (2HG/glutamate threshold between IDH mutant and wild-type IDH samples). Samples with an IDH1 mutation compared to the threshold constant: t(3) = 4.47, uncorrected p=0.021, Bonferroni corrected p=0.063. Samples with an IDH2 mutation compared to the threshold constant: t(3) = 9.19, uncorrected p=0.0027, Bonferroni corrected p=0.0082. Samples with an IDH1 or IDH2 mutation compared to the threshold constant: t(7) = 8.73, uncorrected p=5.20×10−5, Bonferroni corrected p=1.56×10−4. (B) Representative TIC from samples without an IDH mutation (top panel), samples with an IDH1 mutation (middle panel), and samples with an IDH2 mutation (bottom panel). The derivatized organic acids eluting between 31.4 and 34.4 min are shown, including aspartate (31.7 min), 2HG (33.1 min), and glutamate (34.3 min) based on spectra of derivatized commercial standards. Additional details for this experiment can be found at https://osf.io/smdfr/.

DOI: http://dx.doi.org/10.7554/eLife.26030.005

Figure 3.

Figure 3—figure supplement 1. 2HG levels in AML patient samples based on EIC analysis.

Figure 3—figure supplement 1.

This is the same experiment as in Figure 3, but with quantitation of 2HG signal intensity relative to glutamate determined using the EIC for each AML patient sample. (A) Dot plot with means reported as crossbars. One-sample t-tests on log10 transformed data comparing 2HG/glutamate level of samples to a constant of 0.001 (2HG/glutamate threshold between IDH mutant and wild-type IDH samples). Samples with an IDH1 mutation compared to the threshold constant: t(3) = 5.40, uncorrected p=0.012, Bonferroni corrected p=0.037. Samples with an IDH2 mutation compared to the threshold constant: t(3) = 10.93, uncorrected p=0.0016, Bonferroni corrected p=0.0049. Samples with an IDH1 or IDH2 mutation compared to the threshold constant: t(7) = 10.44, uncorrected p=1.61×10−5, Bonferroni corrected p=4.84×10−5. (B) Representative EIC from samples without an IDH mutation (top panel), samples with an IDH1 mutation (middle panel), and samples with an IDH2 mutation (bottom panel) with aspartate, glutamate, and 2HG shown. Additional details for this experiment can be found at https://osf.io/smdfr/.
DOI: http://dx.doi.org/10.7554/eLife.26030.006
Open in a new tab

As outlined in the Registered Report (Fiehn et al., 2016), we planned to conduct three comparisons using the Bonferroni correction to adjust for multiple comparisons, making the a priori Bonferroni adjusted significance threshold 0.0167. We performed one-sample t-tests comparing the 2HG/glutamate level determined by TIC (log10 transformed) to a constant of 0.0024, which was the 2HG/glutamate threshold to delineate between IDH mutant and wild-type IDH samples. The sample size was determined a priori to detect the effects based on the originally reported data, which reported the 2HG/glutamate threshold as 0.01. The test on samples with an IDH2 mutation was statistically significant (t(3) = 9.19, uncorrected p=0.0027, corrected p=0.0082), as was the test on samples with an IDH1 or IDH2 mutation (t(7) = 8.73, uncorrected p=5.20×10−5, corrected p=1.56×10−4). The planned comparison of samples with an IDH1 mutation, however, was not statistically significant (t(3) = 4.47, uncorrected p=0.021, corrected p=0.063). The same analysis was conducted using the 2HG/glutamate levels determined by EIC, where the observed 2HG/glutamate threshold between IDH mutant and wild-type IDH samples was 0.001. All effects were found to be statistically significant (IDH1: t(3) = 5.40, uncorrected p=0.012, corrected p=0.037; IDH2: t(3) = 10.93, uncorrected p=0.0016, corrected p=0.0049; IDH1/IDH2: t(7) = 10.44, uncorrected p=1.61×10−5, corrected p=4.84×10−5). To summarize, for this experiment we found results that were in the same direction as the original study and statistically significant, except on samples with IDH1 mutations when using the same type of chromatograms as the original study. As with all statistical significance testing, these results should take into consideration that whether a result is statistically significant or not is dependent not only on the significance threshold, but also the number of samples (Sullivan and Feinn, 2012).

Meta-analyses of original and replication effects

We performed a meta-analysis using a random-effects model, where possible, to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Fiehn et al., 2016). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. For a one-sample test, Cohen’s d is the difference between the sample mean and the null value using the sample standard deviation. The estimate of the effect size of one study, as well as the associated uncertainty (i.e. confidence interval), compared to the effect size of the other study provides another approach to compare the original and replication results (Errington et al., 2014; Valentine et al., 2011). Importantly, the width of the confidence interval for each study is a reflection of not only the confidence level (e.g. 95%), but also variability of the sample (e.g. SD) and sample size.

To compare the original and replication results for the enzymatic assays, the linear regression slope was determined for the values estimated from the representative experiment reported in Figure 2 of Ward et al. (2010) and compared to the slopes reported in this replication attempt. For the isocitrate-dependent NADPH production assay, the slopes for wild-type IDH2 and vector control from the original study were outside the 95% confidence interval (CI) of the slopes generated during the replication attempt, while the slope for R172K mutant IDH2 from the original study was within the 95% CI of the replication attempt (Figure 1B). For the alpha-ketoglutarate-dependent NADPH consumption assay, the slopes for R172K mutant IDH2 and wild-type IDH2 from the original study fell outside the 95% CI of the slopes generated during the replication attempt, while the slope for vector control from the original study was within the 95% CI of the replication attempt (Figure 1D).

Similar to the enzymatic assays, the intracellular 2HG/glutamate levels determined 48 hr after transfection in the original study were estimated from the representative image reported in Figure 3D of Ward et al. (2010) and compared to the values reported in this replication attempt. For R172K mutant IDH2, the intracellular 2HG/glutamate levels in the original study were within the 95% CI of the replication attempt, while wild-type IDH2 and vector control fell outside the 95% CI (Figure 2A).

There were three comparisons made with the AML patient sample data analyzing 2HG/glutamate levels based on IDH mutational status (Figure 4). For each of the comparisons, the 2HG/glutamate levels were compared to a constant representing the 2HG/glutamate threshold between IDH mutant and wild-type IDH samples detected in the study (original: 0.01; replication: 0.0024). The comparison of samples with an IDH1 mutation resulted in d = 8.40, 95% CI [2.84, 14.09] for the data estimated a priori from Figure 5C in the original study (Ward et al., 2010), whereas d = 2.23, 95% CI [0.26, 4.17] for this study. A meta-analysis of these two effects resulted in d = 4.80, 95% CI [−1.16, 10.76], p=0.114. The original and replication effects were in the same direction, however, the point estimate of the replication effect size was not within the confidence interval of the original result, and vice versa. Further, the large confidence intervals of the meta-analysis along with a statistically significant Cochran’s Q test (p=0.031), suggests heterogeneity between the original and replication studies. The test of samples with an IDH2 mutation resulted in d = 6.75, 95% CI [3.45, 10.04] for the original study, d = 4.60, 95% CI [1.05, 8.22] for this study, with a meta-analysis resulting in d = 5.72, 95% CI [3.42, 8.02], p=1.12×10−6. Similarly, the test of samples with an IDH1 or IDH2 mutation resulted in d = 4.36, 95% CI [2.62, 6.09] for the original study, d = 3.09, 95% CI [1.35, 4.80] for this study, with a meta-analysis resulting in d = 3.71, 95% CI [2.46, 4.96], p=5.73×10−9. For each of these two comparisons, the original and replication effects were in the same direction and the point estimate of the replication effect size was within the confidence interval of the original result, and vice versa.

Figure 4. Meta-analyses of each effect.

Figure 4.

Open in a new tab

Effect size and 95% confidence interval are presented for Ward et al. (2010), this replication study (RP:CB), and a random effects meta-analysis of those two effects. Sample sizes used in Ward et al. (2010) and this replication attempt are reported under the study name. Random effects meta-analysis of AML patient samples with an IDH1 mutation compared to a constant representing the 2HG/glutamate threshold between IDH mutant and wild-type IDH samples detected in the study (original: 0.01; replication: 0.0024) (meta-analysis p=0.114), AML patient samples with an IDH2 mutation compared to the threshold constant (meta-analysis p=1.12×10−6), and AML patient samples with an IDH1 or IDH2 mutation compared to the threshold constant (meta-analysis p=5.73×10−9). Additional details for these meta-analyses can be found at https://osf.io/4m3n8/.

DOI: http://dx.doi.org/10.7554/eLife.26030.007

This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Fiehn et al., 2016). However, this is limited to what was obtainable from the original paper, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as cell line, transfection reagent, and metabolite extraction technique were maintained, others were unknown or not easily controlled for. These include variables such as cell line genetic drift (Hughes et al., 2007; Kleensang et al., 2016), GC-MS parameters such as injection volume (Allwood et al., 2009), and other patient characteristics (Klevorn and Teague, 2016). Furthermore, recent studies since Ward et al. (2010) was published have identified numerous factors, such as hypoxic microenvironments that promote glutamine consumption for glutaminolysis and reductive carboxylation pathways, affecting IDH reactions, specifically alpha-ketoglutarate (Parker and Metallo, 2015). Additionally, enantiomers of 2HG have been found to be produced by different enzymes with distinct metabolisms (Losman and Kaelin, 2013). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.

Materials and methods

As described in the Registered Report (Fiehn et al., 2016), we attempted a replication of the experiments reported in Figures 2, 3D and 5 of Ward et al. (2010). A detailed description of all protocols can be found in the Registered Report (Fiehn et al., 2016). Additional detailed experimental notes, data, and analysis are available on the Open Science Framework (OSF) (RRID:SCR_003238) (https://osf.io/8l4ea/; Showalter et al., 2017).

Cell culture

HEK239T cells (ATCC, cat# CRL-3216, RRID:CVCL_0063) were grown in DMEM supplemented with 10% Fetal Bovine Serum (FBS) (Hyclone, Logan, UT, cat# SH30071.03). Cells were grown at 37°C in a humidified atmosphere at 10% CO2. Quality control data for the cell line are available at https://osf.io/zt996/. This includes results confirming the cell line was free of mycoplasma contamination (MycoAlert PLUS, Lonza, Basel, Switzerland, cat# LT07-701). Additionally, STR DNA profiling of the cell line was performed (DDC Medical, Fairfield, Ohio) and cells were confirmed to be the indicated cell line when queried against STR profile databases.

Plasmids and transfection

To generate pcDNA3.1-IDH2WT (deposited in Addgene, plasmid# 87926) and pcDNA3.1-IDH2R172K (deposited in Addgene, plasmid# 87927), IDH2WT and IDH2R172K were amplified by PCR from pCMV6-Entry vectors (Origene, Rockville, MD, cat# RC201152 and cat# RC400103) and cloned into the BamH1 and EcoR1 sites of the pcDNA3.1 expression vector (Invitrogen, Carlsbad, CA, cat# V790-20). Identity of all vectors were confirmed by sequencing and agarose gel electrophoresis (https://osf.io/zqyyq/). The day before transfection HEK293T cells were plated and allowed to grow overnight. For the assessment of alpha-ketoglutarate dependent NADPH consumption (Figure 1), 6 × 105 cells were seeded per well in six-well plates. For assessing the production of 2HG (Figure 2), 3.5 × 106 cells were seeded per 10 cm plate. Cells were transfected with the appropriate plasmid with Lipofectamine 2000 (Invitrogen, cat# 11668027) according to manufacturer’s instructions. For a 6-well plate, each well was transfected with 1 µg DNA diluted first in 50 µl serum-free, antibiotic-free DMEM and incubated for 3–5 min, and 2.67 µl Lipofectamine 2000 diluted first in 50 µl serum-free, antibiotic-free DMEM and incubated for 3–5 min. Transfections for 10 cm plates included 7.5 µg DNA diluted in 250 µl serum-free, antibiotic-free DMEM, and 20 µl Lipofectamine 2000 diluted in 250 µl serum-free, antibiotic-free DMEM. Following the individual incubations, the diluted DNA and Lipofectamine 2000 was combined and incubated for an additional 10 min before being added to the cells. Medium was replaced 24 hr later.

Cell lysate based enzyme assays

For IDH oxidative and reductive activity assays, transfected cells were harvested 48 hr after transfection in 0.5 ml/well with mammalian protein extraction reagent (mPER) containing protease and phosphatase inhibitor cocktail (4 µg/ml aprotinin, 0.2 mM AEBSF, 4 µg/ml leupeptin, 4 µg/ml pepstatin A, 0.2 mM NaOV, and 1 mM NaF) on ice. Lysate was collected, homogenized using a T-10 homogenizer with S10N-5 tip (IKA, Staufen im Breisgau, Germany) for 1–2 min on ice, and then centrifuged at 14,000xg for 10 min at 4°C. Protein concentration was measured using the DC Protein Assay Kit II (BioRad, Hercules, CA, cat# 5000112) according to manufacturer’s instructions. Lysate was aliquoted for IDH oxidative and reductive activity assays and Western blotting. IDH oxidative activity was determined by mixing 15 µg of each protein lysate with 200 µl of assay buffer (100 mM Tris-HCl, pH 7.5, 1.3 mM MnCl2, 0.33 mM EDTA, 0.4 mM ß-NADP+, 0.4 mM D-(+)-threo-isocitrate). Triplicate reactions were measured every 2.5 min for 30 min at 340 nm absorbance using a microplate spectrophotometer (Molecular Devices, Sunnyvale, CA, FilterMax F5 Multi-Mode Microplate Reader) and Multi-Mode Analysis Software (Molecular Devices, RRID:SCR_014789), version 3.4.0.25. IDH reductive activity was determined by mixing 30 µg of each protein lysate with 200 µl of assay buffer (100 mM Tris-HCl, pH 7.5, 1.3 mM MnCl2, 0.02 mM ß-NADPH, 1 mM alpha-ketoglutarate). Triplicate reactions were measured every 20 min for 3 hr at 340 nm. For all experiments, a mPER only sample lacking lysate protein was mixed with assay buffer to determine the background, which was subtracted from the average triplicate reading for each timepoint. For each biological repeat, the value at time 0 was subtracted from all timepoints to calculate change in absorbance.

Western blot

Sample buffer (50 mM Tris-HCl, pH 6.8, 2% SDS, 10% glycerol, 1% ß-mercaptoethanol, 0.02% bromophenol blue) was added to protein lysates and incubated at 95°C for 5 min. 40 µg of protein along with protein ladder was resolved by SDS-PAGE and transferred to nitrocellulose membrane as described in the Registered Report (Fiehn et al., 2016). Membrane was washed with deionized water and Ponceau stain was used to confirm transfer. The membrane was blocked with 5% w/v nonfat dry milk/0.2% sodium azide in 1X TBS with 0.1% Tween-20 (TBST) for 30 min at room temperature. Incubation with primary antibody was conducted overnight at 4°C. Membranes were probed with goat anti-IDH1 (Santa Cruz Biotechnology, Dallas, TX, cat# sc49996, RRID:AB_2123166); 1:500 dilution in 5% w/v nonfat dry milk/TBST, mouse anti-IDH2 (Abcam, Cambridge, UK, cat# ab55271, RRID:AB_943793); 0.5 µg/ml (1:1,000) dilution in 5% w/v nonfat dry milk/TBST, and mouse anti-alpha-Tubulin (Sigma-Aldrich, St. Louis, MO, cat# T5168, RRID:AB_477579); 1:10,000 dilution in 5% w/v nonfat dry milk/TBST. Each incubation was followed by washes with TBST and the appropriate secondary antibody: HRP-conjugated rabbit anti-goat (Invitrogen, cat# 81–1620, RRID:AB_2534006); 1:5000 dilution in 5% w/v nonfat dry milk/TBST or HRP-conjugated goat anti-mouse (Bio-Rad, cat# 170–5047, RRID:AB_11125753); 1:7500 dilution in 5% w/v nonfat dry milk/TBST. The HRP-conjugated anti-Actin antibody (Cell Signaling, Danvers, MA, cat# 12620) listed in the Registered Report was tried; however, it overlapped with IDH2 and thus could not be utilized. Membranes were washed with TBST and incubated with SuperSignal West Femto Maximum Sensitivity Substrate (Thermo Fisher Scientific, Waltham, MA, cat# 34096) according to the manufacturer’s instructions. Western blots were visualized using an Alpha Innotech (San Leandro, CA) imaging station (FluorChem FC2 system) using the AlphaView software (RRID:SCR_014549), version 3.2.2. Between antibody incubations, HRP was inactivated by incubating with 1 mM sodium azide and subsequently imaged to confirm HRP inactivation before proceeding to the next antibody incubation. All images taken are available at https://osf.io/th3za/.

AML patient samples

Patient samples were obtained from Roswell Park Cancer Institute. The samples were prescreened for IDH genotypic status and the pre-specified sample of each group was determined prior to conducting the study. Approval was obtained from the institutional review board (protocol # CIC 9805) and shared samples and data were de-identified for this study. Cells used for this study were prepared by Ficoll separation of mononuclear cells from peripheral blood or bone marrow and were frozen as viable cells in 10% dimethyl sulfoxide (DMSO).

Metabolite extraction

HEK293T cells were harvested 24 and 48 hr after transfection by removal of medium, while peripheral blood mononuclear cells from frozen AML samples, were centrifuged at 1000xg to remove freezing medium. After removal of medium, cells were rapidly quenched with 1.5 ml of methanol chilled to −80°C. Cells were incubated at −80°C for 15 min and in the case of HEK293T cells scraped off the dish. Cells were centrifuged for 5 min at 2000xg at 4°C to pellet cellular debris. Pellets were re-extracted by an addition of 500 µl of 80% methanol in water, chilled to −80°C, vortexed, then incubated at 4°C for 15 min and centrifuged for 5 min at 1000xg at 4°C.

Supernatants from the multiple extractions were combined and evaporated to dryness using a cold trap concentrator (Labconco, Kansas City, MO). Samples were then resuspended in 200 µl LC-MS grade water. A 2 ml AG-1 × 8 100–200 anion exchange column (Bio-Rad, cat# 731–6211) was washed with five column volumes (10 ml) of 3 N HCl followed by transfer of resuspended extracts to resin column. Metabolites were eluted using 10 ml 3 N HCl. Samples were evaporated to dryness using a cold trap concentrator and resuspended in 100 µl MTBSTFA/ACN (1:1, v/v) and shaken at 60°C for 1 hr using an Orbital Mixing Chilling/Heating Plate (Torrey Pines Scientific Inc., Carlsbad, CA) at a maximum orbital speed of 9. Derivatized extracts were further diluted (1:4) with a MTBSTFA/ACN (1:1, v/v) mixture and transferred to glass vials with micro-inserts and capped immediately prior to GC-MS analysis.

GC-MS analysis

The system consisted of a GC-MS system (Agilent Technologies, Santa Clara, California), a 7693 Series autosampler, a split/splitless injector, a 7890A GC system, and a quadrupole mass spectrometer 5977A. Injection parameters were as follows: injection volume, 0.2 µl; injector temperature, 250°C; helium carrier gas flow, 1 ml/min; splitless period, 1 min. For GC separation a 30 m x 0.25 mm, 0.25 µm HP-5MS UI (Agilent Technologies) capillary column was used with an oven temperature program: 100°C (3 min), 4 °C/min to 230°C (hold 4 min), 30 °C/min to 300°C (hold 5 min). MS detection parameters were as follows: electron ionization, −70 eV; acquisition rate, 2.71 scans/s; mass range, m/z 50–600; MS ion source temperature, 230°C; MS quadrupole temperature, 150°C; electron multiplier voltage, 1651 V. For the data acquisition, the MSD Productivity ChemStation Software (Agilent Technologies), version E.02.00 was used. Method blanks and quality control samples were run with experimental samples for quality control purposes. Both total ion chromatograms (TIC) and extracted ion chromatograms (EIC) are reported. Data files were processed using MassHunter Qualitative Analysis Software (Agilent Technologies), version B.05.00 and MassHunter Quantitative Analysis Software (Agilent Technologies), version B.05.01. Retention times for compounds were as follows: 31.7 min aspartate, 34.3 min glutamate, and 33.1 min 2-hydroxyglutarate. Metabolites were identified based on retention times and electron ionization (EI) spectra matching with authentic standards (Sigma, cat# 2HG:H8378, Asp:A9256, Glu:G1251). This metabolomics data is available at the NIH Common Fund's Data Repository and Coordinating Center (supported by NIH grant, U01-DK097430) website, (http://www.metabolomicsworkbench.org), where it has been assigned a Metabolomics Workbench Project ID: ST000548. The data is directly accessible at http://www.metabolomicsworkbench.org/data/DRCCMetadata.php?Mode=Study&StudyID=ST000548.

Statistical analysis

Statistical analysis was performed with R software (RRID:SCR_001905), version 3.3.2 (R Core Team, 2016). All data, csv files, and analysis scripts are available on the OSF (https://osf.io/8l4ea/). Confirmatory statistical analysis was pre-registered (https://osf.io/t873r/) before the experimental work began as outlined in the Registered Report (Fiehn et al., 2016). Proposed analysis of GC-MS data was conducted on TIC data, with additional exploratory analysis performed using the EIC data. Data were checked to ensure assumptions of statistical tests were met. When described in the results, the Bonferroni correction, to account for multiple testings, was applied to the alpha error or the p-value. The Bonferroni corrected value was determined by dividing the uncorrected value (.05) by the number of tests performed. Although the Bonferroni method is conservative, it was accounted for in the power calculations to ensure sample size was sufficient. A meta-analysis of a common original and replication effect size was performed with a random effects model and the metafor package (Viechtbauer, 2010) (available at: https://osf.io/4m3n8/). The original study data was extracted a priori from the published figures by estimating the value reported. The extracted data from Figures 2A-B of Ward et al. (2010) were used to calculate the mean slope after fitting the data to a linear regression. The extracted data were published in the Registered Report (Fiehn et al., 2016) and was used in the power calculations to determine the sample size for this study.

Deviations from registered report

The expression vector pcDNA3.1 was used instead of pcDNA3 since it was unavailable for order at the time the study was conducted. The concentrations of protein used was increased for the IDH oxidative (0.3 µg to 15 µg) and reductive (3 µg to 30 µg) assays, as well as the amount of substrate (D-(+)-threo-isocitrate: 0.1 mM to 0.4 mM; alpha-ketoglutarate: 0.5 mM to 1 mM). This was due to an inability to detect a signal above the noise with the originally specified protein concentrations, with the need to increase the substrate concentration due to saturation before the end of the time points. Cell lysates were also homogenized instead of sonicated to generate protein extracts. Following this optimization, the same procedure was carried out for the biological repeats reported. For the Western blot, Actin was not used as the additional control since it overlapped with IDH2; Alpha-Tubulin was used instead. For the GC-MS, the Registered Report listed MTBSTFA in the materials, but incorrectly listed MSTFA in the procedure. MTBSTFA was used in this replication attempt, similar to the original study. For the GC-MS run, it was unstated how ramping to 300°C was to be performed, as such 30 °C/min was used. Derivatized extracts were diluted 1:4 to ensure metabolites were in proper concentration range for the GC-MS instrument used since the cell number, derivatization volume, and injection volume were not listed in the original study. Additional materials and instrumentation not listed in the Registered Report, but needed during experimentation are also listed.

Acknowledgements

The Reproducibility Project: Cancer Biology would like to thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses and the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Funding Statement

The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Contributor Information

Jessica K Tyler, Weill Cornell Medicine, United States.

Reproducibility Project: Cancer Biology:

Elizabeth Iorns, Alexandria Denis, Nicole Perfito, and Timothy M Errington

Funding Information

This paper was supported by the following grant:

  • Laura and John Arnold Foundation to .

Additional information

Competing interests

MRS: West Coast Metabolomics Center, University of California, Davis is a Science Exchange associated lab.

TC: West Coast Metabolomics Center, University of California, Davis is a Science Exchange associated lab.

OF: West Coast Metabolomics Center, University of California, Davis is a Science Exchange associated lab.

RP:CB: EI, NP: Employed by and hold shares in Science Exchange Inc.

The other authors declare that no competing interests exist.

Author contributions

MRS, Acquisition of data, Drafting or revising the article.

JH, Acquisition of data, Drafting or revising the article.

TC, Acquisition of data, Drafting or revising the article.

KV, Acquisition of data, Drafting or revising the article.

KLC, RP:CB, Acquisition of data, Drafting or revising the article.

OF, Acquisition of data, Drafting or revising the article.

Ethics

Human subjects: Patient samples were obtained from Roswell Park Cancer Institute. The samples were prescreened for IDH genotypic status and the pre-specified sample of each group was determined prior to conducting the study. Approval was obtained from the institutional review board (protocol # CIC 9805) and shared samples and data were de-identified for this study.

References

  1. Allwood JW, Erban A, de Koning S, Dunn WB, Luedemann A, Lommen A, Kay L, Löscher R, Kopka J, Goodacre R. Inter-laboratory reproducibility of fast gas chromatography-electron impact-time of flight mass spectrometry (GC-EI-TOF/MS) based plant metabolomics. Metabolomics. 2009;5:479–496. doi: 10.1007/s11306-009-0169-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. An Z, Ganji SK, Tiwari V, Pinho MC, Patel T, Barnett S, Pan E, Mickey BE, Maher EA, Choi C. Detection of 2-hydroxyglutarate in brain tumors by triple-refocusing MR spectroscopy at 3T in vivo. Magnetic Resonance in Medicine. 2016;14:1522–2594. doi: 10.1002/mrm.26347. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Andronesi OC, Rapalino O, Gerstner E, Chi A, Batchelor TT, Cahill DP, Sorensen AG, Rosen BR. Detection of oncogenic IDH1 mutations using magnetic resonance spectroscopy of 2-hydroxyglutarate. Journal of Clinical Investigation. 2013;123:3659–3663. doi: 10.1172/JCI67229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Borger DR, Goyal L, Yau T, Poon RT, Ancukiewicz M, Deshpande V, Christiani DC, Liebman HM, Yang H, Kim H, Yen K, Faris JE, Iafrate AJ, Kwak EL, Clark JW, Allen JN, Blaszkowsky LS, Murphy JE, Saha SK, Hong TS, Wo JY, Ferrone CR, Tanabe KK, Bardeesy N, Straley KS, Agresta S, Schenkein DP, Ellisen LW, Ryan DP, Zhu AX. Circulating oncometabolite 2-hydroxyglutarate is a potential surrogate biomarker in patients with isocitrate dehydrogenase-mutant intrahepatic cholangiocarcinoma. Clinical Cancer Research. 2014;20:1884–1890. doi: 10.1158/1078-0432.CCR-13-2649. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Carbonneau M, M Gagné L, Lalonde ME, Germain MA, Motorina A, Guiot MC, Secco B, Vincent EE, Tumber A, Hulea L, Bergeman J, Oppermann U, Jones RG, Laplante M, Topisirovic I, Petrecca K, Huot MÉ, Mallette FA, Gagné M, Guiot A, Secco M-C, Tumber EE, Jones U, Huot K. The oncometabolite 2-hydroxyglutarate activates the mTOR signalling pathway. Nature Communications. 2016;7:12700. doi: 10.1038/ncomms12700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chaturvedi A, Herbst L, Pusch S, Klett L, Goparaju R, Stichel D, Kaulfuss S, Panknin O, Zimmermann K, Toschi L, Neuhaus R, Haegebarth A, Rehwinkel H, Hess-Stumpp H, Bauser M, Bochtler T, Struys EA, Sharma A, Bakkali A, Geffers R, Araujo-Cruz MM, Thol F, Gabdoulline R, Ganser A, Ho AD, von Deimling A, Rippe K, Heuser M, Krämer A. Pan-mutant-IDH1 inhibitor BAY1436032 is highly effective against human IDH1 mutant acute myeloid leukemia in vivo. Leukemia. 2017:46. doi: 10.1038/leu.2017.46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chavez M, Ho J, Tan C. Reproducibility of high-throughput plate-reader experiments in synthetic biology. ACS Synthetic Biology. 2017;6:375–380. doi: 10.1021/acssynbio.6b00198. [DOI] [PubMed] [Google Scholar]
  8. Dang L, White DW, Gross S, Bennett BD, Bittinger MA, Driggers EM, Fantin VR, Jang HG, Jin S, Keenan MC, Marks KM, Prins RM, Ward PS, Yen KE, Liau LM, Rabinowitz JD, Cantley LC, Thompson CB, Vander Heiden MG, Su SM. Cancer-associated IDH1 mutations produce 2-hydroxyglutarate. Nature. 2009;462:739–744. doi: 10.1038/nature08617. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. de la Fuente MI, Young RJ, Rubel J, Rosenblum M, Tisnado J, Briggs S, Arevalo-Perez J, Cross JR, Campos C, Straley K, Zhu D, Dong C, Thomas A, Omuro AA, Nolan CP, Pentsova E, Kaley TJ, Oh JH, Noeske R, Maher E, Choi C, Gutin PH, Holodny AI, Yen K, DeAngelis LM, Mellinghoff IK, Thakur SB. Integration of 2-hydroxyglutarate-proton magnetic resonance spectroscopy into clinical practice for disease monitoring in isocitrate dehydrogenase-mutant glioma. Neuro-Oncology. 2016;18:283–290. doi: 10.1093/neuonc/nov307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. DiNardo CD, Propert KJ, Loren AW, Paietta E, Sun Z, Levine RL, Straley KS, Yen K, Patel JP, Agresta S, Abdel-Wahab O, Perl AE, Litzow MR, Rowe JM, Lazarus HM, Fernandez HF, Margolis DJ, Tallman MS, Luger SM, Carroll M. Serum 2-hydroxyglutarate levels predict isocitrate dehydrogenase mutations and clinical outcome in acute myeloid leukemia. Blood. 2013;121:4917–4924. doi: 10.1182/blood-2013-03-493197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Errington TM, Iorns E, Gunn W, Tan FE, Lomax J, Nosek BA. An open investigation of the reproducibility of cancer biology research. eLife. 2014;3:e04333. doi: 10.7554/eLife.04333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Fathi AT, Nahed BV, Wander SA, Iafrate AJ, Borger DR, Hu R, Thabet A, Cahill DP, Perry AM, Joseph CP, Muzikansky A, Chi AS. Elevation of urinary 2-hydroxyglutarate in IDH-mutant glioma. The Oncologist. 2016;21:214–219. doi: 10.1634/theoncologist.2015-0342. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fathi AT, Sadrzadeh H, Borger DR, Ballen KK, Amrein PC, Attar EC, Foster J, Burke M, Lopez HU, Matulis CR, Edmonds KM, Iafrate AJ, Straley KS, Yen KE, Agresta S, Schenkein DP, Hill C, Emadi A, Neuberg DS, Stone RM, Chen YB. Prospective serial evaluation of 2-hydroxyglutarate, during treatment of newly diagnosed acute myeloid leukemia, to assess disease activity and therapeutic response. Blood. 2012;120:4649–4652. doi: 10.1182/blood-2012-06-438267. [DOI] [PubMed] [Google Scholar]
  14. Fiehn O, Showalter MR, Schaner-Tooley CE, Reproducibility Project: Cancer Biology Registered report: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate. eLife. 2016;5:e12626. doi: 10.7554/eLife.12626. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Ganji SK, An Z, Tiwari V, McNeil S, Pinho MC, Pan E, Mickey BE, Maher EA, Choi C. In vivo detection of 2-hydroxyglutarate in brain tumors by optimized point-resolved spectroscopy (PRESS) at 7T. Magnetic Resonance in Medicine. 2017;77:936–944. doi: 10.1002/mrm.26190. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Giehm L, Otzen DE. Strategies to increase the reproducibility of protein fibrillization in plate reader assays. Analytical Biochemistry. 2010;400:270–281. doi: 10.1016/j.ab.2010.02.001. [DOI] [PubMed] [Google Scholar]
  17. Gross S, Cairns RA, Minden MD, Driggers EM, Bittinger MA, Jang HG, Sasaki M, Jin S, Schenkein DP, Su SM, Dang L, Fantin VR, Mak TW. Cancer-associated metabolite 2-hydroxyglutarate accumulates in acute myelogenous leukemia with isocitrate dehydrogenase 1 and 2 mutations. The Journal of Experimental Medicine. 2010;207:339–344. doi: 10.1084/jem.20092506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hughes P, Marshall D, Reid Y, Parkes H, Gelber C. The costs of using unauthenticated, over-passaged cell lines: how much more data do we need? BioTechniques. 2007;43:575–586. doi: 10.2144/000112598. [DOI] [PubMed] [Google Scholar]
  19. Intlekofer AM, Dematteo RG, Venneti S, Finley LW, Lu C, Judkins AR, Rustenburg AS, Grinaway PB, Chodera JD, Cross JR, Thompson CB. Hypoxia induces production of L-2-Hydroxyglutarate. Cell Metabolism. 2015;22:304–311. doi: 10.1016/j.cmet.2015.06.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Janin M, Mylonas E, Saada V, Micol JB, Renneville A, Quivoron C, Koscielny S, Scourzic L, Forget S, Pautas C, Caillot D, Preudhomme C, Dombret H, Berthon C, Barouki R, Rabier D, Auger N, Griscelli F, Chachaty E, Leclercq E, Courtier MH, Bennaceur-Griscelli A, Solary E, Bernard OA, Penard-Lacronique V, Ottolenghi C, de Botton S. Serum 2-hydroxyglutarate production in IDH1- and IDH2-mutated de novo acute myeloid leukemia: a study by the acute leukemia Frenchassociation group. Journal of Clinical Oncology. 2014;32:297–305. doi: 10.1200/JCO.2013.50.2047. [DOI] [PubMed] [Google Scholar]
  21. Kalinina J, Ahn J, Devi NS, Wang L, Li Y, Olson JJ, Glantz M, Smith T, Kim EL, Giese A, Jensen RL, Chen CC, Carter BS, Mao H, He M, Van Meir EG. Selective detection of the D-enantiomer of 2-Hydroxyglutarate in the CSF of glioma patients with mutated Isocitrate Dehydrogenase. Clinical Cancer Research. 2016;22:6256–6265. doi: 10.1158/1078-0432.CCR-15-2965. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kleensang A, Vantangoli MM, Odwin-DaCosta S, Andersen ME, Boekelheide K, Bouhifd M, Fornace AJ, Li HH, Livi CB, Madnick S, Maertens A, Rosenberg M, Yager JD, Zhaog L, Hartung T. Genetic variability in a frozen batch of MCF-7 cells invisible in routine authentication affecting cell function. Scientific Reports. 2016;6:28994. doi: 10.1038/srep28994. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Klevorn LE, Teague RM. Adapting Cancer immunotherapy models for the real world. Trends in Immunology. 2016;37:354–363. doi: 10.1016/j.it.2016.03.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lemonnier F, Cairns RA, Inoue S, Li WY, Dupuy A, Broutin S, Martin N, Fataccioli V, Pelletier R, Wakeham A, Snow BE, de Leval L, Pujals A, Haioun C, Paci A, Tobin ER, Narayanaswamy R, Yen K, Jin S, Gaulard P, Mak TW. The IDH2 R172K mutation associated with angioimmunoblastic T-cell lymphoma produces 2hg in T cells and impacts lymphoid development. PNAS. 2016;113:15084–15089. doi: 10.1073/pnas.1617929114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lombardi G, Corona G, Bellu L, Della Puppa A, Pambuku A, Fiduccia P, Bertorelle R, Gardiman MP, D'Avella D, Toffoli G, Zagonel V. Diagnostic value of plasma and urinary 2-hydroxyglutarate to identify patients with isocitrate dehydrogenase-mutated glioma. The Oncologist. 2015;20:562–567. doi: 10.1634/theoncologist.2014-0266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lombardi G, Della Puppa A, Zagonel V. Urine 2-Hydroxyglutarate in Glioma. The Oncologist. 2016;21:1026. doi: 10.1634/theoncologist.2016-0123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Losman JA, Kaelin WG. What a difference a hydroxyl makes: mutant IDH, (R)-2-hydroxyglutarate, and Cancer. Genes and Development. 2013;27:836–852. doi: 10.1101/gad.217406.113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. McGehee E, Rakheja D, Oliver D, Chen W, Boriack R, Collins RH. The importance of plasma D-2HG measurement in screening for IDH mutations in acute myeloid leukaemia. British Journal of Haematology. 2016;173:323–326. doi: 10.1111/bjh.13598. [DOI] [PubMed] [Google Scholar]
  29. Mondesir J, Willekens C, Touat M, de Botton S. IDH1 and IDH2 mutations as novel therapeutic targets: current perspectives. Journal of Blood Medicine. 2016;7:171–180. doi: 10.2147/JBM.S70716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Natsumeda M, Igarashi H, Nomura T, Ogura R, Tsukamoto Y, Kobayashi T, Aoki H, Okamoto K, Kakita A, Takahashi H, Nakada T, Fujii Y. Accumulation of 2-hydroxyglutarate in gliomas correlates with survival: a study by 3.0-tesla magnetic resonance spectroscopy. Acta Neuropathologica Communications. 2014;2:158. doi: 10.1186/s40478-014-0158-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Parker SJ, Metallo CM. Metabolic consequences of oncogenic IDH mutations. Pharmacology and Therapeutics. 2015;152:54–62. doi: 10.1016/j.pharmthera.2015.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. R Core Team . R Foundation for Statistical Computing. Vienna, Austria: 2016. [Google Scholar]
  33. Robinson GL, Philip B, Guthrie MR, Cox JE, Robinson JP, VanBrocklin MW, Holmen SL. In vitro visualization and characterization of wild type and mutant IDH homo- and heterodimers using bimolecular fluorescence complementation. Cancer Research Frontiers. 2016;2:311–329. doi: 10.17980/2016.311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Showalter MR, Hatakeyama J, Cajka T, Vandervorst K, Carraway KL, Fiehn O, Iorns E, Denis A, Perfito N, Errington TM. 2017. Study 16: Replication of Ward et al., 2010 (Cancer cell) Open Science Framework. [DOI]
  35. Sullivan GM, Feinn R. Using effect Size-or why the P value is not Enough. Journal of Graduate Medical Education. 2012;4:279–282. doi: 10.4300/JGME-D-12-00156.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Valentine JC, Biglan A, Boruch RF, Castro FG, Collins LM, Flay BR, Kellam S, Mościcki EK, Schinke SP. Replication in prevention science. Prevention Science. 2011;12:103–117. doi: 10.1007/s11121-011-0217-6. [DOI] [PubMed] [Google Scholar]
  37. Vander Heiden MG, DeBerardinis RJ. Understanding the intersections between metabolism and Cancer biology. Cell. 2017;168:657–669. doi: 10.1016/j.cell.2016.12.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Verma G, Mohan S, Nasrallah MP, Brem S, Lee JY, Chawla S, Wang S, Nagarajan R, Thomas MA, Poptani H. Non-invasive detection of 2-hydroxyglutarate in IDH-mutated gliomas using two-dimensional localized correlation spectroscopy (2D L-COSY) at 7 Tesla. Journal of Translational Medicine. 2016;14:274. doi: 10.1186/s12967-016-1035-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Viechtbauer W. Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software. 2010;36:1–48. doi: 10.18637/jss.v036.i03. [DOI] [Google Scholar]
  40. Ward PS, Lu C, Cross JR, Abdel-Wahab O, Levine RL, Schwartz GK, Thompson CB. The potential for isocitrate dehydrogenase mutations to produce 2-hydroxyglutarate depends on allele specificity and subcellular compartmentalization. Journal of Biological Chemistry. 2013;288:3804–3815. doi: 10.1074/jbc.M112.435495. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Ward PS, Patel J, Wise DR, Abdel-Wahab O, Bennett BD, Coller HA, Cross JR, Fantin VR, Hedvat CV, Perl AE, Rabinowitz JD, Carroll M, Su SM, Sharp KA, Levine RL, Thompson CB. The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate. Cancer Cell. 2010;17:225–234. doi: 10.1016/j.ccr.2010.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Willekens C, Micol J-B, Poinsignon V, Quivoron C, Saada V, Broutin S, Terroir-Cassou-Mounat M, Ghez D, Bourhis J-H, Bosq J, Ribrag V, Paci A, Lacronique-Penard V, Botton SD. Serum 2-Hydroxyglutarate Level can predict IDH2 mutation in myeloid sarcoma. Blood. 2015;126:3835 [Google Scholar]
  43. Yan H, Parsons DW, Jin G, McLendon R, Rasheed BA, Yuan W, Kos I, Batinic-Haberle I, Jones S, Riggins GJ, Friedman H, Friedman A, Reardon D, Herndon J, Kinzler KW, Velculescu VE, Vogelstein B, Bigner DD. IDH1 and IDH2 mutations in gliomas. The New England Journal of Medicine. 2009;360:765–773. doi: 10.1056/NEJMoa0808710. [DOI] [PMC free article] [PubMed] [Google Scholar]
eLife. 2017 Jun 27;6:e26030. doi: 10.7554/eLife.26030.009

Decision letter

Editor: Jessica K Tyler1

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Replication Study: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting α-ketoglutarate to 2-hydroxyglutarate" for consideration by eLife. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Vivek Malhotra as the Senior Editor. The reviewers have opted to remain anonymous.

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary

The inherent property of any reproducibility study is that its design is directly related to the original work and set of experiments and cannot extend beyond. This exact situation has arisen with the current reproducibility study, that has been conducted according to the approved plan and results were carefully taken and were adequately statistically treated. It is not surprising that with exception of certain absolute values which did not match to the original publication of Ward et al., the results were reproduced with a high accuracy. The methodical approach is excellent and of the top level, adequate to the task, including derivatization and temperature gradients. Overall, the experiments appear to be performed to a high standard and clearly reproduce the majority of findings from the Ward paper, which is expected based on other independently published studies.

Essential revisions

1) It should be pointed out that specifically precleaning of samples on a column adds to a straightforward analysis. However, it should be verified, whether such precleaning does not withdraw from the sample any desired species to be tested.

2) To bring an additional value besides the simple fact of reproducibility, the authors should discuss why certain other studies have found non zero levels of 2-hydroxyglutarate in cells with wild-type IDH2, or in patients plasma or urine. Discussion (or even the Result section) should include comparison of the absolute values and not only ratios towards glutamate (even if this has been a strict condition of the reproducibility study). Authors may comment on conditions under which the activity of wt IDH2 of mutant IDH2 was surveyed. Since 2010 research on the topic has identified numerous factors affecting IDH2 (and IDH1) reactions, namely affecting 2-oxoglutarate as the primary reactant. Distinct contribution of e.g. one-carbon metabolism, state of aerobic glycolysis vs. oxphos and glutaminolysis, etc., are different even in the same immune cells under different conditions. Moreover, enantiomers of 2-hydroxyglutarate have been found to be produced by different enzymes and having distinct metabolism (e.g. dehydrogenases). In the light of these new views, the current reproducibility study represents just a solid beginning in our understanding.

3) Authors should correct the confused labels in Figure 1:

Figure 1A Title legend NADPH −> NADP+ (which actually stands for consumption whereas figure panel A shows the production) is confused with the Figure 1A Title legend. The legends need to be mutually exchanged, so to be for Figure 1A NADP+−> NADPH and for Figure 1B NADPH −> NADP+

eLife. 2017 Jun 27;6:e26030. doi: 10.7554/eLife.26030.010

Author response

Essential revisions

1) It should be pointed out that specifically precleaning of samples on a column adds to a straightforward analysis. However, it should be verified, whether such precleaning does not withdraw from the sample any desired species to be tested.

Thank you for raising this point. Correct, sample extraction and clean-up steps may lead to unwanted losses of metabolite levels in samples. For this replication study, we used the exact same methods as the original authors as specified in the Registered Report. The extraction and resin methods are found in other publications by the Thompson group1,2. An external validation of the extraction and resin purification methods was not performed as it was outside of the scope of the Replication Study presented here. We have also revised the manuscript to highlight this point.

2) To bring an additional value besides the simple fact of reproducibility, the authors should discuss why certain other studies have found non zero levels of 2-hydroxyglutarate in cells with wild-type IDH2, or in patients plasma or urine. Discussion (or even the Result section) should include comparison of the absolute values and not only ratios towards glutamate (even if this has been a strict condition of the reproducibility study). Authors may comment on conditions under which the activity of wt IDH2 of mutant IDH2 was surveyed. Since 2010 research on the topic has identified numerous factors affecting IDH2 (and IDH1) reactions, namely affecting 2-oxoglutarate as the primary reactant. Distinct contribution of e.g. one-carbon metabolism, state of aerobic glycolysis vs. oxphos and glutaminolysis, etc., are different even in the same immune cells under different conditions. Moreover, enantiomers of 2-hydroxyglutarate have been found to be produced by different enzymes and having distinct metabolism (e.g. dehydrogenases). In the light of these new views, the current reproducibility study represents just a solid beginning in our understanding.

We have revised the results/Discussion section to discuss the low levels of 2HG detected in this replication as well as other studies investigating 2HG as a biomarker in AML and other cancers. We did not perform this replication in a manner to provide accurate quantification of 2HG concentrations in the samples because this would have required standard curves, isotope labeled internal standards and recovery experiments. We therefore present semi-quantified data similar the original paper and as outlined in the replication report. However, we did include comments that raw 2HG peak intensities detected in this study followed the trend of 2HG/glutamate ratios.

We also expanded the final section to include that additional biological factors affecting IDH reactions have been published since the original study, specifically differential metabolism of 2HG enantiomers. Specifically, it has been reported that normal tissues can produce S-2HG under times of hypoxic stress, which is distinct from R-2HG, produced from mutations in IDH1/2. For studies that do not chromatographically separate enantiomers (R- vs. S- 2HG) both species will give the same m/z and spectra, and therefore appear as the same species. It is possible the low levels of S-2HG are contributing to the non-zero values. Additionally, depending on type of mass spectral analysis and standard of data processing for each group, the lowest values reported can vary.

3) Authors should correct the confused labels in Figure 1:

Figure 1A Title legend NADPH −> NADP+ (which actually stands for consumption whereas figure panel A shows the production) is confused with the Figure 1A Title legend. The legends need to be mutually exchanged, so to be for Figure 1A NADP+−> NADPH and for Figure 1B NADPH −> NADP+

Thank you for bringing this error to our attention. We have uploaded the correct version of this figure during resubmission.

1) Wise, D. R. et al. Hypoxia promotes isocitrate dehydrogenase-dependent carboxylation of α-ketoglutarate to citrate to support cell growth and viability. P Natl Acad Sci USA 108, 19611-19616, doi:10.1073/pnas.1117773108 (2011).

2) Lu, C. et al. IDH mutation impairs histone demethylation and results in a block to cell differentiation. Nature 483, 474-U130, doi:10.1038/nature10860 (2012).

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Citations

    1. Showalter MR, Hatakeyama J, Cajka T, Vandervorst K, Carraway KL, Fiehn O, Iorns E, Denis A, Perfito N, Errington TM. 2017. Study 16: Replication of Ward et al., 2010 (Cancer cell) Open Science Framework. [DOI]

    Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd

    RESOURCES