Skip to main content
PMC home page
eLife logoLink to eLife
. 2018 Jan 9;7:e29747. doi: 10.7554/eLife.29747

Replication Study: Systematic identification of genomic markers of drug sensitivity in cancer cells

John P Vanden Heuvel 1,2, Ewa Maddox 1, Samar W Maalouf 1; Reproducibility Project: Cancer Biology✉
Editor: Joaquín M Espinosa3
PMCID: PMC5760202  PMID: 29313488

Abstract

In 2016, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Vanden Heuvel et al., 2016), that described how we intended to replicate selected experiments from the paper ‘Systematic identification of genomic markers of drug sensitivity in cancer cells’ (Garnett et al., 2012). Here we report the results. We found Ewing’s sarcoma cell lines, overall, were more sensitive to the PARP inhibitor olaparib than osteosarcoma cell lines; however, while the effect was in the same direction as the original study (Figure 4C; Garnett et al., 2012), it was not statistically significant. Further, mouse mesenchymal cells transformed with either the EWS-FLI1 or FUS-CHOP rearrangement displayed similar sensitivities to olaparib, whereas the Ewing’s sarcoma cell line SK-N-MC had increased olaparib sensitivity. In the original study, mouse mesenchymal cells transformed with the EWS-FLI1 rearrangement and SK-N-MC cells were found to have similar sensitivities to olaparib, whereas mesenchymal cells transformed with the FUS-CHOP rearrangement displayed a reduced sensitivity to olaparib (Figure 4E; Garnett et al., 2012). We also studied another Ewing’s sarcoma cell line, A673: A673 cells depleted of EWS-FLI1 or a negative control both displayed similar sensitivities to olaparib, whereas the original study reported a decreased sensitivity to olaparib when EWS-FLI1 was depleted (Figure 4F; Garnett et al., 2012). Differences between the original study and this replication attempt, such as the use of different sarcoma cell lines and level of knockdown efficiency, are factors that might have influenced the outcomes. Finally, where possible, we report meta-analyses for each result.

Research organism: Human, Mouse

Introduction

The Reproducibility Project: Cancer Biology (RP:CB) is a collaboration between the Center for Open Science and Science Exchange that seeks to address concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology (Errington et al., 2014). For each of these papers a Registered Report detailing the proposed experimental designs and protocols for the replications was peer reviewed and published prior to data collection. The present paper is a Replication Study that reports the results of the replication experiments detailed in the Registered Report (Vanden Heuvel et al., 2016) for a 2012 paper by Garnett et al., and uses a number of approaches to compare the outcomes of the original experiments and the replications.

In 2012, Garnett et al. reported the results of a large-scale high throughput screen to identify novel interactions between investigational drugs and cancer-derived human cell lines, along with a similar study published at the same time by Barretina and colleagues (Barretina et al., 2012). In addition to capturing expected gene-drug interactions, several unpredicted associations were identified, including an enhanced sensitivity between the EWS-FLI1 translocation of Ewing’s sarcoma family tumors and poly(ADP-ribose) polymerase (PARP) inhibitors (Garnett et al., 2012). Selective inhibition of cell survival and proliferation in Ewing’s sarcoma cell lines was observed with the PARP inhibitor, olaparib, comparable to the observed inhibition in BRCA-deficient cells (Garnett et al., 2012). Further, the EWS-FLI1 translocation was reported to be sufficient for increased sensitivity of cells to olaparib, while transient depletion of EWS-FLI1 from Ewing’s sarcoma cells resulted in partial rescue of olaparib sensitivity, suggesting the sensitivity of Ewing’s sarcoma cells to olaparib might be related to EWS-FLI1 transcriptional activity.

The Registered Report for the 2012 paper by Garnett et al. described the experiments to be replicated (Figure 4C and E–F, and Supplemental Figures 16 and 20), and summarized the current evidence for these findings (Vanden Heuvel et al., 2016). Additional studies have reported hypersensitivity of Ewing’s sarcoma cell lines to PARP inhibitors (Brenner et al., 2012; Engert et al., 2015; Gill et al., 2015; Norris et al., 2014; Ordóñez et al., 2015; Smith et al., 2015a; Stewart et al., 2014). However, studies extending the use of olaparib, or other PARP inhibitors, as monotherapies in xenograft models have reported limited effectiveness (Norris et al., 2014; Ordóñez et al., 2015; Smith et al., 2015a; 2015b; Stewart et al., 2014), consistent with no objective responses from a phase II study of olaparib (Choy et al., 2014). In agreement with these observations, a new methodology for biomarker discovery, that accounts for variability in general levels of drug sensitivity, failed to find a statistically significant association of PARP inhibitors and the EWS-FLI1 translocation (Geeleher et al., 2016). However, studies testing combinatorial treatments of PARP inhibitors with other drugs, such as the DNA alkylating agent temozolomide, have reported enhanced sensitivity of Ewing sarcomas (Brenner et al., 2012; Engert et al., 2015; Gill et al., 2015; Norris et al., 2014; Ordóñez et al., 2015; Smith et al., 2015b; Stewart et al., 2014), with several clinical trials beginning (Pishas and Lessnick, 2016). Furthermore, a recent study reported that cells with SLFN11 inactivation are more resistant to PARP inhibitors, as single agents or in combination with temozolomide; however combination with an ATR inhibitor can overcome this resistance (Murai et al., 2016).

The outcome measures reported in this Replication Study will be aggregated with those from the other Replication Studies to create a dataset that will be examined to provide evidence about reproducibility of cancer biology research, and to identify factors that influence reproducibility more generally.

Results and discussion

Sensitivity of Ewing’s sarcoma cell lines to PARP inhibition

We sought to independently replicate whether Ewing’s sarcoma cell lines were more sensitive to the PARP inhibitor, olaparib, than control cell lines. This experiment is comparable to what was reported in Figure 4C and Supplemental Figure 16 of Garnett et al. (2012) and described in Protocol 1 in the Registered Report (Vanden Heuvel et al., 2016). While the original study included a comparison of Ewing’s sarcoma cells to cell lines from other tumor types, this replication attempt was restricted to osteosarcoma cells. Similar to the original study, a BRCA2-deficient cell line (DoTc2-4510) and a BRCA-proficient cell line (MES-SA) were included to capture differential sensitivity to olaparib across genotypes. Olaparib sensitivity was determined for each cell line using a colony formation assay over a range of concentrations (0.1, 0.32, 1, 3.2, or 10 µM) with the effective concentration determined as the olaparib concentration at which the number of colonies were reduced by at least 90% compared to vehicle control. Similar to the original study, DoTc2-4510 cells were found to be highly sensitive to olaparib (0.1 µM), while MES-SA were largely resistant (10 µM) (Figure 1). The median effective concentration of osteosarcoma cell lines (n = 7) was 3.2 µM (range: 3.2 µM - 10 µM), while the median effective concentration of Ewing’s sarcoma cell lines (n = 5) was 1 µM (range: 0.1 µM - 10 µM) (Figure 1). This compares to the original study that reported a median effective concentration of 3.2 µM (range: 1 µM - 10 µM) for osteosarcoma cell lines (n = 8) and 1 µM (range: 0.32 µM - 1 µM) for Ewing’s sarcoma cell lines (n = 5) (Garnett et al., 2012).

Figure 1. Sensitivity of Ewing’s sarcoma cell lines to olaparib.

Colony formation assays were performed on the indicated cell lines in the presence of a range of olaparib concentrations (0.1, 0.32, 1, 3.2, or 10 µM) or vehicle control (DMSO). Plates were retreated every 3 or 4 days, fixed and stained 7–21 days following plating, and colonies counted. The effective concentration displayed for each cell line is defined as the concentration that reduced colony formation by greater than 90% compared to vehicle control. For G-292 cells, where the highest olaparib concentration tested (10 µM) did not inhibit colony formation by at least 90%, the effective concentration was defined as 10 µM. Wilcoxon-Mann-Whitney test for ordinal data comparing the effective concentrations of Ewing’s sarcoma cell lines to osteosarcoma cell lines; U = 12, p=0.390. Additional details for this experiment can be found at https://osf.io/zy3s5/.

Figure 1.

Figure 1—figure supplement 1. Repeat of colony formation assay.

Figure 1—figure supplement 1.

This is a repeat of the same experiment as in Figure 1. G-292 cells, did not inhibit colony formation by at least 90% at the highest olaparib concentration tested (10 µM) and was thus defined as 10 µM. Wilcoxon-Mann-Whitney test for ordinal data comparing the effective concentrations of Ewing’s sarcoma cell lines to osteosarcoma cell lines; U = 11, p=0.302. Additional details for this experiment can be found at https://osf.io/zy3s5/.
Open in a new tab

To compare the sensitivity of Ewing’s sarcoma cells to osteosarcoma cells, a Wilcoxon-Mann-Whitney test for ordinal data was performed using the effective concentration as outlined in the Registered Report (Vanden Heuvel et al., 2016). Using a sample size determined a priori to detect the effect based on the originally reported data, the comparison of the effective concentrations of Ewing’s sarcoma cell lines to osteosarcoma cell lines was not statistically significant (U = 12, p=0.390). Thus, the null hypothesis that the effective concentration of olaparib is similar for Ewing’s sarcoma and osteosarcoma cell lines can not be rejected. A similar result was obtained with a duplicate set of plates using the same cell lines (Figure 1—figure supplement 1). To summarize, for this experiment we found results that were in the same direction as the original study and not statistically significant.

These results should take into consideration that other methods to assess olaparib sensitivity, such as cellular viability assays, did not result in all Ewing’s sarcoma cell lines tested being acutely sensitive to PARP inhibitors, but rather a majority displaying increased sensitivity with some cell lines relatively insensitive (Garnett et al., 2012; Gill et al., 2015). Importantly, the cellular viability assays utilized a larger sample size compared to the colony formation assay reported here. The sample size used for this replication attempt was determined from the effect reported in the original study between Ewing’s sarcoma and osteosarcoma cell lines. Although the replication study was powered to detect the original effect size estimate with at least 80% power, both the original study and this replication used relatively small sample sizes, which can confound the findings and prevent them from being extrapolated to the overall population (Button et al., 2013; Faber and Fonseca, 2014). Similarly, only five of the cell lines tested were the same between the two studies, largely due to the inability to obtain any of the Ewing’ sarcoma cell lines used in the original study. These factors, among others, influence the research outcome of each designed experiment. Further studies should take into account both of these results, especially when considering the number of cell lines to test.

Sensitivity to olaparib in cells transformed with the EWS-FLI1 rearrangement

The EWS-FLI1 rearrangement is characteristic of Ewing’s sarcoma tumors and in the original study was identified as a statistically significant association with olaparib sensitivity (Garnett et al., 2012). To test whether the sensitivity to olaparib was due to the EWS-FLI1 rearrangement, we independently replicated an experiment comparing olaparib sensitivity in mouse mesenchymal cells transformed with EWS-FLI1, or FUS-CHOP a related liposarcoma-associated translocation (Riggi et al., 2006; 2005). This experiment is similar to what was reported in Figure 4E of Garnett et al. (2012) and described in Protocol 2 in the Registered Report (Vanden Heuvel et al., 2016). Using the same transformed mouse mesenchymal cells as the original study, as well as the human Ewing’s sarcoma cell line SK-N-MC for comparison, sensitivity to olaparib was determined using a cellular viability assay over a range of concentrations. During the course of the assay, the cells continued to proliferate in no drug conditions (Figure 2—figure supplement 1), a necessary condition since olaparib sensitivity relies on cellular proliferation (Dale Rein et al., 2015; Weston et al., 2010). While the SK-N-MC cells displayed sensitivity to olaparib with a mean IC50 of 2.67 µM, 95% CI [3.11–2.23], the EWS-FLI1 and FUS-CHOP transformed mouse mesenchymal cells were both relatively resistant with more than 50% of cells remaining viable at the highest dose tested (Figure 2). This compares to the original study that reported EWS-FLI1 transformed mouse mesenchymal cells displaying olaparib sensitivity similar to SK-N-MC cells (estimated IC50 of 1.1 µM for EWS-FLI1 and 1.5 µM for SK-N-MC) with FUS-CHOP transformed cells remaining relatively resistant (estimated IC50 of 7.8 µM) (Garnett et al., 2012). The analysis plan specified in the Registered Report (Vanden Heuvel et al., 2016) proposed to compare the IC50 values from EWS-FLI1 transformed cells, or SK-N-MC cells, to FUS-CHOP transformed cells, however as stated above this could not be performed because of the inability to determine IC50 values for either of the transformed mouse mesenchymal cells. To summarize, for this experiment we found results that were not in the same direction as the original study.

Figure 2. Olaparib sensitivity in cells transformed with the EWS-FLI1 rearrangement.

Cell viability assays were performed for EWS-FLI1 and FUS-CHOP transformed mouse mesenchymal cells, as well as the human Ewing’s sarcoma cell line (SK-N-MC), which harbors the EWS-FLI1 fusion. Cells were treated with the indicated doses of olaparib and 72 hr later cell viability was determined. Relative viability was calculated as a percentage of vehicle control treated cells. Means reported and error bars represent SD from three independent biological repeats. Additional details for this experiment can be found at https://osf.io/t3dm6/.

Figure 2.

Figure 2—figure supplement 1. Population doubling time of cells and confirmation of EWS-FLI1 rearrangement.

Figure 2—figure supplement 1.

(A) For each cell line, relative viability of untreated and vehicle control treated cells at 24 and 72 hr after treatment with olaparib were used to determine the population doubling times. Box and whisker plot with median represented as the line through the box and whiskers representing values within 1.5 IQR of the first and third quartile. Means as black dot and bold error bars represent 95% CI (n = 6). (B) Relative expression levels of EWS-FLI1 and Actb were determined by qRT-PCR for each cell line. Expression level of EWS-FLI1 transformed mouse mesenchymal cells was assigned a value of 100. Means reported from one biological repeat. Actb, a housekeeping gene for the mouse mesenchymal cells, was not expected to be expressed in SK-N-MC cells, which are human. The EWS-FLI1 transformed mouse mesenchymal cells contain the EWS-FLI1 fusion gene from SK-N-MC cells (Riggi et al., 2005). Additional details for this experiment can be found at https://osf.io/t3dm6/.
Open in a new tab

Olaparib sensitivity after depletion of EWS-FLI1 from A673 cells

To test if EWS-FLI1 is necessary for olaparib sensitivity, we replicated an experiment similar to what was reported in Figure 4F and Supplemental Figure 20 in Garnett et al. (2012) and described in Protocol 3 in the Registered Report (Vanden Heuvel et al., 2016). Using the Ewing’s sarcoma cell line, A673, which harbors the EWS-FLI1 translocation, cells were transfected with either siRNA targeting the EWS-FLI1 fusion or control siRNA and then concomitantly treated with a range of concentrations of olaparib or vehicle control. Knockdown efficiency was examined by quantitative real-time-polymerase chain reaction (qRT-PCR). A673 cells transfected with EWS-FLI1 siRNA resulted in an average reduction of 65% in EWS-FLI1 expression relative to control siRNA whether cells were treated with vehicle control or olaparib (Figure 3B). Cellular viability was decreased by olaparib treatment in a dose dependent manner with EWS-FLI1 and control depleted cells displaying similar sensitivities (Figure 3A, Figure 3—figure supplement 1). This compares to the original study that reported olaparib sensitivity was partially reverted when EWS-FLI1 was depleted from A673 cells (estimated IC50 of 1.4 µM for control siRNA and 2.7 µM for EWS-FLI1 siRNA) (Garnett et al., 2012). The original study also reported an achieved knockdown of ~94% in EWS-FLI1 expression whether cells were treated with olaparib or vehicle control (Garnett et al., 2012). The difference in achieved knockdown between the original study and this replication attempt is a possible reason for the differences in olaparib sensitivity outcomes. The level of knockdown required to yield a given phenotype varies because it is system-dependent (e.g. cell type, assay, function of gene-of-interest), thus, a higher level of knockdown might be required to observe an effect with this experimental design. Importantly, though, observing different outcomes are informative to establish the range of conditions under which a given phenotype can be observed (Bailoo et al., 2014).

Figure 3. Olaparib sensitivity after depletion of EWS-FLI1 from A673 cells.

A673 Ewing’s sarcoma cells transiently transfected with negative control siRNA (siCT) or an siRNA targeting the EWS-FLI1 translocation (siEF1). (A) Cell viability assays were performed with the indicated doses of olaparib, or an equivalent volume of vehicle control (DMSO). After 72 hr of treatment cell viability was determined. Relative viability was calculated as a percentage of untreated cells. Means reported and error bars represent SD from three independent biological repeats. (B) siRNA-mediated depletion of EWS-FLI1 was determined after 72 hr treatment with vehicle control (DMSO) or olaparib. Relative expression levels of EWS-FLI1 expression normalized to RPLP0 (ribosomal protein lateral stalk subunit P0) was determined by qRT-PCR. Expression level of siCT cells treated with DMSO was assigned a value of 100. Means reported and error bars represent SD from three independent biological repeats. Two-way ANOVA main effect for siRNA (siCT or siEF1); F(1,12) = 96, p=4.46×10−7. Pairwise contrast between DMSO treated cells transfected with siCT or siEF1; t(12) = 7.32, uncorrected p=9.17×10−6 with a priori alpha level = 0.025; (Bonferroni corrected p=1.83×10−5). Pairwise contrast between olaparib treated cells transfected with siCT or siEF1; t(12) = 6.53, uncorrected p=2.80×10−5 with a priori alpha level = 0.025; (Bonferroni corrected p=5.60×10−5). Additional details for this experiment can be found at https://osf.io/2w22x/.

Figure 3.

Figure 3—figure supplement 1. Cell viability assays for each biological repeat.

Figure 3—figure supplement 1.

This is the same experiment as in Figure 3. The line graph of each biological repeat is plotted individually. Means reported and error bars represent SD from three technical replicates for each measurement. Additional details for this experiment can be found at https://osf.io/2w22x/.
Open in a new tab

It’s also worth noting that in the original study, olaparib treatment in the control siRNA cells resulted in a relative viability of zero at the highest concentrations of olaparib tested, while in this replication attempt relative viability remained at around 40% (Figure 3A). This could be due to how the lower bound of detection (maximal response) was determined (Sebaugh, 2011) as well as differences in cellular growth conditions that could impact olaparib sensitivity (Stordal et al., 2013; Strese et al., 2013). The analysis plan specified in the Registered Report (Vanden Heuvel et al., 2016) proposed to compare the IC50 values from control siRNA transfected cells to EWS-FLI1 siRNA transfected cells, however this could not be performed because of the inability to determine absolute IC50 values for either condition following published guidelines (Sebaugh, 2011). To summarize, for this experiment we found the cell viability results were not in the same direction as the original study.

Meta-analysis of original and replication effects

We performed a meta-analysis using a random-effects model, where possible, to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Vanden Heuvel et al., 2016). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. Cliff’s delta (d) is a non-parametric estimate of effect size that measures how often a value in one group is larger than the values from another group. The estimate of the effect size of one study, as well as the associated uncertainty (i.e. confidence interval), compared to the effect size of the other study provides another approach to compare the original and replication results (Errington et al., 2014; Valentine et al., 2011). Importantly, the width of the confidence interval for each study is a reflection of not only the confidence level (e.g. 95%), but also variability of the sample (e.g. SD) and sample size.

The comparison of the effective concentration of olaparib for Ewing’s sarcoma cells to osteosarcoma cells resulted in a Cliff’s d = 0.34, 95% CI [−0.24, 0.74] for this study, whereas Cliff’s d = 0.93, 95% CI [0.62, 0.99] for the data estimated a priori from Figure 4C of the original study (Garnett et al., 2012). A meta-analysis (Figure 4) of these two effects resulted in Cliff’s d = 0.63, 95% CI [−0.10, 0.92], which was statistically significant (p=0.029). Importantly, the confidence interval around Cliff’s d is asymmetric, while the p value is calculated using the normal distribution and is thus not well defined; however there is no agreement on how to compute p values from an asymmetric distribution (Dunne et al., 1996; Rohatgi and Saleh, 2000). Both results are consistent when considering the direction of the effect, however the point estimate of the replication effect size was not within the confidence interval of the original result, or vice versa.

Figure 4. Meta-analysis of effect.

Figure 4.

Open in a new tab

Effect size and 95% confidence interval are presented for Garnett et al. (2012), this replication attempt (RP:CB), and a random effects meta-analysis of those two effects. Sample sizes used in Garnett et al. (2012) and this replication attempt are reported under the study name. Random effects meta-analysis of effective concentrations of Ewing’s sarcoma cell lines to osteosarcoma cell lines from colony formation assays (meta-analysis p=0.029). Additional details for this meta-analysis can be found at https://osf.io/whs6e/.

This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Vanden Heuvel et al., 2016). However, this is limited to what was obtainable from the original paper and through communication with the original authors, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as cell lines, number of cells plated/injected, and the specific PARP inhibitor were maintained, others were unknown or not easily controlled for. These include variables such as cell line genetic drift (Hughes et al., 2007; Kleensang et al., 2016), sex of cell lines (Clayton and Collins, 2014), impacts of atmospheric oxygen on cell viability and growth (Boregowda et al., 2012), and differing compound potency resulting from different stock solutions (Kannt and Wieland, 2016) or from variation in cell division rates (Hafner et al., 2016). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.

Materials and methods

As described in the Registered Report (Vanden Heuvel et al., 2016), we attempted a replication of the experiments reported in Figure 4C and E–F, and Supplemental Figures 16 and 20 of Garnett et al. (2012). A detailed description of all protocols can be found in the Registered Report (Vanden Heuvel et al., 2016). Additional detailed experimental notes, data, and analysis are available on the Open Science Framework (OSF) (RRID:SCR_003238) (https://osf.io/nbryi/; Vanden Heuvel et al., 2017).

Cell culture

A673 (ATCC, cat# CRL-1598, RRID:CVCL_0080), TC-71 (Children’s Oncology Group Cell Culture and Xenograft Repository (COGcell), RRID:CVCL_2213) (Whang-Peng et al., 1986), SK-N-MC (ATCC, cat# HTB-10, RRID:CVCL_0530), CHLA-9 (COGcell, RRID:CVCL_M150) (Batra et al., 2004), CHLA-10 (COGcell, RRID:CVCL_6583) (Batra et al., 2004), U-2-OS (ATCC, cat# HTB-96, RRID:CVCL_0042), SJSA-1 (ATCC, cat# CRL-2098, RRID:CVCL_1697), SAOS-2 (ATCC, cat# HTB-85, RRID:CVCL_0548), HOS (ATCC, cat# CRL-1543, RRID:CVCL_0312), MG-63 (ATCC, cat# CRL-1427, RRID:CVCL_0426), 143B (ATCC, cat# CRL-8303, RRID:CVCL_2270), G-292 clone A141B1 (ATCC, cat# CRL-1423, RRID:CVCL_2909), DoTc2-4510 (ATCC, cat# CRL-7920, RRID:CVCL_1181), MES-SA (ATCC, cat# CRL-1976, RRID:CVCL_1404), and EWS-FLI1 and FUS-CHOP transformed mouse mesenchymal progenitor cells (shared by Stamenkovic lab, Centre Hospitalier Universitaire Vaudois (CHUV) from vials frozen in 2006) were maintained in growth medium as described in the Registered Report (Vanden Heuvel et al., 2016) with Fetal Bovine Serum (FBS) (Gibco, cat# 16000–036), Roswell Park Memorial Institute (RPMI) 1640 medium (Gibco, cat# A10491), and Dulbecco’s Modified Eagle’s Medium (DMEM), high-glucose (Gibco, cat# 11965–092) sourced differently than listed. A673 (Giard et al., 1973) and SK-N-MC cells (Biedler et al., 1973) were originally classified as neuroblastoma cell lines, but have since been determined to be Ewing’s sarcoma cells (Martínez-Ramírez et al., 2003; Whang-Peng et al., 1986). Cells were grown at 37°C in a humidified atmosphere at 5% CO2. Quality control data for cell lines are available at https://osf.io/x4zwb/. This includes results confirming the cell lines were free of mycoplasma contamination and common mouse pathogens (IDEXX BioResearch, Columbia, Missouri). Additionally, STR DNA profiling of the cell lines was performed and all cells were confirmed to be the indicated cell lines when queried against STR profile databases.

Therapeutic compounds

10 mM olaparib (Selleckchem, cat# S1060; lot# S106021) was aliquoted into amber vials with O-ring screw caps, sparged with Argon, and stored at −80°C until use. Freeze-thaws were limited to less than five times.

Colony formation assays

Cells were plated at low density (2000 cells per well of 6-well culture plate (Falcon, cat# 353224)) and allowed to adhere overnight. The following day cells were treated with five doses of olaparib (10 µM, 2 µM, 1 µM, 0.32 µM, 0.1 µM) diluted to give a final DMSO concentration of 0.1% v/v or vehicle control (0.1% v/v DMSO). Medium was replaced and cells were retreated with olaparib or vehicle control every 3 or 4 days. When sufficient colonies were visible (greater than ~100) in the vehicle control condition (after 7–21 days), cells were washed with PBS and fixed with ice-cold methanol for 30 min while shaking at room temperature. Methanol was removed and cells were stained with Giemsa stain at 1:20 dilution in deionized water for 4 hr at room temperature (or overnight at 4°C) while shaking. Cells were rinsed with water, dried, and colonies were quantified (ImageJ software (RRID:SCR_003070), version 1.51h (Schneider et al., 2012)) in a blinded manner for each well. The effective concentration for each cell line was determined as the concentration that reduced colony formation by greater than 90% compared to vehicle control. For G-292 cells, where the highest olaparib concentration tested (10 µM) did not inhibit colony formation by at least 90%, the effective concentration was determined as 10 µM. This assay was performed with a duplicate set of plates using the same cell lines. Images of stained plates with cell line and olaparib concentration labeled, similar to Supplemental Figures 16 of Garnett et al. (2012), are available at https://osf.io/whwsk/.

Cell viability assays with mouse mesenchymal progenitor cells

The seeding density of each cell line was empirically determined by seeding between 500 and 1.6 × 104 cells in 96 well plates (Costar, cat# 3917) in 100 µl medium in technical quadruplicate. 48, 72, and 96 hr after seeding, cells were fixed in 4% paraformaldehyde (Affymetrix, cat# 19943) for 30 min at 37°C. Cells were stained with 1 µM Syto 60 red fluorescent nucleic acid stain, diluted in PBS, for 1 hr according to manufacturer’s instructions. Fluorescence signal was quantified using a red fluorescent filter (excitation = 625 nm/emission = 660–720 nm) and a GloMAX Multi Detection System (Promega, model# 9311–011), software version 2.3.2. For each cell line, a log(dose) response curve was fitted using the ‘log(Agonist) vs. response – Find ECanything’ analysis from GraphPad Prism software (San Diego, California, RRID:SCR_002798), version 6.0b. The number of cells seeded that achieved a 70% fluorescent signal (~70% confluent) after 96 hr was determined.

SK-N-MC cells were seeded at 8000 or 12,000 cells/well and EWS-FLI1 and FUS-CHOP transformed mouse mesenchymal progenitor cells were seeded at 3000 or 4,000 cells/well (non-edge wells) into a 96-well plate and incubated overnight. The following day cells were treated with serial dilutions of olaparib to yield six dilutions ranging from 0.39 µM to 12.5 µM to give a final DMSO concentration of 0.1% v/v, as well as vehicle control (0.1% v/v DMSO) and untreated cells that were used to normalize each biological repeat. All conditions were done in technical triplicate. Wells were fixed, stained, and fluorescence signal quantified as described for the seeding density step 24, 48, and 72 hr after start of treatment. Untreated and vehicle control treated wells were used to determine the population doubling time (Figure 2—figure supplement 1) using the formula (doubling time = incubation time*ln(2)/ln(72 hr plate average reading/24 hr plate average reading)). Relative viability was calculated as a percentage of vehicle control treated cells. Spline interpolation was performed on values for each biological repeat, where possible, to determine an estimate IC50 as described in the Registered Report (Vanden Heuvel et al., 2016) with R software (RRID:SCR_001905), version 3.3.2 (R Core Team, 2017).

siRNA transfection and cell viability assays with A673 cells

The A673 Ewing’s sarcoma cell line was seeded at 5,000 cells per well in a 96 well plate (non-edge wells) and immediately reverse transfected with 25 nM AllStars negative control siRNA (Qiagen, cat# 1027281) or an siRNA targeting the EWS-FLI1 translocation (Qiagen, cat# 1027423; custom order: 5’-GGCAGCAGAACCCUUCUUACG-3’) with Lipofectamine RNAiMAX with the cells in suspension according to manufacturer’s instructions. Immediately after siRNA transfection cells were treated with serial dilutions of olaparib to yield nine dilutions ranging from 100 µM to 0.015 µM or an equivalent volume of vehicle control (DMSO). Untreated cells as well as medium alone wells that were used for background subtraction were included. All conditions were done in technical triplicate. After 72 hr, cell viability was determined using Cell Titer 96 Aqueous One Solution Cell Proliferation Assay (Promega, cat# G3582) according to manufacturer’s instructions. After 4 hr incubation at 37°C in a humidified atmosphere at 5% CO2 absorbance was read at 490 nm using a GloMax Multi Detection System. Relative viability was calculated as a percentage of untreated cells after background subtraction (medium only wells).

Gene expression analysis

In parallel to the cell viability assay, A673 cells were seeded at 3 × 104 cells per well of a 24 well plate (Falcon, cat# 353226) and immediately reverse transfected with negative control siRNA or an siRNA targeting the EWS-FLI1 translocation as described above. Immediately after siRNA transfection cells were treated with 1.3 µM olaparib or an equivalent volume of vehicle control (0.013% v/v DMSO). After 72 hr, cells were harvested and RNA isolated using the NucleoSpin RNAII kit (Machery-Nagel, cat# 740955.5) according to manufacturer’s instructions. RNA concentration and purity was determined (data available at https://osf.io/ryvmu/). Total RNA was reverse transcribed into cDNA using High-capacity cDNA reverse transcription kit (Applied Biosystems, cat# 4368814) according to manufacturer’s instructions. qRT-PCR reactions were then performed in technical triplicate using EWS-FLI1 and RPLP0-specific primers (Integrated DNA Technologies: sequences listed in Registered Report [Vanden Heuvel et al., 2016]), and Perfecta SYBR green FastMix (Quantra Biosciences, cat# 95073–012) according to manufacturer’s instructions. PCR cycling conditions were used as follows: [1 cycle 95°C for 10 min – 40 cycles 95°C for 15 s, 60°C 60 s] using a StepOnePlus real-time PCR system (Applied Biosystems, cat# 4376592) and StepOne software (RRID:SCR_014281), version 2.3. EWS-FLI1 transcript levels were normalized to RPLP0 levels in each sample.

Confirmation of EWS-FLI1 rearrangement in mouse mesenchymal progenitor cells

Presence of the EWS-FLI1 rearrangement in the mouse mesenchymal progenitor cells shared for this replication attempt was confirmed by qPCR analysis (Figure 2—figure supplement 1B). SK-N-MC cells and EWS-FLI1 and FUS-CHOP transformed mouse mesenchymal progenitor cells (>3 × 106 cells) were pelleted at 110xg for 5 min, washed once with PBS, and lysed in 400 µl RNA lysis buffer (Promega, cat# Z3051). RNA extraction was performed with the SV96 Total RNA Isolation System (Promega, cat# Z3505) according to manufacturer’s instructions. RNA preps were eluted in 100 µl nuclease free water (Promega, cat# P119E). RNA concentration and purity was determined with a NanoDrop 2000 (Thermo Fisher Scientific, Waltham, Massachusetts) (data available at https://osf.io/2kz7n/). Total RNA was reverse transcribed into cDNA using High-capacity cDNA reverse transcription kit according to manufacturer’s instructions. qRT-PCR were then performed in technical triplicate using EWS-FLI1 and mouse Actb specific primers (EWS-FLI1 primer is same as above; Actb sequence is: Forward: 5’-GACTCATCGTACTCCTGCTTG-3’, Reverse: 5’-GATTACTGCTCTGGCTCCTAG-3’) at 200 nM final concentration, and Perfecta SYBR green FastMix according to manufacturer’s instructions. PCR conditions were the same as described above. Negative controls containing water with no cDNA template, as well as cDNA template without reverse transcriptase were included.

Statistical analysis

Statistical analysis was performed with R software (RRID:SCR_001905), version 3.3.2 (R Core Team, 2017). All data, csv files, and analysis scripts are available on the OSF (https://osf.io/nbryi/). Confirmatory statistical analysis was pre-registered (https://osf.io/wt8df/) before the experimental work began as outlined in the Registered Report (Vanden Heuvel et al., 2016). Data were checked to ensure assumptions of statistical tests were met. When described in the results, the Bonferroni correction, to account for multiple testings, was applied to the alpha error or the p-value. The Bonferroni corrected value was determined by divided the uncorrected value (.05) by the number of tests performed. A meta-analysis of a common original and replication effect size was performed with a random effects model and the metafor R package (Viechtbauer, 2010) (https://osf.io/whs6e/). For the colony formation analysis, to obtain an overall effect size estimate from this replication attempt, the estimates from the two repeats were combined with a fixed effects model. Furthermore, all meta-analyses were performed without weighting, since unweighted Cliff’s d has been reported to reduce bias (Kromrey et al., 2005). The asymmetric confidence intervals for the overall Cliff’s d estimate was determined using the normal deviate corresponding to the (1 - alpha/2)th percentile of the normal distribution (Cliff, 1993). The original study data presented in Figure 4C was extracted a priori from the published figure by determining the height of each bar, while the summary data (mean and standard deviation) pertaining to Figure 4E-F were shared by the original authors. The data and estimated IC50 values for Figure 4E-F were published in the Registered Report (Vanden Heuvel et al., 2016) and used in the power calculations to determine the sample size for this study.

Deviations from registered report

The source of FBS, RPMI 1640, DMEM, high-glucose, and SYBR Green PCR mix were different than what is listed in the Registered Report, with the used source and catalog number listed above. Statistical analysis proposed in the Registered Report for some of the experiments were unable to be performed as described above. We also included an additional test to confirm the EWS-FLI1 rearrangement in the mouse mesenchymal progenitor cells shared for this replication attempt. Additional materials and instrumentation not listed in the Registered Report, but needed during experimentation are also listed.

Acknowledgements

The Reproducibility Project: Cancer Biology would like to thank Dr. Cyril Benes (Massachusetts General Hospital Cancer Center) for sharing critical information and data and Dr. Ivan Stamenkovic (CHUV) for sharing the transformed mouse mesenchymal progenitor cells. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Funding Statement

The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Contributor Information

Reproducibility Project: Cancer Biology, Email: nicole@scienceexchange.com, tim@cos.io.

Joaquín M Espinosa, University of Colorado School of Medicine, United States.

Reproducibility Project: Cancer Biology:

Elizabeth Iorns, Rachel Tsui, Alexandria Denis, Nicole Perfito, and Timothy M Errington

Funding Information

This paper was supported by the following grant:

  • Laura and John Arnold Foundation to .

Additional information

Competing interests

Indigo Biosciences is a Science Exchange associated lab.

Ewa Maddox: Indigo Biosciences is a Science Exchange associated lab.

Samar W Maalouf: Indigo Biosciences is a Science Exchange associated lab.

EI, RT, NP: Employed by and hold shares in Science Exchange Inc.

Author contributions

Acquisition of data, Drafting or revising the article.

Acquisition of data.

Acquisition of data.

Analysis and interpretation of data, Drafting or revising the article.

Additional files

Transparent reporting form
elife-29747-transrepform.docx (245.3KB, docx)
DOI: 10.7554/eLife.29747.009

References

  1. Bailoo JD, Reichlin TS, Würbel H. Refinement of experimental design and conduct in laboratory animal research. ILAR Journal. 2014;55:383–391. doi: 10.1093/ilar/ilu037. [DOI] [PubMed] [Google Scholar]
  2. Barretina J, Caponigro G, Stransky N, Venkatesan K, Margolin AA, Kim S, Wilson CJ, Lehár J, Kryukov GV, Sonkin D, Reddy A, Liu M, Murray L, Berger MF, Monahan JE, Morais P, Meltzer J, Korejwa A, Jané-Valbuena J, Mapa FA, Thibault J, Bric-Furlong E, Raman P, Shipway A, Engels IH, Cheng J, Yu GK, Yu J, Aspesi P, de Silva M, Jagtap K, Jones MD, Wang L, Hatton C, Palescandolo E, Gupta S, Mahan S, Sougnez C, Onofrio RC, Liefeld T, MacConaill L, Winckler W, Reich M, Li N, Mesirov JP, Gabriel SB, Getz G, Ardlie K, Chan V, Myer VE, Weber BL, Porter J, Warmuth M, Finan P, Harris JL, Meyerson M, Golub TR, Morrissey MP, Sellers WR, Schlegel R, Garraway LA. The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature. 2012;483:603–307. doi: 10.1038/nature11003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Batra S, Reynolds CP, Maurer BJ. Fenretinide cytotoxicity for Ewing's sarcoma and primitive neuroectodermal tumor cell lines is decreased by hypoxia and synergistically enhanced by ceramide modulators. Cancer Research. 2004;64:5415–5424. doi: 10.1158/0008-5472.CAN-04-0377. [DOI] [PubMed] [Google Scholar]
  4. Biedler JL, Helson L, Spengler BA. Morphology and growth, tumorigenicity, and cytogenetics of human neuroblastoma cells in continuous culture. Cancer Research. 1973;33:2643–2652. [PubMed] [Google Scholar]
  5. Boregowda SV, Krishnappa V, Chambers JW, Lograsso PV, Lai WT, Ortiz LA, Phinney DG. Atmospheric oxygen inhibits growth and differentiation of marrow-derived mouse mesenchymal stem cells via a p53-dependent mechanism: implications for long-term culture expansion. Stem Cells. 2012;30:975–987. doi: 10.1002/stem.1069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brenner JC, Feng FY, Han S, Patel S, Goyal SV, Bou-Maroun LM, Liu M, Lonigro R, Prensner JR, Tomlins SA, Chinnaiyan AM. PARP-1 inhibition as a targeted strategy to treat Ewing's sarcoma. Cancer Research. 2012;72:1608–1613. doi: 10.1158/0008-5472.CAN-11-3648. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, Munafò MR. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience. 2013;14:365–376. doi: 10.1038/nrn3475. [DOI] [PubMed] [Google Scholar]
  8. Choy E, Butrynski JE, Harmon DC, Morgan JA, George S, Wagner AJ, D'Adamo D, Cote GM, Flamand Y, Benes CH, Haber DA, Baselga JM, Demetri GD. Phase II study of olaparib in patients with refractory Ewing sarcoma following failure of standard chemotherapy. BMC Cancer. 2014;14:813. doi: 10.1186/1471-2407-14-813. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Clayton JA, Collins FS. Policy: NIH to balance sex in cell and animal studies. Nature. 2014;509:282–283. doi: 10.1038/509282a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cliff N. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin. 1993;114:494–509. doi: 10.1037/0033-2909.114.3.494. [DOI] [Google Scholar]
  11. Dale Rein I, Solberg Landsverk K, Micci F, Patzke S, Stokke T. Replication-induced DNA damage after PARP inhibition causes G2 delay, and cell line-dependent apoptosis, necrosis and multinucleation. Cell Cycle. 2015;14:3248–3260. doi: 10.1080/15384101.2015.1085137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Dunne A, Pawitan Y, Doody L. Two-sided P-values from discrete asymmetric distributions based on uniformly most powerful unbiased tests. The Statistician. 1996;45:397. doi: 10.2307/2988542. [DOI] [Google Scholar]
  13. Engert F, Schneider C, Weiβ LM, Probst M, Fulda S. PARP inhibitors sensitize ewing sarcoma cells to temozolomide-induced apoptosis via the mitochondrial pathway. Molecular Cancer Therapeutics. 2015;14:2818–2830. doi: 10.1158/1535-7163.MCT-15-0587. [DOI] [PubMed] [Google Scholar]
  14. Errington TM, Iorns E, Gunn W, Tan FE, Lomax J, Nosek BA. An open investigation of the reproducibility of cancer biology research. eLife. 2014;3:e04333. doi: 10.7554/eLife.04333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Faber J, Fonseca LM. How sample size influences research outcomes. Dental Press Journal of Orthodontics. 2014;19:27–29. doi: 10.1590/2176-9451.19.4.027-029.ebo. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Garnett MJ, Edelman EJ, Heidorn SJ, Greenman CD, Dastur A, Lau KW, Greninger P, Thompson IR, Luo X, Soares J, Liu Q, Iorio F, Surdez D, Chen L, Milano RJ, Bignell GR, Tam AT, Davies H, Stevenson JA, Barthorpe S, Lutz SR, Kogera F, Lawrence K, McLaren-Douglas A, Mitropoulos X, Mironenko T, Thi H, Richardson L, Zhou W, Jewitt F, Zhang T, O'Brien P, Boisvert JL, Price S, Hur W, Yang W, Deng X, Butler A, Choi HG, Chang JW, Baselga J, Stamenkovic I, Engelman JA, Sharma SV, Delattre O, Saez-Rodriguez J, Gray NS, Settleman J, Futreal PA, Haber DA, Stratton MR, Ramaswamy S, McDermott U, Benes CH. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature. 2012;483:570–575. doi: 10.1038/nature11005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Geeleher P, Cox NJ, Huang RS. Cancer biomarker discovery is improved by accounting for variability in general levels of drug sensitivity in pre-clinical models. Genome Biology. 2016;17:190. doi: 10.1186/s13059-016-1050-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Giard DJ, Aaronson SA, Todaro GJ, Arnstein P, Kersey JH, Dosik H, Parks WP. In vitro cultivation of human tumors: establishment of cell lines derived from a series of solid tumors. JNCI: Journal of the National Cancer Institute. 1973;51:1417–1423. doi: 10.1093/jnci/51.5.1417. [DOI] [PubMed] [Google Scholar]
  19. Gill SJ, Travers J, Pshenichnaya I, Kogera FA, Barthorpe S, Mironenko T, Richardson L, Benes CH, Stratton MR, McDermott U, Jackson SP, Garnett MJ. Combinations of PARP inhibitors with temozolomide drive PARP1 trapping and apoptosis in Ewing's sarcoma. PLoS One. 2015;10:e0140988. doi: 10.1371/journal.pone.0140988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hafner M, Niepel M, Chung M, Sorger PK. Growth rate inhibition metrics correct for confounders in measuring sensitivity to cancer drugs. Nature Methods. 2016;13:521–527. doi: 10.1038/nmeth.3853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hughes P, Marshall D, Reid Y, Parkes H, Gelber C. The costs of using unauthenticated, over-passaged cell lines: how much more data do we need? BioTechniques. 2007;43:575–586. doi: 10.2144/000112598. [DOI] [PubMed] [Google Scholar]
  22. Kannt A, Wieland T. Managing risks in drug discovery: reproducibility of published findings. Naunyn-Schmiedeberg's Archives of Pharmacology. 2016;389:353–360. doi: 10.1007/s00210-016-1216-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kleensang A, Vantangoli MM, Odwin-DaCosta S, Andersen ME, Boekelheide K, Bouhifd M, Fornace AJ, Li HH, Livi CB, Madnick S, Maertens A, Rosenberg M, Yager JD, Zhaog L, Hartung T. Genetic variability in a frozen batch of MCF-7 cells invisible in routine authentication affecting cell function. Scientific Reports. 2016;6:28994. doi: 10.1038/srep28994. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kromrey JD, Hogarty KY, Ferron JM, Hines CV, Hess MR. Robustness in Meta-Analysis: An Empirical Comparison of Point and Interval Estimates of Standardized Mean Differences and Cliff’s Delta. Joint Statistical Meetings - American Statistical Association; Minneapolis. 2005. [Google Scholar]
  25. Martínez-Ramírez A, Rodríguez-Perales S, Meléndez B, Martínez-Delgado B, Urioste M, Cigudosa JC, Benítez J. Characterization of the A673 cell line (Ewing tumor) by molecular cytogenetic techniques. Cancer Genetics and Cytogenetics. 2003;141:138–142. doi: 10.1016/S0165-4608(02)00670-2. [DOI] [PubMed] [Google Scholar]
  26. Murai J, Feng Y, Yu GK, Ru Y, Tang SW, Shen Y, Pommier Y. Resistance to PARP inhibitors by SLFN11 inactivation can be overcome by ATR inhibition. Oncotarget. 2016;7:76534–76550. doi: 10.18632/oncotarget.12266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Norris RE, Adamson PC, Nguyen VT, Fox E. Preclinical evaluation of the PARP inhibitor, olaparib, in combination with cytotoxic chemotherapy in pediatric solid tumors. Pediatric Blood & Cancer. 2014;61:145–150. doi: 10.1002/pbc.24697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Ordóñez JL, Amaral AT, Carcaboso AM, Herrero-Martín D, del Carmen García-Macías M, Sevillano V, Alonso D, Pascual-Pasto G, San-Segundo L, Vila-Ubach M, Rodrigues T, Fraile S, Teodosio C, Mayo-Iscar A, Aracil M, Galmarini CM, Tirado OM, Mora J, de Álava E. The PARP inhibitor olaparib enhances the sensitivity of Ewing sarcoma to trabectedin. Oncotarget. 2015;6:18875–18890. doi: 10.18632/oncotarget.4303. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Pishas KI, Lessnick SL. Recent advances in targeted therapy for Ewing sarcoma. F1000Research. 2016;5:2077. doi: 10.12688/f1000research.8631.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. R Core Team . Vienna, Austria: R Foundation for Statistical Computing; 2017. [Google Scholar]
  31. Riggi N, Cironi L, Provero P, Suvà ML, Kaloulis K, Garcia-Echeverria C, Hoffmann F, Trumpp A, Stamenkovic I. Development of Ewing's sarcoma from primary bone marrow-derived mesenchymal progenitor cells. Cancer Research. 2005;65:11459–11468. doi: 10.1158/0008-5472.CAN-05-1696. [DOI] [PubMed] [Google Scholar]
  32. Riggi N, Cironi L, Provero P, Suvà ML, Stehle JC, Baumer K, Guillou L, Stamenkovic I. Expression of the FUS-CHOP fusion protein in primary mesenchymal progenitor cells gives rise to a model of myxoid liposarcoma. Cancer Research. 2006;66:7016–7023. doi: 10.1158/0008-5472.CAN-05-3979. [DOI] [PubMed] [Google Scholar]
  33. Rohatgi VK, Saleh A. Wiley Series in Probability and Statistics. Hoboken: John Wiley & Sons, Inc; 2000. An Introduction to Probability and Statistics: Rohatgi/An Introduction. [DOI] [Google Scholar]
  34. Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nature Methods. 2012;9:671–675. doi: 10.1038/nmeth.2089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sebaugh JL. Guidelines for accurate EC50/IC50 estimation. Pharmaceutical Statistics. 2011;10:128–134. doi: 10.1002/pst.426. [DOI] [PubMed] [Google Scholar]
  36. Smith MA, Hampton OA, Reynolds CP, Kang MH, Maris JM, Gorlick R, Kolb EA, Lock R, Carol H, Keir ST, Wu J, Kurmasheva RT, Wheeler DA, Houghton PJ. Initial testing (stage 1) of the PARP inhibitor BMN 673 by the pediatric preclinical testing program: PALB2 mutation predicts exceptional in vivo response to BMN 673. Pediatric Blood & Cancer. 2015a;62:91–98. doi: 10.1002/pbc.25201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Smith MA, Reynolds CP, Kang MH, Kolb EA, Gorlick R, Carol H, Lock RB, Keir ST, Maris JM, Billups CA, Lyalin D, Kurmasheva RT, Houghton PJ. Synergistic activity of PARP inhibition by talazoparib (BMN 673) with temozolomide in pediatric cancer models in the pediatric preclinical testing program. Clinical Cancer Research. 2015b;21:819–832. doi: 10.1158/1078-0432.CCR-14-2572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Stewart E, Goshorn R, Bradley C, Griffiths LM, Benavente C, Twarog NR, Miller GM, Caufield W, Freeman BB, Bahrami A, Pappo A, Wu J, Loh A, Karlström Å, Calabrese C, Gordon B, Tsurkan L, Hatfield MJ, Potter PM, Snyder SE, Thiagarajan S, Shirinifard A, Sablauer A, Shelat AA, Dyer MA. Targeting the DNA repair pathway in Ewing sarcoma. Cell Reports. 2014;9:829–840. doi: 10.1016/j.celrep.2014.09.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stordal B, Timms K, Farrelly A, Gallagher D, Busschots S, Renaud M, Thery J, Williams D, Potter J, Tran T, Korpanty G, Cremona M, Carey M, Li J, Li Y, Aslan O, O'Leary JJ, Mills GB, Hennessy BT. BRCA1/2 mutation analysis in 41 ovarian cell lines reveals only one functionally deleterious BRCA1 mutation. Molecular Oncology. 2013;7:567–579. doi: 10.1016/j.molonc.2012.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Strese S, Fryknäs M, Larsson R, Gullbo J. Effects of hypoxia on human cancer cell line chemosensitivity. BMC Cancer. 2013;13:331. doi: 10.1186/1471-2407-13-331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Valentine JC, Biglan A, Boruch RF, Castro FG, Collins LM, Flay BR, Kellam S, Mościcki EK, Schinke SP. Replication in prevention science. Prevention Science. 2011;12:103–117. doi: 10.1007/s11121-011-0217-6. [DOI] [PubMed] [Google Scholar]
  42. Vanden Heuvel JP, Bullenkamp J, Reproducibility Project: Cancer Biology Registered report: Systematic identification of genomic markers of drug sensitivity in cancer cells. eLife. 2016;5:e13620. doi: 10.7554/eLife.13620. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Vanden Heuvel JP, Maddox E, Maalouf SW, Iorns E, Tsui R, Denis A, Perfito N, Errington TM. Study 37: Replication of Garnett et al., 2012 (Nature) Open Science Framework. 2017 doi: 10.17605/OSF.IO/NBRYI. [DOI] [Google Scholar]
  44. Viechtbauer W. Conducting Meta-Analyses in R with the metafor Package. Journal of Statistical Software. 2010;36 doi: 10.18637/jss.v036.i03. [DOI] [Google Scholar]
  45. Weston VJ, Oldreive CE, Skowronska A, Oscier DG, Pratt G, Dyer MJ, Smith G, Powell JE, Rudzki Z, Kearns P, Moss PA, Taylor AM, Stankovic T. The PARP inhibitor olaparib induces significant killing of ATM-deficient lymphoid tumor cells in vitro and in vivo. Blood. 2010;116:4578–4587. doi: 10.1182/blood-2010-01-265769. [DOI] [PubMed] [Google Scholar]
  46. Whang-Peng J, Triche TJ, Knutsen T, Miser J, Kao-Shan S, Tsai S, Israel MA. Cytogenetic characterization of selected small round cell tumors of childhood. Cancer Genetics and Cytogenetics. 1986;21:185–208. doi: 10.1016/0165-4608(86)90001-4. [DOI] [PubMed] [Google Scholar]

Decision letter

Editor: Joaquín M Espinosa1

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Replication Study: Systematic identification of genomic markers of drug sensitivity in cancer cells" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Charles Sawyers as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Jenny Barrett (Reviewer #1); Cyril Benes (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary of discussion:

This replication study reports the results of a series of experiments aimed at reproducing findings in an original manuscript by Garnett et al. (Nature 2012) reporting increased sensitivity to PARP inhibitors among Ewing's sarcoma cell lines.

After thorough discussion among reviewers and members of the Editorial board, it was decided to request submission of a revised manuscript that explicitly states the inconclusive nature of the replication study.

The replication study reports increased sensitivity for PARP inhibitors among select Ewing's cell lines, but the difference is not statistically significant as reported in the original publication, which could certainly be attributed to the use of different and fewer sarcoma cell lines. Another key experiment involving knockdown of EWS-FLI is also inconclusive, due to the relatively poor siRNA knockdown of EWS-FLI relative to the original paper.

It is then required that these differences in the experimental approach be made very clear in the revised manuscript, and that the word "inconclusive" be part of the Abstract since the report does not disprove the hypothesis that Ewing's sarcoma cell lines are more sensitive to PARP inhibitors.

Although additional experiments cannot be requested at this point, the revised Discussion should include a statement saying that more cell lines would need to be tested to move beyond the current "inconclusive" nature of the report, as well as a more robust EWS-FLI knockdown.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Replication Study: Systematic identification of genomic markers of drug sensitivity in cancer cells" for further consideration at eLife. Your revised article has been favorably evaluated by Charles Sawyers (Senior Editor) and a Reviewing editor.

The manuscript has been improved and is essentially ready for acceptance provided you make editorial changes to the Abstract acknowledging that some of the results, particularly the siRNA experiments are inconclusive due to difference in knockdown efficiency. It is important that this be explicitly stated to avoid any misinterpretation of the conclusions. (The current concluding sentence in the Abstract, "No single study can provide conclusive evidence for or against a claim…" does not provide the necessary level of clarity.)

eLife. 2018 Jan 9;7:e29747. doi: 10.7554/eLife.29747.013

Author response

Summary of discussion:

This replication study reports the results of a series of experiments aimed at reproducing findings in an original manuscript by Garnett et al. (Nature 2012) reporting increased sensitivity to PARP inhibitors among Ewing's sarcoma cell lines.

After thorough discussion among reviewers and members of the Editorial board, it was decided to request submission of a revised manuscript that explicitly states the inconclusive nature of the replication study.

The replication study reports increased sensitivity for PARP inhibitors among select Ewing's cell lines, but the difference is not statistically significant as reported in the original publication, which could certainly be attributed to the use of different and fewer sarcoma cell lines. Another key experiment involving knockdown of EWS-FLI is also inconclusive, due to the relatively poor siRNA knockdown of EWS-FLI relative to the original paper.

It is then required that these differences in the experimental approach be made very clear in the revised manuscript, and that the word "inconclusive" be part of the Abstract since the report does not disprove the hypothesis that Ewing's sarcoma cell lines are more sensitive to PARP inhibitors.

Although additional experiments cannot be requested at this point, the revised Discussion should include a statement saying that more cell lines would need to be tested to move beyond the current "inconclusive" nature of the report, as well as a more robust EWS-FLI knockdown.

We have revised the manuscript to include further discussion on the sample size and selection of cell lines used in this replication and the original study for the colony survival assay and how this can influence a result. Similarly we revised the siRNA knockdown of EWS-FLI1 to include discussion about what conditions are necessary to observe an outcome, specifically how the knockdown percentage might influence the ability to detect an effect. These experiments are interpretable and the comments the reviewers raised are some of the possible explanations for the difference in results between the two studies and what the necessary conditions are to observe the effect reported in the original study.

We have also revised the Abstract and final paragraph of the Discussion to highlight how this study, just like any single study, is not conclusive evidence for or against an effect, but that the accumulation of evidence provides the support for or against a claim and the conditions necessary to observe it.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

The manuscript has been improved and is essentially ready for acceptance provided you make editorial changes to the Abstract acknowledging that some of the results, particularly the siRNA experiments are inconclusive due to difference in knockdown efficiency. It is important that this be explicitly stated to avoid any misinterpretation of the conclusions. (The current concluding sentence in the Abstract, "No single study can provide conclusive evidence for or against a claim…" does not provide the necessary level of clarity.)

We disagree with stating that results are 'inconclusive', regardless of what study or experiment it is in reference to, particularly if it is in the context of equating 'statistically significant' and 'positive results' with 'conclusive' and 'null' with 'inconclusive'. For example, failing to reject the null hypothesis (that Ewing's Sarcoma and other tested cell lines are equivalently sensitive to olaparib) does not mean we accept the null hypothesis, only that there is insufficient evidence at the predefined level of significance and power (sample size) to reject the null hypothesis. We do agree that the results we report could be attributed to the use of different sarcoma cell lines and a level of siRNA knockdown of EWS-FLI relative to the original study and have revised the Abstract to highlight this.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    Transparent reporting form
    elife-29747-transrepform.docx (245.3KB, docx)
    DOI: 10.7554/eLife.29747.009

    Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd

    RESOURCES