Skip to main content
NCBI home page
eLife logoLink to eLife
. 2014 Dec 10;3:e04180. doi: 10.7554/eLife.04180

Registered report: Melanoma genome sequencing reveals frequent PREX2 mutations

Denise Chroscinski 1, Darryl Sampey 2, Alex Hewitt 3; Reproducibility Project: Cancer Biology*, Elizabeth Iorns 5, William Gunn 6, Fraser Tan 7, Joelle Lomax 8, Timothy Errington 9
Editor: Roger Davis4
PMCID: PMC4270141  PMID: 25490935

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of 50 papers in the field of cancer biology published between 2010 and 2012. This Registered Report describes the proposed replication plan of key experiments from ‘Melanoma genome sequencing reveals frequent PREX2 mutations’ by Berger and colleagues, published in Nature in 2012 (Berger et al., 2012). The key experiments that will be replicated are those reported in Figure 3B and Supplementary Figure S6. In these experiments, Berger and colleagues show that somatic PREX2 mutations identified through whole-genome sequencing of human melanoma can contribute to enhanced lethality of tumor xenografts in nude mice (Figure 3B, S6B, and S6C; Berger et al., 2012). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

DOI: http://dx.doi.org/10.7554/eLife.04180.001

Research organism: mouse

Introduction

Melanoma is a highly aggressive tumor with poor prognosis in the metastatic stage. Based on their association with UV-induced DNA damage, melanomas are often hypermutated and considerable efforts have been made to sequence such tumors in order to better understand their molecular basis. Many well-known oncogenes are frequently involved in melanoma pathogenesis, including BRAF and NRAS, and significant work has been done to develop targeted kinase inhibitors against the protein products of these genes (Kunz, 2014). However, even with treatment, melanoma has an extremely high rate of recurrence; thus, there is great interest in identifying novel candidate genes that promote oncogenesis in melanoma, thereby providing additional therapeutic targets.

One such candidate is Phosphatidylinositol-3,4,5-trisphosphate RAC Exchanger 2 (PREX2), a 183-kDa protein known to inhibit PTEN phosphatase activity, stimulate PI3K signaling, and suspected to regulate the small GTPase RAC1 (Fine et al., 2009; Cerami et al., 2012). Using whole-genome sequencing of 25 metastatic tumors, Berger and colleagues identified PREX2 as being a highly mutated gene in melanoma. Apart from observing a large subset of BRAF and NRAS mutations, the authors found PREX2 to have a mutation frequency of approximately 14%, with 13 detected non-synonymous point mutations, including four nonsense truncation mutations (Berger et al., 2012). In order to demonstrate the biological relevance of specific PREX2 mutations, the authors created transformed melanocyte cell lines that stably expressed various mutated and truncated forms of PREX2. By using these cell lines to create tumor xenografts in nude mice, the authors showed that ectopic expression of mutant PREX2 accelerated tumor formation.

Berger and colleagues chose to analyze six representative PREX2 mutations derived from their whole-genome sequencing screen. These variants included three truncation variants and three non-synonymous point mutations predicted to carry functional impact. These mutant PREX2 constructs were packaged into lentiviruses and transduced into TERT-immortalized human melanocytes engineered to express NRASG12D. Ectopic expression of various mutant PREX2 isoforms was confirmed by Western blot (Figure 6A). These experiments will be replicated in Protocols 1 and 2. Berger and colleagues next transplanted the melanocytic lines into immunodeficient mice alongside control melanocytes expressing either wild-type PREX2 or GFP (green fluorescent protein). They found that overexpression of all three truncated variants, as well as the point mutation G844D, significantly accelerated tumor growth in vivo, thus affirming the biological relevance of their genomic data (Figure 3B, S6B, and S6C). These key experiments, which support the hypothesis that mutant PREX2 promotes oncogenesis in melanoma, will be replicated in Protocol 3.

There is some debate over which mutations observed in various melanoma samples are biologically relevant, including PREX2. Potentially, mutational heterogeneity across tumor samples may contribute to false-positive findings (Lawrence et al., 2013). Various genome-wide screens have yielded conflicting results about which genes are frequently mutated in melanoma. Recently, mutated PREX2 was identified in both the primary tumor and in metastatic tumor tissue from a genomic analysis of a single melanoma patient (Turajlic et al., 2012). However, five studies failed to identify PREX2 in their genome-wide melanoma screens, including a meta-analysis study that analyzed hundreds of published datasets (Hodis et al., 2012; Krauthammer et al., 2012; Ni et al., 2013; Marzese et al., 2014; Xia et al., 2014). To date, there have been no replication attempts assessing the biological significance of PREX2 mutant isoforms in melanoma.

Materials and methods

Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors.

Protocol 1: generation of NRASG12D melanocyte cells expressing various mutated forms of PREX2

This protocol describes the generation of pMEL/hTERT/CDK4(R24C)/p53DD/NRASG12D (NRASG12D) melanocytes that stably express various mutated forms of PREX2. This protocol details the production of lentivirus for each mutated PREX2 isoform, as well as the viral transduction of melanocytes, and selection for stable-expressing lines using antibiotic resistance.

Sampling

  • Outline of experimental endpoints:

    1. At the end of this protocol, we will have generated NRASG12D melanocytes overexpressing the following protein products:

      • GFP vector (control)

      • WT PREX2 (control)

      • PREX2 Q1430* (Truncation mutation)

      • PREX2 G844D (Substitution mutation)

Materials and reagents

Reagent Type Manufacturer Catalog # Comments
GenElute Endotoxin-free
Plasmid Maxiprep Kit		 Reagent Sigma PLEX15-1KT This kit replaces the Qiagen Endo-free Maxiprep kit used by the original authors
pMD2-Gag/Pol Viral packaging vector N/A N/A Reagent being provided by original authors
pMD2 VSVG Viral packaging vector N/A N/A Reagent being provided by original authors
RSV REV Viral packaging vector N/A N/A Reagent being provided by original authors
GFP Expression construct N/A N/A Reagent being provided by original authors
Wild-type PREX2 Expression construct N/A N/A Reagent being provided by original authors
PREX2 Q1430* Expression construct N/A N/A Reagent being provided by original authors
PREX2 G844D Expression construct N/A N/A Reagent being provided by original authors
HEK293T cells Cell line ATCC CRL-3216 Replaces original cells from Life Technologies
NRASG12D melanocytes Cell line N/A N/A Reagent being provided by original authors
Sequencing primers Oligos Sequences provided by original authors; specific brand information will be left up to the discretion of the replicating lab and recorded later
Sequencing reagents Reagent Specific brand information will be left up to the discretion of the replicating lab and recorded later
10 cm tissue culture dishes (plastic) Labware Corning (Sigma-Aldrich) CLS430167 Original brand not specified
10 cm tissue culture dishes (glass) Labware Corning (Sigma-Aldrich) CLS70165101 Additional reagent not used in original study
Fetal bovine serum (FBS) Cell culture reagent Sigma-Aldrich F0392 Replaces Invitrogen cat. no. 26400-036 used in original study
Dulbecco's Modified Eagle's Medium (DMEM) – high glucose Cell culture reagent Sigma-Aldrich D6429 Replaces Invitrogen cat. no. 11995-065 used in original study
Lipofectamine 2000 Transfection reagent Life Technologies 52887
OptiMEM-1 reduced serum medium Cell culture reagent Life Technologies 31985-070
Ham's F10 medium Cell culture reagent Sigma-Aldrich N6908 Replaces Invitrogen cat. no. 11550-043 used in original study
Fetal bovine serum (FBS); heat inactivated Cell culture reagent Sigma-Aldrich F4135 Replaces Invitrogen cat. no. 10082-147 used in original study
Penicillin–Streptomycin solution (100x) stabilized Cell culture reagent Sigma-Aldrich P4333 Replaces Invitrogen cat. no. 15140-122 used in original study
6 cm tissue culture dishes Labware Corning (Sigma-Aldrich) CLS430166 Original brand not specified
Hexadimethrine bromide (Polybrene) Cell culture reagent Sigma-Aldrich 107689 Original brand not specified
Blasticidin S, hydrochloride Antibiotic EMD-Millipore 203350 Original brand not specified
TRI reagent Reagent Sigma-Aldrich T9424 Additional reagent not used in original study
SuperScript III First-Strand Synthesis System cDNA synthesis Life Technologies 18080-051 Additional reagent not used in original study
Nuclease-Free Water (not DEPC treated) Reagent Life Technologies AM9930 Additional reagent not used in original study
RNase AWAY (spray) Reagent Fisher 21-402-178 Additional reagent not used in original study
Open in a new tab

Procedure

Note: all cell lines will be sent for STR profiling and mycoplasma testing.

  1. Grow and prepare endotoxin-free plasmid constructs according to the manufacturer's protocol for the GenElute Endotoxin-free Plasmid Maxiprep Kit.

    • A. Viral packaging vectors:

      • i. pMD2-Gag/Pol (∼25 µg DNA needed for production of 4 viruses)

      • ii. pMD2 VSVG (∼15 µg DNA needed for production of 4 viruses)

      • iii. RSV REV (∼17 µg DNA needed for production of 4 viruses)

    • B. PREX2 expression vectors:

      • i. GFP vector (∼15 µg DNA needed for virus production)

      • ii. WT PREX2 (∼15 µg DNA needed for virus production)

      • iii. PREX2 Q1430* (∼15 µg DNA needed for virus production)

      • iv. PREX2 G844D (∼15 µg DNA needed for virus production)

  2. Sequence PREX2 plasmids to confirm identity and run on gel to confirm vector integrity. Use the following sequencing primers:

    • A. CMV forward: CGCAAATGGGCGGTAGGCGTG

    • B. prex2a-1 forward: ACTGAAATGCTAATGTGTGG

    • C. prex2a-2 forward: CCTTTTTACTCCAGTGATAAGAGAT

    • D. prex2a-3 forward: AGTACAGGCGGCCAACGAAG

    • E. prex2a-4 forward: ATCACAACCATGGCGGCCCCTT

    • F. prex2a-5 forward: GTAGGCTACTCCTGGCTCTT

    • G. prex2a-6 forward: AGCTGCCTGTGCAAACACAG

    • H. prex2a-7 reverse: GACTTCCTTCTGCTTGATAT

    • I. prex2a-8 reverse: TGCTGGTGAAGGAGGCGATG

    • J. prex2a-9 reverse: AGAGAATTTAGGCTGGTACA

    • K. prex2a-10 reverse: ATCCCTTTTCTACCAACTTT

    • L. prex2a-11 reverse: CTTGCTCCATTCCTAATTTT

    • M. prex2a-12 reverse: CCTTCTCATGGTTACTACAATATTC

    • N. V5 reverse: ACCGAGGAGAGGGTTAGGGAT

  3. Using the same primers as above, sequence the endogenous PREX2 gene from cDNA derived from untransfected pMEL/hTERT/CDK4(R24C)/p53DD/NRASG12D melanocytes.

    • A. Melanocytes should be maintained in Ham's F10 medium supplemented with 10% heat inactivated FBS and 1% penicillin/streptomycin at 37°C with 5% CO2.

    • B. Isolate total RNA using TRI reagent, and generate cDNA as described in the manufacturer's protocol for SuperScript III cDNA synthesis kit, using OligoDT primers to enrich for mRNA.

    • C. Use gene-specific primers to sequence the length of the PREX2 gene to determine endogenous mutational status.

  4. On Day 1 of viral production, plate 6 × 106 HEK293T cells in a 10 cm plate. Plate one 10-cm plate for each virus you wish to package (total of 4 plates needed).

    • A. HEK293T cells should be maintained in DMEM supplemented with 10% FBS at 37°C with 5% CO2.

    • B. Note: high titer lentivirus is best packaged in early passage, healthy 293T cells. Avoid continuous growth to/from confluence. Routinely split 293T when culture approaches 80% confluence.

  5. On Day 2, create the transfection master mix: (Tube #1)

    • A. Create a master mix (for the number of transfections being conducted) of Lipofectamine and OptiMEM.

      • i. Each transfection will require 30 µl of Lipofectamine diluted in 720 µl of OptiMEM. Allow mixture to incubate for 5 min at RT.

  6. For each virus, assemble DNA, packaging vectors, and OptiMEM in a 1.5 ml centrifuge tube (Tube #2)

    • A. Plasmid DNA = 10.0 µg

    • B. Packaging vector

      • i. pMD2 Gag/Pol = 5.0 µg

      • ii. pRESREV = 2.5 µg

      • iii. pMD2 VSVG = 3.0 µg

    • C. Bring volume to 750 µl with OptiMEM.

  7. Combine Tube #1 (Lipofectamine/OptiMEM) with Tube #2 (DNA/packaging vector/OptiMEM). After combining, mix by pipetting and allow the mixture to incubate for 20 min at RT.

    • A. While incubating, ‘gently’ aspirate growth medium from HEK293T cells and pipette 8 ml of OptiMEM to each plate.

    • B. Add 1.5 ml of transfection mixture to the plate (pipetting directly into the media) and place into the 37°C incubator.

    • C. Allow minimum 6–8 hr for transfection. After transfection completion, remove OptiMEM media and refresh HEK293T plates with 10 ml of growth media (again pipetting gently onto the side of the plate).

  8. On Day 4 (48 hr post-transfection) and 5 (72 hr post-transfection), collect virus by removing medium and filtering through a 0.45-µm filter into a 50 ml conical tube. Pool fractions from both the days. After the two collections, there is a total of 20 ml of virus. Immediately after collection/filtration (for both time points), put the virus on ice and then transfer to 4°C for short-term or −80°C for long-term storage.

  9. Infect pMEL/hTERT/CDK4(R24C)/p53DD/NRASG12D melanocytes with virus to generate stable cells lines.

    • A. Day 1: seed NRASG12D cells at 50% confluence in 6 cm plates.

      • i. Melanocytes should be maintained in Ham's F10 medium supplemented with 10% heat inactivated FBS and 1% penicillin/streptomycin at 37°C with 5% CO2.

    • B. Day 2: remove media and replace with 3 ml of viral supernatant containing 8 µg/ml polybrene.

      • i. Incubate cells for 24 hr.

    • C. Day 3: remove viral media and replace with fresh growth media.

    • D. Day 4: replace growth media with fresh media containing 5 µg/ml Blastocidin.

    • E. Days 4–9: select cells for ∼5 days, confirming that a plate of non-transduced NRASG12D cells is negatively selected in parallel.

    • F. Day 9: remove Blastocidin media and expand cells into fresh growth media. Collect entire population of transduced cells for further analysis.

Deliverables

  • Data to be collected:

    1. Sequencing information and gel-verification of PREX2 plasmids cloned into the pLenti6.3/V5 vector

    2. Mycoplasma testing of NRASG12D melanocytes

    3. STR profile of NRASG12D melanocytes

  • Sample delivered for further analysis:

    1. NRASG12D melanocytes stably expressing PREX2 mutant isoforms for further analysis (Protocols 2 and 3).

Confirmatory analysis plan

  • Statistical analysis of the Replication Data:

    1. Not applicable.

Known differences from the original study

This replication is only generating stable melanocyte lines for GFP, wild-type PREX2, PREX2 Q1430*, and PREX2 G844D. The original study also included several other PREX2 mutants, including PREX2 K278*, E824*, P948S, and G106E. This replication will include the additional step of sequencing the endogenous PREX2 gene in the NRASG12D melanocyte cell line to determine its mutational status. All known differences in reagents and supplies are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. PREX2 expression constructs obtained from the original authors will be verified for sequence identity and DNA integrity. The endogenous mutational status of PREX2 in NRASG12D melanocytes will be assessed. All data obtained from the experiment will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/jvpnw/).

Protocol 2: confirming ectopic expression of PREX2 mutant isoforms by Western blot

This protocol investigates the expression levels of mutant PREX2 isoforms in virally transduced NRASG12D melanocytes that were generated in Protocol 1. This protocol uses an anti-V5 antibody to recognize tagged forms of wild-type and mutant PREX2 (as well as the GFP control), thus verifying the successful lentiviral transduction of expression constructs and providing information about ectopic protein expression levels (as was demonstrated in Figure 6A). Membranes will also be probed with anti-α-tubulin to provide normalized values of relative protein expression. Three original cell lines produced by the original authors will also be included so that protein expression levels can be compared between the two studies.

Sampling

  1. The original data presented is qualitative and this prevents power calculations being performed a priori to determine sample size (number of biological replicates). Instead, we will be including three cell lines originally derived by the authors and analyzing these cell lines in parallel to the newly derived cell lines from Protocol 1.

  2. Three separate lysates will be prepared from each cell line:

    • GFP vector stable NRASG12D cells (control)

    • Previously generated PREX2 Q1430* stable NRASG12D cells (control from original study authors)

    • Previously generated PREX2 G844D stable NRASG12D cells (control from original study authors)

    • Previously generated WT PREX2 stable NRASG12D cells (control from original study authors)

    • PREX2 WT stable NRASG12D cells (from Protocol 1)

    • PREX2 Q1430* stable NRASG12D cells (from Protocol 1)

    • PREX2 G844D stable NRASG12D cells (from Protocol 1)

  1. Blots will be probed with the following antibodies:

    • 1. Anti-V5 tag

    • 2. Anti-PREX2

    • 3. Anti-alpha tubulin

Materials and reagents

Reagent Type Manufacturer Catalog # Comments
NRASG12D melanocytes expressing GFP Cell line Produced in Protocol 1
NRASG12D melanocytes expressing WT PREX2 Cell line Produced in Protocol 1
NRASG12D melanocytes expressing PREX2 Q1430* Cell line Produced in Protocol 1
NRASG12D melanocytes expressing G844D Cell line Produced in Protocol 1
NRASG12D melanocytes expressing WT PREX2 Cell line Obtained from original authors
NRASG12D melanocytes expressing PREX2 Q1430* Cell line Obtained from original authors
NRASG12D melanocytes expressing G844D Cell line Obtained from original authors
Ham’s F10 medium Cell culture reagent Sigma-Aldrich N6908 Replaces Invitrogen cat. no. 11550-043 used in original study
Fetal bovine serum (FBS); heat inactivated Cell culture reagent Sigma-Aldrich F4135 Replaces Invitrogen cat. no. 10082-147 used in original study
Penicillin–streptomycin solution (100x) stabilized Cell culture reagent Sigma-Aldrich P4333 Replaces Invitrogen cat. no. 15140-122 used in original study
IGEPAL CA-630
(NP-40 substitute)		 Reagent Sigma-Aldrich I8896 Replaces US Biological cat. no. N3500 used in original study
Phenylmethanesulfonyl fluoride (PMSF) Reagent Sigma-Aldrich 78,830 Replaces Pierce cat. no. 36978 used in original study
Protease inhibitor cocktail (mammalian) Reagent Sigma-Aldrich P8340 Replaces Roche cat. no. 11836153001 used in original study
Phosphatase inhibitor cocktail 2 Reagent Sigma-Aldrich P5726 Replaces Roche cat. no. 04906837001 used in original study
Coomassie (Bradford) Protein Assay Kit Reagent Thermo-Fisher (Pierce) PI-23200 Original brand not specified
BCA Protein Assay Kit Reagent Thermo-Fisher (Pierce) 23227 Original brand not specified
10-cm tissue culture dishes Labware Corning (Sigma-Aldrich) CLS430167 Original brand not specified
Novex 4-12% Tris-Glycine, Mini, 1.0 mm, 12-well Reagent Life Technologies EC60352
Novex Tris-Glycine SDS Running Buffer (10X) Reagent Life Technologies LC2675 Original brand not specified
Novex Tris-Glycine SDS Sample Buffer (2X) Reagent Life Technologies LC2676 Original brand not specified
NuPAGE® Sample Reducing Agent (10X) Reagent Life Technologies NP0009 Original brand not specified
ECL DualVue Western Markers (15 to 150 kDa) Reagent Sigma-Aldrich GERPN810 Original brand not specified
BLUEeye prestained protein ladder Reagent Sigma-Aldrich 94964 Original brand not specified
Nitrocellulose membrane Reagent BioRad 162-0113 Original brand not specified
Ponceau S solution Reagent Sigma-Aldrich P7170 Original brand not specified
Mouse anti-V5 tag Antibody Invitrogen 451098
Mouse anti-α-tubulin, clone DM1A Antibody Sigma-Aldrich T9026
Mouse anti-PREX2 Antibody Abcam Ab169027 Additional reagent not used in original study
Horse anti-mouse IgG, HRP-linked antibody Antibody Cell Signaling Technologies (CST) 7076
Tris Buffered Saline (TBS) Reagent Sigma-Aldrich T5912 Replaces Fisher cat. no. BP2471-1 used in original study
Tween 20 Reagent Sigma-Aldrich P1379 Original brand not specified
ECL Prime Western Blotting Detection Reagent Reagent Sigma-Aldrich (GE Healthcare) GERPN2236 Replaces Pierce cat. no. 34075 used in original study
Open in a new tab

Procedure

  1. Maintain NRASG12D melanocyte lines in Ham's F10 medium with 10% heat inactivated FBS and 1% penicillin/streptomycin at 37°C with 5% CO2.

  2. Subculture the four cell lines onto three 10-cm plates each, for a total of 12 plates. These plates constitute replicates for each cell line for eventual quantitation of protein expression. Allow cells to grow to log phase.

  3. Place 10 cm plates of log-phase growing cells on ice. Use a cell-scraper to scrape cells (on ice) into a microcentrifuge tube. Add 250 µl of lysis buffer per 10 cm plate.

    • A. Lysis buffer = 20 mM Tris–HCl, pH 8.0, 150 mM NaCl, 2 mM EDTA, 1% NP40, 1 mM PMSF, 1× protease inhibitor cocktail, and 1× phosphatase inhibitor.

    • B. Prepare three separate lysates for each cell line (one lysate scraped from each plate).

  4. Incubate cells in lysis buffer for 20 min at RT. Centrifuge lysate for 15 min at 14,000 rpm at 4°C. Transfer supernatant to a fresh tube, then add 2× sample loading buffer containing reducing agent.

  5. Load lysate onto a pre-cast polyacrylamide 4–12% Tris–glycine gel with molecular weight ladder.

    • A. Quantify lysate total protein concentration.

    • B. Load 30–50 µg of total protein per well.

  6. Perform electrophoresis in standard Tris–glycine–SDS running buffer.

  7. Transfer the gel onto a nitrocellulose membrane.

    • A. Transfer buffer = 25 mM Tris–HCl, 192 mM glycine, 20% methanol.

    • B. Use standard wet-transfer for 1–2 hr; PREX2 is a relatively large protein (runs about 160 kDa).

    • C. Following protein transfer, stain the membrane with Ponceau-S in order to detect protein levels. Scan image of stained membrane before washing.

  8. Block membrane in 5% milk in 1× TBS with 0.1% Tween-20 (TBS-T) overnight at 4°C on an orbital shaker.

  9. Incubate membranes with primary antibody overnight at 4°C on an orbital shaker. Dilute primary antibodies in 5% bovine serum albumin in TBS-T containing 0.05% sodium azide.

    • A. Mouse anti-V5; dilute at 1:5000.

    • B. Mouse anti-PREX2; use at 1 µg/ml, according to the manufacturer's instructions.

  10. Wash membranes six times for 10 min each with TBS-T at room temperature (RT).

  11. Incubate membranes with secondary antibody for 40 min at RT on an orbital shaker. Dilute secondary antibody in 5% milk in TBS-T.

    • A. Horse anti-mouse IgG; dilute at 1:2000

  12. Wash membranes six times for 10 min each with TBS-T at RT.

  13. Detect chemiluminescent signal with ECL Prime Western blotting detection reagent, according to the manufacturer's instructions.

  14. Strip blots for 15 min in 0.2 M NaOH, then wash membranes six times for 10 min each with TBS-T at RT.

  15. Block membranes in 5% milk in TBS-T for 1 hr at RT on an orbital shaker.

  16. Repeat steps 9–13 for anti-α-tubulin primary antibody. Dilute primary antibody at 1:5000. Use same secondary antibody (at same dilution) as above.

  17. Quantify density of bands and normalize against α-tubulin.

Deliverables

  • Data to be collected:

    1. Images of probed membranes (full images with ladder)

    2. Scanned images of Ponceau-stained membranes, post-transfer

    3. Densitometric analyses of normalized bands, presented in a bar graph showing standard deviation across replicates for each cell line

Confirmatory analysis plan

  • Statistical analysis of the Replication Data:

    1. Means and standard deviations will be computed across replicates for each cell line.

    2. We will perform a 2-way ANOVA (2 × 3 factorial analysis), comparing expression levels of the three PREX2 variants and the two cell-line cohorts (the originally-derived cell lines and the newly-derived cell lines from Protocol 1). This analysis will test two parameters: a) whether the original and replication values are different and b) if the three PREX2 variants are different. Because our hypothesis is that they are all the same, no individual follow-up tests are needed.

Known differences from the original study

This replication is only analyzing protein expression from cell lines engineered to express GFP, wild-type PREX2, PREX2 Q1430*, and PREX2 G844D. The original study also included several other PREX2 mutants, including PREX2 K278*, E824*, P948S, and G106E. This replication includes an antibody probing for PREX2, so that we can better determine its endogenous expression level. Additionally, we are also testing protein expression in the original PREX2 cells lines derived by the original authors, so that we can compare expression levels between the original lines and the replication lines. All known differences in reagents and supplies are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The endogenous expression of PREX2 will be assessed in cell lines not overexpressing PREX2 variants. An image of Ponceau-stained membranes (post-transfer) will be included to verify successful protein transfer. All of the raw data, including the image files and quantified bands from the Western blot, will be uploaded to the project page on the OSF (https://osf.io/jvpnw/) and made publically available. This experiment is also the quality control for the other replication protocols as it assesses the levels of ectopic PREX2 variant expression in the utilized cell lines.

Protocol 3: generation of tumor xenografts expressing mutated forms of PREX2

This protocol assesses the propensity of ectopically expressed PREX2 mutations to accelerate tumor formation of immortalized human melanocytes in vivo. This protocol utilizes stably transfected NRASG12D human melanocyte lines that were previously generated and analyzed in Protocols 1 and 2. The melanocytic lines are transplanted into immunodeficient mice alongside control melanocytes expressing wild-type PREX2 or GFP (green fluorescent protein). Tumor growth is assessed for 16 weeks, and tumor-free survival is monitored, as depicted in Figure 3B, S6B. Further, confirmatory staining and analysis of tumor tissue will be completed, as depicted in Figure 6C.

Sampling

  • These experiments will utilize 7, 8, or 14 mice per treatment group, for a total power of ≥80%.

    1. See Power calculations section for details

  • Outline of experimental conditions:

    1. NCR-NUDE female mice injected subcutaneously with:

  • GFP-vector stable NRASG12D melanocytes (control)

    • ○ n = 14

  • PREX2 WT stable NRASG12D melanocytes (control)

    • ○ n = 7

  • PREX2 Q1430* NRASG12D melanocytes

    • ○ n = 8

  • PREX2 G844D NRASG12D melanocytes

    • ○ n = 14

Materials and reagents

Reagent Type Manufacturer Catalog # Comments
NRASG12D melanocytes expressing GFP Cell line Produced in Protocol 1
NRASG12D melanocytes expressing WT PREX2 Cell line Produced in Protocol 1
NRASG12D melanocytes expressing PREX2 Q1430* Cell line Produced in Protocol 1
NRASG12D melanocytes expressing G844D Cell line Produced in Protocol 1
Ham’s F10 medium Cell culture reagent Sigma-Aldrich N6908 Replaces Invitrogen cat. no. 11550-043 used in original study
Hanks’ balanced salt solution, with sodium bicarbonate, without phenol red, calcium chloride, and magnesium sulfate Cell culture reagent Sigma-Aldrich H6648 Original brand not specified
Matrigel Matrix High Concentration (HC), phenol red-free Cell culture reagent Corning 354262 Original catalog number not specified
Fetal bovine serum (FBS); heat inactivated Cell culture reagent Sigma-Aldrich F4135 Replaces Invitrogen cat. no. 10082-147 used in original study
Penicillin–streptomycin solution (100x) stabilized Cell culture reagent Sigma-Aldrich P4333 Replaces Invitrogen cat. no. 15140-122 used in original study
1 mL syringe; 26 G x 5/8 needle (single-use) Labware BD Biosciences 309597 Original brand not specified
NCR-NUDE mice
(homozygous; NCRNU-F)		 Mouse line Taconic NCRNU-F
Carazzi’s Haematoxylin IHC Stain Specific brand information will be left up to the discretion of the replicating lab and recorded later
Eosin IHC Stain
Permount Mounting medium
Open in a new tab

Procedure

  1. Maintain NRASG12D cell lines in Ham's F10 medium with 10% heat inactivated FBS and 1% penicillin/streptomycin at 37°C with 5% CO2.

  2. Resuspend 1 × 106 cells in a 1:1 ratio of Matrigel and Hanks’ balanced salt solution and keep on ice. The final injection volume should be 100 µl.

  3. Subcutaneously inject 1 × 106 cells with a 26-gauge needle and insulin syringe into 6- to 8-week old female NCR-NUDE mice.

    • A. Mice were housed per IACUC regulations, in barrier housing with standard chow and 12 hr light/dark cycles.

    • B. Anesthetize mice with isofluorane prior to injection.

    • C. Inject mice subcutaneously on the flank.

  4. Monitor mice three times a week for tumor development for 16 weeks.

    • A. Record date when visible tumor is detected.

  5. Measure tumor volume once weekly.

    • A. Measure tumor in two directions with calipers. Calculate tumor volume as (length × width2)/2.

  6. Track survival of mice for 16 weeks, recording dates of euthanasia.

    • A. Sacrifice mice when tumor volume reaches 1.5 cm3 or if mice become moribund or cachectic.

    • B. Sacrifice any surviving mice at the end of the study.

  7. Upon euthanasia, harvest and process tumor tissue for further analysis.

    • A. Harvest one representative tumor per mouse group (total of 4 tumors).

    • B. Fix tissues in 10% neutral buffered formalin for 24 hr.

    • C. Dehydrate tissues through graded alcohols and clear in xylene.

    • D. Infiltrate with, and then embed, tissues in paraffin and section into 5-µm sections.

    • E. Mount sections onto positively charged slides.

  8. Stain tumor section with H&E (total: 1 stained section per tumor = 4 stained sections).

    • A. Perform H&E staining by hand using the following procedure:

      • i. Deparaffinize sections twice in xylene, then rehydrate through graded alcohols (95%, 70%, 50% ETOH) to water.

      • ii. Stain sections with Carazzi's hematoxylin, then rinse slides in water.

      • iii. Stain sections with eosin.

      • iv. Dehydrate sections through graded alcohols (50%, 70%, 90%) and then place in xylene.

      • v. Apply coverslips to slides with Permount and store slides at room temperature.

  9. Blindly image stained sections and have images blindly analyzed by a Board Certified Veterinary Pathologist to verify the tumor composition of the tissue sections.

Data to be collected

  • Deliverables:

    1. Mouse health records (age, time to tumor detection, tumor incidence, date of euthanasia, and cause of termination)

    2. Raw and calculated tumor volume measurements for each date/mouse

    3. Kaplan–Meier curves generated for tumor-free survival of each mouse line

    4. Images of H&E stained tumor sections and pathology report. (compare to Figure 6C)

    5. Pathologist's report of tissue section evaluation

Confirmatory analysis plan

This replication attempt will perform the statistical analyses listed below, compute the effects sizes, compare them against the reported effect size in the original paper, and use a meta-analytic approach to combine the original and replication effects, which will be presented as a Forest plot.

  • Statistical analysis of the Replication Data:

    1. Comparison of Kaplan–Meier survival curves tracking tumor incidence using Bonferroni’s correction for multiple comparisons.

      • The authors originally examined the Kaplan–Meier curves for PREX2 mutants and compared the endpoint values of the mutant curves to the endpoint values of the wild-type PREX2 curve using an unpaired two-tailed t-test. We will replicate their t-tests but also compare the entire survival curves (each mutant curve vs both wild-type and GFP control) using the log-rank Mantel–Cox test with Bonferroni’s alpha correction, which we believe is a more appropriate statistical approach.

    2. Comparison of tumor growth rates

      • We will measure tumor growth rates across all mouse cohorts over the length of the study. These data were collected but not reported or analyzed in the original study. We will plot growth curves for each treatment group and use area under the curve analysis to calculate the mean and std. error. We will then use the means, std. error, and n to perform a 1-way ANOVA. Further, we will perform corrected t-tests (Bonferroni correction) to perform pairwise comparisons between PREX2 mutants and either GFP or wild-type controls.

Known differences from the original study

This replication is only generating and analyzing xenografts based on the stable melanocyte lines for GFP, wild-type PREX2, PREX2 Q1430*, and PREX2 G844D. The original study also generated and analyzed tumor xenografts using other PREX2 mutant-expressing melanocyte lines, including PREX2 K278*, E824*, P948S, and G106E. In order to sufficiently power all experiments and achieve the necessary number of events for Kaplan–Meier analysis, the duration of this replication will be extended from 9 weeks in the original paper to 16 weeks in the replication. All known differences in reagents and supplies are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The genetic integrity, mycoplasma-free purity, and levels of exogenous expression of each NrasG12V melanocyte line used in this experiment have been previously validated in Protocols 1 and 2. All mice will be handled and housed in accordance with the Institutional Animal Care and Use Committee (IACUC). All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/82nfe/)

Power calculations

Protocol 3

Summary of original data
Figure 3B. Kaplan–Meier survival curves Median survival Hazards ratio [to WT] Hazards ratio [to GFP] N
WT PREX2 N/A N/A N/A 10
GFP N/A N/A N/A 10
PREX2 Q1430* 5 weeks 0.08758 0.1243 10
PREX2 G844D 5 weeks 0.1296 0.1952 10
Open in a new tab

Note: Mantel–Haenszel hazard ratios were generated in Graphpad Prism v. 6.0 following analysis of Kaplan–Meier curves with the log-rank (Mantel–Cox) test using the Mantel–Haenszel method.

Test family

  • Log-rank (Mantel–Cox) test with Bonferroni alpha correction for multiple comparisons

Power calculations
Experiment duration A Priori power Total events needed (WT or GFP) Estimated sample size (WT or GFP) Total events needed (PREX2 mutants) Estimated sample size (PREX2 mutant)
Q1430* vs WT 16 weeks ≥80% 1 7 6 7
G844D vs WT 16 weeks ≥80% 1 6 11 12
Q1430* vs GFP 16 weeks ≥80% 3 14 7 8
G844D vs GFP 16 weeks ≥80% 4 14 12 14
Open in a new tab

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Levi Garraway, Lynda Chin, and most especially Yonathan Lissanu Deribe, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We are grateful to Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology: American Type Culture Collection (ATCC), BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, and Sigma-Aldrich.

Funding Statement

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Footnotes

Berger MF, Hodis E, Heffernan TP, Deribe YL, Lawrence MS, Protopopov A, Ivanova E, Watson IR, Nickerson E, Ghosh P, Zhang H, Zeid R, Ren X, Cibulskis K, Sivachenko AY, Wagle N, Sucker A, Sougnez C, Onofrio R, Ambrogio L, Auclair D, Fennell T, Carter SL, Drier Y, Stojanov P, Singer MA, Voet D, Jing R, Saksena G, Barretina J, Ramos AH, Pugh TJ, Stransky N, Parkin M, Winckler W, Mahan S, Ardlie K, Baldwin J, Wargo J, Schadendorf D, Meyerson M, Gabriel SB, Golub TR, Wagner SN, Lander ES, Getz G, Chin L, Garraway LA. 09May2012. Melanoma genome sequencing reveals frequent PREX2 mutations. Nature 3:502–506. doi: 10.1038/nature11071.

Contributor Information

Roger Davis, University of Massachusetts Medical School, United States.

Elizabeth Iorns, Science Exchange, Palo Alto, California.

William Gunn, Mendeley, London, United Kingdom.

Fraser Tan, Science Exchange, Palo Alto, United States.

Joelle Lomax, Science Exchange, Palo Alto, United States.

Timothy Errington, Center for Open Science, Charlottesville, United States.

Funding Information

This paper was supported by the following grant:

  • Laura and John Arnold Foundation to .

Additional information

Competing interests

DC: Noble Life Sciences is a Science Exchange associated lab.

DS: BioFactura is a Science Exchange associated lab.

RP:CB: EI, FT and JL are employed by, and holds shares in, Science Exchange, Inc.

The other authors declare that no competing interests exist.

Author contributions

DC, Conception and design.

DS, Conception and design.

AH, Drafting or revising the article.

RP:CB, Conception and design, Drafting or revising the article.

References

  1. Berger MF, Hodis E, Heffernan TP, Deribe YL, Lawrence MS, Protopopov A, Ivanova E, Watson IR, Nickerson E, Ghosh P, Zhang H, Zeid R, Ren X, Cibulskis K, Sivachenko AY, Wagle N, Sucker A, Sougnez C, Onofrio R, Ambrogio L, Auclair D, Fennell T, Carter SL, Drier Y, Stojanov P, Singer MA, Voet D, Jing R, Saksena G, Barretina J, Ramos AH, Pugh TJ, Stransky N, Parkin M, Winckler W, Mahan S, Ardlie K, Baldwin J, Wargo J, Schadendorf D, Meyerson M, Gabriel SB, Golub TR, Wagner SN, Lander ES, Getz G, Chin L, Garraway LA. Melanoma genome sequencing reveals frequent PREX2 mutations. Nature. 2012;485:502–506. doi: 10.1038/nature11071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Cerami E, Gao J, Dogrusoz U, Gross BE, Sumer SO, Aksoy BA, Jacobsen A, Byrne CJ, Heuer ML, Larsson E, Antipin Y, Reva B, Goldberg AP, Sander C, Schultz N. The cBio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Discovery. 2012;2:401–404. doi: 10.1158/2159-8290.CD-12-0095. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Fine B, Hodakoski C, Koujak S, Su T, Saal LH, Maurer M, Hopkins B, Keniry M, Sulis ML, Mense S, Hibshoosh H, Parsons R. Activation of the PI3K pathway in cancer through inhibition of PTEN by exchange factor P-REX2a. Science. 2009;325:1261–1265. doi: 10.1126/science.1173569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Hodis E, Watson IR, Kryukov GV, Arold ST, Imielinski M, Theurillat JP, Nickerson E, Auclair D, Li L, Place C, Dicara D, Ramos AH, Lawrence MS, Cibulskis K, Sivachenko A, Voet D, Saksena G, Stransky N, Onofrio RC, Winckler W, Ardlie K, Wagle N, Wargo J, Chong K, Morton DL, Stemke-Hale K, Chen G, Noble M, Meyerson M, Ladbury JE, Davies MA, Gershenwald JE, Wagner SN, Hoon DS, Schadendorf D, Lander ES, Gabriel SB, Getz G, Garraway LA, Chin L. A landscape of driver mutations in melanoma. Cell. 2012;150:251–263. doi: 10.1016/j.cell.2012.06.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Krauthammer M, Kong Y, Ha BH, Evans P, Bacchiocchi A, McCusker JP, Cheng E, Davis MJ, Goh G, Choi M, Ariyan S, Narayan D, Dutton-Regester K, Capatana A, Holman EC, Bosenberg M, Sznol M, Kluger HM, Brash DE, Stern DF, Materin MA, Lo RS, Mane S, Ma S, Kidd KK, Hayward NK, Lifton RP, Schlessinger J, Boggon TJ, Halaban R. Exome sequencing identifies recurrent somatic RAC1 mutations in melanoma. Nature Genetics. 2012;44:1006–1014. doi: 10.1038/ng.2359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Kunz M. Oncogenes in melanoma: an update. European Journal of Cell Biology. 2014;93:1–10. doi: 10.1016/j.ejcb.2013.12.002. [DOI] [PubMed] [Google Scholar]
  7. Lawrence MS, Stojanov P, Polak P, Kryukov GV, Cibulskis K, Sivachenko A, Carter SL, Stewart C, Mermel CH, Roberts SA, Kiezun A, Hammerman PS, McKenna A, Drier Y, Zou L, Ramos AH, Pugh TJ, Stransky N, Helman E, Kim J, Sougnez C, Ambrogio L, Nickerson E, Shefler E, Cortes ML, Auclair D, Saksena G, Voet D, Noble M, DiCara D, Lin P, Lichtenstein L, Heiman DI, Fennell T, Imielinski M, Hernandez B, Hodis E, Baca S, Dulak AM, Lohr J, Landau DA, Wu CJ, Melendez-Zajgla J, Hidalgo-Miranda A, Koren A, McCarroll SA, Mora J, Lee RS, Crompton B, Onofrio R, Parkin M, Winckler W, Ardlie K, Gabriel SB, Roberts CW, Biegel JA, Stegmaier K, Bass AJ, Garraway LA, Meyerson M, Golub TR, Gordenin DA, Sunyaev S, Lander ES, Getz G. Mutational heterogeneity in cancer and the search for new cancer-associated genes. Nature. 2013;499:214–218. doi: 10.1038/nature12213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Marzese DM, Scolyer RA, Roque M, Vargas-Roig LM, Huynh JL, Wilmott JS, Murali R, Buckland ME, Barkhoudarian G, Thompson JF, Morton DL, Kelly DF, Hoon DS. DNA methylation and gene deletion analysis of brain metastases in melanoma patients identifies mutually exclusive molecular alterations. Neuro Oncology. 2014;16:1499–1509. doi: 10.1093/neuonc/nou107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Ni TK, Landrette SF, Bjornson RD, Bosenberg MW, Xu T. Low-copy piggyBac transposon mutagenesis in mice identifies genes driving melanoma. Proceedings of the National Academy of Sciences of USA. 2013;110:E3640–E3649. doi: 10.1073/pnas.1314435110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Rubinstein LV, Gail MH, Santner TJ. Planning the duration of a comparative clinical trial with loss to follow-up and a period of continued observation. Journal of Clinical Epidemiology. 1981;34:469–479. doi: 10.1016/0021-9681(81)90007-2. [DOI] [PubMed] [Google Scholar]
  11. Schoenfeld DA. Sample-size formula for the proportional-hazards regression model. Biometrics. 1983;39:499–503. doi: 10.2307/2531021. [DOI] [PubMed] [Google Scholar]
  12. Turajlic S, Furney SJ, Lambros MB, Mitsopoulos C, Kozarewa I, Geyer FC, Mackay A, Hakas J, Zvelebil M, Lord CJ, Ashworth A, Thomas M, Stamp G, Larkin J, Reis-Filho JS, Marais R. Whole genome sequencing of matched primary and metastatic acral melanomas. Genome Research. 2012;22:196–207. doi: 10.1101/gr.125591.111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Xia J, Jia P, Hutchinson KE, Dahlman KB, Johnson D, Sosman J, Pao W, Zhao Z. A meta-analysis of somatic mutations from next generation sequencing of 241 melanomas: a Road Map for the study of genes with Potential clinical relevance. Molecular Cancer Therapeutics. 2014;13:1918–1928. doi: 10.1158/1535-7163.MCT-13-0804. [DOI] [PMC free article] [PubMed] [Google Scholar]
eLife. 2014 Dec 10;3:e04180. doi: 10.7554/eLife.04180.002

Decision letter

Editor: Roger Davis1

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: Melanoma genome sequencing reveals frequent PREX2 mutations” for consideration at eLife. Your article has been favorably evaluated by Charles Sawyers (Senior editor) and 4 reviewers, one of whom is a member of our Board of Reviewing Editors.

The Reviewing editor and the other reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

There are two main conclusions drawn in the Nature paper by Berger et al. First, PREX2 is identified as a frequently (14%) non-synonymously mutated gene in melanoma. Second, that ectopic expression of cancer-associated PREX2 proteins promote melanoma genesis in a xenograft assay in mice. This is an important study because the identification of driver (vs passenger) mutations in cancer is critical for understanding the mechanism of tumorigenesis.

The first conclusion is addressed by Chroscinski and colleagues in the literature summary. The authors note that validation of this conclusion is not supported by several studies, including meta-analysis. However, Chroscinski and colleagues state that two papers support the conclusion that PREX2 is frequently mutated in melanoma. This is not correct. Turajlic et al. do describe a PREX2 mutation, but the analysis is limited to a single patient, while Furney et al. only refer to PREX2 mutations described by Berger et al. Thus, these papers do not support the conclusion that PREX2 is frequently mutated in melanoma. It also appears to be significant that another study of melanoma exome sequencing by many of the same authors published 2 months after Berger et al. (Hodis et al.) does not appear to identify PREX2 as a frequently mutated gene. Together, these considerations do not provide support for the conclusion presented by Berger et al. that PREX2 is frequently mutated in melanoma. This needs to be more directly addressed by Chroscinski and colleagues: they should discuss statistical thresholds for calling mutations significant (e.g., MutSig) and examine PREX2 mutations in public cancer genome portals such as cBioPortal (MSK) or the Broad Tumor Portal algorithms (in addition to reporting on the negative studies).

The second conclusion (that is addressed by re-analysis by Chroscinski and colleagues) is important: that PREX2 is a driver mutation rather than a passenger mutation during melanoma formation. The proposed experimental design appears to replicate the original study. However, a number of issues were raised by the reviewers:

1) The PREX2 mutations reported by Berger et al. are present throughout the PREX2 sequence. This is more consistent with passenger mutation than driver mutation. The restriction of the re-analysis to a limited number of mutations originally examined is therefore problematic. To document that the PREX2 mutations acts as drivers, it would be best to examine the same mutations that are reported in the original study, rather than a sub-set of these mutations.

2) There are a number of problems with the original experimental design that complicate conclusions drawn from the xenograft study. Chroscinski and colleagues should be aware that: a) the genetic status of PREX2 in the cell line that is employed is unknown; b) the relative expression of endogneous PREX2 and ectopically expressed PREX2 is unknown; and c) it has previously been reported that WT PREX2 in a different cells (MCF10A) causes PTEN inhibition, activation of AKT, and increased proliferation (Fine et al. (2009) Science 325, 1261). These deficiencies in the design of the original study will influence the ability to draw sound conclusions from the study, but will be common between the original study and the replication study.

3) Regarding the power calculations, the proposed power calculations take for granted survival and hazard numbers published in the original study. This is fine at this stage, however, we suggest the following improvements.

(a) Cross-study variation should be taken into account to determine expected loos of power computed on published numbers, pre-data collection. This is hard to estimate, but papers by Giovanni Parmigiani and collaborators at the Dana Farber provide some estimates about cross-study variation that could be used for this purpose. The authors should budget some additional variability because of cross-study reproducibility, and increase the sample size on-the-fly, as they deem appropriate as deemed appropriate.

(b) The final report on the replicated study should report the actual power of the tests, based on the standard deviations in the replicated study.

eLife. 2014 Dec 10;3:e04180. doi: 10.7554/eLife.04180.003

Author response

There are two main conclusions drawn in the Nature paper by Berger et al. First, PREX2 is identified as a frequently (14%) non-synonymously mutated gene in melanoma. Second, that ectopic expression of cancer-associated PREX2 proteins promote melanoma genesis in a xenograft assay in mice. This is an important study because the identification of driver (vs passenger) mutations in cancer is critical for understanding the mechanism of tumorigenesis.

The first conclusion is addressed by Chroscinski and colleagues in the literature summary. The authors note that validation of this conclusion is not supported by several studies, including meta-analysis. However, Chroscinski and colleagues state that two papers support the conclusion that PREX2 is frequently mutated in melanoma. This is not correct. Turajlic et al. do describe a PREX2 mutation, but the analysis is limited to a single patient, while Furney et al. only refer to PREX2 mutations described by Berger et al. Thus, these papers do not support the conclusion that PREX2 is frequently mutated in melanoma. It also appears to be significant that another study of melanoma exome sequencing by many of the same authors published 2 months after Berger et al (Hodis et al.) does not appear to identify PREX2 as a frequently mutated gene. Together, these considerations do not provide support for the conclusion presented by Berger et al. that PREX2 is frequently mutated in melanoma. This needs to be more directly addressed by Chroscinski and colleagues: they should discuss statistical thresholds for calling mutations significant (e.g., MutSig) and examine PREX2 mutations in public cancer genome portals such as cBioPortal (MSK) or the Broad Tumor Portal algorithms (in addition to reporting on the negative studies).

We thank the reviewers for these astute suggestions. We have amended the Introduction to more accurately describe the findings of Turajlic et al., and we have removed the incorrectly used reference for Furney et al. We have added a sentence to highlight the possibility of false-positive findings due to tumor heterogeneity, as described by Lawrence, et al. We have also replaced the phrase “significantly mutated” with “frequently mutated” in the Introduction.

In general, the Reproducibility Project: Cancer Biology focuses on generation of new data, not reanalysis of existing datasets. As such, while interesting, we feel it is beyond the scope of this replication study to perform data mining or analyses of PREX2 mutations in various genomic databases to determine prevalence or significance. Rather, we will instead limit our focus to replicating experiments that were performed in the original paper.

The second conclusion (that is addressed by re-analysis by Chroscinski and colleagues) is important: that PREX2 is a driver mutation rather than a passenger mutation during melanoma formation. The proposed experimental design appears to replicate the original study. However, a number of issues were raised by the reviewers:

1) The PREX2 mutations reported by Berger et al. are present throughout the PREX2 sequence. This is more consistent with passenger mutation than driver mutation. The restriction of the re-analysis to a limited number of mutations originally examined is therefore problematic. To document that the PREX2 mutations acts as drivers, it would be best to examine the same mutations that are reported in the original study, rather than a sub-set of these mutations.

We thank the reviewers for this insightful comment. However, ultimately, the goal of the Reproducibility Project: Cancer Biology is not to appraise the biological conclusions and implications imparted by the original authors, but rather to systematically assess the degree to which we can reproduce the methodology and experimental effect sizes described in the original paper. Of note, the original authors also chose to only analyze a subset (6/28) of all of the PREX2 mutations they identified (see Figure 3A).

We agree that all of the experiments included in the original study are important, and choosing which experiments to replicate has been one of the great challenges of this project. We acknowledge that the exclusion of certain experiments limits the scope of what can be analyzed about the project, but we are attempting to identify a balance of breadth of sampling for general inference with sensible investment of resources on replication projects.

Consistent with this mentality, we feel it is beyond the scope of this replication study to assess the original authors’ conclusions and interpretations regarding whether PREX2 is a passenger or driver in melanoma. We believe that replicating a subset of the original data is appropriate to allow us achieve our goal of quantitatively evaluating the ability of the originally reported results to be replicated. To avoid confusion regarding this issue, we have altered the Introduction so that it does not reference the debate over passenger versus driver mutations. We will also refrain from discussing the functional relevance of any of the mutations that we are not directly testing, including those identified but not analyzed in the original paper.

2) There are a number of problems with the original experimental design that complicate conclusions drawn from the xenograft study. Chroscinski and colleagues should be aware that: a) the genetic status of PREX2 in the cell line that is employed is unknown; b) the relative expression of endogneous PREX2 and ectopically expressed PREX2 is unknown; and c) it has previously been reported that WT PREX2 in a different cells (MCF10A) causes PTEN inhibition, activation of AKT, and increased proliferation (Fine et al. (2009) Science 325, 1261). These deficiencies in the design of the original study will influence the ability to draw sound conclusions from the study, but will be common between the original study and the replication study.

We thank the reviewers for these suggestions. We remind the reviewers that this project focuses on direct replication of the experiments as detailed in the original report and with information provided by the original authors. Aspects of an experiment not included in the original study are occasionally added to ensure the quality of the research, but by no means is a requirement of this project; rather, it is an extension of the original work. Adding additional aspects not included in the original study can be of scientific interest, and can be included if it is possible to balance them with the main aim of this project: to perform a direct replication of the original experiment(s).

As such, we agree with the reviewers that there is scientific interest in better understanding some aspects of this biological system. To address the reviewers’ concern (a) about the genetic status of endogenous PREX2, we have added an additional step to Protocol 1, whereby the endogenous PREX2 gene in NRASG12D melanocytes will be sequenced to determine its mutational status. Additionally, to address the reviewers’ concern (b) about the expression levels of endogenous PREX2, we have added an additional step to Protocol 2, whereby we will also blot for PREX2 protein in both the overexpressed PREX2 variants and the GFP vector control, comparing the levels of PREX2 expression across cell lines. However, we will consider these data exploratory and therefore not include them in our statistical analysis. As for the reviewers’ concern (c) regarding the behavior of PREX2 in different cell lines, we believe addressing this concern is beyond the scope of this particular replication study.

3) Regarding the power calculations, the proposed power calculations take for granted survival and hazard numbers published in the original study. This is fine at this stage, however, we suggest the following improvements.

(a) Cross-study variation should be taken into account to determine expected loos of power computed on published numbers, pre-data collection. This is hard to estimate, but papers by Giovanni Parmigiani and collaborators at the Dana Farber provide some estimates about cross-study variation that could be used for this purpose. The authors should budget some additional variability because of cross-study reproducibility, and increase the sample size on-the-fly, as they deem appropriate as deemed appropriate.

We thank the reviewers for these suggestions. The cross-study variation, such as approaches that utilize the 95% confidence interval of the effect size, can be useful in conducting power calculations when planning adequate sample sizes for detecting the true population effect size, which requires a range of possible observed effect sizes. However, the Reproducibility Project: Cancer Biology is designed to conduct replications that have 80% power to detect the point estimate of the originally reported effect size. While this has the limitation of being underpowered to detect smaller effects than what is originally reported, this standardizes the approach across all studies to be designed to detect the originally reported effect size with at least 80% power. Also, while the minimum power guarantee is beneficial for observing a range of possible effect sizes, the experiments in this replication, and all experiments in the project, are designed to detect the originally reported effect size with a minimum power of 80%. Thus, performing power calculations during or after data collection is not necessary in this replication attempt as all studies included are already designed to meet a minimum power or are identified beforehand as being underpowered and thus are not included in the confirmatory analysis plan. The papers by Giovanni Parmigiani and collaborators highlight the importance of accounting for variability that can occur across different studies, specifically gene expression data. While it is possible for a difference in variance between the originally reported results and the replication data, this will be reflected in the presentation of the data and a possible reason for obtaining a different effect size estimate.

(b) The final report on the replicated study should report the actual power of the tests, based on the standard deviations in the replicated study.

As described above, we do not see the value in performing post-hoc power calculations on the obtained data. However, we do agree that reporting the actual power of the tests to detect the originally reported effect size estimate based on the sample size analyzed in the replication study is important and will be reported.

Articles from eLife are provided here courtesy of eLife Sciences Publications, Ltd

RESOURCES