Response to correspondence on “Reproducibility of CRISPR-Cas9 methods for generation of conditional mouse alleles: a multi-center evaluation”

We would like to thank the Yang et al. (2013) authors for their comments and debate on optimal methods for mouse transgenesis. The two Jaenisch laboratory studies published in Cell in 2013 were ground-breaking, demonstrating for the first time proof of principle CRISPR mediated gene editing in the mouse zygote to generate knockout and conditional alleles, and caused much excitement in the transgenic mouse community.

However, over several years and in many laboratories, the reality did not match the excitement when it came to generating conditional alleles in a single step. While it is true that the 2-guides 2-oligo approach can work in certain circumstances (members of our own consortium reported some success with this method), the efficiencies reported in Yang et al. (2013) do not bear out across multiple gene targets. Indeed, as the comments from Yang et al. point out, they themselves have performed further reproducibility experiments on the Mecp2 locus (their point #1) and these unpublished results fail to reproduce the 16% efficiency from their original publication. And our study is not the first time concerns have been raised as to the efficiency of the 2-guides 2-oligo method, with anecdotal reports from others in the transgenic community (Science; 2016. doi:10.1126/science.aal0334 [doi.org]), which stated that “What was disappointing is none of us could reproduce at the efficiencies reported by Jaenisch. … It works at 1% or 2% at JAX and a lot of projects are failing. It’s really not proven to be a robust method.” And the Yang et al. group’s response was that “The paper reported what we found,” Jaenisch says. “Now, we see there are issues”.

In regard to their point #2, regrettably, details of concentrations of reagents used were not reported in Yang et al. (2013). The authors, in this correspondence, now state that they had provided concentrations in their other report [ref #4], but this reference (#4) does not describe generation of conditional alleles, and therefore, the experimental conditions of this paper (ref #4) cannot be extrapolated for generating the conditional alleles. Also, because such critical details were unavailable in the Yang et al. paper (ref #1), some of us had contacted the authors asking for tips on how to get their method to work, but we received no response. This oversight of the authors (in failing to describe the concentrations in the original paper) indeed allowed us to assess efficiencies using a range of conditions on many loci, both lower and greater than the now revealed conditions from Yang et al. (2013), and using different delivery platforms (microinjection, electroporation), the results of which further confirm that the Yang et al. method is not efficient as it was originally reported. We suggest the reader to refer to the extensive data in our additional file 1 (supplementary data file) where we show that the wide range of reagent concentrations does not affect the efficiency of the Yang et al. method. Furthermore, Hatada’s group (Horii et al.) attempted to reproduce the Mecp2 experiments, and they reported either very low efficiency or very high toxicity when the concentration of reagents was in the higher range. See Table 1 in Horii et al.; the concentration of 50/12/100 produced only 2% efficiency whereas the concentration 100/24/200 led to the death of nearly 90% of embryos, and the authors were unable to determine the method’s efficiency at this higher concentration.

In regard to their point #3, the authors speculate that Piezo-driven zygote injection may contribute to the difference of success rates. It would be necessary to examine this speculation by comparing the efficiencies of Piezo-driven and pronuclear injection methods side-by-side for a few loci. Because efficiencies at different genomic loci often vary highly (which the Yang et al. authors state in their paragraph below point #3), it would be ideal to gather such side-by-side data for at least 6 to 10 loci or more to ensure reproducibility. Otherwise, the assumption remains speculative.

Further, the authors in the paragraph below their point #3 suggest that their original method may not be efficient on other loci by stating that “it would be premature for scientific community to assume that their method would work on other loci” indicating that their study was too underpowered for routine use in core facilities. We discussed this specific point (underpower) in our paper: we suggest the reader to refer to the discussion section of our paper from the sentence that reads ‘While many published methods are reproducible (as evidenced by their wide usage), the research community often encounters issues in reproducing some published methods.’

Lastly, our observations call into question the robustness of the approach and its suitability for widespread use. Additionally, we evaluated alternative methods in parallel to report improved efficiencies across several gene targets using one-donor methods. It is vital we hold published methodologies to the highest possible standards, especially in the field of mouse transgenesis, where widespread adoption of low efficiency genetic manipulation strategies can have ethical consequences on the number of animals used in research. Science in general currently has a reproducibility crisis (https://www.nature.com/collections/prbfkwmwvz [nature.com]), and it is our responsibility as scientists that published methods are robustly tested and that the results from higher-powered analyses, which can at times be contradictory, are themselves published.