Great science inspires us to tackle the issue of data reproducibility

This special MBoC edition celebrates the American Society for Cell Biology’s 2015 award winners by featuring essays describing their inspiring scientific journeys and sharing their impressive wisdom. These articles remind us that, while much attention has recently been focused on concerns about research practices and data reproducibility, we live in an era of unprecedented achievements in biomedical research discovery, such as immune checkpoint therapy for cancer, which arose directly out of basic research (Sharma and Allison, 2015).

David G. Drubin

Editor-in-Chief

As the successful careers of these ASCB awardees serve to remind us, it is important that all scientists promote research practices and standards that result in high-quality, reproducible research. If the scientific community cannot convince the public that we have control of this issue, we risk reduced funding and imposition of guidelines developed and enforced by government legislators.

Because peer-reviewed publications are both the product of research and the vehicles for communicating scientific discoveries, journals have a critical role to play, promoting the practices that make research reproducible. It is appropriate that scientist-run journals like MBoC take a lead in this effort. MBoC has always embraced a “back-to-basics” approach to promote research integrity. In other words, major innovations are not needed to promote reproducibility, just an emphasis on sound fundamentals.

Reproducibility must begin with those individuals performing the initial study. First and foremost, it is vital that investigators perform adequate independent replicates and report: 1) how many independent replicates were performed and 2) the variance in the results. Appropriate statistical practices are important but are not a panacea. This is because there is a difference between precision and accuracy. An experimental setup with a systematic design flaw can produce data that are precise but inaccurate. Second, it is vital that the variables in each procedure be altered systematically to determine which parameters are critical for making results reproducible. Having done these two things, it is next essential that the study be communicated in sufficient detail to allow others to reproduce the key findings. Finally, one of the best ways to insure that a result is correct is to get the same answer using at least two independent approaches. In their classic study, Jamieson and Palade (1967) used both cell fractionation and radioautography to discover the intracellular trafficking route.

Journals are part of this reproducibility issue in another way. Many scientists feel that it is necessary to publish in high-profile journals to secure funding, employment, and career advancement. Attempting to publish in such journals, however, may make authors feel the need to oversimplify their results and omit inconvenient data, both of which compromise the integrity of the reported results. Moreover, higher-profile journals tend to have severe constraints on article length, compounding these problems. Vital to making research reproducible is being able to report ALL of the nuances of the experimental procedures and results, including “inconvenient facts” that might not fit perfectly with the major findings of a study. One back-to-basics solution to this problem is to publish research articles in professional society journals like MBoC that are concerned only with results being new and true and not with their pop­ular appeal or flashiness. MBoC and some other professional society journals do not have artificial limits on the number of figures or the length of the text. Such limits impair the ability to provide sufficient detail so others can reproduce published work.

A promising way to address reproducibility issues is through development of field-specific, community standards. Achieving reproducibility can be challenging, because scientific research is difficult and protocols are often complex (Aschwanden and King, 2015). However, this is not an excuse for publication of results that cannot be reproduced. Rather, it is an important acknowledgment of the reality that there are a lot of variables in performing experiments and in collecting and analyzing data. Seemingly insignificant changes in execution or analysis can have profound impacts on results. Because the ways in which complex phenomena are observed, classified, and reported are often research area specific, one-size-fits-all solutions for the reproducibility issue are unattainable. The autophagy field has developed its own standards by consensus (Klionsky et al., 2012), providing a powerful example for others to follow.

Many of these approaches to the data-reproducibility problem are discussed in a report by the ASCB Data Reproducibility Task Force (American Society for Cell Biology, 2015). In the near future, MBoC will be developing strategies to implement the task force’s recommendations in ways that do not place excessive administrative burdens on authors.

In closing, I offer congratulations to the 2015 ASCB award winners! You inspire us all with your creativity and passion, exemplify sound science practices, and remind us that great scientific achievements result when all of these elements are combined.