All together now: how and why scientific communities should develop best practice guidelines

Reproducibility in science—or the lack thereof—has garnered much attention in the past few years. On March 1, 2016, Molecular Biology of the Cell (MBoC) published a Perspective by Daniel Klionsky outlining the importance of scientific communities coming together to develop guidelines that could serve as “best practices” for their field (Klionsky, 2016). This Perspective was invited as a direct follow-up to one of several recommendations made by the American Society for Cell Biology (ASCB) Data Reproducibility Task Force in their white paper on enhancing rigor in basic research (American Society for Cell Biology, 2015). Here we highlight some of the benefits of community-based standards and describe how MBoC and ASCB can assist other scientific communities in developing best practices for their specific fields.

Klionsky detailed his experience within the autophagy community in developing a set of guidelines for the field and provided a step-by-step “how-to” manual so other fields might undertake this process and develop similar resources (Klionsky, 2016). Notably, whereas some scientific best practices, such as the importance of independent replicates and use of valid statistics, are likely to be broadly applicable, “one-size-fits-all” solutions for promoting reproducibility are not possible. This is because each research field has a unique biology, methodology, and terminology. In his Perspective, Klionsky describes several benefits for the scientific community that we reiterate here.

Developing a set of standards and guidelines within a specific scientific area could serve as a tremendous resource for the community at large, providing a set of best practices for established investigators, as well as for trainees and new laboratories coming into the field. The process itself, as Klionsky describes for the autophagy field, is also a great way to build a sense of community and possibly even bring groups together that have become divided by long-standing disagreements over results or methodologies. Although the end result may not always be 100% agreement, the process of coming up with standards at least leads to open discussions regarding the differences, and some consensus may be reached on advantages or limitations of particular assays or methodologies. As with science itself, such guidelines and standards within a scientific community are also intended to be dynamic and evolve as science progresses and new technologies are developed, and thus those in the field will need to continue their discussions and revise their guidelines over time. The autophagy community, for instance, has just published the third version of its guidelines (Klionsky et al., 2016).

Another potential benefit for developing community-based standards is to help scientists in that field avoid common mistakes and possible misinterpretation of data, resulting, it can be hoped, in improved reproducibility of scientific results. Community guidelines can lead to “gold standards,” and in the case of autophagy, community-based standards have been frequently cited by both authors and reviewers during the review of manuscripts. The scientific process often involves hypotheses and results going through periods of self-correction; however, scientific communities should make a strong commitment to minimizing common mistakes and, potentially, irreproducible results. In efforts to advocate for predictable and sustainable federal funding for science, it is not to our benefit if headlines keep highlighting that many scientific studies are not reproducible. Attempts to develop community-based standards in more subspecialties within cell biology could help demonstrate that the scientific community is taking tangible steps to address the reproducibility issue.

As pointed out in a previous editorial, MBoC has always promoted a “back-to-basics” approach in scientific training and in promoting research integrity and sound science (Drubin, 2015). The ASCB Data Reproducibility Task Force also had several discussions on training, including aspects such as good record-keeping in the lab, statistics training, training in fundamentals such as what constitutes a good control, the perils associated with the “beautification” of research data, and the importance of presenting raw data at lab meetings and sharing them with mentors. These are just a few examples, but they highlight the need for a reboot in how we train our students and postdocs. Klionsky’s experience with the autophagy community and also as Editor-In-Chief of the journal Autophagy led him to note that, although one might think that certain practices are or should be obvious, this is often not the case (Klionsky, 2016).

It is easy for us to say that more scientific communities should develop community-based standards, but we acknowledge that it can often take a herculean effort to develop the standards. In addition, we realize that such initiatives cannot be mandated by journals, scientific societies, or federal funding agencies but instead must be grassroots efforts, initiated from within—and developed by—the scientific communities themselves. It also takes a willing leader, or small group of leaders, in the field to step up and drive the effort. The ASCB and MBoC would like to help promote the development of more community-based standards. We can help facilitate workshop(s) and can provide a discussion forum at ASCB annual meetings. We can also help promote and broadly disseminate standards by publishing them in MBoC. As part of our continuing efforts to address data reproducibility issues in the broad cell biology community, ASCB and MBoC are here to assist.