Assessing and Improving Research Quality in Core Facilities

ABSTRACT

Core facilities allow scientists to perform experiments needing specialized technologies in a time- and cost-efficient way. They became increasingly important and now produce a significant amount of research data. Experiments carried out in core facilities are typically shared between the facility staff and the users. However, sharing experiments brings additional challenges to ensure data rigor and reproducibility—for example, in communication, trust, and accountability. We present here an interactive website developed especially for core facilities that offers tools to help them assess, improve, and ensure research quality.

INTRODUCTION

Core facilities are shared resource laboratories that provide access to specialized instrumentation, advanced technologies, and services to scientific investigators. They play an important role in life science as they generate a substantial part of the research data.^1,2 Being specialists in their field, the facility staff can provide useful training and mentoring to their users and help support scientific rigor and reproducibility. In addition, by being independent units, core facilities can offer protection against experimental bias.²

Core facilities have different operating modes, depending on who performs the experiment. In full-service facilities, the experiments are performed exclusively by the staff, whereas self-service facilities provide specialized equipment to their users who perform the experiment themselves. Any intermediate combination is possible in hybrid-service facilities where the staff and users share the experimental procedure sequentially or conjointly.

However, the fact that experiments are shared between staff and users can make the production of quality data more difficult, as shown in the survey on research quality that we conducted at 253 core facilities from different fields of life sciences.³ The survey revealed that communication, management, and responsibility sharing are particularly challenging aspects that can negatively impact on data rigor and reproducibility when experiments are shared.

It is not always obvious how to deal with these issues in a context in which the user turnover is high and the experimental requirements are constantly changing. Some core facilities have implemented different quality systems to help them ensure rigor and reproducibility (for example, ISO9001, PREMIER (Predictiveness and Robustness through Modular Improvement of Experimental Research), or EQIPD (Enhancing Quality in Preclinical Data)).⁴ However, these quality systems require time and resources to implement or even additional qualified staff for their management, which core facilities often cannot afford. In addition, these quality systems usually do not consider the specific needs of core facilities, nor their particular operating modes, which can make their implementation difficult and less effective.

We initiated the Q-CoFa project (Quality at the interface between Core Facilities and their users) to analyze and understand the interactions between core facilities and their users and see how these play a role in the quality of research data. Using the results of our survey, including the free-text field responses, as well as the current norms and literature, we compiled recommendations for the best practices, offering practical solutions that are pertinent for core facilities in academic settings. The recommendations are freely available on the website www.quality-in-core-facilities.org, which is mainly intended as a tool for core facility leaders, but the information will also be useful for core facility staff members and users. Our vision is to create a site providing advice “for core facilities by core facilities” that will engage the whole core facility community in the process of self-improvement and thus benefit from the hands-on experiences from various core facilities settings. Therefore, one part of the website uses MediaWiki, which allows editing of the recommendations and contribution with new information. In the following paragraphs, we will introduce the website and briefly describe the information available.

THE Q-COFA WEBSITE

After an introductory page, the main menu presents 20 different aspects affecting research quality, with each aspect having subpages for self-testing, recommendations, or downloads (Figure 1).  In the main menu, the visitor has the following options: choose a specific aspect and test how he/she performs in that respect (Figure 2A), read the relevant recommendations (Figure 2B), or download concrete templates to help implement it (Figure 2C). Additionally, when one is not sure where to start, the site provides a general test to identify which aspects might need to be improved first (Figure 2D).

Figure 1.

Structure of the Q-CoFa (Quality in Core Facilities) website. An introductory home page leads to current norms, literature, and the main menu, presenting all aspects influencing research quality. For each of these aspects, three further subpages are available: test-yourself, recommendations, and download.

Figure 2.

Four choices of what can be done from the main menu of the Q-CoFa website. (A) Test yourself on a particular aspect of research quality. (B) Read recommendations on how to improve research quality in a certain area. (C) Download templates to help implement a certain aspect. (D) Take a general test to identify which aspects to improve.

Benchmarking

To help the visitors embark on the improvement journey in an efficient way, we provide a general assessment test with 54 questions covering all the aspects of research quality shown in Figure 1. The test result shows one’s performance for each aspect separately and allows a core facility to identify the aspects to consider for improvement.

In addition to the general test, shorter tests specific for each aspect (with 4 to 7 questions each) allow a more detailed assessment of one’s strengths and weaknesses in a selected area and compares one’s own performance to the average results of the previous visitors. All responses are stored anonymously in a database and, together with all the previous tests, used to calculate the average results presented to the next visitor. Like this, an entertaining benchmarking is possible and might encourage the visitor to improve certain aspects in their facility.

Recommendations for how to improve research quality

Once a process to improve has been identified, the detailed recommendations accessible via the main menu provide advice to help the core facilities improve it. They are written in a simple and informal way and make use of direct quotes of core facility leaders from our survey.³ The recommendations are practice oriented and consider the additional difficulties raised by sharing the experimental procedure between the user and the facility staff. Although applicable to all types of core facilities in general, individual recommendations can be adapted to the specific needs of the core facility if necessary. Importantly, we would like to encourage the core facility community to expand the information and contribute with their own experience using the MediaWiki interface (editing is possible after registration).

To illustrate the information available, the “Communication” section, for example, gives ideas for how to structure the first meeting with the user, reminds of the importance of involving the user and the PI (Principal Investigator) from the very beginning, and speaks about communication soft skills and communication management. The “Interaction with users” section uses quotes from core facility leaders to exemplify the problems such as noncooperative users and provides many practical and original solutions on how to deal with them. The “Management” section discusses and compares different management software to reduce the time the core facility staff needs to spend on these issues. Last, we would like to point out the “Quality checkpoint” section that discusses the importance of supervision and checking in order to ensure research quality. We recommend introducing certain checkpoints to maximize data quality while keeping a balance between quality, cost, and manageable procedure. Several time- and cost-effective checkpoints should be placed at strategic steps spanning the whole experimental procedure: 1) Experimental design, 2) Sample quality, 3) Data analysis, and 4) Publication. Introducing these checkpoints does not mean that core facilities should have the entire control over the experiment but rather serve to initiate a discourse and awareness about data rigor at all experimental steps between the user, PI, and core facility.³

Downloads pages

Each area also has an upload/download space where the core facility community can share and exchange useful files, which are modifiable for customization. For example, a ready-to-use template of a feedback form can be found there, as well as a generic data management form, which comes with explanations encouraging further reflection on the procedure that the facility wants to adopt together with the user to guarantee traceable data. Documentation guidelines help to ensure that all aspects of the experimental documentation are complete. Furthermore, a responsibility table can assist and summarize the discussion between the facility staff, the user, and PI when they define how the tasks’ responsibility will be shared between them. Again, we invite the core facility community to share their own templates by uploading them in the relevant section.

In summary, the Q-CoFa website aims to help core facilities self-improve research quality, time efficiency, and customer service. It helps critically review one’s own facility, compare oneself with others, prepare for an audit, or establish a new core facility.

OUTLOOK

Core facilities provide valuable service and expertise to the scientific community. While training their users, they can play a very important role in teaching and spreading good research practices. In addition, they can provide independent quality controls along the experimental process. The Q-CoFa website offers a framework to core facilities on how to self-assess and improve research quality, considering the challenges posed by the close interaction between facility staff and users throughout the shared experimental process. We believe it would be helpful to introduce an incentive to motivate and convince users to follow optimal research practices and procedures. We imagine that core facilities, certified by an independent organization, could reward users having carried out their experiments to the highest standards with a “quality label” in the future. To support this effort, we integrated the recommendations from the Q-CoFa project into the EQIPD quality system and set up a specific framework called “EQIPD NEED for core facilities” (https://eqipd-toolbox.paasp.net/wiki/NEED). This provides a basis for the development of a quality label specific for core facilities. Such a label will be valuable when recognized and trusted by publishers and funding agencies. We hope our Q-CoFa website will encourage the core facilities community to improve further research quality in life sciences.