The RECORD reporting guidelines: meeting the methodological and ethical demands of transparency in research using routinely-collected health data

Abstract

Routinely-collected health data (RCD) are now used for a wide range of studies, including observational studies, comparative effectiveness research, diagnostics, studies of adverse effects, and predictive analytics. At the same time, limitations inherent in using data collected without specific a priori research questions are increasingly recognized. There is also a growing awareness of the suboptimal quality of reports presenting research based on RCD. This has created a perfect storm of increased interest and use of RCD for research, together with inadequate reporting of the strengths and weaknesses of these data resources. The REporting of studies Conducted using Observational Routinely-collected Data (RECORD) statement was developed to address these limitations and to help researchers using RCD to meet their ethical obligations of complete and accurate reporting, as well as improve the utility of research conducted using RCD. The RECORD statement has been endorsed by more than 15 journals, including Clinical Epidemiology. This journal now recommends that authors submit the RECORD checklist together with any manuscript reporting on research using RCD.

Introduction

Information stored in repositories of routinely-collected health data (RCD) – such as health administrative datasets¹ – is increasingly regarded as a potential data source for clinical epidemiological research. The reasons are threefold: 1) data collection platforms are increasingly available for a wide range of data types; 2) with a greater number of sources, the volume of data is concomitantly growing, leading to greater breadth and depth of available data; and 3) primary data collection is increasingly costly, making secondary data analyses potentially cost-effective.

A number of funding agencies, such as the Canadian Institutes of Health Research, have actively endorsed the use of RCD for research, specifically for enhancing patient-oriented research and improving health care effectiveness, safety, and delivery.² Given these potential benefits, RCD are now used for a wide range of studies, including observational studies, comparative effectiveness research, diagnostics, studies of adverse effects, and predictive analytics.^3,4

At the same time, limitations inherent in using data collected without specific a priori research questions are increasingly recognized.^5,6 Concerns have been raised about data errors,⁷ missing data,^8,9 uncontrolled confounding,^10,11 data that are out of date, and data dredging.¹² Moreover, the potential linkage between datasets creates “myriad opportuni ties for the introduction of errors and omissions.”³ The potential for bias is amplified when linkage methods are inaccurate or incomplete,⁴ introducing errors that could have substantial consequences.³ There is also a growing awareness of the suboptimal quality of reports presenting research based on RCD.^13–15 This has created a “perfect storm” of increased interest and use of RCD for research, together with inadequate reporting of the strengths and weaknesses of these data resources.

Multifaceted benefits of improved reporting

Improving the reporting of studies using RCD not only facilitates comprehension and evaluation among readers but also allows replication of studies.¹⁶ Adequate documentation thus is a core standard of reporting.¹⁷ It is also a central ethical principle of clinical research. For example, the Declaration of Helsinki states that researchers have a duty to make the results of their research available and to do so in accordance with accepted guidelines for ethical reporting.¹⁸ This accords with respect for fairness and reciprocity: researchers draw on and should contribute to the accumulating pool of scientific knowledge.¹⁸

With research using RCD, transparent and accurate reporting may have the additional benefit of reducing research waste. Currently, research oversight is lacking for many sources of RCD. While data custodians may require approval procedures, there is no associated review of study questions or research methods to ensure efficient data use and prevent unnecessary duplication of analyses.^19,20

Therefore, accurate and complete reporting is needed to evaluate the clinical validity and utility of findings^4,8 and reduce duplication of effort. Adequate reporting of research also benefits by providing an external indicator that researchers are honest and trustworthy. Reporting guidelines provide a standard – a set of de facto professional norms – against which research can be judged. Investigators who are compliant with reporting guidelines fulfill their “social licence”²¹ to conduct research, demonstrating to the research community and broader public that they are satisfying its ethical requirements.

RECORD: meeting the research and ethics mandate for studies using RCD

This line of reasoning of course raises a key question: what standards should researchers and publishers uphold concerning RCD? Previously, the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement²² served as the standard, given that most research using RCD is observational. However, it has been acknowledged that the use of RCD raises additional issues not covered by STROBE, including description of the databases and validation of diagnostic codes.^23,24 The REporting of studies Conducted using Observational Routinely-collected Data (RECORD) statement²⁵ was developed to address these limitations and help researchers using RCD to meet their ethical obligations.

RECORD consists of a checklist of 13 items that supplement or modify existing STROBE items concerning an article’s title, abstract, introduction, methods, results, and discussion sections, as well as other information required in research reports. The recommendations reflect three broad areas of concern: identification of studies using RCD; evaluation of important methodological components; and information regarding access to and limits imposed on the data. Identifying research as using RCD is also important given the present lack of Medical Subject Heading terms with which to search for these types of studies. Identifying studies is a prerequisite to critiquing and building upon them.

The methods used to develop the RECORD guidelines were published in May 2015²⁶ and the full guideline appeared in October 2015.²⁵ Since then, both reports have been embraced by the scientific community. The RECORD statement has been endorsed by more than 15 journals, including Clinical Epidemiology. This journal now recommends that authors submit the RECORD checklist together with any manuscript reporting on research using RCD.

We consider this progress as just a beginning. We actively encourage ongoing discussion of the RECORD document from interested parties. An open discussion forum has been created within the RECORD statement website (http://www.record-statement.org/forum/)²⁷ as a place for interested individuals and groups to provide comments. We anticipate that this will lead to thoughtful contributions and possible future revisions of the checklist. As such, RECORD represents a living document that can be adapted to reflect changes in the field.

We will also continue to monitor the impact of the RECORD document on the field of clinical epidemiology. Studies of existing reporting guidelines suggest that endorsement by a journal leads to improved adherence among studies published within the journal. Moreover, use of checklists has been linked to improvements in completeness of research reports^28–32 and the quality of published articles.³³ Demonstrating the effectiveness of reporting guidelines is an important step in providing evidence of the benefits of RCD-based research to funding agencies as well as to the broader public and helping researchers justify their work in an era of increasing financial constraints.