Protocol for a scoping review to support development of a CONSORT extension for randomised controlled trials using cohorts and routinely collected health data

Abstract

Introduction

Randomised controlled trials (RCTs) conducted using cohorts and routinely collected health data, including registries, electronic health records and administrative databases, are increasingly used in healthcare intervention research. The development of an extension of the CONsolidated Standards of Reporting Trials (CONSORT) statement for RCTs using cohorts and routinely collected health data is being undertaken with the goal of improving reporting quality by setting standards early in the process of uptake of these designs. To develop this extension to the CONSORT statement, a scoping review will be conducted to identify potential modifications or clarifications of existing reporting guideline items, as well as additional items needed for reporting RCTs using cohorts and routinely collected health data.

Methods and analysis

In separate searches, we will seek publications on methods or reporting or that describe protocols or results from RCTs using cohorts, registries, electronic health records and administrative databases. Data sources will include Medline and the Cochrane Methodology Register. For each of the four main types of RCTs using cohorts and routinely collected health data, separately, two investigators will independently review included publications to extract potential checklist items. A potential item will either modify an existing CONSORT 2010, Strengthening the Reporting of Observational Studies in Epidemiology or REporting of studies Conducted using Observational Routinely collected health Data item or will be proposed as a new item. Additionally, we will identify examples of good reporting in RCTs using cohorts and routinely collected health data.

Ethics and dissemination

The proposed scoping review will help guide the development of the CONSORT extension statement for RCTs conducted using cohorts and routinely collected health data.

Strengths and limitations of this study.

• Our scoping review will be conducted using rigorous methods, with peer-reviewed searches developed by a research librarian that will comply with Institute of Medicine standards and are not limited by language.

• Due to the novelty of randomised controlled trials using cohorts and routinely collected health data, we anticipate identifying only a limited number of methods and reporting articles in our scoping review.

• To supplement articles on methods and reporting, we will review primary trial protocols and reports to identify elements that need reporting and to identify examples of good reporting.

Introduction

Randomised controlled trials (RCTs), when well designed and conducted, are widely acknowledged to be the gold standard for evaluating the effectiveness and harms of medical interventions.^1–3 Important concerns exist, however, about many RCTs, including limitations related to difficulty recruiting sufficiently large and representative samples, limited real-world generalisability and prohibitive costs.^4–12 To attempt to address these and other challenges, trial designs have been developed in which RCTs are conducted within the frameworks cohorts⁴ and routinely collected health data. Routinely collected health data are defined as data collected for administrative and clinical purposes, without specific a priori research questions,¹³ and include registries,¹⁴ electronic health records¹⁵ and health administrative databases.¹⁶

Biomedical research reporting guidelines have been developed to assist authors to report research studies as accurately, transparently and completely as possible. Reporting guidelines typically describe a minimum set of information that should be clearly reported, provide examples of guideline-consistent reporting and include a checklist to facilitate compliance.^{17 18} Multiple existing reporting guidelines include items that are potentially applicable to RCTs conducted using cohorts and routinely collected health data. In addition to the CONsolidated Standards of Reporting Trials (CONSORT) statement for reporting of parallel group RCTs,¹⁹ reporting guidelines with the most direct overlap include the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for the reporting of observational studies, generally,²⁰ and the REporting of studies Conducted using Observational Routinely collected Data (RECORD) guideline²¹ which address reporting specific to observational studies conducted using routinely collected health data.

The development of an extension of the CONSORT statement for RCTs conducted using cohorts and routinely collected health data is being undertaken with the goal of improving long-term reporting quality by setting standards early in the process of uptake of these trial designs.²² To develop this CONSORT extension, information is needed to understand which items from CONSORT, STROBE and RECORD can be used without modification and which should be included with adaptations, as well as aspects of reporting of RCTs conducted using cohorts and routinely collected health data that are not covered adequately in these reporting guidelines and that require new reporting items. In addition, examples of complete and transparent reporting of different aspects of these RCTs are needed.

Relatively little guidance has been published on the methods and reporting of RCTs conducted using cohorts and routinely collected health data. To account for this, the proposed scoping review will identify articles on the methods or reporting of RCTs conducted using cohorts, registries, electronic health records and health administrative databases, as well as examples of protocols and reports of results from these types of RCTs. The objectives of the scoping review are to (1) determine which items from an initial long list of items based on CONSORT, STROBE and RECORD that are being considered for possible inclusion in the CONSORT extension can be included without modification, identify items from the initial list that need adaptation and identify additional reporting considerations to develop new items; and (2) identify examples of complete and transparent reporting of different aspects of these types of RCTs that can be used to support the CONSORT extension.

Methods

The scoping review will be conducted following the approach described by Arksey and O’Malley²³ and will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses: extension for Scoping Reviews (PRISMA-ScR) guidelines.²⁴

Database searches

In separate searches, we will seek publications that describe aspects of methods or reporting or that describe protocols or results from RCTs (including cluster RCTs) using (1) cohorts, (2) registries, (3) electronic health records and (4) health administrative databases. Ovid MEDLINE Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Ovid MEDLINE Daily and Ovid MEDLINE and EBM Reviews—Cochrane Methodology Registry (Final issue, third Quarter 2012) will be searched by an experienced librarian familiar with knowledge synthesis for publications on methods or reporting of these types of RCTs and for examples of these types of RCTs. MEDLINE strategies for the searches were developed by a research librarian with input from the project team and were peer reviewed using the Peer Review of the Electronic Search Strategy standard.²⁵ The MEDLINE strategy was then adapted for the Cochrane Library Methodology Register which includes methodological research available up to its last update in July 2012.

Search strategies comply with Institute of Medicine standards and are not limited by language.²⁶ We will search for articles on methods and reporting and examples of RCTs published in the last 10 years (2008–2018) which will allow us to identify relatively recent reporting practices and focus on challenging aspects of reporting. See online supplementary file 1 for detailed search strategies. In addition to the database searches, references of included studies will be reviewed for additional eligible studies, a web search will be conducted and members of the project team with experience in each type of trial will be consulted to provide additional studies that were not identified in our search.

Study selection

For each search, separately, results will be downloaded into the citation management database RefWorks, and duplicate references will be removed. Following this, references will be transferred into the systematic review software DistillerSR (Evidence Partners, Ottawa, Canada). A coding manual based on eligibility criteria has been developed, and a pilot test of the coding manual will be performed prior to the study’s inception. The initial coding manuals for inclusion and exclusion for all four types of trial designs are shown in online supplementary file 2. Because the trial designs that will be included in the CONSORT extension reflect relatively recent developments, we anticipate that we will identify only a small number of articles on their methodology and reporting. Thus, we will also include publications of trial protocols and results.

We will assess the eligibility of each publication through a two-stage process. In the first stage, two reviewers will independently screen titles and abstracts to identify potentially relevant studies. We will use a liberal accelerated method²⁷ to screen titles and abstracts, meaning that articles deemed eligible by one of the reviewers will be included in full-text review, and only excluded articles will be screened by a second reviewer. Since title and abstract screening is done randomly and concurrently, reviewers will not know if the other reviewer has excluded the reference or not. In the second stage, two investigators will independently conduct a full-text review. Disagreements after full-text review will be resolved by consensus, with a third investigator consulted as necessary. Translators will be consulted to evaluate titles and abstracts and full-text articles for languages other than those for which team members are fluent, if any. See online supplementary file 3 for the preliminary PRISMA flow of studies figures for the four types of trial designs.

Data extraction and verification

To develop a preliminary ‘long list’ of items to consider for the CONSORT extension checklist, as an initial step, items from the CONSORT 2010 will be examined to identify items where modifications will be needed for RCTs conducted using cohorts and routinely collected health data, and items from the STROBE and RECORD reporting guidelines will be examined to identify additional items to complement CONSORT items. Two investigators will independently review these reporting guidelines, and any item deemed possibly relevant to RCTs using cohorts and routinely collected health data by either or both investigators will be included in the ‘long list’. Additional preliminary ‘long-list’ items will be provided by other members of the project team.

For each of the four types of RCTs conducted using cohorts and routinely collected health data, separately, two investigators will independently review included publications to extract additional potential items for the ‘long list’. A potential item will either modify an existing CONSORT 2010, STROBE or RECORD item that has been included in the ‘long list’ or will be proposed as a new item. Potential items will be identified from publications that report information relevant to conducting RCTs using cohorts and routinely collected health data, but that were not included in our initial ‘long list’. In addition, potential items will be suggested based on gaps in reporting identified from primary trial protocols or reports. Data will be extracted and collected in DistillerSR using a standardised data-extraction form. The long list of items will evolve dynamically as potential modifications and new items are added based on the review of publications identified from our literature search using the DistillerSR Dynamic Question function. Thus, reviewers will add a potential item only once to the long list, after which it becomes visible for all reviewers. Reviewers will not duplicate items already provided by other reviewers. This will be done to avoid redundancy, as we expect potential gaps in reporting to occur in multiple publications that will be reviewed. In addition to each proposed item modification or new item, reviewers will add a brief explanation of why the suggested modification or new item is deemed important.

In addition to identifying gaps in reporting, for each item on our long list, we will attempt to identify examples of complete and transparent reporting in RCTs using cohorts, registries, electronic health record and health administrative databases. When examples of complete and transparent reporting for a particular item on the long list are identified, text corresponding to reporting of that item will be inserted in the data-extraction form in DistillerSR.

Prior to data extraction from included studies, all reviewers will assess a sample of trial reports. The results will be compared and discussed among the reviewers in order to ensure consistent application of the data-extraction process.

Patient and public involvement

One of the members of our extension to the CONSORT statement, Maureen Sauvé, is a patient organisation leader. She has been involved in working with researchers to establish a cohort of patients living with the rare disease, scleroderma, which supports RCTs of trials of online rehabilitation, self-management and psychological intervention programmes.²⁸

Conclusion

This scoping review will gather previously published methods and recommendations for the reporting of RCTs using cohorts and routinely collected health data, as well as identify gaps in reporting of these studies. We will identify potential modifications or clarifications of CONSORT 2010, STROBE and RECORD items, as well as potential additional items to develop an extension to the CONSORT statement for reporting RCTs using cohorts and routinely collected health data. Following the scoping review, identified items will be vetted using a three-stage Delphi approach²⁹ and a face-to-face meeting, after which the reporting checklist and explanation and elaboration documents for the CONSORT extension will be finalised. The resulting CONSORT extension will promote transparency, clarity, reduce research waste and provide guidance to researchers on appropriate and consistent reporting of RCTs using cohorts and routinely collected health data.

Ethics and dissemination

The proposed scoping review will help guide the development of the CONSORT extension statement for RCTs conducted using cohorts and routinely collected health data. The findings will be disseminated through peer-reviewed publications and conference presentations.