Reporting and design of randomized controlled trials for COVID-19: A systematic review

Abstract

Background

The novel coronavirus 2019 (COVID-19) pandemic has mobilized global research at an unprecedented scale. While challenges associated with the COVID-19 trial landscape have been discussed previously, no comprehensive reviews have been conducted to assess the reporting, design, and data sharing practices of randomized controlled trials (RCTs).

Purpose

The purpose of this review was to gain insight into the current landscape of reporting, methodological design, and data sharing practices for COVID-19 RCTs.

Data sources

We conducted three searches to identify registered clinical trials, peer-reviewed publications, and pre-print publications.

Study selection

After screening eight major trial registries and 7844 records, we identified 178 registered trials and 38 publications describing 35 trials, including 25 peer-reviewed publications and 13 pre-prints.

Data extraction

Trial ID, registry, location, population, intervention, control, study design, recruitment target, actual recruitment, outcomes, data sharing statement, and time of data sharing were extracted.

Data synthesis

Of 178 registered trials, 112 (62.92%) were in hospital settings, median planned recruitment was 100 participants (IQR: 60, 168), and the majority (n = 166, 93.26%) did not report results in their respective registries. Of 35 published trials, 31 (88.57%) were in hospital settings, median actual recruitment was 86 participants (IQR: 55.5, 218), 10 (28.57%) did not reach recruitment targets, and 27 trials (77.14%) reported plans to share data.

Conclusions

The findings of our study highlight limitations in the design and reporting practices of COVID-19 RCTs and provide guidance towards more efficient reporting of trial results, greater diversity in patient settings, and more robust data sharing.

1. Introduction

The novel coronavirus 2019 (COVID-19) pandemic has mobilized global research at an unprecedented scale. Indeed, billions of dollars in funding have been invested in clinical trial research to facilitate the rapid evaluation of potential therapies and vaccines [1,2]. By July 2020 over 1700 COVID-19 studies had been listed in international clinical trial registries [3].

Despite the sheer volume of ongoing research, the fight against this pandemic has been largely inefficient [[4], [5], [6], [7]]. Few effective treatments have been identified [8]. The use of non-peer reviewed pre-print publishing has also rapidly expanded [9]. Yet, while challenges associated with trial feasibility in the context of the COVID-19 pandemic have been discussed previously [10], no comprehensive evidence reviews have been conducted to assess the reporting, design, and data sharing practices of randomized controlled trials.

The purpose of this study was to evaluate the emerging randomized controlled trial (RCT) COVID-19 evidence with respect to the ability to rapidly disseminate findings, methodological designs, and data sharing practices of RCTs for COVID-19.

2. Methods

This systematic literature review was designed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [11].

2.1. Data sources and search strategies

Three information identification strategies were designed to identify registered clinical trials, peer-reviewed publications, and pre-print (i.e. non-peer reviewed) publications of RCTs of interventions for COVID-19.

To identify trials listed in clinical trial registries, we searched listings in: ClinicalTrials.gov; the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP); the European Clinical Trials Registry; the Chinese Clinical Trial Registry, the German Clinical Trials Register; the Japan Primary Registries Network, the Iranian Clinical Trial Registry, and the Australian New Zealand Clinical Trials Registry. Searches were conducted using the terms ‘COVID-19 OR SARS-CoV-2 OR novel coronavirus 2019’ or database-specific tools to list COVID-19 registered trials, where available, in all clinical trial registries up to 15 July 2020 [3].

Second, we conducted systematic searches in MEDLINE and EMBASE (via Ovid) and the Cochrane Central Register of Controlled Trials (CENTRAL) to identify RCTs for the prevention or treatment of COVID-19 from 1 January 2020 to 12 July 2020 (Supplementary Tables 1–3). Finally, a search was conducted on 15 July 2020 to identify pre-print publications on medRxiv and bioRxiv (Supplementary Table 4) [12]. These three strategies were supplemented by hand searches of the reference lists of full texts identified in the search.

2.2. Trial selection and eligibility

Broad eligibility criteria were applied to select RCTs on the topic of prevention or treatment of patients with COVID-19 irrespective of interventions, controls, or outcomes (Table 1 ). The inclusion and exclusion criteria were applied to both the publications and registered trials (Table 1). Given an anticipated delay from study completion to results dissemination, clinical trial registries with a primary completion date of 1 June 2020 or earlier were eligible for inclusion [13,14]. Publications in languages other than English were excluded.

Table 1.

Population, intervention, comparator, outcomes, and study design (PICOS) criteria for trial selection.

Criteria	Details
Population	People with pre-exposure to SARS-CoV-2 virus People with post-exposure to SARS-CoV-2 virus Patients with COVID-19 disease
Interventions	Any interventions for COVID-19
Comparator	No restrictions
Outcomes	No restrictions
Study design	Randomized controlled trials
Others	Peer-reviewed and non-peer-reviewed publications in the English language Registered randomized controlled trials with primary completion date on 1 June 2020 or earlier⁎

^⁎Trials were included with a primary date of completion by 1 June 2020 or earlier to provide reasonable time for preprints or publications with trial results.

Two reviewers (AD and JJHP) independently reviewed all abstracts and proceedings identified in the literature searches. The full-text publications of potentially relevant abstracts were then retrieved and assessed for eligibility by two independent reviewers (AD and ZL). Trial registries were screened and reviewed by a paired group of six reviewers (NEZ, LD, GH, GS, SK, and OH). Hand searches were performed on the reference lists of full texts identified in the search (AD and ZL). Discrepancies in study selection were resolved by discussion or, when necessary, by a third investigator (KT or EJM).

2.3. Data extraction

Two independent reviewers (AD and ZL) extracted data into a standardized data extraction spreadsheet. For each eligible trial, we extracted the trial identifier, trial registry, study location sites, population of interest, intervention(s), control(s), study design, recruitment target, actual recruitment, and the outcomes to be collected. We also captured any plans to share data or formal data sharing statements, as well as the anticipated time of data sharing. Cross-checking for consistency was conducted by other reviewers (LD and KT). A risk of bias assessment was conducted by two reviewers (AD and ZL) according to the Cochrane Collaboration's risk-of-bias assessment tool [15].

2.4. Data synthesis

We summarized the characteristics of included trials and publications across three broad areas: 1) Completion versus reporting of registered clinical trials; 2) Methodological designs of published clinical trials; and 3) Data sharing agreements of published clinical trials.

3. Results

3.1. Registry and literature search

Across the three data gathering strategies, we identified 178 trials in clinical trial registries, 7830 records in medical literature database, and 14 additional publications through hand searches of bibliographies and trial registries (Fig. 1 ). Of the 7844 abstracts, 319 records were selected for full-text review, with 38 publications (35 trials) satisfying all inclusion criteria. Twenty-five peer reviewed publications were identified, with an additional 13 pre-prints. A complete list of registered trials (Supplementary Table 5), included peer reviewed and preprint publications (Supplementary Tables 6 and 7), excluded peer reviewed and preprint publications (Supplementary Table 8), and risk-of-bias assessments (Supplementary Table 9) are available in the supplemental materials.

Fig. 1.

Study flow diagram.

Using the Cochrane risk of bias tool, the published RCTs were most often judged to have some concerns for randomization, deviations from intervention, measurement of the outcome, and selection of the reported results. The majority of trials were judged to have a low risk of missing outcome data. The overall risk of bias for most published trials was judged to have some concerns (Supplementary Table 9).

3.2. Completion versus reporting of registered clinical trials

We identified 178 RCTs with a primary completion date of 1 June 2020 or earlier (Supplementary Table 10). Across these studies, the median planned enrollment was 100 participants (interquartile range [IQR]: 60, 186). Most trials (n = 112/178, 62.92%) were in hospital settings, compared to 25 trials (14.04%) in outpatient settings and four trials (2.25%) in prevention. Most trials were 2-arm studies (n = 139/178, 78.09%), while 26 (14.61%) were 3-arm studies, and 13 (7.30%) reported four or more arms. Trials were often open-label (n = 79/178, 44.38%), double blind (n = 28/178, 15.73%) or single blind (n = 12/178, 6.74%). Study sites most frequently included China (n = 90/178, 50.56%), Iran (n = 50/178, 28.09%), and the United States (n = 10/178, 5.62%). Seven (3.93%) of the 178 trials have indicated that they suspended their recruitment, 72 (40.45) reported they were recruiting, 43 (24.16%) were not recruiting, 2 (1.12%) reported an unclear status, while 54 trials (30.34%) indicated that they are complete. While twelve trials (6.74%) had linked publications to their respective trial registry entry, the vast majority of registry entries (n = 166/178, 93.26%) had not published any study results by 15 July 2020.

While 35 trials have published their results, only twelve trials (n = 12/35, 34.29%) have linked a publication to a respective trial registry. Five of the published trials (n = 5/35, 14.29%) did not report clinical trial registration and 18 trials (n = 18/35, 51.43%) did not link publications in their respective registries. The vast majority of the 178 registered trials (n = 166/178, 93.25%) did not report study results in their respective registries by 15 July 2020.

3.3. Methodological designs of published clinical trials

Thirty-five RCTs were identified (Table 2 ; Supplementary Tables 10 and 11), consisting of 24 peer-reviewed publications and 11 pre-prints. As observed in the registered trials with primary completion dates of 1 June 2020 or earlier, the majority of published trial evidence is from China (n = 22/35, 62.86%). Similarly, most trials were 2-arm designs (n = 30/35, 85.71%) and were in hospital settings (n = 31/35, 88.57%).

Table 2.

Trial characteristics of published randomized controlled trials for COVID-19.

Trial ID	Trial registry	Region	Population	Intervention	Comparator	Recruitment target	Actual recruitment	Recruitment achieved %
Peer-reviewed articles
RASTAVI [33]	NCT03201185	Spain	Hospitalized	Ramipril	SOC	NR	109	NA
ACTT [17]	NCT04280705	Multinationale	Hospitalized	Remdesivir	Placebo	800	1063d	132.9%
CloroCOVID19 [34]	NCT04323527	Brazil	Hospitalized	HCQ	Placebo	440	81	18.4%
COVID-19 PEP [29]	NCT04308668	USA, Canada	Household or occupational post-exposure	HCQ	Placebo	1500	821	54.7%
Cao 2020A [18]	ChiCTR2000029308	China	Hospitalized	LPV/r	SOC	160	199d	124.4%
Cao 2020B [24]	ChiCTR-OPN-2000029580	China	Hospitalized	Ruxolitinib	Placebo	70	43	61.4%
Chen 2020A [35]	NCT04261517	China	Hospitalized	HCQ	SOC	NR	30	NA
Christensen 2020 [36]	NR	Denmark	Health care workers	Video training for PPE	In-person training	NR	21	NA
Goldman 2020 [37]	NCT04292899	Multinationalf	Hospitalized	Remdesivir 10 days	Remdesivir 5 days	400	402	100.5%
Hu 2020 [38]	ChiCTR-TRC-2000029434	China	Hospitalized	Lianhua Qingwen Capsules	SOC	240	284	118.3%
Hung 2020 [39]	NCT04276688	Hong Kong	Hospitalized	LPV/r + Ribavirin + Interferon-beta-1b	LPV/r	70	127	181.4%
Li 2020A [40]	ChiCTR2000029757	China	Hospitalized	Convalescent plasma	SOC	200	103	51.5%
Li 2020B [25]	NR	China	Hospitalized	Low-dose chest CT	Conventional-dose chest CT	NR	60	NA
Liu 2020A [41]	NR	China	Hospitalized	Progressive muscle relaxation technology	Routine care	NR	51	NA
Liu 2020B [42]	NR	China	Hospitalized	Respiratory muscle training & exercise	SOC	72	72	100.0%
Mitjà 2020 [43]	NCT04304053	Spain	Outpatients	HCQ	SOC	280	293	104.6%
Skipper 2020 [44]	NCT04308668	United States, Canada	Outpatients with high risk exposure	HCQ	SOC	1500	491d	32.7%
Tang 2020 [26]	ChiCTR2000029868	China	Hospitalized	HCQ	SOC	360	150	41.7%
Wang 2020A [20]	NCT04257656	China	Hospitalized	Remdesivir	Placebo	453	237	52.3%
Wei 2020A [45]	NR	China	Hospitalized	Internet-based intervention	Supportive care	NR	26	NA
Wen 2020 [46]	ChiCTR2000029381	China	Hospitalized	1.Xuebijing 50 mL 2.Xuebijing 100 mL	SOC	NR	60	NA
Wu 2020 [21]	ChiCTR2000029658	China	ICU	High-flow nasal oxygenation	SOC	60	60	100.0%
Ye 2020A [47]	ChiCTR2000029418	China	Hospitalized	Chinese herbal medicine + SOC	SOC	NR	42	NA
GRECCO-19 [48]	NCT04326790	Greece	Hospitalized	Colchicine	SOC	NR	105	NA
Pre-print articles
RECOVERY [16]	NCT04381936; ISRCTN 50189673	United Kingdom	Hospitalized	Dexamethasone	SOC	6000b	6425d	107.1%
Yuan 2020A [49]	ChiCTR2000029431	China	Hospitalized	Tc-MDP + SOC	SOC	NR	21	NA
ELACOI [27]	NCT04252885	China	Hospitalized	1.LPV/r 2.Arbidol	SOC	125	86	68.8%
Gharbharan 2020 [19]	NCT04342182	Netherlands	Hospitalized	Convalescent plasma + SOC	SOC	426	86a	20.2%
Chen 2020B [50]	ChiCTR2000029559	China	Hospitalized	HCQ	SOC	NR	62	NA
Chen 2020C [51]	ChiCTR2000030054	China	Hospitalized	1.Chloroquine 2.HCQ	SOC	100	94	94.0%
Chen 2020D [52]	ChiCTR2000030254	China	Hospitalized	Favipiravir	Arbidol	240	240	100.0%
Zhong 2020 [53]	ChiCTR2000029851	China	Hospitalized	a-Lipoic acid	Placebo	NR	17	NA
Zheng 2020 [54]	ChiCTR2000029496	China	Hospitalized	1.Novaferon 2.LPV/r + Novaferon	LPV/r	NR	89	NA
Lou 2020 [55]	ChiCTR2000029544	China	Hospitalized	1.Favipiravir 2.Baloxavir marboxil	SOCc	NR	30	NA
Davoudi-Monfared 2020 [56]	IRCT20100228003449N28	Iran	Hospitalized	Interferon β-11a	SOC	NR	81	NA

NR – Not reported; NA – Not applicable; HCQ – (Hydroxy)chloroquine; LPV/r – Lopinavir/ritonavir; SOC – Standard of care; PPE – Personal protective equipment; ICU – Intensive care unit; CT – Computed tomography; Tc-MDP – Technetium (99mTc) medronic acid.

^aThe trial was halted prematurely due to concerns about the potential benefit of convalescent plasma.

^bThis applies to the dexamethasone + SOC arms of this adaptive trial. The preprint does not include all arms of the RECOVERY Trial (n = 12,022 as of 9 July 2020).

^cThe control group had existing antiviral treatment including LPV/r or darunavir/cobicistat and arbidol.

^dSample size reassessment was done during the trial.

^eUSA, Denmark, UK, Greece, Germany, Korea, Mexico, Spain, Japan, and Singapore.

^fUSA, Italy, Spain, Germany, Hong Kong, Singapore, South Korea, Taiwan.

For the published trials, planned recruitment varied from 60 to 6000 participants with a median of 260 participants (IQR: 118.75, 443.25) while actual recruitment numbers varied from 21 to 6425 participants with a median of 86 participants (IQR: 55.5, 218). Ten clinical trials (n = 10/35, 28.57%) did not reach their recruitment target, with actual recruitment ranging from 18.4% to 94.0%, compared to 100.0% to 181.4% for the trials that did achieve their enrollment targets. However, the RECOVERY Trial, ACTT Trial, and ChiCTR2000029308 completed sample size reassessments during the trials and subsequently reached their recruitment target [[16], [17], [18]]. Of the ten clinical trials that did not reach recruitment targets, eight (n = 8/35, 22.86%) were due to feasibility constraints. The hydroxychloroquine inpatient clinical trial (NCT04342182) in Brazil was halted prematurely due to concerns about the potential benefit of the intervention [19]. Actual recruitment was limited in one trial due to ethical concerns which resulted in early stopping [19].

3.4. Data sharing agreements of published clinical trials

As presented in Table 3 , most trials have reported plans to share data (n = 27/35, 77.14%). Of these, the mechanism of sharing is often upon individual request (n = 21/27, 77.78%). For instance, two trials conducted in China have reported that approval is required from the Human Genetic Resources Administration of China prior to data sharing [20,21]. The stated time of data sharing varied from immediately upon trial completion to up to one year after publication.

Table 3.

Data sharing agreement of primary published randomized controlled trials for COVID-19.

Trial ID	Registry number	Plans to share data	Data sharing mechanism	Time of data sharing
Peer-reviewed articles
RASTAVI	NCT03201185	NR	NR	NR
ACTT	NCT04280705	Yes	Email with the corresponding author	After finalization of clinical study report
CloroCOVID19	NCT04323527	No	Not available	Not available
COVID-19 PEP	NCT04308668	Yes	Available upon request	Within 1 month of publication for up to 3 years
Cao 2020A	ChiCTR2000029308	Yes	Contact with the corresponding author	1 year after publication
Cao 2020B	ChiCTR-OPN-2000029580	NR	NR	NR
Chen 2020A	NCT04261517	Undecided	Undecided	Undecided
Christensen 2020	NR	Yes	The dataset supporting the conclusions of this article is included within the article	Immediate
Goldman 2020	NCT04292899	Yes	Available upon request	Within 18 months of trial completion
Hu 2020	ChiCTR-TRC-2000029434	Yes	Available upon request	6 months after trial completion
Hung 2020	NCT04276688	Yes	Can be obtained by submitting a valid research proposal to the corresponding author	Upon request
Li 2020B	NR	Yes	Available upon request	Upon request
Li 2020A	ChiCTR2000029757	Yes	Available with publication	Immediate
Liu 2020A	NR	NR	NR	NR
Liu 2020B	NR	NR	NR	NR
Mitjà 2020	NCT04304053	NR	NR	NR
Skipper 2020	NCT04308668	Yes	Open access	Beginning 22 July 2020
Tang 2020	ChiCTR2000029868	Yes	Available upon request	Within 6 weeks of trial completion
Wang 2020A	ChiCTR2000029868	Yes	Approval from Human Genetic Resources Administration of China required	Upon request
Wei 2020A	NR	NR	NR	NR
Wen 2020	NCT04257656	Undecided	Undecided	Undecided
Wu 2020	ChiCTR2000029658	Yes	Available with approval from the Human Genetic Resources Administration of China	Upon request
Ye 2020A	ChiCTR2000029381	Yes	Available upon request by contact with the corresponding author	Upon request
GRECCO-19	ChiCTR2000029418	Yes	Available upon request by contact with the corresponding author	Upon request
Pre-print articles
RECOVERY	NCT04326790	Yes	Available upon request	Available with publication
Yuan 2020A	ChiCTR2000029431	Yes	Available upon request	Upon request
ELACOI	NCT04252885	Yes	Requests should be directed to the lead contact	Upon request
Gharbharan 2020	NCT04342182	Yes	Available upon request to non-for-profit organizations	Upon request
Chen 2020B	ChiCTR2000029559	Yes	The dataset supporting the conclusions of this article is included within the article.	Immediate
Chen 2020C	ChiCTR2000030054	Yes	Available upon request by contact with the corresponding author	Upon request
Chen 2020D	ChiCTR2000030254	Yes	With the permission of the corresponding author, we can provide participant data, statistical analysis	Upon request
Zhong 2020	ChiCTR2000029851	Yes	All data referred to in the manuscript was available	Immediate
Zheng 2020	ChiCTR2000029496	Yes	Written requests need to be submitted to corresponding authors.	Upon request
Lou 2020	ChiCTR2000029544	Yes	Available 1 year after publication with no time limit	1 year after publication
Davoudi-Monfared 2020	IRCT20100228003449N28	Yes	Available upon request	Upon request

NR – Not reported.

4. Discussion

To our knowledge, this is the first systematic review of registered clinical trials, peer-reviewed publications, and pre-print publications of COVID-19 RCTs that focuses specifically on reporting, methodological designs, and data sharing practices. While we accommodated for a time lag between study completion and results dissemination, the vast majority of the 178 registered trials had not yet published findings, either in peer-reviewed journals or in pre-print repositories. This finding highlights a need for more rapid and robust reporting practices, as effective dissemination is essential to reduce duplicated research efforts while providing much-needed guidance for future research, practice, and policy [22,23].

Nearly all published trials were conducted with hospitalized patients, highlighting a lack of evidence emerging in the outpatient as well as pre- or post-exposure prophylaxis settings. While there is undoubtedly value in evaluating interventions for the most severely ill patients, there are several public health motivations to direct research efforts and funding to managing patients in the community or mitigating the risk of infection altogether. As most trials did not reach recruitment targets due to feasibility constraints, there are also concerns to be raised regarding the statistical underpowering of studies and the validity of findings from these investigations [[24], [25], [26], [27]]. However, this should be considered in light of trials or trial arms which were terminated early for ethical reasons and that the widespread limitations imposed by the pandemic may have impacted recruitment practices [19,28].

In most published trials, investigators indicated that data would be made available upon request, with timelines for such inquiries varying from immediately following publication to one year after findings were disseminated. While some trials have shared de-identified individual patient data, most trials have not yet made such data available [29]. Timely and robust data sharing is critical to ensuring that the efforts of both patients and investigators is sufficiently leveraged to yield potential health benefits, provide real-time guidance, and facilitate collaboration within the scientific community [22,23]. Clinical trial protocols with robust and rapid data sharing are particularly warranted in this time of global health crisis. Our results highlight opportunities for enhanced data sharing across the scientific community. Such collaborations may advance our understanding of prevention and treatment of COVID-19, with rapid, real-world applications and meaningful implications for addressing the health, social, and economic burden of the virus [[30], [31], [32]].

Our conclusions are based on a rigorous review of ongoing and completed trials in COVID-19, including systematic searches in international clinical registries, major medical literature databases, and pre-print repositories. The inclusion of pre-print publications afforded a more complete picture of COVID-19 trial reporting practices, as this acknowledges the delays inherent to publishing through a peer-reviewed process. As the evidence base for COVID-19 interventions continues to evolve, with new registered trials and completed trials reporting their findings, the conclusions drawn based on our review may change. However, we sought to provide a timely analysis of the early COVID-19 RCT research landscape to identify limitations and opportunities for individual researchers and the broader research community to improve reporting practices and enter into stronger, more effective collaborations.

4.1. Limitations

This study has several limitations. First, not all RCTs are necessarily registered and our review of the 178 trials was limited to the most recently updated data available in the respective registries. Second, the phases of the trials varied and due to the small sample, there was likely high heterogeneity in the 35 published trials. Third, given the early nature of this study, the majority of published evidence was from China, thus the included trials are not representative of the conduct of trials globally. Fourth, our search strategy included hand searching to supplement our database searches and this introduces subjectivity. However, we sought to address this with two independent reviewers. Finally, we used a limited time interval to examine clinical trials from 1 January 2020 to 1 June 2020, thus the included trials are not representative of all active trials studying COVID-19. However, the purpose of this interval was to allow us to examine and report on the early clinical trial practices in response to COVID-19.

5. Conclusions

The findings of our study highlight the limitations of the reporting and feasibility of COVID-19 randomized controlled trials. This systematic review provides guidance for future trials, including a need for more efficient reporting of clinical trial results, greater diversity of clinical trial patient settings, and robust data sharing practices for meaningful and rapid real-world application to the COVID-19 pandemic.

Funding

No funding was received for this study.

Contributors

AD, JJHP, and EJM conceptualized the study. AD, JJHP, MZ, NEZ, ZL, LD, GH, GS, SK, OH, KT, and EJM contributed to data curation; formal analysis; investigation; methodology; validation; visualization; and writing - review & editing. AD, JJHP, and MZ contributed to writing - original draft. JJHP, KT, and EJM provided project administration; resources; software; and supervision.

Declaration of Competing Interest

The authors do not have any competing interests.

Appendix A. Supplementary data

Supplementary material