Abstract
This cross-sectional study evaluates the characteristics and expected strength of evidence of coronavirus disease 2019 studies registered on ClinicalTrials.gov.
The coronavirus disease 2019 (COVID-19) pandemic has led to a massive activation of clinical research. The methodological strength of these studies is not well characterized but has implications for the quality of evidence produced. We evaluated the characteristics and expected strength of evidence of COVID-19 studies registered on ClinicalTrials.gov.
Methods
For this cross-sectional analysis, we searched ClinicalTrials.gov on May 19, 2020, using the terms COVID-19, SARS-CoV-2, 2019-nCov, 2019 novel coronavirus, and severe acute respiratory syndrome coronavirus 2 and extracted all structured data fields.1 We excluded withdrawn, suspended, terminated, or expanded-access studies. We categorized reported outcomes and graded studies using the 2011 Oxford Centre for Evidence-Based Medicine (OCEBM) level of evidence framework.2 A single reviewer (K.P.) verified studies for inclusion and removed duplicates, and 2 reviewers (A.C.P. and M.P.T.) audited results. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.
Results
We identified 1551 studies registered from March 1, 2011, to May 19, 2020, meeting inclusion criteria: 911 (58.7%) interventional (including 664 randomized clinical trials [RCTs]) and 640 (41.3%) observational studies (Table); 1180 (76.1%) were single center. Frequently reported primary and secondary outcomes include mortality (526 [33.9%]), ventilation requirement (413 [26.6%]), and treatment complications (359 [23.1%]). Of the 1551 studies, 451 (29.1%) could potentially yield OCEBM level 2 evidence, or the highest level of individual study evidence.2
Table. COVID-19 Studies Registered on ClinicalTrials.gov.
| Study characteristic | Study type, No. (%)a | ||
|---|---|---|---|
| All (n = 1551) | Observational (n = 640) | RCTs (n = 664) | |
| Study size, No. of participants | |||
| 0-100 | 580 (37.4) | 180 (28.1) | 231 (34.8) |
| 101-1000 | 731 (47.1) | 308 (48.1) | 358 (53.9) |
| >1000 | 240 (15.5) | 152 (23.8) | 75 (11.3) |
| Funding source | |||
| NIH or federal | 30 (1.9) | 17 (2.7) | 6 (0.9) |
| Industry | 223 (14.4) | 42 (6.6) | 148 (22.3) |
| Study centers | |||
| 1 | 1180 (76.1) | 517 (80.8) | 461 (69.4) |
| 2 | 104 (6.7) | 27 (4.2) | 54 (8.1) |
| ≥3 | 267 (17.2) | 96 (15.0) | 149 (22.4) |
| Recruitment status | |||
| Not yet recruiting | 588 (37.9) | 186 (29.1) | 314 (47.3) |
| Active, not recruiting | 58 (3.7) | 35 (5.5) | 13 (2.0) |
| Recruiting or enrolling | 837 (54.0) | 370 (57.8) | 329 (49.5) |
| Completed | 68 (4.4) | 49 (7.7) | 8 (1.2) |
| Study design features | |||
| Placebo group | NA | 0 | 203 (30.6) |
| Blinded | NA | NA | 364 (54.8) |
| Intervention type | |||
| Behavioral | 63 (4.1) | 21 (3.3) | 32 (4.8) |
| Critical care interventions | 27 (1.7) | 6 (0.9) | 16 (2.4) |
| Diagnostics | 149 (9.6) | 109 (17.0) | 2 (0.3) |
| Drugs and biologic compounds | 599 (38.6) | 28 (4.4) | 486 (73.2) |
| Hydroxychloroquine or chloroquine | 174 (11.2) | 4 (0.6) | 155 (23.3) |
| Remdesivir | 9 (0.6) | 1 (0.2) | 7 (1.1) |
| Other antivirals | 55 (3.5) | 0 | 48 (7.2) |
| Other antimicrobials | 73 (4.7) | 1 (0.2) | 66 (9.9) |
| Tocilizumab | 31 (2.0) | 2 (0.3) | 21 (3.2) |
| Corticosteroids | 23 (1.5) | 1 (0.2) | 20 (3.0) |
| Convalescent plasma | 71 (4.6) | 1 (0.2) | 39 (5.9) |
| Oxygenation | 25 (1.6) | 2 (0.3) | 15 (2.3) |
| Stem cells | 30 (1.9) | 0 | 21 (3.2) |
| Vaccination | 28 (1.8) | 3 (0.5) | 18 (2.7) |
| OCEBM level of evidence | |||
| 2 | 451 (29.1) | 87 (13.6) | 364 (54.8) |
| 3 | 908 (58.5) | 361 (56.4) | 300 (45.2) |
| 4 | 192 (12.4) | 192 (30.0) | 0 |
| Measured outcomesb | |||
| Mortality | 526 (33.9) | 131 (20.5) | 330 (49.7) |
| Intensive care requirement | 249 (16.1) | 64 (10.0) | 153 (23.0) |
| Hospitalization requirement | 340 (21.9) | 86 (13.4) | 211 (31.8) |
| Ventilation requirement | 413 (26.6) | 73 (11.4) | 284 (42.8) |
| Treatment complications | 359 (23.1) | 52 (8.1) | 233 (35.1) |
Abbreviations: COVID-19, coronavirus disease 2019; NA, not applicable; NIH, National Institutes of Health; OCEBM, Oxford Center for Evidence Based Medicine, 2011 edition; RCT, randomized clinical trial.
Percentages have been rounded and may not total 100.
Studies have more than 1 measured outcome.
Across 664 RCTs, the primary outcome most frequently pertained to clinical course (323 [48.6%]); 51 (7.7%) had a primary outcome of mortality, and 42 (6.3%) had a composite end point including mortality (Figure). Blinding (required for OCEBM level 2 evidence) was reported for 364 RCTs, of which 195 (29.3%) were placebo-controlled, 238 (35.8%) planned enrollment of more than 100 participants, and 113 (17.0%) reported at least 2 study centers or sites. Only 75 RCTs (11.3%) were placebo-controlled and blinded with at least 2 study centers (60 with enrollment >100 participants; 24 with >500 participants). Most RCTs evaluated drugs and biologic compounds (486 [73.2%]); 155 (23.3%), hydroxychloroquine or chloroquine; 7 (1.1%), remdesivir; 48 (7.2%), other antivirals; 21 (3.2%), tocilizumab; and 20 (3.0%), corticosteroids.
Figure. Primary Outcome of Randomized Clinical Trials (RCTs) by Enrollment Size.
Primary listed outcome of 664 RCTs organized by intended enrollment size. Most studies have intended enrollment fewer than 500 patients, and the most common primary end points address (1) clinical course, (2) infection rates, or (3) persistence of viral detection.
Of the 640 observational studies, 517 (80.8%) were single center and 123 (19.2%) were multicenter, 36 of which had 10 or more centers. Eighty-seven studies (13.6%) were prospective cohort studies that could yield level 2 evidence.
Discussion
Although a few large multicenter trials may generate high-quality evidence, the large proportion of studies with an expected low level of evidence is concerning. Rapid dissemination of studies with low-quality evidence studies can influence public opinion, government actions, and clinical practice in potentially harmful ways,3 especially with a rising tide of COVID-19 study dissemination via preprint or other strategies ahead of peer review.
A number of measures can mitigate these issues. Preprint results could be accompanied by transparent data sharing ahead of peer review. Rapidly deployable systems for multicenter registries and trials should be created but with an emphasis on quality, not just speed. These systems could be activated for global health crises, leading to streamlined operations for international study coordination, data sharing, and central institutional review boards. These systems can also combine and harmonize similar observational studies into large multicenter studies or embed randomization or pragmatic features when possible. The World Health Organization’s 100-country, adaptive Solidarity trial comparing 4 treatment arms uses a common data platform and operations, but results may not be available for months.4,5 Finally, we urge institutional review boards to work with investigators to ensure that experimental research involving human participants is sufficiently well designed to achieve the goal of generating clinically meaningful evidence.
Our study has important limitations. Current regulations only require drug, device, or biological studies to register with ClinicalTrials.gov. Half of non-US studies are estimated to not be registered with ClinicalTrials.gov,6 and OCEBM is most accurately applied to completed studies.
This cross-sectional study found that despite the marked rise in COVID-19 studies, only 29.1% of those registered in ClinicalTrials.gov have the potential to result in OCEBM level 2 evidence. Of the RCTs, only 29.3% are placebo-controlled, blinded studies. Global decline in new cases could also stall enrollment. Even before results are known, most studies likely will not yield meaningful scientific evidence at a time when rapid generation of high-quality knowledge is critical.
References
- 1.ClinicalTrials.gov. COVID-19 search details. Accessed April 8, 2020. https://clinicaltrials.gov/ct2/results/details?cond=COVID-19
- 2.OCEBM Levels of Evidence Working Group. OCEBM levels of evidence. Published May 1, 2016. Accessed April 9, 2020. https://www.cebm.net/index.aspx?o=5653
- 3.Kim AHJ, Sparks JA, Liew JW, et al. A rush to judgment? rapid reporting and dissemination of results and its consequences regarding the use of hydroxychloroquine for COVID-19. Ann Intern Med. 2020;172(12):819-821. doi: 10.7326/M20-1223 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mullard A. Flooded by the torrent: the COVID-19 drug pipeline. Lancet. 2020;395(10232):1245-1246. doi: 10.1016/S0140-6736(20)30894-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.World Health Organization. “Solidarity” clinical trial for COVID-19 treatments. Updated June 4, 2020. Accessed May 20, 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments
- 6.Banno M, Tsujimoto Y, Kataoka Y. Studies registered in non-ClinicalTrials.gov accounted for an increasing proportion of protocol registrations in medical research. J Clin Epidemiol. 2019;116:106-113. doi: 10.1016/j.jclinepi.2019.09.005 [DOI] [PubMed] [Google Scholar]